{"title": "Efficient Online Portfolio with Logarithmic Regret", "book": "Advances in Neural Information Processing Systems", "page_first": 8235, "page_last": 8245, "abstract": "We study the decades-old problem of online portfolio management and propose the first algorithm with logarithmic regret that is not based on Cover's Universal Portfolio algorithm and admits much faster implementation. Specifically Universal Portfolio enjoys optimal regret $\\mathcal{O}(N\\ln T)$ for $N$ financial instruments over $T$ rounds, but requires log-concave sampling and has a large polynomial running time. Our algorithm, on the other hand, ensures a slightly larger but still logarithmic regret of $\\mathcal{O}(N^2(\\ln T)^4)$, and is based on the well-studied Online Mirror Descent framework with a novel regularizer that can be implemented via standard optimization methods in time $\\mathcal{O}(TN^{2.5})$ per round. The regret of all other existing works is either polynomial in $T$ or has a potentially unbounded factor such as the inverse of the smallest price relative.", "full_text": "Ef\ufb01cient Online Portfolio with Logarithmic Regret\n\nHaipeng Luo\n\nDepartment of Computer Science\nUniversity of Southern California\n\nChen-Yu Wei\n\nDepartment of Computer Science\nUniversity of Southern California\n\nhaipengl@usc.edu\n\nchenyu.wei@usc.edu\n\nKey Laboratory of Machine Perception, MOE, School of EECS, Peking University\nCenter for Data Science, Peking University, Beijing Institute of Big Data Research\n\nKai Zheng\n\nzhengk92@pku.edu.cn\n\nAbstract\n\nWe study the decades-old problem of online portfolio management and propose\nthe \ufb01rst algorithm with logarithmic regret that is not based on Cover\u2019s Universal\nPortfolio algorithm and admits much faster implementation. Speci\ufb01cally Universal\nPortfolio enjoys optimal regret O(N ln T ) for N \ufb01nancial instruments over T\nrounds, but requires log-concave sampling and has a large polynomial running time.\nOur algorithm, on the other hand, ensures a slightly larger but still logarithmic\nregret of O(N 2(ln T )4), and is based on the well-studied Online Mirror Descent\nframework with a novel regularizer that can be implemented via standard optimiza-\ntion methods in time O(T N 2.5) per round. The regret of all other existing works\nis either polynomial in T or has a potentially unbounded factor such as the inverse\nof the smallest price relative.\n\n1\n\nIntroduction\n\nWe consider the well-known online portfolio management problem [8], where a learner has to\nsequentially decide how to allocate her wealth over a set of N \ufb01nancial instruments in order to\nmaximize her return, importantly under no assumptions at all on how the market behaves. Speci\ufb01cally,\nfor each trading period t = 1, . . . , T , the learner \ufb01rst decides the proportion of her wealth to invest\non each stock, and then by the end of the period observes the return of each stock and continues to\ninvest with her total wealth. The goal of the learner is to maximize the ratio between her total wealth\nafter T rounds and the total wealth of the best constant-rebalanced portfolio (CRP) which always\nrebalances the wealth to ensure a \ufb01xed proportion of investment for each stock. Equivalently, the\nlearner aims to minimize her regret, which is the negative logarithm of the aforementioned ratio.\nThe minimax optimal regret for this problem is O(N ln T ), achieved by Cover\u2019s Universal Portfolio\nalgorithm [8]. This algorithm requires sampling from a log-concave distribution and all known\nef\ufb01cient implementations have large polynomial (in N and T ) running time, such as O(T 14N 4) [15].1\nOnline Newton Step (ONS) [12], on the other hand, follows the well-studied framework of Online\nMirror Descent (OMD) with a simple time-varying regularizer and admits much faster implementation\nvia standard optimization methods. The regret of ONS is O(GN ln T ) where G is the largest gradient\n(cid:96)\u221e-norm encountered over T rounds (formally de\ufb01ned in Section 1.1) and can be arbitrarily large\nmaking the bound meaningless. A typical way to prevent unbounded gradient is to mix the output\n\n1Recent improvements on log-concave sampling such as [7, 16, 17] may lead to improved running time, but\n\nit is still a large polynomial.\n\n32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montr\u00e9al, Canada.\n\n\fTable 1: Comparisons of regret and running time of different algorithms. Note that G is potentially\nunbounded. For running time, we assume Interior Point Method is used to solve the involved\noptimization problems, and omit all logarithmic (in N and T ) factors.\n\nAlgorithm\n\nUniversal Portfolio [8, 15]\n\nONS [12]\nFTRL [3]\nEG [14]\n\nSoft-Bayes [19]\n\nADA-BARRONS (this work)\n\nRegret\nN ln T\nGN ln T\n\u221a\nG2N ln(N T )\n\u221a\nG\n\nT ln N\nN T ln N\nN 2(ln T )4\n\nTime (per round)\n\nT 14N 4\nN 3.5\nT N 2.5\n\nN\nN\n\nT N 2.5\n\n\u221a\n\n\u221a\n\n\u221a\nT ln N ) and O(G\n\nof ONS with a small amount of uniform distribution, which after optimal trade-off can at best\nlead to a regret bound of O(N\nT ln T ). An earlier work [3] achieves a worse regret bound of\nO(G2N ln(N T )) via an ef\ufb01cient algorithm based on another well-known framework Follow-the-\nRegularized-Leader (FTRL).\nThere are also extremely ef\ufb01cient approaches with time complexity O(N ) or O(N ln N ) per round,\n\u221a\nsuch as exponentiated gradient [14], online gradient descent [22], and Soft-Bayes from recent work\nof [19]. The \ufb01rst two achieve regret of order O(G\nT ) respectively2 while the\nlast one achieves O(\nN T ln N ) without the dependence on G. Despite being highly ef\ufb01cient, all of\nthese approaches fail to achieve the optimal logarithmic dependence on T for the regret.\nAs one can see, earlier works all exhibit a trade-off between regret and time complexity. A long-\nstanding open question is how fast an algorithm with optimal regret can be. Speci\ufb01cally, are there\nalgorithms with optimal regret and similar or even better time complexity compared to ONS?\nIn this work, we make a signi\ufb01cant step toward answering this question by proposing a simple\nalgorithm with regret O(N 2(ln T )4) and time complexity O(T N 2.5) per round. To the best of our\nknowledge, this is the \ufb01rst algorithm with logarithmic regret (and no dependence on G) that is\nnot based on Cover\u2019s algorithm and admits fast implementation comparable to ONS and [3]. As a\ncomparison, we show in Table 1 the regret and time complexity of existing works and ours, where\nfor OMD/FTRL-type algorithms we do a naive calculation of the running time based on the Interior\nPoint Method [18] (to solve the key optimization problems involved), despite the possibility of even\nfaster implementation.\nOur algorithm is parameter-free and deterministic. It follows the OMD framework with a novel\nregularizer that is a mixture of the one used in ONS and the so-called log-barrier (a.k.a. Burg\nentropy) [10, 2, 21].3 Critically, our algorithm also relies on an increasing learning rate schedule for\nthe log-barrier similar to recent works on bandit problems [2, 21], as well as another sophisticated\nadaptive tuning method for the learning rate of the ONS regularizer, which resembles the standard\ndoubling trick but requires new analysis since monotonicity does not hold for our problem.\n\n1.1 Notation and Setup\n\nThe online portfolio problem \ufb01ts into the well-studied online learning framework (see for exam-\nple [13]). Formally, the problem proceeds for T rounds (for some T > N). On each round\nt = 1, . . . , T , the learner \ufb01rst decides a distribution xt \u2208 \u2206N where \u2206N is the (N \u2212 1)-dimensional\nsimplex. After that the learner observes the price relative vector rt \u2208 RN\n+ so that her total wealth\nchanges by a factor of (cid:104)xt, rt(cid:105). Taking the negative logarithm, this corresponds to observing a loss\nfunction ft(x) = \u2212 ln(cid:104)xt, rt(cid:105) for x \u2208 \u2206N , chosen arbitrarily by an adversary. The regret of the\n\n2For online gradient descent, G is the largest gradient (cid:96)2-norm.\n3A recent work [6] uses a mixture of the Shannon entropy and log-barrier as the regularizer for a different\n\nproblem.\n\n2\n\n\fAlgorithm 1: BARrier-Regularized Online Newton Step (BARRONS)\n1 Input: 0 < \u03b2 \u2264 1\n2 De\ufb01ne: \u00af\u2206N = {x \u2208 \u2206N : xi \u2265 1\n3 Initialize: x1 = 1\n\n2 , 0 < \u03b7 \u2264 1\nN 1, A0 = N IN where 1 is the all-one vector and IN is the N \u00d7 N identity\n\nN T , \u2200i}\n\nmatrix.\n\n4 for t = 1, 2, . . . do\n5\n6\n\nPredict xt and observe loss function ft(x) = \u2212 ln(cid:104)x, rt(cid:105).\nMake updates\n\n(1)\n\n(2)\n\nAt = At\u22121 + \u2207t\u2207(cid:62)\n\n\u03b7t,i = \u03b7 exp\n\n(cid:18)\n\nt\n\n(cid:19)\nmax\ns\u2208[t]\n(cid:104)x,\u2207t(cid:105) + D\u03c8t(x, xt)\n\nN xs,i\n\nlogT\n\n1\n\nxt+1 = argmin\nx\u2208 \u00af\u2206N\n2 (cid:107)x(cid:107)2\n\nAt\n\n+(cid:80)N\n\n1\n\u03b7t,i\n\nln 1\nxi\n\n.\n\ni=1\n\nwhere \u2207t = \u2207ft(xt) and \u03c8t(x) = \u03b2\n\nlearner against a CRP parameterized by u \u2208 \u2206N is then de\ufb01ned as\n\nT(cid:88)\n\nft(xt) \u2212 T(cid:88)\n\nt=1\n\nt=1\n\nReg(u) =\n\nft(u) = \u2212 ln\n\nt=1 (cid:104)xt, rt(cid:105)\n\u03a0T\nt=1 (cid:104)u, rt(cid:105) ,\n\u03a0T\n\nwhich is exactly the negative logarithm of the ratio of total wealth per dollar invested between\nthe learner and the CRP u. Our goal is to minimize the regret against the best CRP, that is, to\nminimize maxu\u2208\u2206N Reg(u). This setup is also useful for other non-\ufb01nancial applications such as\ndata compression [9, 19].\nNote that the regret is invariant to the scaling of each rt and thus without loss of generality we assume\nmaxi\u2208[N ] rt,i = 1 for all t where we use the notation [n] to represent the set {1, . . . , n}. It is now\nclear what the aforementioned largest gradient norm G formally is: G = maxt\u2208[T ] (cid:107)\u2207ft(xt)(cid:107)\u221e =\n}, which in general can be unbounded. To control\nmaxt\u2208[T ],i\u2208[N ]\nits magnitude, previous works [3, 4, 12, 14] either explicitly force xt,i to be lower bounded, which\nleads to worse regret, or make the so-called no-junk-bonds assumption (that is, mint,i rt,i is not too\nsmall), which might make sense for the portfolio problem but not other applications [19]. Our main\ntechnical contribution is to show how this term can be automatically canceled by a negative regret\nterm obtained from the log-barrier regularizer with increasing learning rates.\n\n(cid:104)xt,rt(cid:105) \u2264 min{\n\nmint,i,xt,i\n\nmint,i rt,i\n\nrt,i\n\n1\n\n1\n\n,\n\n2 Barrier-Regularized Online Newton Step\n\nRecall that for a sequence of convex regularizers \u03c8t, the outputs of Online Mirror Descent are de\ufb01ned\nby xt+1 = argminx\u2208\u2206N (cid:104)x,\u2207t(cid:105) + D\u03c8t(x, xt) where \u2207t is a shorthand for \u2207ft(xt), D\u03c8t(x, y) =\n\u03c8t(x) \u2212 \u03c8t(y) \u2212 (cid:104)\u2207\u03c8t(y), x \u2212 y(cid:105) is the Bregman divergence associated with \u03c8t, and we start with\nx1 being the uniform distribution. The intuition is that we would like xt+1 to have small loss with\nrespect to a linear approximation of ft, and at the same time to be close to the previous decision xt to\nensure stability of the algorithm.\nAlthough not presented in this form originally, Online Newton Step [12] is an instance of OMD with\n\u03c8t(x) = \u03b2\nt (for some A0) is the gradient covariance\nmatrix and \u03b2 is a parameter. The analysis of [12] shows that the regret of ONS for the portfolio\nproblem is Reg(u) = O( N ln T\nan oracle tuning, by H\u00f6lder inequality this gives O(GN ln T ) as mentioned.\nTo get rid of the dependence on G, we observe the following. Since \u2207t = \u2212 rt\n(cid:104)xt,rt(cid:105), its (cid:96)\u221e-norm is\nlarge only when there is a stock with high reward rt,i while the learner puts a small weight xt,i on it.\nHowever, the reason that the weight xt,i is small is because the learner \ufb01nds it performing poorly\n\n) as long as \u03b2 \u2264 min(cid:8) 1\n\n(cid:9). Even assuming\n\n2 x(cid:62)Atx where At = At\u22121 + \u2207t\u2207(cid:62)\n\n8|(u\u2212xs)(cid:62)\u2207s|\n\n2 , mins\u2208[T ]\n\n2 (cid:107)x(cid:107)2\n\n= \u03b2\n\nAt\n\n\u03b2\n\n1\n\n3\n\n\fAt\n\ni=1\n\n1\n\u03b7t,i\n\nln 1\nxi\n\n2 (cid:107)x(cid:107)2\n\n+(cid:80)N\n\nprior to round t, which means that the learner had better choices and actually should have performed\nbetter than stock i previously (that is, negative regret against stock i). Now as stock i becomes good\nat time t and potentially in the future, as long as the learner can pick up this change quickly, the\noverall regret should not be too large.\nSimilar observations were made in previous work [2, 21] for different problems in the bandit setting,\nwhere they introduced the log-barrier regularizer with increasing learning rate to explicitly ensure a\nlarge negative regret term based on the intuition above. This motivates us to add an extra log-barrier\nregularizer to ONS for our problem. Speci\ufb01cally, we de\ufb01ne our regularizer to be the following\nmixture: \u03c8t(x) (cid:44) \u03b2\n, where \u03b7t,i is individual and time-varying learning\nrate. Different from previous work, we propose a more adaptive tuning schedule for these learning\nrates based on Eq. (1) (instead of a doubling schedule [2, 21]), but the key idea is the same: increase\nthe learning rate for a stock when its weight is small so that the algorithm learns faster in case the\nstock becomes better in the future. Another modi\ufb01cation is that we force the decision set to be\n\u00af\u2206N = {x \u2208 \u2206N : xi \u2265 1\nN T , \u2200i} instead of \u2206N to ensure an explicit lower bound for xt,i. We call\nthis algorithm BARrier-Regularized Online Newton Step (BARRONS) (see Algorithm 1).\nUnder the same condition on \u03b2 as for ONS, we prove the following key theorem for BARRONS\nwhich highlights the important negative regret term obtained from the extra log-barrier regularizer.\nNote that it is enough to provide a regret bound only against smooth CRP u \u2208 \u00af\u2206N since one can\nverify that the total loss of any CRP u \u2208 \u2206N can be approximated by a smooth CRP in \u00af\u2206N up to an\nadditive constant of 2 (Lemma 10 in Appendix B).\n\nTheorem 1. For any u \u2208 \u00af\u2206N , if \u03b2 \u2264 \u03b1T (u) for \u03b1t(u) (cid:44) min(cid:8) 1\nN(cid:88)\n\n(cid:18) N ln T\n\n(cid:9), then\n\nBARRONS ensures\n\n1\n\n8|(u\u2212xs)(cid:62)\u2207s|\n\n2 , mins\u2208[t]\n\n8N ln T\n\n(cid:19)\n\n1\n\nReg(u) \u2264 O\n\n\u03b7\n\n+\n\n\u03b2\n\n\u2212\n\n8(ln T )\u03b7\n\ni=1\n\nmax\nt\u2208[T ]\n\nui\nxt,i\n\n.\n\n(3)\n\nThe second term of Eq. (3) comes from ONS while the rest comes from the log-barrier. To see\nwhy the negative term is useful, for a moment assume that we were able to pick \u03b2 such that\n2 \u03b1T (u\u2217) \u2264 \u03b2 \u2264 \u03b1T (u\u2217) where u\u2217 is the best (smoothed) CRP. Then by setting \u03b7 =\n1024N (ln T )2 ,\n1\nthe regret against u\u2217 can be upper bounded by\n\n1\n\n\u03b7\n\n+\n\nO\n\n(cid:19)\n(cid:18) N ln T\n(cid:19)\n(cid:18) N ln T\n(cid:19)\n(cid:18) N ln T\n\u2264 O(cid:0)N 2(ln T )3(cid:1) ,\n\n\u2264 O\n\n\u2264 O\n\n\u03b7\n\n\u03b7\n\n16N ln T\n\u03b1T (u\u2217)\n\n\u2212\n\n1\n\n8(ln T )\u03b7\n\n+ 128N (ln T ) max\nt\u2208[T ]\n\n(cid:18)\n\n+ 128N (ln T )\n\nmax\nt\u2208[T ],i\n\ni=1\n\nmax\nt\u2208[T ]\n\nN(cid:88)\n(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:104)rt, u\u2217 \u2212 xt(cid:105)\n(cid:19)\n\n(cid:104)rt, xt(cid:105)\nu\u2217\ni\nxt,i\n\n+ 1\n\nu\u2217\ni\nxt,i\n\n(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) + 32N ln T \u2212\n\n1\n\n8(ln T )\u03b7\n\n+ 32N ln T \u2212 128N (ln T )\n\nN(cid:88)\nN(cid:88)\n\ni=1\n\ni=1\n\nu\u2217\ni\nxt,i\nu\u2217\ni\nxt,i\n\nmax\nt\u2208[T ]\n\nmax\nt\u2208[T ]\n\nwhich completely eliminates the dependence on the largest gradient norm G!\nThe problem is, of course, it is not clear at all how to tune \u03b2 in this way ahead of time. On a\ncloser look, it is in fact not even clear whether such \u03b2 exists since \u03b1T (u\u2217) depends on the sequence\nx1, . . . , xT and thus also on \u03b2 itself (see Appendix A for more discussions). Assuming its existence,\na natural idea would be to run many copies of BARRONS with different \u03b2 and to choose them\nadaptively via another online learning algorithm such as Hedge [11]. We are unable to analyze this\nmethod due to some technical challenges (discussed in Appendix A), let alone the fact that it leads to\nmuch higher time complexity making the algorithm impractical. In the next section, however, we\ncompletely resolve this issue via an adaptive tunning scheme for \u03b2.\n\n3 ADA-BARRONS\n\nOur main idea to resolve the parameter tuning issue is based on a more involved doubling trick. As\ndicussed we would like to set \u03b2 to be roughly \u03b1T (u\u2217\ns\u2264t fs(u). A standard\n\nT ) where u\u2217\n\nt = minu\n\n(cid:80)\n\n4\n\n\fAlgorithm 2: ADA-BARRONS\n\n1 Initialize: \u03b2 = 1\n2 Run BARRONS with parameter \u03b2 and \u03b7, where after each round t, if the following holds:\n\n2048N (ln T )2 , \u03b3 = 1\n\n2 , \u03b7 =\n\n25\n\n1\n\nwith \u03b1t de\ufb01ned in Theorem 1 and\n\nut = argmin\nu\u2208 \u00af\u2206N\n\n\u03b2 > \u03b1t(ut),\n\nt(cid:88)\n\ns=1\n\nfs(u) +\n\nN(cid:88)\n\ni=1\n\n1\n\u03b3\n\nln\n\n1\nui\n\n,\n\nthen set \u03b2 \u2190 \u03b2\n\n2 , and rerun BARRONS from Line 2 with time index reset to 1.\n\n(4)\n\n(5)\n\nt ) and then restart the algorithm.\n\nt ) is not monotone in t, standard analysis of doubling trick does not work.\n\ndoubling trick would suggest halving \u03b2 whenever it is larger than \u03b1t(u\u2217\nHowever, since \u03b1t(u\u2217\nFortunately, due to the special structure of our problem, we are able to analyze a slight variant of the\nabove proposal where we halve the value of \u03b2 whenever it is larger than \u03b1t(ut), for the regularized\nleader ut (de\ufb01ned in Eq. (5)) instead of the actual leader u\u2217\nt . The regularization used here to compute\nut is again the log-barrier, but the purpose of using log-barrier is simply to ensure the stability of ut\nas discussed later. In fact, ut is exactly the prediction of the FTRL approach of [3], up to a different\nvalue of the parameter \u03b3. Here we only use ut to assist the tunning of \u03b2.\nWe call the \ufb01nal algorithm ADA-BARRONS (see Algorithm 2). Note that for notational simplicity,\nwe reset the time index back to 1 at the beginning of each rerun, that is, the algorithm forgets all the\nprevious data.\nTo see why this works, suppose condition (4) holds at time t and triggers the restart. Then we know\n\u03b1t(ut) < \u03b2 \u2264 \u03b1t\u22121(ut\u22121). On one hand, this implies that the condition of Theorem 1 holds at\ntime t \u2212 1 for ut\u22121, so the regret bound (3) holds for ut\u22121; on the other hand, this also implies\nthat the term N ln T\n\u03b1t(ut), which further admits an upper bound in terms\n\u03b2\nby the same calculation shown after Theorem 1. Therefore, if we can show\nof maxs\u2208[t],i\u2208[N ]\n\u2248 maxs\u2208[t\u22121],i\u2208[N ]\nut,i\n, then the same cancellation will happen which leads\nmaxs\u2208[t],i\u2208[N ]\nxs,i\nto small regret against ut\u22121 for this period. It is also not hard to see that ut\u22121 will have similar total\nloss compared to the actual best CRP u\u2217\nIndeed, we show in Appendix B that both xt and ut enjoy a certain kind of stability, which then\nimplies the following lemma.\nLemma 2. If condition (4) holds at time t, then maxs\u2208[t\u22121],i\u2208[N ]\n\nt\u22121, leading to the desired regret bound overall.\n\nin Eq. (3) is bounded by N ln T\n\nut\u22121,i\nxs,i\n\n\u2265 1\n\n2 maxs\u2208[t],i\u2208[N ]\n\nut,i\nxs,i\n\nut\u22121,i\nxs,i\n\nut,i\nxs,i\n\n.\n\nCall the period between two restart an epoch and use the notation epoch(\u03b2) to indicate the epoch that\nruns with parameter \u03b2. We then prove the following key lemma based on the discussions above.\nLemma 3. For any u \u2208 \u00af\u2206N , if epoch(\u03b2) is not the last epoch, then we have\n\n(cid:88)\n\n(fs(xs) \u2212 fs(u)) \u2264 O(cid:0)N 2(ln T )3(cid:1) \u2212 8N ln T\n(fs(xs) \u2212 fs(u)) \u2264 O(cid:0)N 2(ln T )3(cid:1) +\n\n8N ln T\n\n\u03b2\n\n;\n\n.\n\n\u03b2\n\notherwise,\n\ns\u2208epoch(\u03b2)\n\n(cid:88)\n\ns\u2208epoch(\u03b2)\n\nWith this key lemma, we \ufb01nally prove the claimed regret bound of ADA-BARRONS.\n\nTheorem 4. ADA-BARRONS ensures Reg(u) \u2264 O(cid:0)N 2(ln T )4(cid:1) for any u \u2208 \u2206N .\n\n5\n\n\fN ln T = B \u00d7 O(N 2(ln T )3) + 16N ln T.\n\nProof. Again by Lemma 10 it suf\ufb01ces to consider u \u2208 \u00af\u2206N . Let the number of epochs be B. When\nB = 1, the bound holds trivially by Lemma 3. Otherwise, the regret is upper bounded by\n\nB \u00d7 O(cid:0)N 2(ln T )3(cid:1) +\n\n(cid:32)B\u22121(cid:88)\n\nb=1\n\n(cid:33)\n\n8\n2\u2212B\n\n\u22128\n2\u2212b +\n\u03b1t(u) \u2264 O(maxs\u2208[t],i\n\n) \u2264 O(N T ), which means \u03b1t(u) \u2265\nN T ), condition (4) cannot hold after O(ln(N T )) epochs. Therefore B = O(ln(N T )) = O(ln T )\n\nSince for any t \u2208 [T ] and u \u2208 \u00af\u2206N ,\n\u2126( 1\n(since T > N) and the regret bound follows.\n\nut,i\nxs,i\n\n1\n\nsolve these two problems. It takes time O(cid:16)\n\nComputational complexity. It is clear that the computational bottleneck of our algorithm is to solve\nthe optimization problems de\ufb01ned by Eq. (2) and Eq. (5). Suppose we use Interior Point Method to\nto obtain 1 \u2212 \u0001 accuracy where M is the\ntime complexity to compute the gradient and Hessian inverse of the objective [5], which in our case\nis O(N 3) for solving xt and O(T N 2 + N 3) for solving ut. As T > N, the complexity per round is\ntherefore O(T N 2.5) ignoring logarithmic factors. We note that this is only a pessimistic estimation\nand faster implementation is highly possible, especially for solving ut (given ut\u22121) in light of the\nef\ufb01cient implementation discussed in [1] for similar problems.\n\nN log N\n\u0001\n\n(cid:17)\n\n\u221a\n\nM\n\n4 Detailed Analysis\n\nIn this section we provide the key proofs for our results.\n\nAnalysis of BARRONS The proof of Theorem 1 is a direct combination of the following three\nlemmas, where the \ufb01rst one is by standard OMD analysis and analysis from [12] and the proof is\ndeferred to Appendix B.\nLemma 5. Under the condition of Theorem 1, BARRONS ensures for any u \u2208 \u2206N\n\nt=1\n\n(cid:104)\u2207t, xt \u2212 xt+1(cid:105) + D\u03c8t(u, xt) \u2212 D\u03c8t(u, xt+1) \u2212 \u03b2\n2\n(cid:19)\n2 and \u03b7 \u2264 1 guarantees\n\n(cid:19)\n\nT(cid:88)\n\nLemma 6. BARRONS with parameters \u03b2 \u2264 1\n\n(cid:18)\nD\u03c8t (u, xt) \u2212 D\u03c8t(u, xt+1) \u2212 \u03b2\n2\n\n(cid:104)\u2207t, xt \u2212 u(cid:105)2\n\n\u2264 O\n\nt=1\n\n, \u03d5t(x) = (cid:80)N\n\n2 (cid:107)x(cid:107)2\n\nProof. De\ufb01ne \u03c6t(x) = \u03b2\nD\u03c8t = D\u03c6t + D\u03d5t. Note that D\u03c6t(x, y) = \u03b2\nh(z) = z \u2212 1 \u2212 ln z. For notation simplicity, we also de\ufb01ne \u03b70,i = \u03b71,i for all i. Now we have\n\n. Then \u03c8t(x) = \u03c6t(x) + \u03d5t(x) and\nwhere\n\n2 (cid:107)x \u2212 y(cid:107)2\n\nln 1\nxi\n\n1\n\u03b7t,i\n\n1\n\u03b7t,i\n\ni=1\n\ni=1\n\nAt\n\nAt\n\nyi\n\nReg(u) \u2264 T(cid:88)\n\n(cid:18)\n\n(cid:19)\n\n(cid:104)\u2207t, xt \u2212 u(cid:105)2\n\n.\n\nui\nxt,i\n\n.\n\n(cid:19)\n\n\u03b7\n\n1\n\ni=1\n\n\u2212\n\nmax\nt\u2208[T ]\n\n(cid:17)\n\n8(ln T )\u03b7\n\n(cid:18) N ln T\nN(cid:88)\n(cid:16) xi\nand D\u03d5t(x, y) =(cid:80)N\nT(cid:88)\n(cid:0)D\u03c8t(u, xt) \u2212 D\u03c8t\u22121 (u, xt)(cid:1)\n(cid:18) ui\n(cid:19)\n(cid:17)\nN(cid:88)\nT(cid:88)\n(cid:19)\n\n(cid:18) 1\n(cid:18) ui\n\n\u2212 1\n\n\u03b7t\u22121,i\n\n\u03b7t,i\n\nt=1\n\ni=1\n\nh\n\nh\n\nxt,i\n\n\u03b7t\u22121,i\n\nh\n\nxt,i\n\n(cid:19)\n(cid:19)\n\nh\n\n(cid:18) ui\n\n(cid:19)\n\n\u2212 1\n\n\u03b7t\u22121,i\n\n.\n\n(6)\n\nT(cid:88)\n\n(D\u03c8t(u, xt) \u2212 D\u03c8t(u, xt+1)) \u2264 D\u03c80(u, x1) +\n\nt=1\n\nt=1\n\n\u03b2\n2\n\n\u2264 D\u03c80 (u, x1) +\n\nT(cid:88)\nT(cid:88)\n(cid:19)\nIt remains to deal with(cid:80)T\n\n= D\u03c80 (u, x1) +\n\n= O\n\n(cid:18)\n\nN ln T\n\n\u03b2N +\n\n\u03b2\n2\n\nt=1\n\n\u03b7\n\n+\n\n(cid:16)(cid:107)u \u2212 xt(cid:107)2\n\nAt\n\n(cid:104)\u2207t, u \u2212 xt(cid:105)2 +\n\nT(cid:88)\n(cid:80)N\n\nt=1\n\n\u03b2\n2\n\n(cid:16) 1\n\n\u2212 (cid:107)u \u2212 xt(cid:107)2\n\nT(cid:88)\n\nN(cid:88)\n\nt=1\n\ni=1\n\n(cid:104)\u2207t, u \u2212 xt(cid:105)2 +\n\n+\n\nt=1\n\n\u03b7t,i\n\nAt\u22121\n\n\u2212 1\n\n(cid:18) 1\n(cid:18) 1\nN(cid:88)\nT(cid:88)\n(cid:17)\n(cid:16) ui\n\ni=1\n\n(cid:17)\n\n\u03b7t,i\n. Fix i, let t = s1, s2, . . . , sM \u2208 [2, T ]\nbe the rounds where \u03b7t,i (cid:54)= \u03b7t\u22121,i. De\ufb01ne s0 = 1 and let \u03b7(m) = \u03b7sm,i and x(m) = xsm,i for\n\n\u2212 1\n\n\u03b7t\u22121,i\n\nxt,i\n\nxt,i\n\nt=2\n\nt=2\n\ni=1\n\n\u03b7t,i\n\nh\n\n6\n\n\fh\n\nm=1\n\nm=1\n\n\u03b7(m)\n\n=\n\nm=1\n\nh\n\n=\n\n\u03b7(m)\n\nm=1\n\n\u2212\n\n1\n\n\u03b7t,i\n\nt=2\n\n1\n\nh\n\nx(m)\n\nx(m)\n\nx(m)\n\n(cid:17)\n\n(cid:16)\n\n(cid:19)\n\n\u03b7t\u22121,i\n\nlog2 T\n\n(cid:46)\n\n\u2212 1\n\n\u03b7(m\u22121)\n\nx(m\u22121)\nx(m)\n\nx(m\u22121)\nx(m)\n\nlogT\n\u03b7(m)\n\n(cid:18) ui\n\nxt,i\nx(m\u22121)\nx(m)\n\n(cid:16) ui\n(cid:17)\n(cid:16) ui\n\nT(cid:88)\nM(cid:88)\n\u2264 M(cid:88)\n\n(cid:18) 1\n\uf8eb\uf8ed 1 \u2212 exp\n\uf8eb\uf8ed\u2212 log2\n\nnotational convenience. Note that by de\ufb01nition \u03b7(m) = \u03b7 exp(logT\nWe thus have\n\nN x(m) ) \u2264 \u03b7 exp(logT T ) = \u03b7e.\n(cid:19)\n(cid:19)\n(cid:18) 1\n(cid:16) ui\n(cid:17)\nM(cid:88)\n(cid:17)\n\uf8f6\uf8f8 h\n(cid:33)\n(cid:32)\u2212 logT\n(cid:17) \u2264 M(cid:88)\n(cid:16) ui\n\uf8f6\uf8f8 h\n(cid:32)\n(cid:33)\n(cid:17) \u2264 M(cid:88)\n(cid:16) ui\nWe \ufb01rst consider the case when x(M ) \u2264 min(cid:8) 1\n(cid:9). Because x(M ) \u2264 1\n(cid:32)\n(cid:16) ui\n(cid:17) \u2264 M(cid:88)\n(cid:16) ui\n(cid:16) ui\n(cid:17) \u2264 \u2212\n(cid:16) ui\n(cid:17)(cid:17)\n(cid:18) ln(N T ui)\n(cid:19)\n\n2N = x(0)\ndecreasing in m, there must exist an m\u2217 \u2208 {1, 2, . . . , M} such that x(m\u2217\u22121)\nx(M ) \u2265 2 and x(m\u2217 )\n(cid:33)\nlast expression can thus be further bounded by\n\n(cid:33)\n(cid:16) ui\n\n2 and x(m) is\nx(M ) \u2264 2. The\n\n(cid:18) 1 + N ui\n\n(x(m\u2217\u22121) \u2265 2x(M ))\n\nx(m\u2217\u22121)\nx(M )\n4(ln T )\u03b7\n\nm=m\u2217\n= \u2212 log2\n\n2x(M )\n\u2212 1 \u2212 ln\n\nx(m\u22121)\nx(m)\n4(ln T )\u03b7\n\nx(m\u22121)\nx(m)\n4(ln T )\u03b7\n\nx(m\u22121)\nx(m)\n4(ln T )\u03b7\n\n(cid:16) ui\n\nM(cid:88)\n\n\u2212 log2\n\n\u2212 log2\n\n\u2212 log2\n\n2N , ui\n\n4(ln T )\u03b7\n\n4(ln T )\u03b7\n\n= \u2212\n\n(cid:32)\n\n(cid:19)\n\nm=m\u2217\n\n2x(M )\n\n2x(M )\n\n2x(M )\n\n(cid:17)\n\n(cid:17)\n\nx(m)\n\nx(m)\n\nx(m)\n\n1\n\nh\n\nh\n\n.\n\n1\n\n1\n\nm=1\n\n\u03b7e\n\nh\n\nh\n\nh\n\n2\n\n= \u2212\n\n8(ln T )\u03b7\n\n2x(M )\nui\nxt,i\n\nmax\nt\u2208[T ]\n\n+ O\n\n\u2264 \u2212\n\n\u03b7 ln T\n\n8(ln T )\u03b7\n\nwhere in the \ufb01rst inequality we use the fact for m \u2265 m\u2217, ui\nis positive and increasing when y \u2265 1.\nOn the other hand, if x(M ) \u2265 1\nthus\n\n2 , we have maxt\u2208[T ]\n\n,\n\n\u03b7\n\n1\n\nui\nxt,i\n\n+ O\nmax\nt\u2208[T ]\nx(m) \u2265 ui\n2x(M ) \u2265 1, and that h(y)\nx(m\u2217 ) \u2265 ui\nx(M ) \u2264 2N ui + 2 and\n(cid:18)\n\n= ui\n\n(cid:19)\n\nui\nxt,i\n\n1\n\n8(ln T )\u03b7\n\n\u2212 max\nt\u2208[T ]\n\nui\nxt,i\n\n+ 2N ui + 2\n\n.\n\n(cid:33)\n\n(cid:104)\u2207t, u \u2212 xt(cid:105)2\n\n\u2212 max\nt\u2208[T ]\n\nui\nxt,i\n\n+ 2N ui + 2\n\n+\n\n(cid:19)\n\n(cid:18) 1 + N ui\n\n(cid:19)\n\n\u03b7\n\nN(cid:88)\n\ni=1\n\nO\n\nConsidering both cases and Eq. (6), we get\n\nh\n\nt=2\n\n\u03b7t,i\n\nxt,i\n\n(cid:19)\n\n\u03b7t\u22121,i\n\n\u2212 1\n\n\u2264 0 \u2264\n\n(cid:18) 1\n\n(cid:18) ui\n\n2N or x(M ) \u2265 ui\n(cid:19)\n\nT(cid:88)\n(cid:32)\nT(cid:88)\nT(cid:88)\nD\u03c8t(u, xt) \u2212 D\u03c8t(u, xt+1) \u2212 \u03b2\n(cid:18)\n(cid:18)\nN(cid:88)\n2\n(cid:18) N ln T\n\n8(ln T )\u03b7\n\n(cid:19)\n\n(cid:19)\n\nN ln T\n\n\u03b2N +\n\nt=1\n\nt=1\n\ni=1\n\n+\n\n1\n\n\u03b7\n\n\u2264 O\n\nN(cid:88)\nLemma 7. BARRONS guarantees(cid:80)T\n\n\ufb01nishing the proof.\n\n8(ln T )\u03b7\n\n\u2212\n\ni=1\n\n1\n\n\u03b7\n\nmax\nt\u2208[T ]\n\nui\nxt,i\n\n,\n\n\u2264 O\n\nt=1 (cid:104)\u2207t, xt \u2212 xt+1(cid:105) \u2264 8N ln T\n\n\u03b2\n\n.\n\nProof. De\ufb01ne Ft(x) (cid:44) (cid:104)x,\u2207t(cid:105) + D\u03c8t(x, xt). Using Taylor\u2019s expansion and \ufb01rst order optimality of\nxt+1 we have\n\nFt(xt) \u2212 Ft(xt+1) = \u2207Ft(xt+1)(cid:62)(xt \u2212 xt+1) +\n\n(xt \u2212 xt+1)(cid:62)\u22072Ft(\u03bet)(xt \u2212 xt+1) =\n\n\u2265 1\n2\n\n1\n2\n\n(xt \u2212 xt+1)(cid:62)\u22072Ft(\u03bet)(xt \u2212 xt+1)\n(cid:107)xt \u2212 xt+1(cid:107)2\u22072Ft(\u03bet) ,\n\n1\n2\n\n7\n\n\fwhere \u03bet is some point that lies on the line segment joining xt and xt+1. On the other hand, by the\nde\ufb01nition of Ft, nonnegativity of Bregman divergence, and H\u00f6lder inequality, we have\nFt(xt) \u2212 Ft(xt+1) = (cid:104)xt \u2212 xt+1,\u2207t(cid:105) \u2212 D\u03c8t(xt+1, xt) \u2264 (cid:107)xt \u2212 xt+1(cid:107)\u22072Ft(\u03bet) (cid:107)\u2207t(cid:107)\u2207\u22122Ft(\u03bet) .\nCombining the above two inequalities we get (cid:107)xt \u2212 xt+1(cid:107)\u22072Ft(\u03bet)\n(cid:54) 2(cid:107)\u2207t(cid:107)\u2207\u22122Ft(\u03bet), and thus\n\n(cid:104)\u2207t, xt \u2212 xt+1(cid:105) \u2264 (cid:107)\u2207t(cid:107)\u2207\u22122Ft(\u03bet) (cid:107)xt \u2212 xt+1(cid:107)\u22072Ft(\u03bet) \u2264 2(cid:107)\u2207t(cid:107)2\u2207\u22122Ft(\u03bet)\n\n= 2\u2207T\n\nt (\u03b2At + \u22072\u03d5t(\u03bet))\u22121\u2207t \u2264 2\n\u03b2\n\n\u2207(cid:62)\nt A\u22121\n\nt \u2207t,\n\nT(cid:88)\n\n2\n\u03b2\n\nwhere \u03d5t is the log-barrier regularizer de\ufb01ned in the proof of Lemma 6 (whose Hessian is clearly\npositive semi-de\ufb01nite). Using Lemma 11 in [12], we continue with\n\nt=1\n\nwhere the second inequality uses the fact ln|A0| = N ln N and by AM-GM inequality\nln|AT| \u2264 N ln Tr(AT )\n2 \u2264\n\n(cid:17) \u2264 N ln(N + N T 3) since (cid:107)\u2207t(cid:107)2\n\n\u2264 N ln\n\nN +\n\n2\n\nN\n\nN\n\n\u2207(cid:62)\nt A\u22121\n\nln\n\nt \u2207t \u2264 2\n\u03b2\n(cid:16)\n\n|AT|\n|A0| \u2264 2N ln(1 + T 3)\n(cid:80)T\nt=1(cid:107)\u2207t(cid:107)2\n\n\u03b2\n\n\u2264 8N ln T\n\n,\n\n\u03b2\n\nN 2T 2((cid:80)\n\nt,i)/((cid:80)\n\ni r2\n\ni rt,i)2 \u2264 N 2T 2. This \ufb01nishes the proof.\n\nAnalysis of ADA-BARRONS To prove Lemma 2, we make use of the following stability lemmas\nwhose proofs are deferred to Appendix B.\nLemma 8. In ADA-BARRONS, if \u03b3 \u2264 1\nLemma 9. In ADA-BARRONS, if \u03b7 \u2264 1\n\n\u221a\n\u03b3\n2 for all t and i.\n\u221a\n\n25 , then 1 \u2212 \u221a\n300 , then 1 \u2212 \u221a\n\n2 \u2264 ut+1,i\n2 \u2264 xt+1,i\n\n\u2264 1 +\n\u2264 1 +\n\nfor all t and i.\n\n3\u03b7\n2\n\nut,i\n\n3\u03b7\n\n\u03b3\n\nxt,i\n\n(cid:16) ut\u22121,i(cid:48)\n(cid:17)(cid:16) xt,i(cid:48)\n\nut,i(cid:48)\n\n(cid:17)\n(cid:17)\n\n1+\n\n1+\n\nut,i\nxs,i\n\nxs(cid:48),i(cid:48) =\n\nat\n\u221a\n\u03b3\n1\u2212 \u221a\n2\n3\u03b7\n\u221a\n2\n\u03b3\n2\n\n\u2265 1\nat \u2265 1\n\n2 at. This concludes the proof.\n\nProof of Lemma 2. Denote at (cid:44) maxs\u2208[t],i\u2208[N ]\n(i.e., at = ut,i(cid:48)\n\n. Suppose at attains its max at s = s(cid:48) and i = i(cid:48)\nat \u2265\nat \u2265\n\n(cid:16) ut\u22121,i(cid:48)\nxs(cid:48) ,i(cid:48) ), then when s(cid:48) \u2264 t \u2212 1, we have by Lemma 8 at\u22121 \u2265 ut\u22121,i(cid:48)\n2 at; when s(cid:48) = t, we have by Lemma 8 and 9 at\u22121 \u2265 ut\u22121,i(cid:48)\n(cid:80)N\n(cid:18) N ln T\n\nProof of Lemma 3. De\ufb01ne \u0393t(u) =(cid:80)t\n\u0393t\u22121(u) \u2264 \u0393t\u22121(ut\u22121) \u2264 t\u22121(cid:88)\n\n. Suppose condition\n(4) holds at some time t at the end of epoch(\u03b2) and cause the algorithm to restart. Then we know\nthat \u03b2 \u2264 \u03b1t\u22121(ut\u22121) and \u03b2 > \u03b1t(ut). The \ufb01rst condition guarantees that Eq. (3) holds for ut\u22121 at\ntime t \u2212 1. Also, note that ut\u22121 is the maximizer of \u0393t\u22121. Together they imply for any u \u2208 \u00af\u2206N ,\n\ns=1 fs(xs)\u2212(cid:80)t\n\nfs(xs) \u2212 t\u22121(cid:88)\n\nfs(ut\u22121) \u2264 O\n\ns=1 fs(u)\u2212 1\n\nxt\u22121,i(cid:48) =\n\ni=1 ln 1\nui\n\n8N ln T\n\n(cid:19)\n\nxt\u22121,i(cid:48)\n\nut,i(cid:48)\n\n+\n\n\u03b3\n\n\u2212 at\u22121\n8(ln T )\u03b7\n\n,\n\n\u03b7\n\n\u03b2\n\ns=1\n\ns=1\n\nwhere we recall the notation at (cid:44) maxs\u2208[t],i\u2208[N ]\n\n. The second condition implies\n\n(cid:110)\n\n1\n\u03b2\n\n1\n\n(cid:110)\n\n\u03b1t(ut)\n\n<\n\u2264 max\n\n= max\n\n2, 8 max\n\ns\u2208[t],i\u2208[N ]\n\n8|\u2207(cid:62)\n\n+ 8\n\n2, max\ns\u2208[t]\nut,i\nxs,i\n\n(cid:110)\n\nut,i\nxs,i\n\ns (ut \u2212 xs)|(cid:111)\n(cid:111)\n\n= max\n\n2, max\ns\u2208[t]\n= 8at + 8 \u2264 16at\u22121 + 8,\n\n8\n\n(cid:12)(cid:12)(cid:12)(cid:104)xs, ut \u2212 xs(cid:105)\n\n(cid:104)xs, rs(cid:105)\n\n(cid:12)(cid:12)(cid:12)(cid:111)\n\n(7)\n\n(8)\n\nwhere we apply Lemma 2 for the last step. Further combining this with Eq. (7), and noting that\nft(x) \u2212 ft(u) \u2264 maxi ln ui\n\n\u2264 ln(N T ) for any x \u2208 \u00af\u2206N , we have for any u \u2208 \u00af\u2206N ,\n\nxi\n\nt(cid:88)\n\nfs(xs) \u2212 t(cid:88)\n\nfs(u) \u2264 ln(N T ) +\n\nt\u22121(cid:88)\n\nfs(xs) \u2212 t\u22121(cid:88)\n\ns=1\n\ns=1\n\ns=1\n\ns=1\n\n8\n\nfs(u) \u2264 ln(N T ) + \u0393t\u22121(u) +\n\nN ln(N T )\n\n\u03b3\n\n\f(cid:19)\n\n(cid:18) N ln T\n\u2264 O(cid:0)N 2(ln T )3(cid:1) \u2212 8N ln T\n\n\u2264 O\n\n8N ln T\n\n\u2212\n\n.\n\n+\n\n\u03b7\n\n\u03b2\n\n\u03b2\n\n(cid:18) 1\n\n1\n\n8(ln T )\u03b7\n\n16\u03b2\n\n(cid:19)\n\n\u2212 1\n2\n\n(by (7) and (8))\n\nFor the last epoch, we can apply Theorem 1 over the entire epoch and simply discard the negative\nterm to obtain the claimed bound.\n\n5 Conclusions and Open Problems\n\nWe have shown that our new algorithm ADA-BARRONS achieves logarithmic regret of\nO(N 2(ln T )4) for online portfolio with much faster running time compared to Universal Port-\nfolio, the only previous algorithm with truly logarithmic regret. A natural open problem is whether it\nis possible to further improve either the regret (from N 2 to N) or the computational ef\ufb01ciency without\nhurting the other. It is conjectured in [20] that FTRL with log-barrier [3] (i.e. our ut\u2019s) might also\nachieve logarithmic regret without dependence on G. On the pessimistic side, it is also a conjecture\nthat it might be impossible to have the optimal regret with O(N ) computations per round [19].\n\nAcknowledgements. The authors would like to thank Tim van Erven for introducing the problem\nand to thank Tim van Erven, Dirk van der Hoeven, and Wouter Koolen for helpful discussions\nthroughout the projects, especially on the FTRL approach. The work was done while KZ visited\nthe University of Southern California. KZ gratefully acknowledges \ufb01nancial support from China\nScholarship Council. HL and CYW are grateful for the support of NSF Grant #1755781.\n\n9\n\n\fReferences\n[1] Jacob D Abernethy, Elad Hazan, and Alexander Rakhlin.\n\nInterior-point methods for full-\ninformation and bandit online learning. IEEE Transactions on Information Theory, 58(7):4164\u2013\n4175, 2012.\n\n[2] Alekh Agarwal, Haipeng Luo, Behnam Neyshabur, and Robert E Schapire. Corralling a band\n\nof bandit algorithms. In Conference on Learning Theory, pages 12\u201338, 2017.\n\n[3] Amit Agarwal and Elad Hazan. Ef\ufb01cient algorithms for online game playing and universal\nportfolio management. Electronic Colloquium on Computational Complexity, TR06-033, 2005.\n\n[4] Amit Agarwal, Elad Hazan, Satyen Kale, and Robert E Schapire. Algorithms for portfolio\nmanagement based on the newton method. In Proceedings of the 23rd international conference\non Machine learning, pages 9\u201316, 2006.\n\n[5] Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press,\n\n2004.\n\n[6] S\u00e9bastien Bubeck, Michael B. Cohen, and Yuanzhi Li. Sparsity, variance and curvature in\n\nmulti-armed bandits. In International Conference on Algorithmic Learning Theory, 2018.\n\n[7] S\u00e9bastien Bubeck, Ronen Eldan, and Joseph Lehec. Sampling from a log-concave distribution\n\nwith projected langevin monte carlo. arXiv preprint arXiv:1507.02564, 2015.\n\n[8] Thomas M Cover. Universal portfolios. Mathematical Finance, 1(1):1\u201329, 1991.\n\n[9] Thomas M Cover. Universal data compression and portfolio selection. In Foundations of\nComputer Science, 1996. Proceedings., 37th Annual Symposium on, pages 534\u2013538. IEEE,\n1996.\n\n[10] Dylan J Foster, Zhiyuan Li, Thodoris Lykouris, Karthik Sridharan, and Eva Tardos. Learning in\ngames: Robustness of fast convergence. In Advances in Neural Information Processing Systems,\npages 4734\u20134742, 2016.\n\n[11] Yoav Freund and Robert E. Schapire. A decision-theoretic generalization of on-line learning\nand an application to boosting. Journal of Computer and System Sciences, 55(1):119\u2013139,\nAugust 1997.\n\n[12] Elad Hazan, Amit Agarwal, and Satyen Kale. Logarithmic regret algorithms for online convex\n\noptimization. Machine Learning, 69(2-3):169\u2013192, 2007.\n\n[13] Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends R(cid:13) in\n\nOptimization, 2(3-4):157\u2013325, 2016.\n\n[14] David P Helmbold, Robert E Schapire, Yoram Singer, and Manfred K Warmuth. On-line\nportfolio selection using multiplicative updates. Mathematical Finance, 8(4):325\u2013347, 1998.\n\n[15] Adam Kalai and Santosh Vempala. Ef\ufb01cient algorithms for universal portfolios. Journal of\n\nMachine Learning Research, 3(Nov):423\u2013440, 2002.\n\n[16] L\u00e1szl\u00f3 Lov\u00e1sz and Santosh Vempala. Fast algorithms for logconcave functions: Sampling,\nrounding, integration and optimization. In Foundations of Computer Science, 2006. FOCS\u201906.\n47th Annual IEEE Symposium on, pages 57\u201368. IEEE, 2006.\n\n[17] Hariharan Narayanan and Alexander Rakhlin. Random walk approach to regret minimization.\n\nIn Advances in Neural Information Processing Systems, pages 1777\u20131785, 2010.\n\n[18] Yurii Nesterov and Arkadii Nemirovskii.\n\nprogramming, volume 13. Siam, 1994.\n\nInterior-point polynomial algorithms in convex\n\n[19] Laurent Orseau, Tor Lattimore, and Shane Legg. Soft-bayes: Prod for mixtures of experts with\nlog-loss. In International Conference on Algorithmic Learning Theory, pages 372\u2013399, 2017.\n\n[20] Tim van Erven, Dirk van der Hoeven, and Wouter Koolen. personal communication, 2018.\n\n10\n\n\f[21] Chen-Yu Wei and Haipeng Luo. More adaptive algorithms for adversarial bandits. In Conference\n\non Learning Theory, 2018.\n\n[22] Martin Zinkevich. Online convex programming and generalized in\ufb01nitesimal gradient ascent.\nIn Proceedings of the 20th International Conference on Machine Learning, pages 928\u2013936,\n2003.\n\n11\n\n\f", "award": [], "sourceid": 5033, "authors": [{"given_name": "Haipeng", "family_name": "Luo", "institution": "University of Southern California"}, {"given_name": "Chen-Yu", "family_name": "Wei", "institution": "University of Southern California"}, {"given_name": "Kai", "family_name": "Zheng", "institution": "Peking University"}]}