This paper investigates the problem of non-stationary linear bandits, where the unknown regression parameter is evolving over time. Existing studies develop various algorithms and show that they enjoy an $\widetilde{\mathcal{O}}(T^{2/3}P_T^{1/3})$ dynamic regret, where $T$ is the time horizon and $P_T$ is the path-length that measures the fluctuation of the evolving unknown parameter. In this paper, we discover that a serious technical flaw makes their results ungrounded, and then present a fix, which gives an $\widetilde{\mathcal{O}}(T^{3/4}P_T^{1/4})$ dynamic regret without modifying original algorithms. Furthermore, we demonstrate that instead of using sophisticated mechanisms, such as sliding window or weighted penalty, a simple restarted strategy is sufficient to attain the same regret guarantee. Specifically, we design an UCB-type algorithm to balance exploitation and exploration, and restart it periodically to handle the drift of unknown parameters. Our approach enjoys an $\widetilde{\mathcal{O}}(T^{3/4}P_T^{1/4})$ dynamic regret. Note that to achieve this bound, the algorithm requires an oracle knowledge of the path-length $P_T$. Combining the bandits-over-bandits mechanism by treating our algorithm as the base learner, we can further achieve the same regret bound in a parameter-free way. Empirical studies also validate the effectiveness of our approach.
翻译:本文调查了非静止线性匪徒的问题, 未知回归参数正在逐渐演变。 现有研究开发了各种算法, 并显示它们享有$( 全范围T$Tilde_mathcal{O} (T ⁇ 2/3}P_T ⁇ 1/3}P_T ⁇ 1/3}) 动态遗憾, 其中T$是时间范围, $P_T$是衡量变化中的未知参数波动的路径长度。 本文中, 我们发现一个严重的技术缺陷导致其结果没有根据, 然后提出一个修正, 这使各种算法得到 $( 全范围T ⁇ 3/3/4}P_T ⁇ 1/4} 的动态遗憾, 而没有修改原始算法。 此外, 我们证明, 简单重启的战略不是使用复杂的机制, 而是用来测量正在变化的参数的波动。 我们设计一个UCB型算法, 来平衡开发和探索, 并定期重新启用它来处理未知参数的漂移。 我们的方法拥有一个 $( 全范围T ⁇ _(T ⁇ 3/4} P_T ⁇ _T ⁇ _1/4} 动态遗憾, 我们的算法的轨算法, 也要求我们这个直观的轨法 学习一个动态的路径。