We study the problem of online learning (OL) from revealed preferences: a learner wishes to learn a non-strategic agent's private utility function through observing the agent's utility-maximizing actions in a changing environment. We adopt an online inverse optimization setup, where the learner observes a stream of agent's actions in an online fashion and the learning performance is measured by regret associated with a loss function. We first characterize a special but broad class of agent's utility functions, then utilize this structure in designing a new convex loss function. We establish that the regret with respect to our new loss function also bounds the regret with respect to all other usual loss functions in the literature. This allows us to design a flexible OL framework that enables a unified treatment of loss functions and supports a variety of online convex optimization algorithms. We demonstrate with theoretical and empirical evidence that our framework based on the new loss function (in particular online Mirror Descent) has significant advantages in terms of regret performance and solution time over other OL algorithms from the literature and bypasses the previous technical assumptions as well.
翻译:我们从披露的偏好中研究在线学习(OL)的问题:学习者希望通过观察代理人在变化环境中的效用最大化行动来学习非战略代理人的私人公用事业功能。我们采用了在线反优化设置,学习者以在线方式观察代理人的一系列行动,学习表现是通过与损失功能有关的遗憾来衡量的。我们首先将代理人的效用功能分为一个特殊但广泛的类别,然后在设计新的 convex损失功能时利用这一结构。我们确定,对新损失功能的遗憾也使我们对文献中所有其他通常的损失功能感到遗憾。这使我们能够设计一个灵活的OL框架,以便能够统一处理损失功能,并支持各种在线的convex优化算法。我们用理论和实验证据证明,我们基于新的损失功能(特别是在线Mira Crainfor)的框架在遗憾表现和解决时间方面比文献中的其他OL算法有很大的优势,并绕过以前的技术假设。