Unbiased Learning to Rank (ULTR) studies the problem of learning a ranking function based on biased user interactions. In this framework, ULTR algorithms have to rely on a large amount of user data that are collected, stored, and aggregated by central servers. In this paper, we consider an on-device search setting, where users search against their personal corpora on their local devices, and the goal is to learn a ranking function from biased user interactions. Due to privacy constraints, users' queries, personal documents, results lists, and raw interaction data will not leave their devices, and ULTR has to be carried out via Federated Learning (FL). Directly applying existing ULTR algorithms on users' devices could suffer from insufficient training data due to the limited amount of local interactions. To address this problem, we propose the FedIPS algorithm, which learns from user interactions on-device under the coordination of a central server and uses click propensities to remove the position bias in user interactions. Our evaluation of FedIPS on the Yahoo and Istella datasets shows that FedIPS is robust over a range of position biases.
翻译:不带偏见的学习到排名(Luke)研究学习基于有偏见的用户互动的排名函数的问题。在这个框架内,Luke算法必须依赖由中央服务器收集、储存和汇总的大量用户数据。在本文中,我们考虑在设计上搜索设置,用户在本地设备上对照个人组合进行搜索,目标是从有偏见的用户互动中学习排序功能。由于隐私限制、用户询问、个人文件、结果列表和原始互动数据不会离开他们的设备,而Luke算法必须通过联邦学习(FL)来进行。直接将现有的Luke算法应用于用户的装置可能因为当地互动有限而受培训数据不足的影响。为了解决这一问题,我们提议FedIPS算法,在中央服务器的协调下学习用户在设计上的互动,并点击驱动器来消除用户互动中的位置偏差。我们对Yahoo和Istella数据集的FedIPS的评估表明,FedIPS在一系列立场上都存在偏差。