Unbiased Learning to Rank~(ULTR) that learns to rank documents with biased user feedback data is a well-known challenge in information retrieval. Existing methods in unbiased learning to rank typically rely on click modeling or inverse propensity weighting~(IPW). Unfortunately, the search engines are faced with severe long-tail query distribution, where neither click modeling nor IPW can handle well. Click modeling suffers from data sparsity problem since the same query-document pair appears limited times on tail queries; IPW suffers from high variance problem since it is highly sensitive to small propensity score values. Therefore, a general debiasing framework that works well under tail queries is in desperate need. To address this problem, we propose a model-based unbiased learning-to-rank framework. Specifically, we develop a general context-aware user simulator to generate pseudo clicks for unobserved ranked lists to train rankers, which addresses the data sparsity problem. In addition, considering the discrepancy between pseudo clicks and actual clicks, we take the observation of a ranked list as the treatment variable and further incorporate inverse propensity weighting with pseudo labels in a doubly robust way. The derived bias and variance indicate that the proposed model-based method is more robust than existing methods. Finally, extensive experiments on benchmark datasets, including simulated datasets and real click logs, demonstrate that the proposed model-based method consistently performs outperforms state-of-the-art methods in various scenarios.
翻译:在信息检索方面,一个众所周知的挑战就是将文件排在有偏向的用户反馈数据排位的位置上。在信息检索方面,一个众所周知的挑战就是不偏倚的排名现有方法通常依赖于点击模型或反偏向的偏向权重(IPW) 。 不幸的是,搜索引擎面临严重的长尾查询分布, 无论是点击模型还是IPW都无法很好地处理。 点击模型都存在数据宽度问题, 因为同一对查询文件的尾端查询时间有限; IPW 存在差异性高的问题, 因为它对小偏向分分值非常敏感。 因此, 一个在尾端查询下运行良好的一般偏向性排序框架非常需要。 为了解决这个问题, 我们提议了一个基于模型的不偏向性学习到排位框架。 具体地说, 我们开发了一个通用的有色用户模拟模拟器, 用于为未观测的排名列表生成假点击, 从而解决数据宽度问题。 此外,考虑到假点击与实际点击之间存在差异, 我们观察一个排名列表列表作为处理模型变量, 并且进一步在尾考查中采用更稳健的偏差的模型。