Neural ranking models (NRMs) have achieved promising results in information retrieval. NRMs have also been shown to be vulnerable to adversarial examples. A typical Word Substitution Ranking Attack (WSRA) against NRMs was proposed recently, in which an attacker promotes a target document in rankings by adding human-imperceptible perturbations to its text. This raises concerns when deploying NRMs in real-world applications. Therefore, it is important to develop techniques that defend against such attacks for NRMs. In empirical defenses adversarial examples are found during training and used to augment the training set. However, such methods offer no theoretical guarantee on the models' robustness and may eventually be broken by other sophisticated WSRAs. To escape this arms race, rigorous and provable certified defense methods for NRMs are needed. To this end, we first define the \textit{Certified Top-$K$ Robustness} for ranking models since users mainly care about the top ranked results in real-world scenarios. A ranking model is said to be Certified Top-$K$ Robust on a ranked list when it is guaranteed to keep documents that are out of the top $K$ away from the top $K$ under any attack. Then, we introduce a Certified Defense method, named CertDR, to achieve certified top-$K$ robustness against WSRA, based on the idea of randomized smoothing. Specifically, we first construct a smoothed ranker by applying random word substitutions on the documents, and then leverage the ranking property jointly with the statistical property of the ensemble to provably certify top-$K$ robustness. Extensive experiments on two representative web search datasets demonstrate that CertDR can significantly outperform state-of-the-art empirical defense methods for ranking models.
翻译:神经等级模型(NRM)在信息检索方面已经取得了可喜的成果。 NRM 也显示它很容易成为对抗性例子。最近提出了一项针对NRM的典型的单词替代排序攻击(WSRA)建议,其中攻击者通过在文本中添加人无法察觉的扰动,促进排名中的目标文件。这引起了在现实应用中部署NRMs时的担忧。因此,开发针对NRMs这类攻击的防御技术非常重要。在经验性平稳防御中,在培训中发现了对抗性辩论的例子,用来增强培训集。然而,这种方法对模型的稳健性没有提供理论保证,最终可能被其他精密的WSRA(WRA)击碎。为了逃避这种军备竞赛,需要为NRMMs添加严格和可证实的防御方法。为了这个目的,我们首先在现实世界应用程序中为排序模型确定 kt- K美元(Top- K$),因为用户主要在现实世界情景中关注最高级的替代结果。我们称的排序模式是“Top-$ Robustt” 美元,在排序列表列表中,我们从头版的SDRDRDRI 开始使用正价。