Industrial search and recommendation systems mostly follow the classic multi-stage information retrieval paradigm: matching, pre-ranking, ranking, and re-ranking stages. To account for system efficiency, simple vector-product based models are commonly deployed in the pre-ranking stage. Recent works consider distilling the high knowledge of large ranking models to small pre-ranking models for better effectiveness. However, two major challenges in pre-ranking system still exist: (i) without explicitly modeling the performance gain versus computation cost, the predefined latency constraint in the pre-ranking stage inevitably leads to suboptimal solutions; (ii) transferring the ranking teacher's knowledge to a pre-ranking student with a predetermined handcrafted architecture still suffers from the loss of model performance. In this work, a novel framework AutoFAS is proposed which jointly optimizes the efficiency and effectiveness of the pre-ranking model: (i) AutoFAS for the first time simultaneously selects the most valuable features and network architectures using Neural Architecture Search (NAS) technique; (ii) equipped with ranking model guided reward during NAS procedure, AutoFAS can select the best pre-ranking architecture for a given ranking teacher without any computation overhead. Experimental results in our real world search system show AutoFAS consistently outperforms the previous state-of-the-art (SOTA) approaches at a lower computing cost. Notably, our model has been adopted in the pre-ranking module in the search system of Meituan, bringing significant improvements.
翻译:工业搜索和建议系统大多遵循传统的多阶段信息检索模式:匹配、预排、排名、排名和重新排名等。考虑到系统效率,简单的矢量产品模型通常在排名前阶段部署。最近的工作考虑将大型排名模型的高级知识提炼为小型排名前模型,以提高效益。但是,在排名前系统中仍然存在两大挑战:(一) 预先界定的排名前阶段的潜伏限制没有明确模拟性能增益和计算成本,不可避免地导致排名前阶段的偏差性改进;(二) 将高级教师的知识转让给一个预设的手制架构的预年级学生,仍然因模型性能损失而受到影响。在这项工作中,提出了一个新的AutoFAS框架,共同优化排名前模型的效率和效力:(一) AutoFAS首次选择最有价值的特征和网络结构,同时使用神经结构模型搜索(NAS)技术;(二) 在NAS程序期间,配备了排名前指导改进的评级模型,AutoFAS可以选择最高级的排名前系统结构,在不连续进行世界级教师头等成本计算的情况下,连续地将一个教师头等计算机系统升级。