In this work we introduce an incremental learning framework for Click-Through-Rate (CTR) prediction and demonstrate its effectiveness for Taboola's massive-scale recommendation service. Our approach enables rapid capture of emerging trends through warm-starting from previously deployed models and fine tuning on "fresh" data only. Past knowledge is maintained via a teacher-student paradigm, where the teacher acts as a distillation technique, mitigating the catastrophic forgetting phenomenon. Our incremental learning framework enables significantly faster training and deployment cycles (x12 speedup). We demonstrate a consistent Revenue Per Mille (RPM) lift over multiple traffic segments and a significant CTR increase on newly introduced items.
翻译:在这项工作中,我们引入了“点击走廊(CTR)”预测的渐进学习框架,并展示了它对于塔波拉大规模推荐服务的有效性。我们的方法通过从先前部署的模型温热启动和对“更新”数据进行微调,能够快速捕捉新趋势。过去的知识通过师生模式得以保存,教师在这种模式中充当蒸馏技术,减轻灾难性的遗忘现象。我们的递增学习框架使得培训和部署周期大大加快(x12加速 ) 。我们展示了Per Mile(RPM)在多个交通环节上的持续收入和新引入项目的大幅增长。