Click-Through Rate prediction (CTR) is a crucial task in recommender systems, and it gained considerable attention in the past few years. The primary purpose of recent research emphasizes obtaining meaningful and powerful representations through mining low and high feature interactions using various components such as Deep Neural Networks (DNN), CrossNets, or transformer blocks. In this work, we propose the Deep Multi-Representation model (DeepMR) that jointly trains a mixture of two powerful feature representation learning components, namely DNNs and multi-head self-attentions. Furthermore, DeepMR integrates the novel residual with zero initialization (ReZero) connections to the DNN and the multi-head self-attention components for learning superior input representations. Experiments on three real-world datasets show that the proposed model significantly outperforms all state-of-the-art models in the task of click-through rate prediction.
翻译:最近研究的主要目的是利用深神经网络(DNN)、十字网(CrossNets)或变压器块等各种组成部分,通过采矿低高特征互动,获取有意义和强大的代表性。在这项工作中,我们提议了深多代表模型(Deep-多代表模型),以联合培训两种强大的特征学习组合,即DNN和多头自控。此外,深海MR整合了带有零初始化(ReZero)连接到DNN和多头自控组件以学习高级输入演示的新残余。关于三个真实世界数据集的实验显示,拟议的模型在点击通率预测任务中大大优于所有最先进的模型。