©作者 | 钱浩
单位 | 北邮 GAMMA Lab
研究方向 | 图神经网络
论文标题:
Uncovering the Structural Fairness in Graph Contrastive Learning
NeurIPS 2022
https://arxiv.org/abs/2210.03011
实验表明,FwSeqBlock 能够在 item 特征数量持续增加的情况下获得稳定的指标增益。同时 FwSeqBlock 具有可插拔的特点,能够无负担地与目前主流 SOTA 序列推荐模型结合并且获得正向的指标增益。
▲ 图1. FwSeqBlock模型结构图
在当前工业界的序列建模应用中,对于用户历史行为表征的聚合主要采用 pooling 或者 shallow transformation 方法。本研究认为以上方案并不能很好的建模行为表征中不同细粒度特征间的关系(如购买品牌与购买时间),从而导致最终的用户兴趣表征的质量较为平庸。
为了解决以上问题,FwSeqBlock 提出利用参数化的 field memory 矩阵来显式的刻画不同细粒度特征之间的重要性。具体地讲,首先我们将商品特征与历史行为表征表示成以下 field-wise 形式:
其中维度 , 为自定义超参数。为了捕捉用户历史行为与待推荐商品特征之间的交互,我们引入 field memory 矩阵 显式地学习不同 field 间的联系,其计算过程如下:
接着,我们引入 field-wise attention 机制动态地聚合单个用户历史行为中重要的表征:
最后,我们借鉴了 Skip Connection 和 Layer Normalization 的方法,目标是使训练过程更加顺畅同时避免过拟合的风险,计算如下:
经过以上讨论,FwSeqBlock 聚焦于用户历史行为表征的生成中,具有可插拔的特点,因此可以很方便的与目前 SOTA 序列建模模型相结合。
如下表所示,在基准方法中增加 FwSeqBlock 模块后,所有方法均能获得一致的指标提升。具体指标上,在 Taobao 数据集(6 个特征)中,FwSeqBlock 能够带来 0.49%~0.94% 的 AUC 增益;在业务数据(17 个特征)中,FwSeqBlock 能够获得更加显著的离线指标 AUC 增益。我们认为这样的实验结果能够充分证明 FwSeqBlock 在用户历史行为表征建模中的有效性。
相比 GRU4Rec、Caser、DIN、Bert4Rec 这样专注于聚合用户行为表征的序列建模方法,在增加 FwSeqBlock 后 AUC 指标提升在 0.49~1.69%。因此,我们认为在聚合方法的研究之外,关注行为表征的质量也非常关键。
相比 CSAN,CAN 这样上线文信息相关的模型,FwSeqBlock 的引入仍可以带来一定的提升,证明了对用户历史行为做 field-wise 建模的有效性。
Mean pooling (CSAN)
Sum pooling (CSAN)
Concatenation (Bert4Rec, DIN, DIEN)
▲ 图2. 消融实验效果图
参考文献
[1] Qingyao Ai, Keping Bi, Jiafeng Guo, and W. Bruce Croft. 2018. Learning a Deep Listwise Context Model for Ranking Refinement. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2018). [1] Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer Normaliza- tion. CoRR abs/1607.06450 (2016). arXiv:1607.06450
[2] Alex Beutel,Paul Covington,Sagar Jain,Can Xu,Jia Li,Vince Gatto, and Ed H. Chi. 2018. Latent Cross: Making Use of Context in Recurrent Recommender Systems. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (Marina Del Rey, CA, USA) (WSDM ’18). Association for Computing Machinery, New York, NY, USA, 46–54.
[3] Qiwei Chen, Huan Zhao, Wei Li, Pipei Huang, and Wenwu Ou. 2019. Behavior Sequence Transformer for E-Commerce Recommendation in Alibaba. , Article 12 (2019), 4 pages
[4] Junyoung Chung, Caglar Gulcehre, Kyung Hyun Cho, and Yoshua Bengio.2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv:1412.3555 (2014).
[5] Tom Fawcett. 2006. An introduction to ROC analysis. Pattern Recognition Letters 27, 8 (2006), 861–874.
[6] Yufei Feng, Fuyu Lv, Weichen Shen, Menghan Wang, Fei Sun, Yu Zhu, and Keping Yang. 2019. Deep Session Interest Network for Click-through Rate Prediction. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (Macao, China) (IJCAI’19). AAAI Press, 2301–2307.
[7] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and PatternRecognition(CVPR).770–778.
[8] Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based Recommendations with Recurrent Neural Networks. arXiv:1511.06939 (2016).
[9] Xiaowen Huang, Shengsheng Qian, Quan Fang, Jitao Sang, and Changsheng Xu. 2018. CSAN: Contextual Self-Attention Network for User Sequential Recommen- dation. In Proceedings of the 26th ACM International Conference on Multimedia (MM ’18). 447–455.
[10] Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, Ankur Teredesai, Vipin Kumar, Ying Li, Rómer Rosales, Evimaria Terzi, and George Karypis(Eds.).ACM,2671–2679.
[11] Qi Pi, Guorui Zhou, Yujing Zhang, Zhe Wang, Lejian Ren,Ying Fan, Xiaoqiang Zhu, and Kun Gai. 2020. Search-Based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction. In Proceedings of the 29th ACM International Conference on Information Knowledge Management (Virtual Event, Ireland) (CIKM ’20). Association for Computing Machinery, New York, NY, USA, 2685–2692.
[12] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J. Mach. Learn. Res. 15, 1, 1929–1958.
[13] Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential Recommendation with Bidirectional En- coder Representations from Transformer. In Proceedings of the 28th ACM In- ternational Conference on Information and Knowledge Management (CIKM ’19). Association for Computing Machinery, New York, NY, USA, 1441–1450.
[14] Jiaxi Tang and Ke Wang. 2018. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM ’18). 565–573.
[15] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. [n. d.]. Attention is All you Need. In Advances in Neural Information Processing Systems. Curran Associates, Inc.
[16] Kai Zhang, Hao Qian, Qing Cui, Qi Liu, Longfei Li, Jun Zhou, Jianhui Ma, and Enhong Chen. 2021. Multi-Interactive Attention Network for Fine-Grained Feature Learning in CTR Prediction. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining (Virtual Event, Israel) (WSDM ’21). Association for Computing Machinery, New York, NY, USA, 984–992.
[17] Guorui Zhou, Weijie Bian, Kailun Wu, Lejian Ren, Qi Pi, Yujing Zhang, Can Xiao, Xiang-Rong Sheng, Na Mou, Xinchen Luo, Chi Zhang, Xianjie Qiao, Shiming Xiang, Kun Gai, Xiaoqiang Zhu, and Jian Xu. 2020. CAN: Revisiting Feature Co-Action for Click-Through Rate Prediction. arXiv: 2011.05625 (2020).
[18] Guorui Zhou, Na Mou,Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep Interest Evolution Network for Click-Through Rate Prediction. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01, 5941–5948.
[19] Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu,XiaoMa,Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click- Through Rate Prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery Data Mining (London, United Kingdom) (KDD ’18). Association for Computing Machinery, New York, NY, USA, 1059–1068.
更多阅读
#投 稿 通 道#
让你的文字被更多人看到
如何才能让更多的优质内容以更短路径到达读者群体,缩短读者寻找优质内容的成本呢?答案就是:你不认识的人。
总有一些你不认识的人,知道你想知道的东西。PaperWeekly 或许可以成为一座桥梁,促使不同背景、不同方向的学者和学术灵感相互碰撞,迸发出更多的可能性。
PaperWeekly 鼓励高校实验室或个人,在我们的平台上分享各类优质内容,可以是最新论文解读,也可以是学术热点剖析、科研心得或竞赛经验讲解等。我们的目的只有一个,让知识真正流动起来。
📝 稿件基本要求:
• 文章确系个人原创作品,未曾在公开渠道发表,如为其他平台已发表或待发表的文章,请明确标注
• 稿件建议以 markdown 格式撰写,文中配图以附件形式发送,要求图片清晰,无版权问题
• PaperWeekly 尊重原作者署名权,并将为每篇被采纳的原创首发稿件,提供业内具有竞争力稿酬,具体依据文章阅读量和文章质量阶梯制结算
📬 投稿通道:
• 投稿邮箱:hr@paperweekly.site
• 来稿请备注即时联系方式(微信),以便我们在稿件选用的第一时间联系作者
• 您也可以直接添加小编微信(pwbot02)快速投稿,备注:姓名-投稿
△长按添加PaperWeekly小编
🔍
现在,在「知乎」也能找到我们了
进入知乎首页搜索「PaperWeekly」
点击「关注」订阅我们的专栏吧