Nowadays, deep learning models are widely adopted in web-scale applications such as recommender systems, and online advertising. In these applications, embedding learning of categorical features is crucial to the success of deep learning models. In these models, a standard method is that each categorical feature value is assigned a unique embedding vector which can be learned and optimized. Although this method can well capture the characteristics of the categorical features and promise good performance, it can incur a huge memory cost to store the embedding table, especially for those web-scale applications. Such a huge memory cost significantly holds back the effectiveness and usability of EDRMs. In this paper, we propose a binary code based hash embedding method which allows the size of the embedding table to be reduced in arbitrary scale without compromising too much performance. Experimental evaluation results show that one can still achieve 99\% performance even if the embedding table size is reduced 1000$\times$ smaller than the original one with our proposed method.
翻译:目前,深层次学习模式被广泛采用于推荐人系统和在线广告等网络规模的应用中。在这些应用中,嵌入绝对特征对于深层学习模式的成功至关重要。在这些模型中,标准的方法是给每个绝对特征值指定一个独特的嵌入矢量,可以学习和优化。虽然这种方法可以很好地捕捉绝对特征的特征,并有望取得良好的业绩,但存储嵌入表,特别是这些网络规模的应用程序,可能会产生巨大的记忆成本。这样的巨大的记忆成本大大地抑制了 EDRMs 的有效性和可用性。在本文中,我们提出了一个基于二元代码的 Hash 嵌入方法,允许任意缩小嵌入表的大小,同时不影响太多的性能。实验性评估结果显示,即使嵌入表的大小缩小了1000美元\ 时间,但人们仍然可以达到99 ⁇ 的性能。