Models of acoustic word embeddings (AWEs) learn to map variable-length spoken word segments onto fixed-dimensionality vector representations such that different acoustic exemplars of the same word are projected nearby in the embedding space. In addition to their speech technology applications, AWE models have been shown to predict human performance on a variety of auditory lexical processing tasks. Current AWE models are based on neural networks and trained in a bottom-up approach that integrates acoustic cues to build up a word representation given an acoustic or symbolic supervision signal. Therefore, these models do not leverage or capture high-level lexical knowledge during the learning process. In this paper, we propose a multi-task learning model that incorporates top-down lexical knowledge into the training procedure of AWEs. Our model learns a mapping between the acoustic input and a lexical representation that encodes high-level information such as word semantics in addition to bottom-up form-based supervision. We experiment with three languages and demonstrate that incorporating lexical knowledge improves the embedding space discriminability and encourages the model to better separate lexical categories.
翻译:声词嵌入模型(AWES)学会在固定维度矢量表示中绘制可变长的口头文字部分图,从而在嵌入空间附近预测同一词的不同声学模拟器。除了其语音技术应用外,AWES模型还展示了在各种听觉词汇处理任务中预测人类性能的模型。当前的AWES模型以神经网络为基础,并经过自下而上的方法培训,该模型结合声音信号,以建立一个字表达式,并给出声音或象征性监督信号。因此,这些模型在学习过程中没有利用或捕捉到高层次的词汇知识。在本文中,我们提出了一个多任务学习模型,将自上而下的词汇知识纳入AWES的培训程序。我们的模式学习了声学投入和词汇表达法之间的图谱,该模型在以自下而上的形式监督之外,将词词词语学等高级信息编码。我们实验了三种语言,并证明纳入词汇知识可以改进空间扰动性,并鼓励模型更好地区分不同词汇类别。