We study the problem of word-level confidence estimation in subword-based end-to-end (E2E) models for automatic speech recognition (ASR). Although prior works have proposed training auxiliary confidence models for ASR systems, they do not extend naturally to systems that operate on word-pieces (WP) as their vocabulary. In particular, ground truth WP correctness labels are needed for training confidence models, but the non-unique tokenization from word to WP causes inaccurate labels to be generated. This paper proposes and studies two confidence models of increasing complexity to solve this problem. The final model uses self-attention to directly learn word-level confidence without needing subword tokenization, and exploits full context features from multiple hypotheses to improve confidence accuracy. Experiments on Voice Search and long-tail test sets show standard metrics (e.g., NCE, AUC, RMSE) improving substantially. The proposed confidence module also enables a model selection approach to combine an on-device E2E model with a hybrid model on the server to address the rare word recognition problem for the E2E model.
翻译:我们研究了在基于小字的终端到终端自动语音识别(ASR)模型中的字级信任度估计问题。虽然先前的工程提议为ASR系统培训辅助信任度模型,但并不自然地扩展到以字件(WP)作为其词汇的系统,特别是,培训信任度模型需要地面真理WP正确度标签,但从文字到WP的非单一象征性标签导致产生不准确的标签。本文件提出并研究两种越来越复杂的信任度模型,以解决这一问题。最后的模型利用自我意识直接学习字级信任度,而不需要小字标记,并利用多个假设的完整环境特征来提高信任度。语音搜索实验和长尾试验组显示标准指标(例如NCE、AUC、ARME)大幅改进。拟议的信任模块还使得示范选择方法能够将“点E2E模型”模型与服务器上的混合模型结合起来,以解决E2E模型的稀有文字识别问题。