The recent prevalence of pretrained language models (PLMs) has dramatically shifted the paradigm of semantic parsing, where the mapping from natural language utterances to structured logical forms is now formulated as a Seq2Seq task. Despite the promising performance, previous PLM-based approaches often suffer from hallucination problems due to their negligence of the structural information contained in the sentence, which essentially constitutes the key semantics of the logical forms. Furthermore, most works treat PLM as a black box in which the generation process of the target logical form is hidden beneath the decoder modules, which greatly hinders the model's intrinsic interpretability. To address these two issues, we propose to incorporate the current PLMs with a hierarchical decoder network. By taking the first-principle structures as the semantic anchors, we propose two novel intermediate supervision tasks, namely Semantic Anchor Extraction and Semantic Anchor Alignment, for training the hierarchical decoders and probing the model intermediate representations in a self-adaptive manner alongside the fine-tuning process. We conduct intensive experiments on several semantic parsing benchmarks and demonstrate that our approach can consistently outperform the baselines. More importantly, by analyzing the intermediate representations of the hierarchical decoders, our approach also makes a huge step toward the intrinsic interpretability of PLMs in the domain of semantic parsing.
翻译:最近受过训练的语言模型(PLM)的流行急剧改变了语义分析模式的范式,即从自然语言表达到结构化逻辑形式的绘图现在被设计成Seq2Seqeq的任务。尽管表现良好,以前基于PLM的方法往往由于忽视了该句所载结构信息而出现幻觉问题,这基本上是逻辑形式的关键语义。此外,大多数工作都把PLM当作黑盒,在这个黑盒中,目标逻辑形式的生成过程隐藏在解码模块之下,这大大妨碍了该模型的内在解释性。为了解决这两个问题,我们提议将目前的PLMS纳入一个等级解码网络。我们建议,通过将第一个原则结构作为语义支柱,我们提议两项新型的中间监督任务,即Semantic Anchor Expliton 和Semmantic Achor 匹配,以培训等级解码和验证模型中间表述过程,在微调化过程中,我们用几部语义解码化的本位基准进行密集实验,并展示我们最大的等级方法。