汉语文语转换中语义与表现力联合建模

项目名称： 汉语文语转换中语义与表现力联合建模

项目编号： No.60875015

项目类型： 面上项目

立项/批准年度： 2009

项目学科： 金属学与金属工艺

项目作者： 杨鸿武

作者单位： 西北师范大学

项目金额： 29万元

中文摘要： 高表现力的语音合成是人机交互领域的一个重要研究课题。本研究起始于由文本内容表达所确定的语音表现力的描述，重点研究了表现力语音声学特征的建模与转换。主要研究成果如下： 1．抽取了描述信息表达模式的文本表现力特征集（ETFS），提出了一种基于文本表现力特征表示文本表现力的方法。采用三维PAD情感模型量化描述文本的表现力特征。定义了一套对话活动的D值和关键概念的（P,A）值的标注规则标注了韵律词级和语句级的表现力特征。 2．提出并建立了一种表现力声学特征预测模型（EAPM）。利用非线性回归获得模型，通过文本表现力特征的D值和（P,A）值预测表现力声学特征的变化率。实验结果表明，该模型输出的声学特征有效描述了语音的表现力。 3．提出一种基于合成分析法建立表现力声学特征叠加模型（EASM）的方法。利用支持向量回归建立模型，采用合成分析法，利用模型的预测误差迭代优化模型参数。 4．在歌词到歌曲的转换、情感语音合成、方言的韵律转换和语音信号处理等方面展开了研究，提出了基于五度字调模型的方言语音韵律转换方法，利用GMM建立了语音的频谱模型，采用CS理论实现语音增强，利用FPGA实现了语图仪。

中文关键词： 语义概念;表现力;语音合成;文语转换;表现力模型

英文摘要： Expressive speech synthesis in the field of human-computer interaction is an important research topic. The research begins with the description of expressivity determined by message content, focusing on modeling acoustic correlates of expressive speech and transferring of acoustic features of neutral speech in accordance with their expressive version. Main achievements and originalities are as follow. Firstly, by proposing an expressive textual feature set (ETFS) for text based on expression pattern of a message, the work establishes a method for describing expressivity of message content with expressive textual feature. The work describes expressivity of message content by annotating dialog act of sentences and key concept of prosodic words. Three dimensional PAD emotional model is employed to annotate expressivity of dialog act and key concept of message content. The work establishes a set of annotating rules to describe D values of dialog act and (P,A) values of key concept. Secondly, the work proposes and establishes a non-linear expressive acoustic feature prediction model (EAPM) for acoustic correlates of expressive textual feature. The model uses the D value and (P,A) value to predict the increases of acoustic features of expressive speech. Thirdly, based on analysis-by-synthesis, an expressive acoustic feature superposing model (EASM) is proposed. EASM consists of three models-a neutral component model, a sentence level expressive component model and a prosodic word level expressive component model. The neutral component model predicts neutral acoustic features with textual context features. The model is trained with Support Vector Regression. The parameters of the model are optimized with the prediction errors of the model using analysis-by-synthesis iteratively. Finally, the work also studied lyrics to singing voice conversion, emotional speech synthesis, prosodic conversion of Mandarin Dialects, as well as speech signal processing. The GMM was used to model spectrum of speech for lyrics to singing voice conversion. A five degree tone model was employed for Mandarin Dialects conversion. The work also conducted speech signal processing researches such as speech enhancement based on CS theory and realization a spectrogram with the FPGA for real time speech analysis

英文关键词： Semantic Prior; Expressivity;Speech Synthesis; Text-to-Speech Synthesis; Expressive Model