Conveying the linguistic content and maintaining the source speech's speaking style, such as intonation and emotion, is essential in voice conversion (VC). However, in a low-resource situation, where only limited utterances from the target speaker are accessible, existing VC methods are hard to meet this requirement and capture the target speaker's timber. In this work, a novel VC model, referred to as MFC-StyleVC, is proposed for the low-resource VC task. Specifically, speaker timbre constraint generated by clustering method is newly proposed to guide target speaker timbre learning in different stages. Meanwhile, to prevent over-fitting to the target speaker's limited data, perceptual regularization constraints explicitly maintain model performance on specific aspects, including speaking style, linguistic content, and speech quality. Besides, a simulation mode is introduced to simulate the inference process to alleviate the mismatch between training and inference. Extensive experiments performed on highly expressive speech demonstrate the superiority of the proposed method in low-resource VC.
翻译:在低资源情况下,现有越共方法很难满足这一要求并捕捉到目标发言者的木材。在这项工作中,为低资源VC任务提议了一个称为MFC-StyleVC的新型越共模式。具体地说,群集方法产生的发言者小字节限制是新提出的,以指导不同阶段的演讲者小字节学习。与此同时,为了防止过分适应目标发言者的有限数据,概念正规化限制明确保持特定方面的示范性表现,包括发言风格、语言内容和语言质量。此外,还采用模拟模式模拟推论过程,以缓解培训与推论之间的不匹配。在高显性演讲上进行的广泛实验显示了低资源VC的拟议方法的优越性。</s>