VoiceCraft-X：统一多语言、语音克隆语音合成与语音编辑 (VoiceCraft-X: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editing)

We introduce VoiceCraft-X, an autoregressive neural codec language model which unifies multilingual speech editing and zero-shot Text-to-Speech (TTS) synthesis across 11 languages: English, Mandarin, Korean, Japanese, Spanish, French, German, Dutch, Italian, Portuguese, and Polish. VoiceCraft-X utilizes the Qwen3 large language model for phoneme-free cross-lingual text processing and a novel token reordering mechanism with time-aligned text and speech tokens to handle both tasks as a single sequence generation problem. The model generates high-quality, natural-sounding speech, seamlessly creating new audio or editing existing recordings within one framework. VoiceCraft-X shows robust performance in diverse linguistic settings, even with limited per-language data, underscoring the power of unified autoregressive approaches for advancing complex, real-world multilingual speech applications. Audio samples are available at https://zhishengzheng.com/voicecraft-x/.

翻译：我们介绍了VoiceCraft-X，一种自回归神经编解码器语言模型，它统一了跨11种语言（英语、普通话、韩语、日语、西班牙语、法语、德语、荷兰语、意大利语、葡萄牙语和波兰语）的多语言语音编辑和零样本文本到语音（TTS）合成。VoiceCraft-X利用Qwen3大语言模型进行无音素跨语言文本处理，并采用一种新颖的、时间对齐的文本和语音令牌重排序机制，将两项任务作为单一序列生成问题来处理。该模型能生成高质量、自然逼真的语音，在一个框架内无缝创建新音频或编辑现有录音。VoiceCraft-X在多样化的语言环境中展现出稳健的性能，即使在每种语言数据有限的情况下也是如此，这凸显了统一的自回归方法在推进复杂、现实世界的多语言语音应用方面的强大能力。音频样本可在 https://zhishengzheng.com/voicecraft-x/ 获取。