Recent advancements in video generation highlight that realistic audio-visual synchronization is crucial for engaging content creation. However, existing video editing methods largely overlook audio-visual synchronization and lack the fine-grained spatial and temporal controllability required for precise instance-level edits. In this paper, we propose AVI-Edit, a framework for audio-sync video instance editing. We propose a granularity-aware mask refiner that iteratively refines coarse user-provided masks into precise instance-level regions. We further design a self-feedback audio agent to curate high-quality audio guidance, providing fine-grained temporal control. To facilitate this task, we additionally construct a large-scale dataset with instance-centric correspondence and comprehensive annotations. Extensive experiments demonstrate that AVI-Edit outperforms state-of-the-art methods in visual quality, condition following, and audio-visual synchronization. Project page: https://hjzheng.net/projects/AVI-Edit/.
翻译:近期视频生成领域的进展表明,逼真的视听同步对于创作引人入胜的内容至关重要。然而,现有视频编辑方法大多忽视了视听同步问题,且缺乏实现精确实例级编辑所需的细粒度空间与时间可控性。本文提出AVI-Edit框架,用于实现音频同步的视频实例编辑。我们设计了一种粒度感知掩码优化器,能够将用户提供的粗糙掩码迭代优化为精确的实例级区域。进一步开发了自反馈音频智能体,用于筛选高质量音频引导信号,实现细粒度时序控制。为支撑该任务,我们还构建了大规模数据集,包含以实例为中心的对应关系及全面标注。大量实验表明,AVI-Edit在视觉质量、条件遵循度及视听同步性方面均优于现有先进方法。项目页面:https://hjzheng.net/projects/AVI-Edit/。