We present a novel camera path optimization framework for the task of online video stabilization. Typically, a stabilization pipeline consists of three steps: motion estimating, path smoothing, and novel view rendering. Most previous methods concentrate on motion estimation, proposing various global or local motion models. In contrast, path optimization receives relatively less attention, especially in the important online setting, where no future frames are available. In this work, we adopt recent off-the-shelf high-quality deep motion models for the motion estimation to recover the camera trajectory and focus on the latter two steps. Our network takes a short 2D camera path in a sliding window as input and outputs the stabilizing warp field of the last frame in the window, which warps the coming frame to its stabilized position. A hybrid loss is well-defined to constrain the spatial and temporal consistency. In addition, we build a motion dataset that contains stable and unstable motion pairs for the training. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art online methods both qualitatively and quantitatively and achieves comparable performance to offline methods.
翻译:我们为在线视频稳定化的任务提出了一个新型的相机路径优化框架。 通常, 稳定化管道由三步组成: 运动估算、 道路平滑和新颖的视图。 多数先前的方法都集中在运动估算上, 提出各种全球或本地运动模型。 相反, 路径优化相对较少受到关注, 特别是在重要的在线环境, 没有未来框架。 在这项工作中, 我们采用了最新的现成高品质高品质的运动模型, 用于运动估算, 以恢复相机轨迹, 并关注后两个步骤。 我们的网络在滑动窗口中采用一个短短的 2D 相机路径, 作为输入和输出窗口最后一个框架的稳定扭曲场, 将即将到的框扭曲到其稳定位置。 混合损失的定义非常明确, 以限制空间和时间的一致性 。 此外, 我们构建了一个运动数据集, 包含稳定的和不稳定的运动配对培训。 广泛的实验表明, 我们的方法在质量和数量上都大大超越了最先进的在线方法, 并实现了与离线方法相似的业绩 。