We propose a Deep Unsupervised Trajectory-based stabilization framework (DUT) in this paper\footnote{Our code is available at https://github.com/Annbless/DUTCode.}. Traditional stabilizers focus on trajectory-based smoothing, which is controllable but fragile in occluded and textureless cases regarding the usage of hand-crafted features. On the other hand, previous deep video stabilizers directly generate stable videos in a supervised manner without explicit trajectory estimation, which is robust but less controllable and the appropriate paired data are hard to obtain. To construct a controllable and robust stabilizer, DUT makes the first attempt to stabilize unstable videos by explicitly estimating and smoothing trajectories in an unsupervised deep learning manner, which is composed of a DNN-based keypoint detector and motion estimator to generate grid-based trajectories, and a DNN-based trajectory smoother to stabilize videos. We exploit both the nature of continuity in motion and the consistency of keypoints and grid vertices before and after stabilization for unsupervised training. Experiment results on public benchmarks show that DUT outperforms representative state-of-the-art methods both qualitatively and quantitatively.
翻译:我们建议在本页\ footoot{我们的代码可在https://github.com/Annbless/DUTCode.}中建立一个基于深层无监督的轨迹稳定框架(DUT),传统稳定器侧重于基于轨迹的平滑,在使用手工制作特征的隐蔽和无纹的案例中,这种平滑是可以控制的,但很脆弱。另一方面,以前的深层视频稳定器直接以监督的方式生成稳定的视频,没有明确的轨迹估计,这是稳健的,但控制不那么容易获得,适当的对齐数据很难获得。要建立一个可控和稳健的稳定器,DUT首次尝试稳定不稳定的视频,方法是以不受监督的深层学习方式明确估算和平滑轨迹,由基于 DNNN 的键点探测器和运动估计器组成,以产生基于网格的轨迹的轨迹,以及基于 DNN 的轨迹平滑器来稳定视频。我们利用运动的连续性性质以及关键点和网格的一致性脊椎,在前和电网网格的脊椎中首次尝试,通过明确评估和稳定地展示公共质量测试方法,展示结果。