We present a robust theoretical framework for the characterisation and manipulation of optical flow, i.e 2D vector fields, in the context of their use in motion estimation algorithms and beyond. The definition of two frames of reference guides the mathematical derivation of flow field application, inversion, evaluation, and composition operations. This structured approach is then used as the foundation for an implementation in Python 3, with the fully differentiable PyTorch version oflibpytorch supporting back-propagation as required for deep learning. We verify the flow composition method empirically and provide a working example for its application to optical flow ground truth in synthetic training data creation. All code is publicly available.
翻译:我们为光学流(即2D矢量场)的特征化和操作提供了一个坚实的理论框架,即2D矢量场在运动估计算法和范围外的应用中使用。两个参照框架的定义指导了流体应用、反转、评价和组成操作的数学衍生。然后,这一结构化方法被用作在Python 3实施的基础,其基础是完全不同的Pytorch版本的libtytorch,它支持深层学习所需要的反向调整。我们以经验方式核查流体构成方法,并为在合成培训数据创建中将其应用于光学流动地面真理提供了一个工作范例。所有代码都可以公开查阅。