The rise of variational autoencoders for image and video compression has opened the door to many elaborate coding techniques. One example here is the possibility of conditional interframe coding. Here, instead of transmitting the residual between the original frame and the predicted frame (often obtained by motion compensation), the current frame is transmitted under the condition of knowing the prediction signal. In practice, conditional coding can be straightforwardly implemented using a conditional autoencoder, which has also shown good results in recent works. In this paper, we provide an information theoretical analysis of conditional coding for inter frames and show in which cases gains compared to traditional residual coding can be expected. We also show the effect of information bottlenecks which can occur in practical video coders in the prediction signal path due to the network structure, as a consequence of the data-processing theorem or due to quantization. We demonstrate that conditional coding has theoretical benefits over residual coding but that there are cases in which the benefits are quickly canceled by small information bottlenecks of the prediction signal.
翻译:用于图像和视频压缩的变式自动编码器的上升为许多精心设计的编码技术打开了大门。 这里的一个例子是有条件的跨框架编码的可能性。 这里, 而不是传输原始框架和预测框架( 通常通过运动补偿获得) 之间的剩余部分, 目前的框架是在了解预测信号的条件下传输的。 实际上, 有条件的自动编码可以直接地使用一个有条件的自动编码器来实施, 这在近期的作品中也显示了良好的结果。 在本文中, 我们提供了对跨框架有条件编码的信息理论分析, 并表明与传统的剩余编码相比, 有可能在哪些情况下出现增益。 我们还展示了由于网络结构而导致的预测信号路径中的实际视频编码器中可能出现的信息瓶颈效应, 其原因是数据处理的定理器或者由于四分化。 我们证明, 有条件的编码对剩余编码具有理论上的好处, 但有些情况下, 与预测信号的小型信息瓶颈很快抵消了这些好处。