无线深深深视频语义传输 (Wireless Deep Video Semantic Transmission)

In this paper, we design a new class of high-efficiency deep joint source-channel coding methods to achieve end-to-end video transmission over wireless channels. The proposed methods exploit nonlinear transform and conditional coding architecture to adaptively extract semantic features across video frames, and transmit semantic feature domain representations over wireless channels via deep joint source-channel coding. Our framework is collected under the name deep video semantic transmission (DVST). In particular, benefiting from the strong temporal prior provided by the feature domain context, the learned nonlinear transform function becomes temporally adaptive, resulting in a richer and more accurate entropy model guiding the transmission of current frame. Accordingly, a novel rate adaptive transmission mechanism is developed to customize deep joint source-channel coding for video sources. It learns to allocate the limited channel bandwidth within and among video frames to maximize the overall transmission performance. The whole DVST design is formulated as an optimization problem whose goal is to minimize the end-to-end transmission rate-distortion performance under perceptual quality metrics or machine vision task performance metrics. Across standard video source test sequences and various communication scenarios, experiments show that our DVST can generally surpass traditional wireless video coded transmission schemes. The proposed DVST framework can well support future semantic communications due to its video content-aware and machine vision task integration abilities.

翻译：在本文中,我们设计了新的一类高效的深层源-通道联合编码方法,通过无线频道实现终端到终端视频传输。拟议方法利用非线性变换和有条件编码结构,通过视频框架以适应性方式提取语义特征,并通过深层联合源-通道编码传输无线频道的语义特征域示意图。我们的框架是以深层视频语义传输(DVST)的名称收集的。特别是,利用地域环境先前提供的强势时间,学习的非线性变换功能在时间上变得适应性强,从而形成一个更丰富和更准确的连接模式,指导当前框架的传输。因此,开发了一个新型速调适传输机制,对视频源的深度联合源-通道编码进行定制化。它学会在视频框架中和在视频框架中分配有限的频道带宽度带宽,以最大限度地提高整个传输性能。整个DVST设计是一个优化问题,其目标是最大限度地减少终端到终端传输率率率-扭曲性功能,在视觉质量指标或机器愿景任务性度度度度度指标下,从而指导当前框架的传输。因此,将开发出一个新的速率适应适应性传输机制的传输机制。超越视频-通道-通道-通道-通道-通道-通道-通道-通道-通道-通道-通道-通道-视频-视频-视频-视频-视频-视频-视频-视频-通道/视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频-视频