Semantic communications has received growing interest since it can remarkably reduce the amount of data to be transmitted without missing critical information. Most existing works explore the semantic encoding and transmission for text and apply techniques in Natural Language Processing (NLP) to interpret the meaning of the text. In this paper, we conceive the semantic communications for image data that is much more richer in semantics and bandwidth sensitive. We propose an reinforcement learning based adaptive semantic coding (RL-ASC) approach that encodes images beyond pixel level. Firstly, we define the semantic concept of image data that includes the category, spatial arrangement, and visual feature as the representation unit, and propose a convolutional semantic encoder to extract semantic concepts. Secondly, we propose the image reconstruction criterion that evolves from the traditional pixel similarity to semantic similarity and perceptual performance. Thirdly, we design a novel RL-based semantic bit allocation model, whose reward is the increase in rate-semantic-perceptual performance after encoding a certain semantic concept with adaptive quantization level. Thus, the task-related information is preserved and reconstructed properly while less important data is discarded. Finally, we propose the Generative Adversarial Nets (GANs) based semantic decoder that fuses both locally and globally features via an attention module. Experimental results demonstrate that the proposed RL-ASC is noise robust and could reconstruct visually pleasant and semantic consistent image, and saves times of bit cost compared to standard codecs and other deep learning-based image codecs.
翻译:语义通信已受到越来越多的关注,因为它可以显著减少在不缺少关键信息的情况下传输的数据数量。 多数现有作品探索语言编码和文本传输, 并应用自然语言处理( NLP) 的语义编码技术来解释文本的含义 。 在本文中, 我们为图像数据设想语义通信, 在语义和带宽敏感度方面更加丰富。 我们提议了基于强化学习的稳健的语义编码( RL- ASC) 方法, 将图像编码超过像素水平。 首先, 我们定义了图像数据的语义概念, 包括作为代表单位的类别、 空间安排和视觉特征, 并提议了对语言的语义编码编码技术。 其次, 我们提出图像重建的标准标准代码, 从传统的像素相似性到感知性性性功能。 我们设计了一个基于语义的语义和感知性比值分配模式, 其奖赏是将某种语义的语义代码化性表现与视觉功能性功能化概念化概念化概念化概念化概念化, 将某种语义性语义化的语义语义化语义编码编码编码编码编码学概念化概念化概念化概念化概念化概念化概念化概念化,, 。 因此,,, 以不断的数学-, 的数学- 和数学- 校正化的数学- 校正解解解解解,,,,, 校正 校正 校正 校正 校正 校正 校正 校正 建 建 建 建 建 建 基 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建 建