The efficient communication of noisy data has applications in several areas of machine learning, such as neural compression or differential privacy, and is also known as reverse channel coding or the channel simulation problem. Here we propose two new coding schemes with practical advantages over existing approaches. First, we introduce ordered random coding (ORC) which uses a simple trick to reduce the coding cost of previous approaches. This scheme further illuminates a connection between schemes based on importance sampling and the so-called Poisson functional representation. Second, we describe a hybrid coding scheme which uses dithered quantization to more efficiently communicate samples from distributions with bounded support.
翻译:噪音数据的高效交流在机器学习的若干领域,如神经压缩或有差别的隐私,也被称为反向通道编码或频道模拟问题。在这里,我们提出了两个与现有方法相比具有实际优势的新编码办法。首先,我们引入了命令随机编码(ORC),使用简单手段降低先前方法的编码成本。这个办法进一步说明了基于重要性取样和所谓的Poisson功能代表的组合之间的联系。第二,我们描述了一种混合编码办法,它利用抖动的定量法,在受约束的支持下更有效地传送分布的样品。