In this work, we propose solving the Information bottleneck (IB) and Privacy Funnel (PF) problems with Douglas-Rachford Splitting methods (DRS). We study a general Markovian information-theoretic Lagrangian that includes IB and PF into a unified framework. We prove the linear convergence of the proposed solvers using the Kurdyka-{\L}ojasiewicz inequality. Moreover, our analysis is beyond IB and PF and applies to any convex-weakly convex pair objectives. Based on the results, we develop two types of linearly convergent IB solvers, with one improves the performance of convergence over existing solvers while the other can be independent to the relevance-compression trade-off. Moreover, our results apply to PF, yielding a new class of linearly convergent PF solvers. Empirically, the proposed IB solvers IB obtain solutions that are comparable to the Blahut-Arimoto-based benchmark and is convergent for a wider range of the penalty coefficient than existing solvers. For PF, our non-greedy solvers can characterize the privacy-utility trade-off better than the clustering-based greedy solvers.
翻译:在这项工作中,我们建议解决Douglas-Rachford分解方法(DRS)的信息瓶颈(IB)和隐私漏斗(PF)问题。我们研究一个包括IB和PF的统一框架的通用Markovian信息理论拉格朗吉将军的信息理论(Lagrangian)问题。我们用Kurdyka-L}jasiewicz的不平等来证明拟议的解决者的线性趋同。此外,我们的分析超出了IB和PF的范围,适用于任何以Blahut-Arimoto为基础的对等目标。根据结果,我们开发了两类线性趋同的 IB解答器,其中一种改进了现有解决者之间的趋同性,而另一种则可以独立于相关压缩交易。此外,我们的结果适用于PFPF, 产生了一个新的线性趋同的PFF解溶剂类别。 想象的是,拟议的IB获得与Blahut-Arimoto-bol为基础的基准相当的解决方案,对于比现有解决者更广泛的惩罚系数范围更为一致。