Sliced mutual information (SMI) is defined as an average of mutual information (MI) terms between one-dimensional random projections of the random variables. It serves as a surrogate measure of dependence to classic MI that preserves many of its properties but is more scalable to high dimensions. However, a quantitative characterization of how SMI itself and estimation rates thereof depend on the ambient dimension, which is crucial to the understanding of scalability, remain obscure. This work provides a multifaceted account of the dependence of SMI on dimension, under a broader framework termed $k$-SMI, which considers projections to $k$-dimensional subspaces. Using a new result on the continuity of differential entropy in the 2-Wasserstein metric, we derive sharp bounds on the error of Monte Carlo (MC)-based estimates of $k$-SMI, with explicit dependence on $k$ and the ambient dimension, revealing their interplay with the number of samples. We then combine the MC integrator with the neural estimation framework to provide an end-to-end $k$-SMI estimator, for which optimal convergence rates are established. We also explore asymptotics of the population $k$-SMI as dimension grows, providing Gaussian approximation results with a residual that decays under appropriate moment bounds. All our results trivially apply to SMI by setting $k=1$. Our theory is validated with numerical experiments and is applied to sliced InfoGAN, which altogether provide a comprehensive quantitative account of the scalability question of $k$-SMI, including SMI as a special case when $k=1$.
翻译:片断的相互信息(SMI)的定义是,在单维随机变量随机预测的一维随机值之间的相互信息(MI)平均值(MI)值,它是对传统的MI的替代度,它保存着它的许多属性,但更可伸缩到高维。然而,对SMI本身及其估计率的定量定性取决于环境层面,这对理解可缩放性至关重要。这项工作在名为 美元-SMI的更广泛框架下,从多方面说明SMI对维度的依赖性。这个框架考虑对美元-SMI子空间的预测。我们利用对2-Wasserstein 公标的差分星座的连续性的新结果,对基于Monte Car(MC)的美元-SMI的估计数及其估计率的误差进行了清晰的界限。 明确依赖美元和环境维度,揭示了它们与样品的相互作用。 然后,我们将MISI的内值和内值的内值估算框架结合起来,我们用S-K美元-Simal-al-al-deal-deal-al-sal-mailational-sal-sal-sal-slation exlational-slation ex exlation exluplation ex exlations exlationslations exlations ex exlations exluplup ex ex ex ex ex ex ex ex ex ial ex exlus ial ial ial-slationslational-slational-slation ivalut ex ex exx i ex ex ex ex ex ex-s ial-s-s i i i i i exal exal exal i ex ex ex ex ex ex ex ex ex ex ex exal ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ial ial ial ial ial ial ial ial ex ial ex ial ial