Partial Information Decomposition via Deficiency for Multivariate Gaussians

Partial information decompositions (PIDs) extend the concept of mutual information to three (or more) random variables: they decompose the mutual information between a "message" and two other random variables into information components that are unique to each variable, redundantly present in both, and synergistic. This paper focuses on the PID of three jointly Gaussian random vectors. Barrett (2015) previously characterized the Gaussian PID for a scalar message in closed form - we examine the case where the message is a vector. Specifically, we provide a necessary and sufficient condition for the existence of unique information in the fully multivariate Gaussian PID. We do this by drawing a connection between the notion of Blackwell sufficiency from statistics and the idea of stochastic degradedness of broadcast channels from communication theory. Our first result shows that Barrett's closed form expression extends to the case of vector messages only very rarely. To compute the Gaussian PID in all other cases, we provide a convex optimization approach for approximating the PID, analyze its properties, and evaluate it empirically on randomly generated Gaussian systems.