Graph Convex Hull Bounds as generalized Jensen Inequalities

Jensen's inequality is ubiquitous in measure and probability theory, statistics, machine learning, information theory and many other areas of mathematics and data science. It states that, for any convex function $f\colon K \to \mathbb{R}$ on a convex domain $K \subseteq \mathbb{R}^{d}$ and any random variable $X$ taking values in $K$, $\mathbb{E}[f(X)] \geq f(\mathbb{E}[X])$. In this paper, sharp upper and lower bounds on $\mathbb{E}[f(X)]$, termed "graph convex hull bounds", are derived for arbitrary functions $f$ on arbitrary domains $K$, thereby strongly generalizing Jensen's inequality. Establishing these bounds requires the investigation of the convex hull of the graph of $f$, which can be difficult for complicated $f$. On the other hand, once these inequalities are established, they hold, just like Jensen's inequality, for any random variable $X$. Hence, these bounds are of particular interest in cases where $f$ is fairly simple and $X$ is complicated or unknown. Both finite- and infinite-dimensional domains and codomains of $f$ are covered, as well as analogous bounds for conditional expectations and Markov operators.