We study a multivariate version of trend filtering, called Kronecker trend filtering or KTF, for the case in which the design points form a lattice in $d$ dimensions. KTF is a natural extension of univariate trend filtering (Steidl et al., 2006; Kim et al., 2009; Tibshirani, 2014), and is defined by minimizing a penalized least squares problem whose penalty term sums the absolute (higher-order) differences of the parameter to be estimated along each of the coordinate directions. The corresponding penalty operator can be written in terms of Kronecker products of univariate trend filtering penalty operators, hence the name Kronecker trend filtering. Equivalently, one can view KTF in terms of an $\ell_1$-penalized basis regression problem where the basis functions are tensor products of falling factorial functions, a piecewise polynomial (discrete spline) basis that underlies univariate trend filtering. This paper is a unification and extension of the results in Sadhanala et al. (2016, 2017). We develop a complete set of theoretical results that describe the behavior of $k^{\mathrm{th}}$ order Kronecker trend filtering in $d$ dimensions, for every $k \geq 0$ and $d \geq 1$. This reveals a number of interesting phenomena, including the dominance of KTF over linear smoothers in estimating heterogeneously smooth functions, and a phase transition at $d=2(k+1)$, a boundary past which (on the high dimension-to-smoothness side) linear smoothers fail to be consistent entirely. We also leverage recent results on discrete splines from Tibshirani (2020), in particular, discrete spline interpolation results that enable us to extend the KTF estimate to any off-lattice location in constant-time (independent of the size of the lattice $n$).
翻译:我们研究一个多变的趋势过滤版本,称为 Kronecker 趋势过滤或 KTF, 用于设计点形成以美元为单位的拉特值。 KTF 是一个单向趋势过滤的自然延伸( Steidl 等人, 2006; Kim 等人, 2009; Tibshirani, 2014), 定义方法是尽量减少一个受罚的最低方位问题, 其惩罚期将每个协调方向估算参数的绝对( 更高级) 差数相加。 相应的刑罚操作员可以写成以美元为单位的Kronecker 产品, 以美元为单位的 univariate 趋势过滤操作员为单位, 从而命名 Kronecker 趋势过滤器 。 Qreckeraltermal 范围, 可以用美元为美元为单位的平流利差值 ; K-lickral 水平为单位, 直径比值为美元( 美元为美元 美元) 直径直径直径直径直径直值 。