Phase transitions in nonparametric regressions: a curse of exploiting higher degree smoothness assumptions in finite samples

When the regression function belongs to a smooth class consisting of univariate functions with derivatives up to the $(\gamma+1)$th order bounded in absolute values for a finite $\gamma$, it is generally viewed that exploiting higher degree smoothness assumptions helps reduce the estimation error. This paper shows that the minimax optimal mean integrated squared error (MISE) increases in $\gamma$ when the sample size $n$ is small relative to the order of $\left(\gamma+1\right)^{2\gamma+3}$, and decreases in $\gamma$ when $n$ is large relative to the order of $\left(\gamma+1\right)^{2\gamma+3}$. In particular, this phase transition property is shown to be achieved by common nonparametric procedures. Consider $\gamma_{1}$ and $\gamma_{2}$ such that $\gamma_{1}<\gamma_{2}$, where the $(\gamma_{2}+1)$th degree smoothness class is a subset of the $(\gamma_{1}+1)$th degree class. What is surprising about our results is that they imply, if $n$ is small relative to the order of $\left(\gamma_{1}+1\right)^{2\gamma_{1}+3}$, the optimal rate achieved by the estimator constrained to be in the smoother class (to exploit the $(\gamma_{2}+1)$th degree smoothness) is slower. In data sets with fewer than hundreds-of-thousands observations, our results suggest that one should not exploit beyond the third or fourth degree of smoothness. To some extent, our results provide a theoretical basis for the widely adopted practical recommendations given by Gelman and Imbens (2019). The building blocks of our minimax optimality results are a set of metric entropy bounds we develop in this paper for smooth function classes. Some of our bounds are original, and some of them improve and/or generalize the ones in the literature.