General P-splines for non-uniform B-splines

P-spline represents an unknown univariate function with uniform B-splines on equidistant knots and penalizes their coefficients using a simple difference matrix for smoothness. But for non-uniform B-splines on unevenly spaced knots, such difference penalty fails, and the conventional derivative penalty is hitherto the only choice. We proposed a general P-spline estimator to lift this restriction by deriving a general difference penalty for non-uniform B-splines. We also established a sandwich formula between derivative and general difference penalties for a better understanding of their connections. Simulations show that both P-spline variants have close MSE performance in general. But in practice, one can yield a more satisfactory fit than the other. For example, the bone mineral content (BMC) data favor general P-spline, while the fossil shell data favor standard P-spline. We therefore believe both variants to be useful tools for practical modeling. To implement our general P-spline, we developed two new R packages: gps and gps.mgcv. The latter creates a new "gps" smooth class for mgcv, so that a general P-spline can be specified as s(x, bs = "gps") in a model formula and estimated in the framework of generalized additive models.