Many machine learning tasks that involve predicting an output response can be solved by training a weighted regression model. Unfortunately, the predictive power of this type of models may severely deteriorate under low sample sizes or under covariate perturbations. Reweighting the training samples has aroused as an effective mitigation strategy to these problems. In this paper, we propose a novel and coherent scheme for kernel-reweighted regression by reparametrizing the sample weights using a doubly non-negative matrix. When the weighting matrix is confined in an uncertainty set using either the log-determinant divergence or the Bures-Wasserstein distance, we show that the adversarially reweighted estimate can be solved efficiently using first-order methods. Numerical experiments show that our reweighting strategy delivers promising results on numerous datasets.
翻译:许多涉及预测产出响应的机器学习任务可以通过培训一个加权回归模型来解决。 不幸的是,这类模型的预测力可能会在低样本大小或共变扰动下严重恶化。 将培训样本作为有效缓解这些问题的有效战略,对培训样本进行重新加权。 在本文中,我们提出了一个新颖和连贯的内核加权回归计划,通过使用双重的非负负矩阵对样本重量进行重新校正。 当加权矩阵被限制在不确定性中时,使用对数-确定差异或布雷斯-沃瑟斯坦距离来设置时,我们表明对称的重加权估计可以用第一阶方法有效解决。 数字实验表明,我们的重加权战略在众多数据集上带来了有希望的结果。