In this study, the effects of eight representation regularization methods are investigated, including two newly developed rank regularizers (RR). The investigation shows that the statistical characteristics of representations such as correlation, sparsity, and rank can be manipulated as intended, during training. Furthermore, it is possible to improve the baseline performance simply by trying all the representation regularizers and fine-tuning the strength of their effects. In contrast to performance improvement, no consistent relationship between performance and statistical characteristics was observable. The results indicate that manipulation of statistical characteristics can be helpful for improving performance, but only indirectly through its influence on learning dynamics or its tuning effects.
翻译:在本研究中,调查了8种代表制正规化方法的影响,包括两个新开发的级别管理者(RR),调查表明,在培训期间,可以按预期操纵相关性、宽度和级别等代表的统计特征;此外,可以仅仅通过尝试所有代表制规范化者并微调其效果的强度来改进基线绩效;与业绩改进相比,业绩与统计特征之间没有一致的关系;结果显示,操纵统计特征有助于改进业绩,但只能通过对学习动态或其调试效果的影响来间接地加以改进。