Anderson acceleration (AA) is a well-known method for accelerating the convergence of iterative algorithms, with applications in various fields including deep learning and optimization. Despite its popularity in these areas, the effectiveness of AA in classical machine learning classifiers has not been thoroughly studied. Tabular data, in particular, presents a unique challenge for deep learning models, and classical machine learning models are known to perform better in these scenarios. However, the convergence analysis of these models has received limited attention. To address this gap in research, we implement a support vector machine (SVM) classifier variant that incorporates AA to speed up convergence. We evaluate the performance of our SVM with and without Anderson acceleration on several datasets from the biology domain and demonstrate that the use of AA significantly improves convergence and reduces the training loss as the number of iterations increases. Our findings provide a promising perspective on the potential of Anderson acceleration in the training of simple machine learning classifiers and underscore the importance of further research in this area. By showing the effectiveness of AA in this setting, we aim to inspire more studies that explore the applications of AA in classical machine learning.
翻译:安德森加速(AA)是加速迭代算法汇合的著名方法,其应用领域包括深层次学习和优化。尽管这些领域受到欢迎,但古典机器学习分类法AA在古典机器学习分类法中的效力尚未得到彻底研究。特别是,表格数据对深层次学习模型提出了独特的挑战,而古典机器学习模型在这些情况中表现得较好。然而,这些模型的汇合分析受到的关注有限。为了弥补研究中的这一差距,我们实施了一种支持矢量机分类法变异器(SVM),其中包括AA,以加快汇合速度。我们评估了我们的SVM在生物学领域若干数据集上的性能和不加速Anderson的性能,并表明随着迭代数的增加,AA的使用大大改进了趋同并减少了培训损失。我们的调查结果为安德森在培训简单机器学习分类师方面加快速度的潜力提供了很有希望的观点,并强调了在这一领域进一步开展研究的重要性。通过展示AAA在这种背景下的有效性,我们的目标是鼓励进行更多的研究,探索AA在古典机器学习中的应用。