An effective weighting scheme for training samples is essential for learning tasks. Numerous weighting schemes have been proposed. Some schemes take the easy-first mode, whereas some others take the hard-first one. Naturally, an interesting yet realistic question is raised. Which samples should be learned first given a new learning task, easy or hard? To answer this question, both theoretical analyses and experimental verification are conducted. First, a general optimized objective function is proposed, revealing the relationship between the difficulty distribution and the difficulty-based sample weights. Second, on the basis of the optimized objective function, theoretical answers are obtained. Besides the easy-first and hard-first modes, there are two other priority modes, namely, medium-first and two-ends-first. The prior mode does not necessarily remain unchanged during the training process. Third, an effective and universal solution is proposed to select the optimal priority mode when there is no prior knowledge or theoretical clues. The four modes, namely, easy/medium/hard/two-ends-first, can be flexibly switched in the proposed solution. Fourth, a wide range of experiments is conducted under various scenarios to further compare the weighting schemes in different modes. On the basis of these works, reasonable and comprehensive answers are obtained. Factors including the distribution of samples' learning difficulties and the validation data determine which samples should be learned first in a learning task.
翻译:培训样本的有效加权办法对于学习任务至关重要。许多加权办法已经提出。有些方案采用简单第一和硬第一模式,而另一些方案则采用简单第一模式。自然,提出一个有趣但现实的问题。在新的学习任务(容易或困难)中,哪些样本应该首先学习?为了回答这个问题,进行了理论分析和实验性核查。首先,提出了一个总体优化的目标功能,揭示困难分布和困难抽样加权之间的关系。第二,根据优化的客观功能,获得了理论答案。除了简单第一和硬第一模式外,还有另外两种优先模式,即中一级和两端第一模式。在培训过程中,前一种模式不一定保持不变。第三,提出一个有效而普遍的解决办法,在没有事先知识或理论线索的情况下选择最佳优先模式。四种模式,即简单/中/硬/两端第一模式,可以在拟议解决方案中灵活地转换。第四,在各种假设下进行广泛的实验,进一步比较加权办法的中期和二端第一模式。前一种模式不一定保持不变。在培训过程中,前一种模式不一定保持不变。第三,提出一个有效和普遍的解决办法是选择最佳的优先模式,即简单/中/硬/两端/两端-端-端-端-端-先,在拟议解决办法中可以灵活地转换。第四,在各种假设下进行广泛的试验,进一步比较加权办法的处理办法,在不同的分析,包括所学的抽样分析。在不同的分析中,学习方法中,然后是学习困难。在分析中,首先是学习过程的学习。