Active learning (AL) is a prominent technique for reducing the annotation effort required for training machine learning models. Deep learning offers a solution for several essential obstacles to deploying AL in practice but introduces many others. One of such problems is the excessive computational resources required to train an acquisition model and estimate its uncertainty on instances in the unlabeled pool. We propose two techniques that tackle this issue for text classification and tagging tasks, offering a substantial reduction of AL iteration duration and the computational overhead introduced by deep acquisition models in AL. We also demonstrate that our algorithm that leverages pseudo-labeling and distilled models overcomes one of the essential obstacles revealed previously in the literature. Namely, it was shown that due to differences between an acquisition model used to select instances during AL and a successor model trained on the labeled data, the benefits of AL can diminish. We show that our algorithm, despite using a smaller and faster acquisition model, is capable of training a more expressive successor model with higher performance.
翻译:积极学习(AL)是减少培训机器学习模式所需的批注努力的突出技术。深层学习为在实践中部署AL提供若干基本障碍的解决方案,但引入了许多其他障碍。其中一个问题是,培训购置模型和估计其在未贴标签的人才库中实例的不确定性所需的过度计算资源。我们建议了两种技术来解决这一问题,用于文本分类和标记任务,大大缩短AL的迭代时间和深层获取模型引入的计算间接费用。我们还表明,利用伪标签和蒸馏模型的算法克服了文献中以前揭示的基本障碍之一。也就是说,由于在AL期间用于选择案例的购置模型与在标签数据上培训的继任模型之间存在差异,AL的好处可以减少。我们表明,我们的算法,尽管使用较小和更快的获取模型,但能够培训一种表现更高的更清晰的替代模型。