Classification systems are often deployed in resource-constrained settings where labels must be assigned to inputs on a budget of time, memory, etc. Budgeted, sequential classifiers (BSCs) address these scenarios by processing inputs through a sequence of partial feature acquisition and evaluation steps with early-exit options. This allows for an efficient evaluation of inputs that prevents unneeded feature acquisition. To approximate an intractable combinatorial problem, current approaches to budgeted classification rely on well-behaved loss functions that account for two primary objectives (processing cost and error). These approaches offer improved efficiency over traditional classifiers but are limited by analytic constraints in formulation and do not manage additional performance objectives. Notably, such methods do not explicitly account for an important aspect of real-time detection systems -- the fraction of "accepted" predictions satisfying a confidence criterion imposed by a risk-averse monitor. We propose a problem-specific genetic algorithm to build budgeted, sequential classifiers with confidence-based reject options. Three objectives -- accuracy, processing time/cost, and coverage -- are considered. The algorithm emphasizes Pareto efficiency while accounting for a notion of aggregate performance via a unique scalarization. Experiments show our method can quickly find globally Pareto optimal solutions in very large search spaces and is competitive with existing approaches while offering advantages for selective, budgeted deployment scenarios.
翻译:在资源紧张的环境中,往往会部署分类系统,因为必须分配标签用于对时间、记忆等预算的投入,因此必须分配时间、记忆等预算、记忆等预算投入。 预算、按顺序分类者处理这些假设情况,办法是通过一系列部分特征获取和评价步骤,处理投入,并采用提前退出选项。这样可以对投入进行高效评价,从而避免不必要的特征获取。为了处理棘手的组合问题,目前预算分类方法依赖于妥善管理的损失功能,其中考虑到两个主要目标(处理成本和错误)。这些方法比传统分类者提高了效率,但受到制定过程中分析性限制的限制,没有管理其他业绩目标。值得注意的是,这些方法没有明确说明实时检测系统的一个重要方面 -- -- 即 " 接受的 " 预测的一小部分满足风险偏差监测所强加的信任标准。我们建议了一种针对具体问题的遗传算法,以建立预算的、有基于信任的拒绝选项(处理成本和覆盖范围)的分类方法。三种目标 -- -- 准确性、处理时间/成本和范围 -- -- 得到考虑。算法强调效率,同时通过独特的比例化解决方案核算综合业绩概念,同时进行独特的大规模搜索。