Widespread applications of deep neural networks (DNNs) benefit from DNN testing to guarantee their quality. In the DNN testing, numerous test cases are fed into the model to explore potential vulnerabilities, but they require expensive manual cost to check the label. Therefore, test case prioritization is proposed to solve the problem of labeling cost, e.g., activation-based and mutation-based prioritization methods. However, most of them suffer from limited scenarios (i.e. high confidence adversarial or false positive cases) and high time complexity. To address these challenges, we propose the concept of the activation graph from the perspective of the spatial relationship of neurons. We observe that the activation graph of cases that triggers the models' misbehavior significantly differs from that of normal cases. Motivated by it, we design a test case prioritization method based on the activation graph, ActGraph, by extracting the high-order node features of the activation graph for prioritization. ActGraph explains the difference between the test cases to solve the problem of scenario limitation. Without mutation operations, ActGraph is easy to implement, leading to lower time complexity. Extensive experiments on three datasets and four models demonstrate that ActGraph has the following key characteristics. (i) Effectiveness and generalizability: ActGraph shows competitive performance in all of the natural, adversarial and mixed scenarios, especially in RAUC-100 improvement (~1.40). (ii) Efficiency: ActGraph does not use complex mutation operations and runs in less time (~1/50) than the state-of-the-art method.
翻译:深度神经网络(DNN)的广泛应用得益于DNN测试,以保证其质量。在DNN测试中,许多测试案例被输入模型,以探索潜在的脆弱性,但是它们需要昂贵的人工成本来检查标签。因此,建议测试案例的优先排序,以解决标签成本问题,例如,基于启动和突变的优先排序方法。然而,大多数测试案例都存在有限的情景(即,高度信心对抗性或虚假正面案例)和高时间复杂性。为了应对这些挑战,我们从神经神经的空间关系的角度提出启动图表的概念。我们观察到,触发模型错误行为的案例的启动图与正常案例明显不同。因此,我们根据启动图(Aclagraph)设计了一个测试案例优先排序方法。Acgraph解释了用于解决情景限制的测试案例之间的差异(Acgraph),没有突变操作容易实施,特别是导致时间复杂性较低。我们观察到,启动案例的启动图的启动图的启动图与正常情况相比,运行率(AGraph 4) 展示了三种关键性模型。