论文标题

Actgraph:基于深神经网络激活图的测试用例的优先级

ActGraph: Prioritization of Test Cases Based on Deep Neural Network Activation Graph

论文作者

Chen, Jinyin, Ge, Jie, Zheng, Haibin

论文摘要

深神经网络(DNN)的广泛应用受益于DNN测试,以确保其质量。在DNN测试中,许多测试用例被馈入模型以探索潜在的漏洞,但它们需要昂贵的手动成本来检查标签。因此,提出了测试案例的优先级来解决标记成本的问题,例如基于激活和基于突变的优先级方法。但是,他们中的大多数情况都受到有限的情况(即高信心对抗或假阳性案例)和高时间复杂性。为了应对这些挑战,我们从神经元的空间关系的角度提出了激活图的概念。我们观察到,触发模型不当行为的病例的激活图与正常情况有显着不同。由IT激励,我们通过提取激活图的高阶节点特征以优先级来设计基于激活图ActGraph的测试案例优先方法。 Actgraph解释了解决方案限制问题的测试用例之间的区别。没有突变操作,ActGraph易于实施,从而导致时间复杂性较低。在三个数据集和四个模型上进行的广泛实验表明,ActGraph具有以下关键特征。 (i)有效性和概括性:ActGraph在所有自然,对抗和混合场景中都显示出竞争性能,尤其是在Rauc-100改善(〜1.40)中。 (ii)效率:ActGraph不使用复杂的突变操作,并且在更少的时间内(〜1/50)运行,而不是最先进的方法。

Widespread applications of deep neural networks (DNNs) benefit from DNN testing to guarantee their quality. In the DNN testing, numerous test cases are fed into the model to explore potential vulnerabilities, but they require expensive manual cost to check the label. Therefore, test case prioritization is proposed to solve the problem of labeling cost, e.g., activation-based and mutation-based prioritization methods. However, most of them suffer from limited scenarios (i.e. high confidence adversarial or false positive cases) and high time complexity. To address these challenges, we propose the concept of the activation graph from the perspective of the spatial relationship of neurons. We observe that the activation graph of cases that triggers the models' misbehavior significantly differs from that of normal cases. Motivated by it, we design a test case prioritization method based on the activation graph, ActGraph, by extracting the high-order node features of the activation graph for prioritization. ActGraph explains the difference between the test cases to solve the problem of scenario limitation. Without mutation operations, ActGraph is easy to implement, leading to lower time complexity. Extensive experiments on three datasets and four models demonstrate that ActGraph has the following key characteristics. (i) Effectiveness and generalizability: ActGraph shows competitive performance in all of the natural, adversarial and mixed scenarios, especially in RAUC-100 improvement (~1.40). (ii) Efficiency: ActGraph does not use complex mutation operations and runs in less time (~1/50) than the state-of-the-art method.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源