论文标题
保持突变测试套件一致并与长期突变体相关
Keeping Mutation Test Suites Consistent and Relevant with Long-Standing Mutants
论文作者
论文摘要
突变测试已被证明是测试仪工具包中最强大的错误透视工具之一。以前的许多工作隐含地认为这足以重新计算每个释放的突变套件。可悲的是，这使突变结果不一致。每个版本的突变分数无法直接比较，因此很难测量测试改进。此外，常规代码更改意味着突变套件的相关性自然会随着时间的流逝而自然降解。我们在4个非平凡的系统中发现了与143,500个突变体相关性的降解，发现超额降解了52％。我们引入了一种突变的脆性措施，并将其用于审核软件系统及其突变套件。我们还证明了如何通过任意测试套件的突变体相关性提高了10倍的长期突变套件如何一致的长期突变套件。我们的结果表明，研究界应避免突变套件的重新占据重复，而要专注于长期存在的突变体，从而提高突变测试的一致性和相关性。
Mutation testing has been demonstrated to be one of the most powerful fault-revealing tools in the tester's tool kit. Much previous work implicitly assumed it to be sufficient to re-compute mutant suites per release. Sadly, this makes mutation results inconsistent; mutant scores from each release cannot be directly compared, making it harder to measure test improvement. Furthermore, regular code change means that a mutant suite's relevance will naturally degrade over time. We measure this degradation in relevance for 143,500 mutants in 4 non-trivial systems finding that, on overage, 52% degrade. We introduce a mutant brittleness measure and use it to audit software systems and their mutation suites. We also demonstrate how consistent-by-construction long-standing mutant suites can be identified with a 10x improvement in mutant relevance over an arbitrary test suite. Our results indicate that the research community should avoid the re-computation of mutant suites and focus, instead, on long-standing mutants, thereby improving the consistency and relevance of mutation testing.
