论文标题
破坏失败率的假设测试
Breaking hypothesis testing for failure rates
论文作者
论文摘要
我们描述了点过程和故障率的效用以及对失败率建模的最常见的点过程,即泊松点过程。接下来,我们描述了比较单方面测试的两个泊松点过程速率的最强大测试(此后称为“费率测试”)。反对使用此测试的一个普遍论点是,现实世界数据很少遵循泊松点过程。因此,我们调查当违反此类测试的分布假设并仍应用测试时会发生什么。我们找到一个非病理学示例(使用二项式复合的复合泊松分布上的速率测试),其中违反了速率测试的分布假设使其表现更好(较低的错误率)。我们还发现,如果我们用其他任意分布替换了零假设下的测试统计量的分布,则测试的性能(用假负率对假阳性利率权衡的描述)仍然完全相同。接下来,我们将速率测试的性能与定制为负二项点过程的WALD测试的版本进行比较,并发现其性能非常相似,同时更加通用和通用。最后,我们讨论了Microsoft Azure的应用程序。执行的所有实验的代码都是开源的,并在简介中链接。
We describe the utility of point processes and failure rates and the most common point process for modeling failure rates, the Poisson point process. Next, we describe the uniformly most powerful test for comparing the rates of two Poisson point processes for a one-sided test (henceforth referred to as the "rate test"). A common argument against using this test is that real world data rarely follows the Poisson point process. We thus investigate what happens when the distributional assumptions of tests like these are violated and the test still applied. We find a non-pathological example (using the rate test on a Compound Poisson distribution with Binomial compounding) where violating the distributional assumptions of the rate test make it perform better (lower error rates). We also find that if we replace the distribution of the test statistic under the null hypothesis with any other arbitrary distribution, the performance of the test (described in terms of the false negative rate to false positive rate trade-off) remains exactly the same. Next, we compare the performance of the rate test to a version of the Wald test customized to the Negative Binomial point process and find it to perform very similarly while being much more general and versatile. Finally, we discuss the applications to Microsoft Azure. The code for all experiments performed is open source and linked in the introduction.