论文标题
朝着驾驶导向度量的巷道检测模型
Towards Driving-Oriented Metric for Lane Detection Models
论文作者
论文摘要
在2017年Tusimple Lane检测挑战之后，其基于准确性和F1评分的数据集和评估已成为衡量车道检测方法性能的事实上的标准。尽管他们在改善车道检测方法的性能方面发挥了重要作用，但该评估方法在下游任务中的有效性尚未得到充分研究。在这项研究中，我们设计了2个新的面向驾驶的指标，用于泳道检测：端到端横向偏差度量（E2E-LD）是根据自主驾驶的要求直接制定的，这是自动驾驶的要求，这是泳道检测的核心下游任务；人均模拟横向偏差度量（PSLD）是E2E-LD的轻量级替代指标。为了评估指标的有效性，我们在Tusimple数据集上使用4种主要类型的车道检测方法进行了大规模的经验研究，以及我们新建的数据集Comma2K19-LD。我们的结果表明，常规指标与E2E-LDS具有强烈的负相关性（$ \ leq $ -0.55），这意味着，最近针对传统指标的一些最近的改进可能并没有导致自动驾驶的有意义改进，但实际上可能通过使传统的指标过度拟合而变得更糟。由于自动驾驶是一种安全/安全关键系统，因此鲁棒性的低估阻碍了实用车道检测模型的合理发展。我们希望我们的研究能够帮助社区获得更多下游的任务感知评估，以进行泳道检测。
After the 2017 TuSimple Lane Detection Challenge, its dataset and evaluation based on accuracy and F1 score have become the de facto standard to measure the performance of lane detection methods. While they have played a major role in improving the performance of lane detection methods, the validity of this evaluation method in downstream tasks has not been adequately researched. In this study, we design 2 new driving-oriented metrics for lane detection: End-to-End Lateral Deviation metric (E2E-LD) is directly formulated based on the requirements of autonomous driving, a core downstream task of lane detection; Per-frame Simulated Lateral Deviation metric (PSLD) is a lightweight surrogate metric of E2E-LD. To evaluate the validity of the metrics, we conduct a large-scale empirical study with 4 major types of lane detection approaches on the TuSimple dataset and our newly constructed dataset Comma2k19-LD. Our results show that the conventional metrics have strongly negative correlations ($\leq$-0.55) with E2E-LD, meaning that some recent improvements purely targeting the conventional metrics may not have led to meaningful improvements in autonomous driving, but rather may actually have made it worse by overfitting to the conventional metrics. As autonomous driving is a security/safety-critical system, the underestimation of robustness hinders the sound development of practical lane detection models. We hope that our study will help the community achieve more downstream task-aware evaluations for lane detection.
