论文标题
检测令牌分类数据中的标签错误
Detecting Label Errors in Token Classification Data
论文作者
论文摘要
标签错误的示例是现实世界数据中的一个常见问题，尤其是对于诸如代币分类之类的任务，必须以细粒度选择许多标签。在这里，我们考虑找到包含标签分类数据集中标签错误的句子的任务。我们研究了11种不同的直接方法，这些方法是根据（任何）令牌分类模型（通过任何过程训练的）基于预测的类概率输出来评分令牌/句子的不同直接方法。在基于CONLL-2003实体识别数据中的现实世界标签错误的Precision-Recall评估中，我们确定了一种简单有效的方法，该方法始终检测到使用不同令牌分类模型应用的那些包含标签错误的句子。
Mislabeled examples are a common issue in real-world data, particularly for tasks like token classification where many labels must be chosen on a fine-grained basis. Here we consider the task of finding sentences that contain label errors in token classification datasets. We study 11 different straightforward methods that score tokens/sentences based on the predicted class probabilities output by a (any) token classification model (trained via any procedure). In precision-recall evaluations based on real-world label errors in entity recognition data from CoNLL-2003, we identify a simple and effective method that consistently detects those sentences containing label errors when applied with different token classification models.
