论文标题
选择语言探针的信息理论观点
An information theoretic view on selecting linguistic probes
论文作者
论文摘要
在评估神经表征中编码的语言知识方面，人们兴趣越来越大。一种流行的方法是附加诊断分类器（或“探测”），以从内部表示形式执行监督分类。但是，如何在辩论中选择一个好的探测。 Hewitt和Liang（2019）表明，诊断分类本身的高性能是不够的，因为它可以归因于“富有知识的表示”或“探针学习任务”，Pimentel等人。 （2020）受到挑战。我们表明，这种二分法是从理论上讲是有效的信息。此外，我们发现，构造和选择两篇论文提出的良好探针的方法 *控制任务 *（Hewitt和Liang，2019年）和 *控制功能 *（Pimentel等，2020）是等效的 - 其方法的错误是相同的（Modulo Irlevelvant项）。从经验上讲，这两个选择标准导致结果彼此高度同意。
There is increasing interest in assessing the linguistic knowledge encoded in neural representations. A popular approach is to attach a diagnostic classifier -- or "probe" -- to perform supervised classification from internal representations. However, how to select a good probe is in debate. Hewitt and Liang (2019) showed that a high performance on diagnostic classification itself is insufficient, because it can be attributed to either "the representation being rich in knowledge", or "the probe learning the task", which Pimentel et al. (2020) challenged. We show this dichotomy is valid information-theoretically. In addition, we find that the methods to construct and select good probes proposed by the two papers, *control task* (Hewitt and Liang, 2019) and *control function* (Pimentel et al., 2020), are equivalent -- the errors of their approaches are identical (modulo irrelevant terms). Empirically, these two selection criteria lead to results that highly agree with each other.
