论文标题
探测标记的依赖树
Probing for Labeled Dependency Trees
论文作者
论文摘要
探测已成为分析自然语言处理（NLP）表示表示的重要工具。对于诸如依赖性解析之类的图形NLP任务，线性探针当前仅限于提取未捕获完整任务的无方向性或未标记的解析树。这项工作介绍了Depprobe，这是一种线性探针，可以从嵌入中提取标有和定向的依赖性解析树，而使用较少的参数和计算，而不是先前的方法。利用其完整的任务覆盖范围和轻量级参数化，我们研究了其预测能力，以选择培训Biaffine注意的最佳转移语言。在13种语言中，我们提出的方法在94％的时间内确定了最佳的源树库，表现优于竞争基准和先前的工作。最后，我们在上下文嵌入中分析了特定于任务的子空间的信息，以及在哪些解析器的非线性参数化提供的有益的情况下。
Probing has become an important tool for analyzing representations in Natural Language Processing (NLP). For graphical NLP tasks such as dependency parsing, linear probes are currently limited to extracting undirected or unlabeled parse trees which do not capture the full task. This work introduces DepProbe, a linear probe which can extract labeled and directed dependency parse trees from embeddings while using fewer parameters and compute than prior methods. Leveraging its full task coverage and lightweight parametrization, we investigate its predictive power for selecting the best transfer language for training a full biaffine attention parser. Across 13 languages, our proposed method identifies the best source treebank 94% of the time, outperforming competitive baselines and prior work. Finally, we analyze the informativeness of task-specific subspaces in contextual embeddings as well as which benefits a full parser's non-linear parametrization provides.
