论文标题
很酷，上下文远景及其在问答和其他自然语言处理任务的应用
COOL, a Context Outlooker, and its Application to Question Answering and other Natural Language Processing Tasks
论文作者
论文摘要
远见者改善了视觉变形金刚的性能，后者通过添加前景注意力（一种当地关注的形式）来实现自我发挥的机制。 在自然语言处理中，就像计算机视觉和其他域一样，基于变压器的模型构成了大多数处理任务的最新模型。在这个领域，许多作者也争论并证明了当地环境的重要性。 我们为自然语言处理提供了一种前景注意机制。与现有方法使用的动态卷积相比，在基于变压器的模型的自我发项层层面上添加了酷炫的自我发项层层。 对基于变压器的不同模型对Cool实施的比较经验绩效评估证实了仅使用原始模型来改进基线的机会，仅对各种自然语言处理任务，包括问题答案。所提出的方法通过在某些任务上采用现有的最新方法来实现竞争性能。
Vision outlooker improves the performance of vision transformers, which implements a self-attention mechanism by adding an outlook attention, a form of local attention. In natural language processing, as has been the case in computer vision and other domains, transformer-based models constitute the state-of-the-art for most processing tasks. In this domain, too, many authors have argued and demonstrated the importance of local context. We present an outlook attention mechanism, COOL, for natural language processing. COOL, added on top of the self-attention layers of a transformer-based model, encodes local syntactic context considering word proximity and more pair-wise constraints than dynamic convolution used by existing approaches. A comparative empirical performance evaluation of an implementation of COOL with different transformer-based models confirms the opportunity for improvement over a baseline using the original models alone for various natural language processing tasks, including question answering. The proposed approach achieves competitive performance with existing state-of-the-art methods on some tasks.
