论文标题
探索任务可传递性在大型多任务学习中的作用
Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning
论文作者
论文摘要
最近的工作发现，具有大量不同任务的多任务培训可以统一地改善看不见的目标任务的下游表现。相反，有关任务可传递性的文献已经确定，中间任务的选择会严重影响下游任务绩效。在这项工作中，我们旨在解散多任务表示学习中任务的规模和相关性的影响。我们发现，平均而言，就任务数量而言，增加了多任务学习的规模，实际上比较小的多任务设置会产生更好的学习表示形式。但是，如果提前知道目标任务，那么对一组相关任务的训练就可以以降低的计算成本进行大规模的多任务培训竞争。
Recent work has found that multi-task training with a large number of diverse tasks can uniformly improve downstream performance on unseen target tasks. In contrast, literature on task transferability has established that the choice of intermediate tasks can heavily affect downstream task performance. In this work, we aim to disentangle the effect of scale and relatedness of tasks in multi-task representation learning. We find that, on average, increasing the scale of multi-task learning, in terms of the number of tasks, indeed results in better learned representations than smaller multi-task setups. However, if the target tasks are known ahead of time, then training on a smaller set of related tasks is competitive to the large-scale multi-task training at a reduced computational cost.
