论文标题
分发嵌入网络从一系列分类任务中进行概括
Distribution Embedding Networks for Generalization from a Diverse Set of Classification Tasks
论文作者
论文摘要
我们建议使用小数据分类的分布嵌入网络（DEN）。本着元学习的同样精神，丹从各种各样的培训任务中学习，目标是概括地看不见目标任务。与需要培训和目标任务输入的现有方法不同，具有相同的维度具有相同的分布，DEN允许培训和目标任务在异质输入空间中生活。这对于稀缺的标记数据的表格数据任务特别有用。 DEN使用三个块体系结构：协变量转换块，然后是分布嵌入块，然后进行分类块。我们提供理论见解，以表明这种体系结构允许在对各种任务进行预训练后修复嵌入和分类块；对于每个新任务，只需要对具有相对较少参数的协变量转换块进行微调。为了促进培训，我们还提出了一种合成二进制分类任务的方法，并证明DEN在数值研究中的许多合成和真实任务中都超越了现有方法。
We propose Distribution Embedding Networks (DEN) for classification with small data. In the same spirit of meta-learning, DEN learns from a diverse set of training tasks with the goal to generalize to unseen target tasks. Unlike existing approaches which require the inputs of training and target tasks to have the same dimension with possibly similar distributions, DEN allows training and target tasks to live in heterogeneous input spaces. This is especially useful for tabular-data tasks where labeled data from related tasks are scarce. DEN uses a three-block architecture: a covariate transformation block followed by a distribution embedding block and then a classification block. We provide theoretical insights to show that this architecture allows the embedding and classification blocks to be fixed after pre-training on a diverse set of tasks; only the covariate transformation block with relatively few parameters needs to be fine-tuned for each new task. To facilitate training, we also propose an approach to synthesize binary classification tasks, and demonstrate that DEN outperforms existing methods in a number of synthetic and real tasks in numerical studies.
