论文标题
Debinet：具有非线性过度参数化神经网络的线性模型
DebiNet: Debiasing Linear Models with Nonlinear Overparameterized Neural Networks
论文作者
论文摘要
近年来，过度参数化的神经网络对各种任务以及理论的许多进步，例如通用近似和可证明的融合到全球最低限度。在本文中，我们将过度参数化的神经网络纳入半参数模型，以弥合推理和预测之间的差距，尤其是在高维线性问题中。通过这样做，我们可以利用一类广泛的网络来近似滋扰功能，并始终如一地估计感兴趣的参数。因此，我们可能会提供两个世界中最好的：来自神经网络的通用近似能力以及经典普通线性模型的解释性，从而导致有效的推理和准确的预测。我们展示了使这一点成为可能并通过数值实验证明的理论基础。此外，我们提出了一个框架，即Debinet，其中我们将任意特征选择方法插入半参数神经网络。 Debinet可以根据序列推理和概括误差来表达正则化估计器（例如Lasso）并表现良好。
Recent years have witnessed strong empirical performance of over-parameterized neural networks on various tasks and many advances in the theory, e.g. the universal approximation and provable convergence to global minimum. In this paper, we incorporate over-parameterized neural networks into semi-parametric models to bridge the gap between inference and prediction, especially in the high dimensional linear problem. By doing so, we can exploit a wide class of networks to approximate the nuisance functions and to estimate the parameters of interest consistently. Therefore, we may offer the best of two worlds: the universal approximation ability from neural networks and the interpretability from classic ordinary linear model, leading to both valid inference and accurate prediction. We show the theoretical foundations that make this possible and demonstrate with numerical experiments. Furthermore, we propose a framework, DebiNet, in which we plug-in arbitrary feature selection methods to our semi-parametric neural network. DebiNet can debias the regularized estimators (e.g. Lasso) and perform well, in terms of the post-selection inference and the generalization error.
