论文标题
依赖点依赖性估计的神经方法
Neural Methods for Point-wise Dependency Estimation
论文作者
论文摘要
自成立以来，相互信息的神经估计（MI）已经证明了建模高维随机变量之间的预期依赖性的经验成功。但是，MI是一个汇总统计量，不能用于测量不同事件之间的点依赖性。在这项工作中，我们专注于估计点依赖性（PD），而不是估计预期的依赖性，该依赖性（PD）可以定量地衡量两个结果的共同发生的可能性。我们表明，当我们优化MI神经变异界限时，我们可以自然获得PD。但是，由于其实践中的较大差异，优化这些界限是具有挑战性的。为了解决此问题，我们开发了两种方法（不使用优化MI变化界限）：概率分类器和密度比率拟合。我们在1）MI估计中证明了方法的有效性，2）自我监督的表示学习和3）跨模式检索任务。
Since its inception, the neural estimation of mutual information (MI) has demonstrated the empirical success of modeling expected dependency between high-dimensional random variables. However, MI is an aggregate statistic and cannot be used to measure point-wise dependency between different events. In this work, instead of estimating the expected dependency, we focus on estimating point-wise dependency (PD), which quantitatively measures how likely two outcomes co-occur. We show that we can naturally obtain PD when we are optimizing MI neural variational bounds. However, optimizing these bounds is challenging due to its large variance in practice. To address this issue, we develop two methods (free of optimizing MI variational bounds): Probabilistic Classifier and Density-Ratio Fitting. We demonstrate the effectiveness of our approaches in 1) MI estimation, 2) self-supervised representation learning, and 3) cross-modal retrieval task.
