论文标题
关于使用差异隐私和经典正则化技术的实用性和保护优化的效用和保护
On the utility and protection of optimization with differential privacy and classic regularization techniques
论文作者
论文摘要
如今，深度学习模型的所有者和开发人员必须考虑其培训数据的严格隐私保护规则，通常是人群来源且保留敏感信息。如今，深入学习模型执行隐私保证的最广泛采用的方法依赖于实施差异隐私的优化技术。根据文献，这种方法已被证明是针对多种模型的隐私攻击的成功辩护，但其缺点是对模型的性能的实质性下降。在这项工作中，我们比较了差异私有的随机梯度下降（DP-SGD）算法与使用正则化技术的标准优化实践的有效性。我们分析了所得模型的实用程序，培训性能以及成员推理和模型反转攻击对学识渊博的模型的有效性。最后，我们讨论了差异隐私的缺陷和限制，并在经验上证明了辍学和L2型规范的卓越保护特性。
Nowadays, owners and developers of deep learning models must consider stringent privacy-preservation rules of their training data, usually crowd-sourced and retaining sensitive information. The most widely adopted method to enforce privacy guarantees of a deep learning model nowadays relies on optimization techniques enforcing differential privacy. According to the literature, this approach has proven to be a successful defence against several models' privacy attacks, but its downside is a substantial degradation of the models' performance. In this work, we compare the effectiveness of the differentially-private stochastic gradient descent (DP-SGD) algorithm against standard optimization practices with regularization techniques. We analyze the resulting models' utility, training performance, and the effectiveness of membership inference and model inversion attacks against the learned models. Finally, we discuss differential privacy's flaws and limits and empirically demonstrate the often superior privacy-preserving properties of dropout and l2-regularization.
