论文标题
剪辑 - 拆卸：深视网网络中神经元表示的自动描述
CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networks
论文作者
论文摘要
在本文中，我们提出了一种剪贴板，这是一种新技术，可以自动描述视觉网络中单个隐藏神经元的功能。剪辑 - 拆卸利用多模式视觉/语言模型的最新进展，可以将内部神经元标记开放式概念，而无需任何标记的数据或人类示例。我们表明，剪贴板截断提供了比现有方法的现有方法更准确的描述，在该神经元中可用地面真相以及隐藏层神经元的定性描述。此外，我们的方法非常灵活：它是模型的不可知论，可以轻松处理新概念，并且可以扩展以利用将来更好的多模型模型。最终，剪辑截止值在计算上是有效的，并且可以在短短4分钟内从五层Resnet-50上标记所有神经元，这比现有方法快10倍以上。我们的代码可从https://github.com/trustworthy-ml-lab/clip-dissect获得。最后，众包用户研究结果可在附录B上获得，以进一步支持我们方法的有效性。
In this paper, we propose CLIP-Dissect, a new technique to automatically describe the function of individual hidden neurons inside vision networks. CLIP-Dissect leverages recent advances in multimodal vision/language models to label internal neurons with open-ended concepts without the need for any labeled data or human examples. We show that CLIP-Dissect provides more accurate descriptions than existing methods for last layer neurons where the ground-truth is available as well as qualitatively good descriptions for hidden layer neurons. In addition, our method is very flexible: it is model agnostic, can easily handle new concepts and can be extended to take advantage of better multimodal models in the future. Finally CLIP-Dissect is computationally efficient and can label all neurons from five layers of ResNet-50 in just 4 minutes, which is more than 10 times faster than existing methods. Our code is available at https://github.com/Trustworthy-ML-Lab/CLIP-dissect. Finally, crowdsourced user study results are available at Appendix B to further support the effectiveness of our method.
