论文标题

用于RGB-D显着对象检测的多级跨模式交互网络

Multi-level Cross-modal Interaction Network for RGB-D Salient Object Detection

论文作者

Huang, Zhou, Chen, Huai-Xin, Zhou, Tao, Yang, Yun-Zhi, Liu, Bi-Yuan

论文摘要

具有富裕空间信息的深度线索已被证明有益于增强显着对象检测(SOD),而深度质量直接影响随后的SOD性能。但是,由于其采集设备的局限性,不可避免地要获得一些低质量的深度线索,这可以抑制SOD性能。此外,现有方法倾向于将RGB图像和深度线索组合在直接融合或简单的融合模块中,这使得它们无法有效利用两个源之间的复杂相关性。此外,很少有方法设计适当的模块以完全融合多级功能,从而导致跨层次特征相互作用不足。为了解决这些问题,我们为基于RGB-D的SOD提出了一个新型的多级跨模式相互作用网络(MCINET)。我们的MCI-NET包括两个关键组件:1)跨模式特征学习网络,该网络用于学习RGB图像和深度线索的高级特征,有效地可以利用两个源之间的相关性; 2)多层交互式集成网络,该网络集成了多级跨模式功能以提高SOD性能。在六个基准数据集上进行的广泛实验证明了我们的MCI-NET优于14种最先进的方法,并验证了MCI-NET中不同组件的有效性。更重要的是,我们的MCI-NET显着提高了SOD性能以及更高的FPS。

Depth cues with affluent spatial information have been proven beneficial in boosting salient object detection (SOD), while the depth quality directly affects the subsequent SOD performance. However, it is inevitable to obtain some low-quality depth cues due to limitations of its acquisition devices, which can inhibit the SOD performance. Besides, existing methods tend to combine RGB images and depth cues in a direct fusion or a simple fusion module, which makes they can not effectively exploit the complex correlations between the two sources. Moreover, few methods design an appropriate module to fully fuse multi-level features, resulting in cross-level feature interaction insufficient. To address these issues, we propose a novel Multi-level Cross-modal Interaction Network (MCINet) for RGB-D based SOD. Our MCI-Net includes two key components: 1) a cross-modal feature learning network, which is used to learn the high-level features for the RGB images and depth cues, effectively enabling the correlations between the two sources to be exploited; and 2) a multi-level interactive integration network, which integrates multi-level cross-modal features to boost the SOD performance. Extensive experiments on six benchmark datasets demonstrate the superiority of our MCI-Net over 14 state-of-the-art methods, and validate the effectiveness of different components in our MCI-Net. More important, our MCI-Net significantly improves the SOD performance as well as has a higher FPS.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源