论文标题

使用深度学习的快照相机图像的高光谱示例性

Hyperspectral Demosaicing of Snapshot Camera Images Using Deep Learning

论文作者

Wisotzky, Eric L., Daudkhane, Charul, Hilsmann, Anna, Eisert, Peter

论文摘要

在过去的几十年中,光谱成像技术已经迅速发展。最近开发用于高光谱成像的单相机一击技术允许同时捕获多个光谱带(3x3、4x4或5x5 Mosaic),从而开放了广泛的应用。例子包括术中成像,农业现场检查和食品质量评估。为了捕获跨频谱范围内的图像,即为了实现高光谱分辨率,传感器设计牺牲了空间分辨率。随着镶嵌大小的增加,这种影响变得越来越有害。此外,Demosaicing具有挑战性。在插值过程中,如果不包含边缘,形状和对象信息,则可能出现在获得的图像中。最近的方法使用神经网络进行表述,从而可以从图像数据中直接提取信息。但是,获得这些方法的培训数据也带来了挑战。这项工作提出了一个基于平行的神经网络的表演过程,该过程在一个新的地面真相数据集中训练,该数据集在受控环境中捕获的高光谱快照摄像头具有4x4马赛克模式。该数据集是真实捕获的场景的组合,以及适用于4x4马赛克模式的公开数据的图像。为了获得现实世界的基础真实数据,我们执行了具有1像素换档的多个摄像头捕获,以构成整个数据立方体。实验表明,所提出的网络的表现优于最先进的网络。

Spectral imaging technologies have rapidly evolved during the past decades. The recent development of single-camera-one-shot techniques for hyperspectral imaging allows multiple spectral bands to be captured simultaneously (3x3, 4x4 or 5x5 mosaic), opening up a wide range of applications. Examples include intraoperative imaging, agricultural field inspection and food quality assessment. To capture images across a wide spectrum range, i.e. to achieve high spectral resolution, the sensor design sacrifices spatial resolution. With increasing mosaic size, this effect becomes increasingly detrimental. Furthermore, demosaicing is challenging. Without incorporating edge, shape, and object information during interpolation, chromatic artifacts are likely to appear in the obtained images. Recent approaches use neural networks for demosaicing, enabling direct information extraction from image data. However, obtaining training data for these approaches poses a challenge as well. This work proposes a parallel neural network based demosaicing procedure trained on a new ground truth dataset captured in a controlled environment by a hyperspectral snapshot camera with a 4x4 mosaic pattern. The dataset is a combination of real captured scenes with images from publicly available data adapted to the 4x4 mosaic pattern. To obtain real world ground-truth data, we performed multiple camera captures with 1-pixel shifts in order to compose the entire data cube. Experiments show that the proposed network outperforms state-of-art networks.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源