A Real-Time RGB-NIR Imaging System Using a Single Image Sensor
1
A Real-Time RGB-NIR Imaging System Using a Single Image Sensor Kazunori Yoshizaki 1 Munenori Fukunishi 1 Yasuhiro Komiya 1 Yusuke Monno 2 Masayuki Tanaka 2 Masatoshi Okutomi 2 Steven Lansel 3 1 Olympus R&D Group 2 Tokyo Institute of Technology 3 Olympus Communication Technology of America Abstract In recent years, many applications using a pair of RGB and near-infrared (NIR) images have been proposed in com- puter vision and image processing communities. However, the acquisition of the pair of RGB and NIR images is still a challenging task. In this demonstration, we present a prac- tical real-time RGB-NIR imaging system that can capture RGB and NIR images simultaneously using a single image sensor. Figure 1 shows our developed RGB-NIR imaging system. To acquire high-quality RGB and NIR images, we proposed a novel RGB-NIR filter array [1] that features high sampling density of the G band like the Bayer color filter array. Figure 2 shows our demonstration system that displays the captured RGB and NIR images at 30 fps. In the demonstration, we present several applications including indocyanine green (ICG) measurements. Figure 3 shows ex- ample images captured by our developed RGB-NIR imaging system. The bottom example shows the ICG measurements that emit fluorescence in NIR wavelengths. Our developed system can effectively capture both the standard RGB image and the NIR image including the emission of fluorescence, which is very useful for medical applications. System overview R G B G G N G N B G R G G N G N Proposed pattern Microscope image Spectral sensitivity 0 5 10 15 20 25 30 35 400 600 800 1000 Sensitivity Wavelength (nm) Figure 1. Overview of our developed RGB-NIR imaging system. Our RGB-NIR camera Laptop Main display RGB NIR Captured scene Sub display Our real-time demo system for RGB-NIR image acquisition Figure 2. Overview of our real-time demonstration system. RGB NIR Figure 3. Examples of the captured RGB-NIR image. References [1] H. Teranaka, Y. Monno, M. Tanaka, and M. Okutomi. Single-sensor RGB and NIR image acquisition: Toward optimal performance by taking account of CFA pattern, demosaicking, and color correction. Proc. of IS&T Electronic Imaging, 2016. Acknowledgment. This work was partly supported by the MIC/SCOPE #141203024. 1
Transcript of A Real-Time RGB-NIR Imaging System Using a Single Image Sensor
A Real-Time RGB-NIR Imaging System Using a Single Image Sensor
Kazunori Yoshizaki1 Munenori Fukunishi1 Yasuhiro Komiya1 Yusuke Monno2
Masayuki Tanaka2 Masatoshi Okutomi2 Steven Lansel3
1Olympus R&D Group2Tokyo Institute of Technology
3Olympus Communication Technology of America
Abstract
In recent years, many applications using a pair of RGBand near-infrared (NIR) images have been proposed in com-puter vision and image processing communities. However,the acquisition of the pair of RGB and NIR images is still achallenging task. In this demonstration, we present a prac-tical real-time RGB-NIR imaging system that can captureRGB and NIR images simultaneously using a single imagesensor. Figure1 shows our developed RGB-NIR imagingsystem. To acquire high-quality RGB and NIR images, weproposed a novel RGB-NIR filter array [1] that featureshigh sampling density of the G band like the Bayer colorfilter array. Figure2 shows our demonstration system thatdisplays the captured RGB and NIR images at 30 fps. In thedemonstration, we present several applications includingindocyanine green (ICG) measurements. Figure3 shows ex-ample images captured by our developed RGB-NIR imagingsystem. The bottom example shows the ICG measurementsthat emit fluorescence in NIR wavelengths. Our developedsystem can effectively capture both the standard RGB imageand the NIR image including the emission of fluorescence,which is very useful for medical applications.
System overview
R G B G
G N G N
B G R G
G N G NProposed patternMicroscope image
Spectral sensitivity
0
5
10
15
20
25
30
35
400 600 800 1000
Sen
siti
vity
Wavelength (nm)
�
�
�
���
Figure 1.Overview of our developed RGB-NIR imaging system.
Our RGB-NIRcamera
Laptop
Maindisplay
RGB NIR
Captured scene
Subdisplay Our real-time demo system for
RGB-NIR image acquisition
Figure 2.Overview of our real-time demonstration system.
RGB NIRFigure 3.Examples of the captured RGB-NIR image.
References[1] H. Teranaka, Y. Monno, M. Tanaka, and M. Okutomi. Single-sensor
RGB and NIR image acquisition: Toward optimal performance bytaking account of CFA pattern, demosaicking, and color correction.Proc. of IS&T Electronic Imaging, 2016.
Acknowledgment. This work was partly supported by theMIC/SCOPE #141203024.
1
![NIR SIG Administration, by Toshio Tachibana [APNIC 38 / NIR SIG]](https://static.fdocument.pub/doc/165x107/58e949311a28ab262c8b4ecb/nir-sig-administration-by-toshio-tachibana-apnic-38-nir-sig.jpg)
