Adaptive Residual Interpolation for Color Image DemosaickingYusuke Monno, Daisuke Kiku, Masayuki Tanaka, and Masatoshi Okutomi

Tokyo Institute of Technology 2015

Introduction・ Color image demosaicking plays an important role for

acquiring high-quality color images.

・ Bayer CFA is a de-facto standard CFA.

Bayer CFA・Many algorithms have been proposed

for the Bayer CFA every year.

APSA

AHD

DLM LPADFPD SSD

GBTF

IGD

NAT

LSLC

CSRI

ECC

IRIMLRI

ARI

35.5

36.0

36.5

37.0

37.5

38.0

38.5

39.0

39.5

2001 2003 2005 2007 2009 2011 2013 2015

CP

SNR

fo

r IM

AX

+Ko

dak

30

imag

es

Publication year

Our method

in this paper

Our Idea ofResidual Interpolation

・ Interpolation is performed in a residual domain.

RGB image

・Motivated by two observations;1. The residual image becomes smooth. 2. Interpolation is more accurate for smooth images.

-125

-100

-75

-50

-25

0

25

50

75

100

125

0 100 200 300 400 500

pix

el v

alu

e

pixel index

-125

-100

-75

-50

-25

0

25

50

75

100

125

0 100 200 300 400 500

pix

el v

alu

e

pixel index

G image Tentative R image

R image

Residual image

(R – Tentative R)

Color difference image

(R - G)

Sliced plot of

color difference image

Sliced plot of

residual image

Residual Interpolation Based Algorithms

・ RI: original residual interpolation [30] (ours in ICIP2013)

Direct minimization of residuals by guided filtering [33]

・MLRI: minimized-Laplacian residual interpolation [31] (ours in EI2014)

Minimization of “Laplacian” energy of residuals by modified guided filtering

𝑎, 𝑏 = min𝑎,𝑏

∆ 𝑎𝐺 + 𝑏 − 𝑅 2

Laplacian operator

・ IRI: iterative residual interpolation [32] (Ye et. al. in ICIP2014)

Iterative update of the tentative estimate

・ Proposed ARI: adaptive residual interpolation (This paper in ICIP2015)

1. Adaptive (local) selection of the best iteration number2. Adaptive combination of the RI and the MLRI

𝑎, 𝑏 = min𝑎,𝑏

𝑎𝐺 + 𝑏 − 𝑅 2RI:

Cost function of the guided filtering

MLRI:

Results

・ CPSNR for Kodak and IMAX 30 dataset

・ Visual Comparison

Ground truth LPA[24]DLM[22]AHD[18]SA[26]AP[25]

DFPD[19] LSLC[14]NAT[16]IGD[28]GBTF[20]SSD[15]

CS[27] ProposedIRI[32]MLRI[31]RI[30]ECC[29]

Source Code Available !!http://www.ok.ctrl.titech.ac.jp/res/DM/RI.html

Conclusion

Overall flow of G interpolation

・ The ARI adaptively selects the best iteration number andcombines the RI and the MLRI at each pixel.

・ Experimental results demonstrate a clear improvement over existing algorithms.

References

[30] D. Kiku, Y. Monno, M. Tanaka, and M. Okutomi, “Residual interpolation for color image demosaicking,”

in ICIP, 2013.

[31] D. Kiku, Y. Monno, M. Tanaka, and M. Okutomi, “Minimized-Laplacian residual interpolation for color

image demosaicking,” in EI, 2014.

[32] W. Ye and K. K. Ma, “Image demosaicing by using iterative residual interpolation,” in ICIP, 2014.

[33] K. He et. al., “Guided image filtering,” TPAMI, 2013.

High-frequency

component is

reduced.

Step (i): Iterative directional interpolation (horizontal)

Find the best iteration number

𝑘𝑜𝑝𝑡 = min𝑘

𝑐 𝑖,𝑗 ,𝑘ℎ

= min𝑘

𝑑 𝑖,𝑗 ,𝑘ℎ ∙ 𝛿𝑑 𝑖,𝑗 ,𝑘

ℎ

𝑑 𝑖,𝑗 ,𝑘𝑅ℎ

= 𝑅 𝑖,𝑗 ,𝑘ℎ − 𝑅 𝑖,𝑗 ,𝑘−1

ℎ

𝑑 𝑖,𝑗 ,𝑘𝐺ℎ

= 𝐺 𝑖,𝑗 ,𝑘ℎ − 𝐺 𝑖,𝑗 ,𝑘−1

ℎ

Cost of tentative estimation

Gradient

𝑑 𝑖,𝑗 ,𝑘ℎ = 𝑑 𝑖,𝑗 ,𝑘

𝑅ℎ+ 𝑑 𝑖,𝑗 ,𝑘

𝐺ℎ

Where

Tentative estimate

at k-th iteration

Previous

result

Weight Calculation

𝑤𝑖,𝑗ℎ,𝑟𝑖 = 1/𝑐 𝑖,𝑗

ℎ,𝑟𝑖

𝑤𝑖,𝑗ℎ,𝑚𝑙 = 1/𝑐 𝑖,𝑗

ℎ,𝑚𝑙

𝑤𝑖,𝑗𝑣,𝑟𝑖 = 1/𝑐 𝑖,𝑗

𝑣,𝑟𝑖

𝑤𝑖,𝑗𝑣,𝑚𝑙 = 1/𝑐 𝑖,𝑗

𝑣,𝑚𝑙

CPSNR performance for standard dataset

Source Code: http://www.ok.ctrl.titech.ac.jp/res/DM/RI.html