Patch-based Image Deconvolution via Joint
Modeling of Sparse Priors
Chao Jia and Brian L. EvansThe University of Texas at Austin
12 Sep 2011
1
Non-blind Image Deconvolution Reconstruct natural image from blurred version
Camera shake; astronomy; biomedical image reconstruction
2D convolution matrix H and Gaussian additive noise vector n
Maximum a-posteriori (MAP) estimation for vector X
Prior model for p(X) for natural images? [Elad 2007] Optimization method?
2
Analysis-based modeling [Krishnan 2009]
Prior based on hyper-Laplacian distribution of the spatial derivative of natural images
Linear filtering to compute spatial derivative Fit (0.5-0.8) and (normalization factor) to empirical data
3
Patch-based modeling Sparse coding of patches
Spatial receptive fields of visual cortex [Olshausen 1997] For 10 10 patches Learn an overcomplete dictionary from natural images.
Application in image restoration Denoising, superresolution
[Yang 2010] Localized algorithm: patches can
overlap Use this model in deconvolution?
[Lee 2007]
4
Prior model in natural images From local to global
Slow convergence (EM Algorithm)
Patches should not overlap (Why?) boundary artifacts
5
Joint modeling Take advantage of patch-based sparse representation
while resolving the problems in? Combine analysis-based prior and synthesis-based prior
Sparse spatial gradient
Patch-based sparse coding
Accelerate convergence
Keep consistency on the boundary of adjacent patches
Keep details and textures
6
Joint modeling Discard the generative model
Prior probability
After training, we fix the parameters for all images
sparsity of representation
coefficients
compatibility term
sparsity of gradients
7
MAP estimation using the joint model Problem:
Iteratively updating w and X until convergence w sub-problem small-scale L1 regularized square loss minimization X sub-problem Half-quadratic splitting [Krishnan 2009]
likelihood prior
8
Experimental results Initialization: Wiener estimates / blurred images Dictionary: learned from Berkeley Segmentation database
Patch size 12 12 Prior parameters: Runtime: (Matlab) 16s with Intel Core2 Duo CPU @2.26GHz Experiment settings:
9
Experimental results
test 1
test 2
test 3
test 4
2 3 4 5 6 7 8
ISNR comparison
proposed
[Portilla 2009]
[Krishnan 2009]
test 1
test 2
test 3
test 4
0.8 0.82 0.84 0.86 0.88 0.9 SSIM comparison
proposed
[Portilla 2009]
[Krishnan 2009]
PASCAL Visual Object Classes
Challenge (VOC) 2007 database
10
Experimental results
11
Experimental results
keeps more brick
textures
[Krishnan 2009]
Original image
Blurred image Proposed12
[Portilla 2009]
Experimental results
13
Textures zoomed in
[Krishnan 2009]
Original image
Proposed[Portilla 2009]
Conclusions Global model for MAP estimation
Able to solve general non-blind image deconvolution Joint model of image pixels and representation
coefficients Sparsity of spatial derivative (analysis-based) Sparsity of representation of patches in overcomplete
dictionary (synthesis-based) Iterative algorithm
converges in a few iterations Matlab code for the proposed method is available at http://users.ece.utexas.edu/~bevans/papers/2011/sparsity/
14
References [Elad 2007] M. Elad, P. Milanfar and R. Rubinstein, “Analysis versus
synthesis in signal priors”, Inverse Problems, vol. 23, 2007. [Krishnan 2009] D. Krishnan and R. Fergus, “Fast image deconvolution using
hyper-Laplacian priors,” Advances in Neural Information Processing Systems, vol. 22, pp. 1-9, 2009.
[Olshausen 1997] B.A. Olshausen and D.J. Field, “Sparse coding with an overcomplete basis set: a strategy employed by V1,” Vision Research, vol. 37, no. 23, pp. 3311-3325, 1997.
[Portilla 2009] J. Portilla, “Image restoration through L0 analysis-based sparse optimization in tight frames,” in Proc. IEEE Int. Conf. on Image Processing, 2009, pp. 3909-3912.
[Yang 2010] J. yang, J. Wright, T.S. Huang and Y. Ma, “Image super-resolution via sparse representation,” IEEE Trans. on Image Processing, vol. 19, no. 11, pp. 2861-2873, 2010.
15
Thank you!
16
w sub-problem
patches do not overlap
small-scale l1 regularized square loss minimization
17
X sub-problem
Conjugate gradientiteratively reweighted least squares
Half-quadratic splitting [Krishnan 2009]
auxiliary variable
No need to solve the equationcomponent-wise
quartic function18
MAP estimation using the joint modelblurred image;
noise level; blurring kernel; initialization of recovered image
Update the coefficient of patches
(w sub-problem)
Set α=α0 α>αmax ?
Update auxiliary variable Y
(quartic equation)
Update image X (FFT)
α=kα
X converges?
finish
X sub-problem
19
Image Quality Assessment
20
Full reference metric ISNR -- increment in PSNR (peak signal-to-noise ratio)
SSIM -- structural similarity [Wang 2004]
Prior model of natural images Analysis-based prior
Fast convergence Over smooth the images
Synthesis-based prior (patch-based sparse representation) Dictionary well adapted to nature images Captures textures well Slow convergence Boundary artifacts
21
Computational complexity
22
Computational complexity For each iteration: N is the total number of pixels in the image
Average runtime comparison[Krishnan
2009][Portilla 2009]
Proposed
2s 15s 16s
Top Related