3 Image Enhancement in the Spatial Domain - …140.115.154.40/vclab/teacher/2014DIP/3 Image...
Transcript of 3 Image Enhancement in the Spatial Domain - …140.115.154.40/vclab/teacher/2014DIP/3 Image...
3 Image Enhancement in
the Spatial Domain
Chih-Wei Tang (唐唐唐唐 之之之之 瑋瑋瑋瑋)
Department of Communication Engineering
National Central University
JhongLi, Taiwan
C.E., NCU, Taiwan 2
Outline
� Gray level transformations
� Histogram processing
� Enhancement using arithmetic operations
� Spatial filtering
Background
)],([),( yxfTyxg =
� Spatial domain methods
� operate directly on the pixels
� T operates over some neighborhood of (x,y)
� neighborhood shape: square & rectangular arrays are the most predominant due to the ease of implementation
� mask processing/filtering
� masks/filters/kernels/templates/windows
� e.g., image sharpening
� the center of the window
moves from pixel to pixel
� the simplest form:
gray-level transformation
� s=T(r)
� T can operate on a set of input images
� e.g., sum of input images for
noise reduction3C.E., NCU, Taiwan
C.E., NCU, Taiwan 4
Point Processing
� Examples
� contrast stretching: produce an image of higher contrast than the original
� darken the levels below m & brighten the levels above m
� the limiting case of contrast stretching: thresholding function
Contrast Stretching Thresholding Function
Gray Level Transformations –
Image Negatives
� Reverse the intensity levels
� Enhance white/gray detail embedded in dark regions of an image, especially when the black areas are dominant in size
rLs −−= 1
Original image Negative image
6C.E., NCU, Taiwan
Gray Level Transformations –
Log Transformations� s=clog(1+r), c: constant, r≥0
� Map a narrow range of low gray-level values in the input image into a wider range of output levels
� Compress the higher-level values
� An example: Fourier spectrum of wide range ( )
� a significant degree of detail would lost in the display
Fourier spectrum( )
After log transformation(c=1� s:0~6.2)
610~0
6105.1~0 ×
Display in a 8-bit display
7C.E., NCU, Taiwan 7
Gray Level Transformations –
Power-Law Transformations
� , c & γ: positive constants
� fractional γ : map a narrow range of dark input values into a wider range of output values
� a family of possible transformation curves: varying γ
� A variety of devices used for image capture, printing & display respond according to a power law
� device-dependent value of gamma
γ= crs
8C.E., NCU, Taiwan
Example #1 of Power-Law
Transformations – Gamma Correction� Correct the power-law response phenomena
� an example: The intensity-to-voltage response of CRT devices is a power function (exponents : 1.8~2.5)
� darker than intended
5.2=γ
4.05.2/1 ==γ
9C.E., NCU, Taiwan
Example #2 of Power-Law
Transformations – Contrast Manipulation
Magnetic resonance (MR) image of a fractured human spine
Original image c=1, γ=0.6
�c=1, γ=0.4 c=1, γ=0.3
10
Example #3 of Power-Law
Transformations – Contrast Manipulation
A Washed-out appearance: a compression of gray levels is desirable
Original image c=1, γ=3.0
�c=1, γ=4.0 c=1, γ=5.0
11
Piecewise-Linear Transformation
Functions
� Advantage: arbitrarily complex
� Disadvantage: the specification requires considerably more user inputs
� Example: contrast-stretching transformation
� increase the dynamic range of the gray levels of the input image
r1=r2=m (Thresholding function)
(r1,s1)=(rmin,0)(r2,s2)= (rmax,L-1) 12
Gray-Level Slicing
� Highlight a specific range of gray levels is often desired
� Various ways to accomplish this
� highlight some range and reduce all others to a constant level
� highlight some range but preserve all other levels
Original image Transformed image by (a)
13C.E., NCU, Taiwan
Bit-plane Slicing
� Instead of highlighting gray-level ranges, highlighting the contribution made to total image appearance by specific bits
� Higher-order bits: the majority of the visually significant data
� Lower-order bits: subtle details
� Obtain bit-plane images
� thresholding gray-level transformation function
More significant
More significant
14C.E., NCU, Taiwan
C.E., NCU, Taiwan 15
Outline
� Gray level transformations
� Histogram processing
� Enhancement using arithmetic operations
� Spatial filtering
C.E., NCU, Taiwan 16
Histogram Processing
� Histogram: A plot vs.
� : kth gray level
� : number of pixels in the image having gray level
� k=0,1,..,L-1
� Normalized histogram
� Purposes� image enhancement
� image compression
� segmentation
� Simple to calculate� economic hardware implementation
� proper for real-time image processing
kr kk nrH =)(
kr
knkr
nnrp kk /)( =
Histogram Equalization/
Histogram Linearization (I)
� Having gray-level values automatically cover the entire gray scale
� controlling the probability density function (pdf) of its gray levels via the transformation function T(r), where
� Continuous transformation S=T(r): assume T(r) satisfies
� (a.1) singled-valued: guarantee that the inverse transformation will exist
� (a.2) monotonically increasing: preserves the increasing order from black to white in the output image
� not monotonically increasing: at least a section of the intensity range being inverted
� (b) : the output gray levels will be in the same range as the input levels
∑ ∑= =
−====k
j
k
j
j
jrkk Lkn
nrprTs
0 0
1,...,2,1,0,)()(
10for 10 ≤≤≤≤ rTr
17C.E., NCU, Taiwan
C.E., NCU, Taiwan 18
Histogram Equalization (II)
� According to an elementary probability theory, if and T(r) are known and satisfies condition (a), then the probability density function of the transformed variable s can be obtained by
� The probability density function of the transformed variable s is determined by the gray-level pdf of the input image and by the chosen transformation function
ds
drrpsp rs )()( =
)(sps
)(rpr
)(1
sT−
C.E., NCU, Taiwan 19
Histogram Equalization (III)
� Consider the transformation function
∫==r
r dwwprTs0
)()(
)(])([)(
)(
0rpdwwp
dr
d
rd
rdT
dr
dsr
r
r === ∫
1s0 ,1]1[)(
1)()(
)()(
1
1
≤≤==
= −
−=
=sTr
sTrr
rsrp
rpsp
1. Performing the transformation yields a random variable scharacterized by a uniform probability density function.
2. T(r) depends on , but the resulting always is uniform, independent of .
)(rpr
)(sps
)(rpr
Histogram Matching
(Histogram Specification) (1/2)� Motivation
� sometimes, it is useful to specify the shape of the histogram of the processed image instead of uniform histogram
� the original image & histogram specification are given
∫==r
r dwwprTs0
)()(
sdttpzGz
z == ∫0 )()(, define
)]([)(11
rTGsGz−− ==
)()( zTzG =Obtain T(r)
Obtain G(z))(zpzGive
Give input image r,
Continuous gray level r & z
Discrete gray level
1,...,2,1,0),(1 −== −
LksGz kk
Histogram Equalization
22C.E., NCU, Taiwan
Local Enhancement� Motivation
� to enhance details over small areas in an image
� the computation of a global transformation does not guarantee the desired local enhancement
� Solution
� define a square/rectangular neighborhood
� move the center of this area from pixel to pixel
� histogram equalization in each neighborhood region
OriginalImage
After GlobalEnhancement
After LocalEnhancement 24
C.E., NCU, Taiwan 25
Use of Histogram Statistics for Image
Enhancement
� Provide simple while powerful image enhancement
� Histogram statistics
� global mean
� a measure of average gray level in an image
� global variance
� a measure of average contrast
� local mean
� local variance
∑−
=
=1
0
)(L
i
ii rprm
∑−
=
−=µ1
0
22 )()()(
L
i
ii rpmrr
∑∈
=xy
xySts
tstss rprrm),(
,, )()(
∑∈
−=σxy
xyxySts
tsStsS rpmr),(
,2
,2
)(][
∑−
=
−=µ1
0
)()()(L
i
in
in rpmrr
nth moment of r
C.E., NCU, Taiwan 26
Outline
� Gray level transformations
� Histogram processing
� Enhancement using arithmetic operations
� Spatial filtering
Enhancement Using Arithmetic
Operations – Image Subtraction
�
� The enhancement of the differences between images
� to bring out more details, contrast stretching can be performed
� histogram equalization
� power-law transformation
Original imageOriginal image
Zero the 4 lower-order bit plane
Zero the 4 lower-order bit plane
),(),(),( yxhyxfyxg −=
27C.E., NCU, Taiwan
Image Subtraction –
Mask Mode Radiography� The mask is an X-ray image of a region of a patient’s body
� The image to be subtracted is taken after injection of a contrast medium into the bloodstream
� The bright arterial paths carrying the medium are unmistakably enhanced
� The overall background is much darken than the mask image
Mask imageMask image After inject of a contrast medium into the bloodstreamAfter inject of a contrast medium into the bloodstream28C.E., NCU, Taiwan 28
C.E., NCU, Taiwan 29
The Range of the Image Values
After Image Subtraction
� For 8-bit image: -255~255
� Two principle ways to scale a different image
� add 255 to every pixel & divide by 2
� limitation: the full range of the display may not be utilized
� the value of the minimum difference is obtained & its negative added to all the pixels in the difference image
� scale to the interval [0, 255]
� more complex
C.E., NCU, Taiwan 30
Outline
� Gray level transformations
� Histogram processing
� Enhancement using arithmetic operations
� Spatial filtering
Basics of Spatial Filtering (1/2)
)1,1()1,1(),1()0,1(),()0,0(
...),1()0,1()1,1()1,1(
+++++++
+−−+−−−−=
yxfwyxfwyxfw
yxfwyxfwR
L )1,1()1,1(),1()0,1(),()0,0(
...),1()0,1()1,1()1,1(
+++++++
+−−+−−−−=
yxfwyxfwyxfw
yxfwyxfwR
L
� The concept of filtering comes from the use of the Fourier transform for signal processing in the frequency domain.
� At each point (x,y), the response of the filter at that point is calculated using a predefined relationship.
� Linear spatial filtering: the response is the sum of products
3131
C.E., NCU, Taiwan 32
Basics of Spatial Filtering (2/2)
� Linear spatial filtering often is referred to as “convolving a mask with an image.”
� filtering mask / convolution mask / convolution kernel
� Nonlinear spatial filtering
� the filtering operation is based conditionally on the values of the pixels in the neighborhood under consideration, but not explicitly use coefficients in the sum-of-products manner
� e.g., median filter for noise reduction: compute the median gray-level value in the neighborhood
When the Center of the Filter Approaches the
Border of the Image ?
� One or more rows/columns of the mask will be located outside the image plane
� Solution
� limit the excursions of the center of the mask to be at a distance no less than (n-1)/2 pixels from the border
� the resulting filtered image is smaller than the original
� partial filter mask: filter all pixels only with the section of the mask that is fully contained in the image
� Padding
� add rows & columns of 0’s (or other constant gray level), or replicate rows/columns
� the padding is stripped off at the end of the process
� side effect: an effect near the edges that becomes more prevalent as the size of the mask increases
33C.E., NCU, Taiwan 33
C.E., NCU, Taiwan 34
Smoothing Linear Filters (1/2)
� Averaging filters/low pass filters: replace the value of every pixel in an image by the average of the gray levels of the pixels contained in the neighborhood of the filter mask
� Reduce sharp transitions in gray levels
� side effect: blur edges
� Reduce irrelevant detail in an image and get a gross representation of objects of interest
� irrelevant: pixel regions that are small with respect to the size of the filter mask
� Applications
� noise reduction
� smooth false contouring
Smoothing Linear
Filters (2/2)
� Computationally efficient: instead of being 1/9 (1/16), the coefficients of the filter are all 1’s
� Box filter: a spatial averaging filter in which all coefficients are equal
� Weighted average: give more importance (weight) to some pixels
� reduce blurring in the smoothing process
� the pixel at the center of the mask is multiplied by a higher value than any other
� the other pixels are inversely weighted as a function of their distance from the center of the mask
� The attractive feature of “16”: integer power of 2 for computer implementation
35C.E., NCU, Taiwan 35
Examples of Smoothing
Linear Filters (1/2)
� n=3: general slight blurring through the entire image, but details are of approximately the same size as the filter mask are affected considerably more
� n=9: the significant further smoothing of the noisy rectangles
� n=15 & 35: generally eliminate small objects from an image
� n=35: black order due to zero padding
36
Examples of Smoothing Linear Filters (2/2)
� Goal: get a gross representation of objects of interest, such that the intensity of smaller objects blends with the background and larger objects become bloblike
� the size of the mask
� Smoothing linear filtering + thresholding to eliminate objects based on their intensity
37
C.E., NCU, Taiwan 38
Order-Statistics Filters
� Nonlinear spatial filters whose response is based on ordering (ranking) the pixels in the area to be filtered
� Examples: max filter, min filter & median filter
� Median filter
� replace the value of a pixel by the median of the gray levels in the neighborhood of that pixel
� force points with distinct gray levels to be more like their neighbors
� for 3x3 neighborhood
(10,20,20,20,15,20,20,25,100) -> median is 20
� excellent noise-reduction capabilities, with considerably less blurring than linear smoothing filters of similar size
� particular effective for impulse noise (salt-and-pepper noise
C.E., NCU, Taiwan 39
An Example of Applying Median Filters
� The image processed with the averaging filter has less visible noise, but the price paid is significant blurring.
C.E., NCU, Taiwan 40
Foundations of Sharpening Spatial
Filters (1/2)
� Highlight fine detail in an image or to enhance detail that has been blurred
� Averaging is analogous to integration, and sharpening can be accomplished by spatial differentiation
� For our use, what we require for a first derivative are
� zero in flat areas
� nonzero at the onset of a gray-level step or ramp
� nonzero along ramps
� For our use, what we require for a second derivative
� zero in flat areas
� nonzero at the onset and end of a gray-level step or ramp
� zero along ramps of constant slope
C.E., NCU, Taiwan 41
Foundations of Sharpening Spatial
Filters (2/2)
� The first-order derivative of a 1-D function f(x)
� The second-order derivative of f(x)
� The two definitions satisfy the conditions stated previously
)()1( xfxfx
f−+=
∂
∂
)(2)1()1(2
2
xfxfxfx
f−−++=
∂
∂
An Example of Applying Sharpening
Spatial Filters (2/3)� Ramp: the first-order derivative is nonzero along the entire
ramp, while the second-order derivative is nonzero only at the onset and end of the ramp.
� first-order derivatives produce thick edges and second-order derivatives, much finer ones
� Isolated noise point: the response at and around the point is much stronger for the second- than for the first-order derivative.
� second-order derivative enhances much more than a first-order derivative does.
� Thin line: essentially the same difference between the two derivatives
� Gray-level step: the response of the two derivatives is the same
� double-edge effect: The second derivative has a transition from positive back to negative. 43
C.E., NCU, Taiwan 44
Foundation of
Sharpening Spatial Filters (3/3)
� In most applications, the second derivative is better suited than the first derivative for image enhancement.
� The principle use of first derivatives in image processing: edge extraction.
Second Derivatives for
Enhancement – The Laplacian
� Isotropic filters
� rotation invariant: rotating the image and applying the filter gives the same result as applying the filter to the image first and then rotating the result
� The simplest isotropic derivative operator: Laplacian
� Laplacian is a linear operator : because derivatives of any order are linear operator
y
f
x
ff
2
2
2
22
∂
∂+
∂
∂=∇
),(2)1,()1,(
),(2),1(),1(
2
2
2
2
yxfyxfyxfx
f
yxfyxfyxfx
f
−−++=∂
∂
−−++=∂
∂
),(4)]1,()1,(),1(),1([2
yxfyxfyxfyxfyxff −−+++−++=∇45
Filter Masks – The Laplacian (1/2)
Isotropic for increments of 45 deg.
Isotropic for increments of 90 deg.
4646
C.E., NCU, Taiwan 47
Filter Masks – The Laplacian (2/2)
� The Laplacian is a derivative operator
� highlights gray-level discontinuities in an image & deemphasizes regions with slowly varying gray levels
� Being used frequently for sharpening digital images
� The Lapalcian for image enhancement
∇+
∇−=
),(),(
),(),(),(
2
2
yxfyxf
yxfyxfyxg
If the center coefficient of theLaplacian mask is negativeIf the center coefficient of the Lapalcian mask is positive
Examples of the Laplacian
Enhanced Image (1/2)
� Notice that the Laplacian image may contain both positive and negative values
� scaling is desired
� Adding the original image to the Laplacian image
� small details were enhanced
� the background tonality is reasonably preserved
48C.E., NCU, Taiwan 48
� The results obtainable with the mask containing the diagonal terms usually are a little sharper than those obtained with the more basic mask
Examples of the Laplacian
Enhanced Image (2/2)
49C.E., NCU, Taiwan
C.E., NCU, Taiwan 50
First Derivatives for Enhancement –
The Gradient (1/2)
� First derivatives in image processing are implemented using the magnitude of the gradient
� gradient is defined as the 2-D column vector
� the magnitude of this vector (gradient)
� nonlinear operator
� isotropic
∂
∂∂
∂
=
=∇
y
fx
f
G
G
y
xf
2/122
2/122][)f(
∂
∂+
∂
∂=+=∇=∇
y
f
x
fGGmagf yx
First Derivatives for Enhancement –
The Gradient (2/2)
� Problem: the computational burden of implementing the gradient
� The approximation of the magnitude of the gradient
� the isotropic properties of the digital gradient are preserved only for a limited number of rotational increments (90 deg.)
� Digital approximation of the gradient
yx GG +≈∇f
5658 , zzGzzG yx −=−=
6859 , zzGzzG yx −=−=
1
2
51C.E., NCU, Taiwan
� Roberts cross-gradient operators
6859 zzzzf −+−≈∇
)2()2(
)2()2(
741963
321987
zzzzzz
zzzzzzf
++−++
+++−++≈∇
(From )2
Since even size are awkward toimplement ���� 3x3 Sobel Operator
A 3x3 Region of an Image & Masks
for Computing the Gradient
derivative in xderivative in y52C.E., NCU, Taiwan
C.E., NCU, Taiwan 53
An Example of Using Sobel Gradient
� An optical image of a contact lens illuminated by a lighting arrangement to highlight imperfections
� Sobel gradient: the edge defects are quite visible, and the slowly varying shades of gray are eliminated.
� simplify the computational task required for automated inspection.