digital image processing130.243.105.49/~lilien/dip/lectures/dip_2007_02.pdf · digital image...
TRANSCRIPT
Digital Image Processing
Course IntroductionLecture 1
Digital Image ProcessingPart 2: Image Enhancement
in the Spatial Domain
Achim J. Lilienthal
AASS Learning Systems Lab, Teknik
Room T1216
Course Book Chapter 3
Achim J. Lilienthal
Contents
1. Image Enhancement in the Spatial Domain
2. Grey Level Transformations
3. Histogram Processing
4. Operations Involving Multiple Images
5. Spatial Filtering
Achim J. Lilienthal
→ Contents
Image Enhancement in the Spatial Domain
Achim J. Lilienthal
Image Enhancement in the Spatial Domain1
Image Enhancementimage processing
the result is supposed to be "more suitable"
"more suitable" according to a certain application
more suitable for visual interpretation
Achim J. Lilienthal
Image Enhancement in the Spatial Domain1
We want to create an image which is "better" in some sense.
helps visual interpretation (brightening, sharpening…)⇒ subjective
pre-processing for a subsequent image analysis algorithm⇒ performance metric (performance of a task)
make the image more "specific" ⇒ application dependent
f(x,y) g(x,y)
original image(or set of images)
new image
T
Achim J. Lilienthal
Image Enhancement in the Spatial Domain1
Spatial Domain versus Frequency Domainspatial domain
direct manipulation of the pixels
⇒ discussed in this lecture
three types of transformations in the spatial domain:
• pixel brightness transformations, point processing(depend only on the pixel value itself)
• spatial filters, local transformations or local processing(depend on a small neighbourhood around the pixel)
• geometric transformations
frequency domain: modifications of the Fourier transform⇒ discussed in the next lectures
Achim J. Lilienthal
Image Enhancement in the Spatial Domain1
Transformations in the Spatial Domain
standard approach:T is applied to a sub-image centred at (x,y)
sub-image is called mask(kernel, filter, template, window)
mask processing or filtering
)],([),( yxfTyxg =
Achim J. Lilienthal
→ Contents
Gray Level Transformations
Achim J. Lilienthal
Grey Level Transformations2
Grey Level Transformationssimplest case: each pixel in the output image depends only on the corresponding pixel in the input image
1x1 neighbourhood (point processing)
examples(contraststretching)
)(rTs =)(rTs =
Achim J. Lilienthal
Grey Level Transformations2
Grey Level Transformations
contrast stretching
thresholding
Achim J. Lilienthal
Grey Level Transformations2
Common Grey Level Transformations (Single Image)linear
identityinverse
power lawn. powern. root
logarithmic
... with more than one input imagesum, meanstatistical operations (variance, t-test …)
Achim J. Lilienthal
Grey Level Transformations2
Common Grey Level Transformations (Single Image)linear
identity
inverse (negative)
power lawn. power
n. root
logarithmic
Achim J. Lilienthal
Grey Level Transformations2
Common Grey Level Transformations (Single Image)inverse transform
Achim J. Lilienthal
→ Contents
Histogram Processing
Achim J. Lilienthal
Histogram Processing3
Grey Scale Histogramshows the number of pixels per grey level
Achim J. Lilienthal
Histogram Processing3
Grey Scale Histogramneutral transform
Achim J. Lilienthal
Histogram Processing3
Grey Scale Histogramneutral transform
inverse transform
Achim J. Lilienthal
Histogram Processing3
Grey Scale Histogramneutral transform
inverse transform
logarithmic transform
Achim J. Lilienthal
Histogram Processing3
Grey Scale Histogrambrightness (addition / subtraction)
Achim J. Lilienthal
Histogram Processing3
Grey Scale Histogrambrightness (addition / subtraction)
contrast (histogram stretching)
Achim J. Lilienthal
Histogram Processing3
Grey Scale Histogrambrightness (addition / subtraction)
contrast (histogram stretching)
Achim J. Lilienthal
Grey Level Transformations2
Contrast Stretchingpiecewise linear function
Achim J. Lilienthal
Grey Level Transformations2
Contrast Stretchingpiecewise linear function
power law transformation(gamma transformation)
γcrs =
Achim J. Lilienthal
Histogram Processing3
Histogram Equalizationcontrast / brightness adjustments sometimes need to be automatised
"optimal" contrast for an image?⇒ flat histogram
histogram normalizationto get a given shape for the histogram (see GW 3.3.2)
Achim J. Lilienthal
Histogram Processing3
∫=r
r dprT0
)()( ωω
Histogram Equalizationconsider the continuous case:
probability density functions (PDFs) of s and r are related by
transformation function = cumulative density function (CDF)
⇒ ⇒
]1,0[, ∈rs
)(1)()()(sT
rpdsdrrpsp rrs ′
==
)()()(0
rpdpdrdrT
drds
r
r
r =⎥⎦
⎤⎢⎣
⎡=′= ∫ ωω 1)( =sps
Achim J. Lilienthal
Histogram Processing3
Histogram Equalizationdiscrete case
does not generally produce a uniform PDF
tends to spread the histogram
enables automatic contrast stretching
∑∑==
===k
j
jk
jjrkk n
nrprTs
00)()(
nnrp k
kr =)(
Achim J. Lilienthal
Histogram Processing3
Histogram Equalization
CDF
Achim J. Lilienthal
Histogram Processing3
Histogram Equalization
Achim J. Lilienthal
Histogram Processing3
Adaptive / Localized Histogram Equalizationtransform a single pixel by histogram equalization calculated over a square or rectangular neighbourhood
original image
global histogram equalization
Achim J. Lilienthal
Histogram Processing3
Adaptive / Localized Histogram Equalizationtransform a single pixel by histogram equalization calculated over a square or rectangular neighbourhood
original image
local histogram equalization (radius = 100)
Achim J. Lilienthal
Histogram Processing3
Adaptive / Localized Histogram Equalizationtransform a single pixel by histogram equalization calculated over a square or rectangular neighbourhood
original image
local histogram equalization (radius = 50)
Achim J. Lilienthal
Histogram Processing3
Adaptive / Localized Histogram Equalizationtransform a single pixel by histogram equalization calculated over a square or rectangular neighbourhood
original image
local histogram equalization (radius = 25)
Achim J. Lilienthal
Histogram Processing3
Adaptive / Localized Histogram Equalizationtransform a single pixel by histogram equalization calculated over a square or rectangular neighbourhood
original image
local histogram equalization (radius = 12)
Achim J. Lilienthal
Histogram Processing3
Use of Histogram Statistics for Image Enhancementidea: instead of using the image histogram directly, we can instead use statistical parameters derived from it
mean: measure of average grey level in the image / neighbourhood
variance: measure of average contrast in the image / neighbourhood
example in GW (see chapter 3.3.4)compare local with global mean and variance and decide whether the pixel is processed or not
⇒⎩⎨⎧ ≤≤≤⋅
=otherwiseANDif
),(),(
),( 210 ,, GSGGS kkmkmyxfyxfE
yxg yxyxσσσ
Achim J. Lilienthal
→ Contents
Operations Involving Multiple Images
Achim J. Lilienthal
Operations Involving Multiple Images4
Operations Between Two or More Imagesimage subtraction
Angiography
tracking
Achim J. Lilienthal
Operations Involving Multiple Images4
Image SubtractionDSA (Digital Subtraction Angiography)
DSA imagemask image live image
Achim J. Lilienthal
Operations Involving Multiple Images4
Image Subtractiontracking with a stationary camera
background image live image difference image
Achim J. Lilienthal
Operations Involving Multiple Images4
Operations Between Two or More Imagesimage subtraction
Angiography
tracking
image averaging (GW 3.4.2)noise reduction
background modeling
⇒
Achim J. Lilienthal
→ Contents
Spatial Filtering
Achim J. Lilienthal
Spatial Filtering5
Neighbourhood Relations Between Pixelsa pixel has 4 or 8 neighbours in 2D depending on the neighbour definition:
4-neighborhoodeach neighbor must sharean edge with the pixel
8- neighborhoodeach neighbor must share an edge or a corner with the pixel
Achim J. Lilienthal
Spatial Filtering5
Basics of Spatial Filteringthe pixel value in the output image is calculated from a local neighbourhood in the input image
the local neighbourhood is described by a mask with a typical size of 3x3, 5x5, 7x7, … pixels
filtering is performed by moving the mask over the image
the centre pixel in the output image is given a value that depends on the input image and the weights of the mask
Achim J. Lilienthal
Spatial Filtering5
Basics of Spatial Filteringfilter subimage defines coefficients w(s,t)
used to updatepixel at (x,y)
111
11
1
111
Achim J. Lilienthal
Spatial Filtering5
Linear Spatial Filteringfilter subimage definescoefficients w(s,t)
response of the filterat point (x,y) is givenby a sum of products
also called convolution(convolution mask)
∑∑−= −=
++=a
as
b
bttysxftswyxg ),(),(),(
(1,1)(0,1)(-1,1)
(1,0)(0,0)(-1,0)
(1,-1)(0,-1)(-1,-1)
Achim J. Lilienthal
Spatial Filtering5
Linear Spatial Filtering – Implementationgeneric code:
How to Deal With the Border?limit excursion of the centre of the mask → smaller image
set outside pixel value zero → border effects
mirroring border pixel values → border effects
modify filter size along the border → slower
for P(x,y) in imagefor F(u,v) in filter
Q(x,y) += F(u,v) × P(x-u,y-v)end
end
Achim J. Lilienthal
Spatial Filtering5
Smoothing Spatial Filters (Averaging Filters)for blurring
removal of small (irrelevant) details, bridging smallgaps
for noise reduction
Smoothing Spatial Filters – Mean Filterneed for normalizationto conserve the total “energy”of the image (sum of all greylevels)quickresults in severe edge blurring
111
111
111
x 1/9
Achim J. Lilienthal
Spatial Filtering5
Smoothing Spatial Filters – Mean Filter
original Mean 11x11Mean 5x5
Achim J. Lilienthal
Spatial Filtering5
Smoothing Spatial Filters – Gaussian Filterweighted average
2D Gaussian kernel
higher weight in the centre to decrease blurring
Why a Gaussian?simple model of blurring in optical systems
smooth
121
242
121
x 1/16
Achim J. Lilienthal
Spatial Filtering5
Smoothing Spatial Filters – Median Filtertake the values of the input image corresponding to the desired sub-window (3x3, 5x5,…)
sort them
take the middle value (example: 3x3 → the 5th largest)
forces pixels with distinct grey levels to be more like their neighbours
very good at reduce salt-and-pepper noise
less blurring than linear filters of the same size
Achim J. Lilienthal
Spatial Filtering5
Smoothing Spatial Filters – Median Filtertake the median value over the sub-window
Average 3x3 Median 3x3X ray image of a circuit board
Achim J. Lilienthal
Spatial Filtering5
Smoothing Spatial Filters – Median Filtertake the median value over the sub-window
mean 3x3 mean 5x5 mean 11x11original image
Achim J. Lilienthal
Spatial Filtering5
Order Statistics Filters (Fractile Filters)median
min, maxuseful in mathematical morphology
percentilegeneralization of median, min, max
1 1 2 2 3 4 5 7 8
min (0%)
25% percentile
median (50%) max (100%)128
542
731
Achim J. Lilienthal
Spatial Filtering5
Order Statistics Filters (Fractile Filters)median
min, maxuseful in mathematical morphology
percentilegeneralization of median, min, max
order statistics filters are nonlinear filters
order statistics do not have an equivalent in the frequency domain
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filtershighlight fine detail (also noise)
enhance edges
uses image differentiation
Sharpening Spatial Filters – 1Dapproximation to 1st Order Derivation
equivalent to the 1D convolution mask
)()1( xfxfxf
−+≈∂∂
000
1-10
000
Achim J. Lilienthal
Spatial Filtering5
Gradient and Magnitude of the Gradient
Sharpening Spatial Filters – Based on the GradientRoberts
Sobel
Prewitt
…
yf
xf
yf
xff
∂∂
+∂∂
≈⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
+⎟⎠⎞
⎜⎝⎛∂∂
=∇22
⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
∂∂
=∇yf
xff ,
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial FiltersRoberts (cross gradient operators)
2 masks approximate | Gx | and | Gy | in yx GGyf
xff +=
∂∂
+∂∂
≈∇
10
0-1
01
-10
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial FiltersSobel Operators
2 masks approximate | Gx | and | Gy | in
detects horizontal and vertical edges
rotations give the other 6 convolution masks, including diagonal edges
yx GGyf
xff +=
∂∂
+∂∂
≈∇
10-1
20-2
10-1
1-2-1
000
121
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filters – Sobel Operatorsweight 2 is supposed to smooth by emphasizing the centre
10-1
20-2
10-1
1-2-1
000
121
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filters – Sobel Operatorsdetection of vertical dark-light edges
10-1
20-2
10-1
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filters – Sobel Operatorscombination of all the directional responses
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial FiltersPrewitt gradient edge detector
2 masks approximate | Gx | and | Gy | in yx GGyf
xff +=
∂∂
+∂∂
≈∇
10-1
10-1
10-1
1-1-1
000
111
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filterscomparison between Sobel and Prewitt operator
Sobel (~ |Gx| + |Gy|) Prewitt (~ |Gx| + |Gy|)
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filtershighlight fine detail (also noise)
enhance edges
uses image differentiation
Sharpening Spatial Filters – 1Dapproximation to 1st order derivation
approximation to 2nd order derivation
equivalent to the 1D convolution mask000
1-21
000)(2)1()1(2
2
xfxfxfxf
−−++=∂∂
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filters – Laplace FilterLaplacian (second order derivative)
thinner edges
not so strong response to a step
better response to fine details
double response to edges
rotation independent ⇒ one mask for all edges
2
2
2
22
yf
xf
∂∂
+∂∂
=∇
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filters – Laplace FilterLaplacian (second order derivative)
filter masks to implement the Laplacianadd the "digital implementation" of the two terms in the Laplacian(90° rotation symmetry)
add also diagonal terms(45° rotation symmetry)
negative values → re-scale
2
2
2
22
yf
xf
∂∂
+∂∂
=∇
010
1-41
010
111
1-81
111
)(2)1()1(2
2
xfxfxfxf
−−++=∂∂
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filters – Laplace Filterdetection of edges independent of direction
isotropic with respect to 90° rotations
0-10
-14-1
0-10
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filters – Laplace FilterLaplace filter + original image ⇒ sharpening
0-10
-15-1
0-10
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial FiltersLaplace filter + original image ⇒ sharpening
Achim J. Lilienthal
Spatial Filtering5
Sharpening Spatial Filters – Unsharp Maskinganalog equivalent used in publishing industry
basic idea: subtract blurred version of an image from original image to generate the edges