part 2: image enhancement in the spatial...

2011-04-12

Digital Image Processing

Achim J. Lilienthal

AASS Learning Systems Lab, Dep. Teknik

Room T1209 (Fr, 11-12 o'clock)

[email protected]

Course Book Chapter 3

Part 2: Image Enhancement

in the Spatial Domain

Achim J. Lilienthal

1. Image Enhancement in the Spatial Domain

2. Grey Level Transformations

3. Histogram Processing

4. Operations Involving Multiple Images

5. Spatial Filtering

Contents

Achim J. Lilienthal

Spatial Filtering

→ Contents

Achim J. Lilienthal

Spatial Filtering5

Neighbourhood Relations Between Pixels a pixel has 4 or 8 neighbours in 2D

depending on the neighbour definition: 4-neighborhood

each neighbor must sharean edge with the pixel

8- neighborhood each neighbor must share an edge or a corner with the pixel

Achim J. Lilienthal

Spatial Filtering5

Basics of Spatial Filtering the pixel value in the output image

is calculated from a local neighbourhood in the input image

the local neighbourhood is described by a mask with a typical size of 3x3, 5x5, 7x7, … pixels

filtering is performed by moving the mask over the image

the centre pixel in the output image is given a value that depends on the input image and the weights of the mask

Achim J. Lilienthal

Spatial Filtering5

Basics of Spatial Filtering filter subimage defines

coefficients w(s,t)

used to updatepixel at (x,y)

111

11

1

111

Achim J. Lilienthal

Spatial Filtering5

Linear Spatial Filtering filter subimage defines

coefficients w(s,t)

response of the filterat point (x,y) is givenby a sum of products

also called convolution(convolution kernel)

∑∑−= −=

++=a

as

b

bttysxftswyxg ),(),(),(

(1,1)(0,1)(-1,1)

(1,0)(0,0)(-1,0)

(1,-1)(0,-1)(-1,-1)

Achim J. Lilienthal

Spatial Filtering5

Linear Spatial Filtering – Implementation generic code:

How to Deal With the Border? limit excursion of the centre of the mask → smaller image

set outside pixel value zero → border effects

mirroring border pixel values → border effects

modify filter size along the border → slower

for P(x,y) in imagefor F(u,v) in filterQ(x,y) += F(u,v) × P(x-u,y-v)

endend

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters (Averaging Filters) for blurring

removal of small (irrelevant) details, bridging small gaps

for noise reduction but: edges are also blurred

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters (Averaging Filters) for blurring

removal of small (irrelevant) details, bridging small gaps

for noise reduction

Smoothing Spatial Filters – 3x3 Mean Filter / Box Filter need for normalization

to conserve the total “energy” of the image (sum of all greylevels)

can cause "ringing" no good model of blurring in a defocused camera

turns a single "point" into a "box"

111

111

111

x 1/9

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Mean Filter

original Mean 5x5 Mean 11x11

Achim J. Lilienthal

Spatial Filtering5

Linear Spatial Filtering in Matlab

filter matrix w filtering modes

'corr' or 'conv' only important in the case of asymmetric filters 'corr' (no mirroring) is the default

f = imread('bubbles.tif');g = imfilter(f, w,

filtering_mode,boundary_options,size_options);

Achim J. Lilienthal

Spatial Filtering5


boundary options P – padding with 0 (default) 'replicate' – replicate values at the outer border 'symmetric' – mirror reflecting across the outer border 'circular' – repeating the image like a periodic function



Achim J. Lilienthal

Spatial Filtering5


size options 'same' – same size as the input image (cropped padded image) 'full' – full size of the padded image default is 'same'



Achim J. Lilienthal

Spatial Filtering5

Linear Spatial Filtering in Matlabf = imread('bubbles.tif');g = imfilter(f, ones(8)/64, 'replicate'); imshow(g);

720 px

Achim J. Lilienthal

Spatial Filtering5

Linear Spatial Filtering in Matlabf = imread('bubbles.tif');g = imfilter(f, fspecial('average',32), 'replicate'); imshow(g);

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Mean Filter square box filter generates defects

axis aligned streaks

blocky results

input

output

example from "A Gentle Introduction to Bilateral Filtering and its Applications", Sylvain Paris, Pierre Kornprobst, Jack Tumblin, and Frédo Durand, SIGGRAPH 2008

Achim J. Lilienthal

Spatial Filtering5

box profile

from "A Gentle Introduction to Bilateral Filtering and its Applications", S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, SIGGRAPH 2008

unrelated pixelsunrelated pixels related pixels

pixelposition

pixelweight

Achim J. Lilienthal

Spatial Filtering5

strategy to solve problems with box filters use an isotropic (i.e. circular) window

use a window with a smooth falloff


box kernel Gaussian kernel

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Gaussian Filter weighted average

2D Gaussian kernel

higher weight in the centre to decrease blurring

Why a Gaussian? simple model of blurring

in optical systems

smooth

also a Gaussian in the frequency domain

121

242

121

1/16 x

Achim J. Lilienthal

Spatial Filtering5 input


Achim J. Lilienthal

Spatial Filtering5 inputbox average


Achim J. Lilienthal

Spatial Filtering5 inputbox average


Gaussian blur

Achim J. Lilienthal

Spatial Filtering5

Gaussian profile


unrelated pixelsunrelated pixels uncertain pixelsuncertain pixels related pixels

pixelposition

pixel weight

Achim J. Lilienthal

Spatial Filtering5

Gaussian profile spatial parameter σ


small σ large σ

limited smoothing strong smoothing

input

Achim J. Lilienthal

Spatial Filtering5

Gaussian profile spatial parameter σ

how to set σ? depends on the application

common strategy: proportional to image size

• e.g. 2% of the image diagonal

• property: independent of image resolution

depends on image content

• smooth "object areas" ⇒ larger σ

• but don't smooth edges ⇒ smaller σ


Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Median Filter take the values of the input image corresponding

to the desired sub-window (3x3, 5x5,…) sort them take the middle value (example: 3x3 → the 5th largest) forces pixels with distinct grey levels

to be more like their neighbours very good at reducing salt-and-pepper noise less blurring than linear filters of the same size

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Median Filter take the median value over the sub-window

Average 3x3 Median 3x3X ray image of a circuit board

Achim J. Lilienthal

Spatial Filtering5

Median Filter in Matlab

original image median 8x8 median 32x32 median 32x2

f = imread('bubbles.tif');g = medfilt2(f, [32 32]); imshow(g);

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Median Filter take the median value over a sub-window

original image median 3x3 median 5x5 median 11x11

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Median Filter take the values of the input image corresponding

to the desired sub-window (3x3, 5x5,…) sort them take the middle value (example: 3x3 → the 5th largest) forces pixels with distinct grey levels

to be more like their neighbours very good at reducing salt-and-pepper noise less blurring than linear filters of the same size nonlinear filter (order statistics filter) no equivalent in the frequency domain (order statistics filter)

Achim J. Lilienthal

Spatial Filtering5

Order Statistics Filters (Fractile Filters) median

min, max useful in mathematical morphology

percentile generalization of median, min, max

128

542

731

1 1 2 2 3 4 5 7 8

min (0%)

25% percentile

median (50%) max (100%)

Achim J. Lilienthal

Spatial Filtering5

Order Statistics Filters (Fractile Filters) median

min, max useful in mathematical morphology

percentile generalization of median, min, max

order statistics filters are nonlinear filters

order statistics filters do not have an equivalent in the frequency domain

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters highlight fine detail (also noise)

enhance edges

use image differentiation (1st order)

( , ) ( , )limf f x y f x yx ε

εε→∞

∂ + −=

∂

Achim J. Lilienthal

Spatial Filtering5


enhance edges

use image differentiation (1st order)

( , ) ( , )limf f x y f x yx ε

εε→∞

∂ + −=

∂ 1, ,i j i jf f fx +∂

≈ −∂

⇒

Achim J. Lilienthal

Spatial Filtering5


enhance edges

use image differentiation

Sharpening Spatial Filters – 1D approximation to 1st Order Derivation

equivalent to the 1D convolution mask

)()1( xfxfxf

−+≈∂∂

000

1-10

000

Achim J. Lilienthal

Spatial Filtering5


enhance edges

use image differentiation

Sharpening Spatial Filters – 1D approximation to 1st Order Derivation

equivalent to the 1D convolution mask

( 1) ( 1)f f x f xx∂

≈ + − −∂

000

10-1

000

Achim J. Lilienthal

Spatial Filtering5

Gradient and Magnitude of the Gradient

Sharpening Spatial Filters – Based on the Gradient Roberts

Prewitt

Sobel

…

yf

xf

yf

xff

∂∂

+∂∂

≈

∂∂

+

∂∂

=∇22

∂∂

∂∂

=∇yf

xff ,

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters Prewitt gradient edge detector

2 masks approximate | Gx | and | Gy | in yx GGyf

xff +=

∂∂

+∂∂

≈∇

10-1

10-1

10-1

1-1-1

000

111

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters Sobel Operators

2 masks approximate | Gx | and | Gy | in

detects horizontal and vertical edges

10-1

20-2

10-1

-1-2-1

000

121

yx GGyf

xff +=

∂∂

+∂∂

≈∇

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Sobel Operators weight 2 is supposed to smooth by emphasizing the centre

10-1

20-2

10-1

1-2-1

000

121

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Sobel Operators detection of vertical dark-light edges

10-1

20-2

10-1

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Sobel Operators combination of all the directional responses

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters comparison between Sobel and Prewitt operator

Sobel (~ |Gx| + |Gy|) Prewitt (~ |Gx| + |Gy|)

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters Roberts (cross gradient operators)

2 masks approximate | Gx | and | Gy | in

10

0-1

01

-10

yx GGyf

xff +=

∂∂

+∂∂

≈∇

Achim J. Lilienthal

Spatial Filtering5


enhance edges

uses image differentiation

Sharpening Spatial Filters – 1D approximation to 1st order derivation

approximation to 2nd order derivation

equivalent to the 1D convolution mask000

1-21

000)(2)1()1(2

2

xfxfxfx

f−−++=

∂∂

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Laplace Filter Laplacian (second order derivative)

2

2

2

22

yf

xf

∂∂

+∂∂

=∇ )(2)1()1(2

2

xfxfxfx

f−−++=

∂∂

Achim J. Lilienthal

Spatial Filtering5


filter masks to implement the Laplacian add the "digital implementation"

of the two terms in the Laplacian(90° rotation symmetry)

2

2

2

22

yf

xf

∂∂

+∂∂

=∇

010

1-41

010

)(2)1()1(2

2

xfxfxfx

f−−++=

∂∂

Achim J. Lilienthal

Spatial Filtering5




add also diagonal terms(45° rotation symmetry)

2

2

2

22

yf

xf

∂∂

+∂∂

=∇

010

1-41

010

111

1-81

111

)(2)1()1(2

2

xfxfxfx

f−−++=

∂∂

Achim J. Lilienthal

Spatial Filtering5




add also diagonal terms(45° rotation symmetry)

negative values → re-scale

2

2

2

22

yf

xf

∂∂

+∂∂

=∇

010

1-41

010

111

1-81

111

)(2)1()1(2

2

xfxfxfx

f−−++=

∂∂

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Laplace Filter detection of edges independent of direction

isotropic with respect to 90° rotations

0-10

-14-1

0-10

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Laplace Filter Laplace filter + original image ⇒ sharpening

0-10

-15-1

0-10

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters Laplace filter + original image ⇒ sharpening

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – 2nd order vs. 1st order Laplacian (second order derivative)

thinner edges

not so strong response to a step

better response to fine details

double response to edges

rotation independent ⇒ one mask for all edges

2

2

2

22

yf

xf

∂∂

+∂∂

=∇ )(2)1()1(2

2

xfxfxfx

f−−++=

∂∂

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Unsharp Masking analog equivalent used in publishing industry

basic idea: subtract blurred version of an image from original image to generate the edges

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters in Matlabf = imread('bubbles.tif');g1 = imfilter(f, fspecial('laplacian',0.5)); g2 = imfilter(f, fspecial('unsharp', 0.5));% ...

Achim J. LilienthalAASS Learning Systems Lab, Örebro University

Agenda

1. Image Smoothing Revisited

2. Bilateral Filtering

3. Applications of Bilateral Filtering

4. Efficient ImplementationHeavily based on: "A Gentle Introduction to Bilateral Filtering and its Applications", Sylvain Paris, Pierre Kornprobst, Jack Tumblin, and Frédo Durand, SIGGRAPH 2008, see http://people.csail.mit.edu/sparis/siggraph07_courseand "Bilateral Filtering for Gray and Color Images", C. Tomasi, R. Manduchi, Proc. Int. Conf. Computer Vision

Agenda

Image Smoothing Revisited

1 Image Smoothing Revisited

General Strategy for Smoothing Images adjacent pixels tend to belong to the same object

images typically vary slowly over space

non-smoothness due to noise

noise values less correlated than the signal

smoothing = making adjacent pixels look more similar smoothing strategy: pixel average over its neighbors

however, basic assumption not true at edges

we are interested in edge preserving image smoothing


Mean Filter Profile

unrelated pixelsunrelated pixels related pixels

pixelposition

pixelweight


Gaussian Filter Profile

unrelated pixelsunrelated pixels uncertain pixelsuncertain pixels related pixels

pixelposition

pixel weight


Gaussian Filter linear convolution

weights independent of spatial location

well-known operation

can be computed efficiently

does smooth images but … … smoothes too much: edges are blurred

… only spatial distance matters, no edge term

Agenda

Bilateral Filtering

2 Definition of the Bilateral Filter

Blur from Averaging Across Edges

*

*

*

input output

same Gaussian kernel everywhere


Bilateral Filter – Limits Averaging Across Edges

input output*

*

*kernel shape depends on image content

[Aurich 95, Smith 97, Tomasi 98]


Bilateral Filter – The Additional Edge Term again: weighted average of pixels.

space weight

not new

range weight

I

new

normalizationfactor

new

( ) ( )∑∈

−−=S

IIIGGW

IBFq

qqpp

p qp ||||||1][rs σσ

please note the different notation: I ↔ f

2 1D Illustration

Bilateral Filter – 1D Illustration 1D image = line of pixels

better visualized as a plot

pixelintensity

pixel position

2 1D Illustration

space

space rangenormalization

Gaussian blur

( ) ( )∑∈

−−=S

IIIGGW

IBFq

qqpp

p qp ||||||1][rs σσ

Bilateral filter[Aurich 95, Smith 97, Tomasi 98]

space

spacerange

p

p

q

q

( )∑∈

−=S

IGIGBq

qp qp ||||][ σ

Is This a Linear Filter?

( ) ( )∑∈

−−=S

IIIGGW

IBFq

qqpp

p qp ||||||1][rs σσ

Bilateral Filtering2

No! BF[I1]p + BF[I2]p ≠ BF[I1+I2]p

( ) ( )∑∈

−−=S

IIIGGW

IBFq

qqpp

p qp ||||||1][rs σσ

Bilateral Filtering2

2 2D Illustration

qp

output input

( ) ( )∑∈

−−=S

IIIGGW

IBFq

qqpp

p qp ||||||1][rs σσ

p

reproduced from [Durand 02]

part 2: image enhancement in the spatial...

Documents