digital image processing130.243.105.49/~lilien/dip/lectures/dip_2007_02.pdf · digital image...

Digital Image Processing

Course IntroductionLecture 1

Digital Image ProcessingPart 2: Image Enhancement

in the Spatial Domain

Achim J. Lilienthal

AASS Learning Systems Lab, Teknik

Room T1216

achim.lilienthal@tech.oru.se

Course Book Chapter 3

Achim J. Lilienthal

Contents

1. Image Enhancement in the Spatial Domain

2. Grey Level Transformations

3. Histogram Processing

4. Operations Involving Multiple Images

5. Spatial Filtering

Achim J. Lilienthal

→ Contents

Image Enhancement in the Spatial Domain

Achim J. Lilienthal

Image Enhancement in the Spatial Domain1

Image Enhancementimage processing

the result is supposed to be "more suitable"

"more suitable" according to a certain application

more suitable for visual interpretation

Achim J. Lilienthal

Image Enhancement in the Spatial Domain1

We want to create an image which is "better" in some sense.

helps visual interpretation (brightening, sharpening…)⇒ subjective

pre-processing for a subsequent image analysis algorithm⇒ performance metric (performance of a task)

make the image more "specific" ⇒ application dependent

f(x,y) g(x,y)

original image(or set of images)

new image

T

Achim J. Lilienthal

Image Enhancement in the Spatial Domain1

Spatial Domain versus Frequency Domainspatial domain

direct manipulation of the pixels

⇒ discussed in this lecture

three types of transformations in the spatial domain:

• pixel brightness transformations, point processing(depend only on the pixel value itself)

• spatial filters, local transformations or local processing(depend on a small neighbourhood around the pixel)

• geometric transformations

frequency domain: modifications of the Fourier transform⇒ discussed in the next lectures

Achim J. Lilienthal

Image Enhancement in the Spatial Domain1

Transformations in the Spatial Domain

standard approach:T is applied to a sub-image centred at (x,y)

sub-image is called mask(kernel, filter, template, window)

mask processing or filtering

)],([),( yxfTyxg =

Achim J. Lilienthal

→ Contents

Gray Level Transformations

Achim J. Lilienthal

Grey Level Transformations2

Grey Level Transformationssimplest case: each pixel in the output image depends only on the corresponding pixel in the input image

1x1 neighbourhood (point processing)

examples(contraststretching)

)(rTs =)(rTs =

Achim J. Lilienthal

Grey Level Transformations2

Grey Level Transformations

contrast stretching

thresholding

Achim J. Lilienthal

Grey Level Transformations2

Common Grey Level Transformations (Single Image)linear

identityinverse

power lawn. powern. root

logarithmic

... with more than one input imagesum, meanstatistical operations (variance, t-test …)

Achim J. Lilienthal

Grey Level Transformations2

Common Grey Level Transformations (Single Image)linear

identity

inverse (negative)

power lawn. power

n. root

logarithmic

Achim J. Lilienthal

Grey Level Transformations2

Common Grey Level Transformations (Single Image)inverse transform

Achim J. Lilienthal

→ Contents

Histogram Processing

Achim J. Lilienthal

Histogram Processing3

Grey Scale Histogramshows the number of pixels per grey level

Achim J. Lilienthal

Histogram Processing3

Grey Scale Histogramneutral transform

Achim J. Lilienthal

Histogram Processing3

Grey Scale Histogramneutral transform

inverse transform

Achim J. Lilienthal

Histogram Processing3

Grey Scale Histogramneutral transform

inverse transform

logarithmic transform

Achim J. Lilienthal

Histogram Processing3

Grey Scale Histogrambrightness (addition / subtraction)

Achim J. Lilienthal

Histogram Processing3

Grey Scale Histogrambrightness (addition / subtraction)

contrast (histogram stretching)

Achim J. Lilienthal

Histogram Processing3

Grey Scale Histogrambrightness (addition / subtraction)

contrast (histogram stretching)

Achim J. Lilienthal

Grey Level Transformations2

Contrast Stretchingpiecewise linear function

Achim J. Lilienthal

Grey Level Transformations2

Contrast Stretchingpiecewise linear function

power law transformation(gamma transformation)

γcrs =

Achim J. Lilienthal

Histogram Processing3

Histogram Equalizationcontrast / brightness adjustments sometimes need to be automatised

"optimal" contrast for an image?⇒ flat histogram

histogram normalizationto get a given shape for the histogram (see GW 3.3.2)

Achim J. Lilienthal

Histogram Processing3

∫=r

r dprT0

)()( ωω

Histogram Equalizationconsider the continuous case:

probability density functions (PDFs) of s and r are related by

transformation function = cumulative density function (CDF)

⇒ ⇒

]1,0[, ∈rs

)(1)()()(sT

rpdsdrrpsp rrs ′

==

)()()(0

rpdpdrdrT

drds

r

r

r =⎥⎦

⎤⎢⎣

⎡=′= ∫ ωω 1)( =sps

Achim J. Lilienthal

Histogram Processing3

Histogram Equalizationdiscrete case

does not generally produce a uniform PDF

tends to spread the histogram

enables automatic contrast stretching

∑∑==

===k

j

jk

jjrkk n

nrprTs

00)()(

nnrp k

kr =)(

Achim J. Lilienthal

Histogram Processing3

Histogram Equalization

CDF

Achim J. Lilienthal

Histogram Processing3

Histogram Equalization

Achim J. Lilienthal

Histogram Processing3

Adaptive / Localized Histogram Equalizationtransform a single pixel by histogram equalization calculated over a square or rectangular neighbourhood

original image

global histogram equalization

Achim J. Lilienthal

Histogram Processing3

Adaptive / Localized Histogram Equalizationtransform a single pixel by histogram equalization calculated over a square or rectangular neighbourhood

original image

local histogram equalization (radius = 100)

Achim J. Lilienthal

Histogram Processing3

Adaptive / Localized Histogram Equalizationtransform a single pixel by histogram equalization calculated over a square or rectangular neighbourhood

original image

local histogram equalization (radius = 50)

Achim J. Lilienthal

Histogram Processing3

Adaptive / Localized Histogram Equalizationtransform a single pixel by histogram equalization calculated over a square or rectangular neighbourhood

original image

local histogram equalization (radius = 25)

Achim J. Lilienthal

Histogram Processing3

Adaptive / Localized Histogram Equalizationtransform a single pixel by histogram equalization calculated over a square or rectangular neighbourhood

original image

local histogram equalization (radius = 12)

Achim J. Lilienthal

Histogram Processing3

Use of Histogram Statistics for Image Enhancementidea: instead of using the image histogram directly, we can instead use statistical parameters derived from it

mean: measure of average grey level in the image / neighbourhood

variance: measure of average contrast in the image / neighbourhood

example in GW (see chapter 3.3.4)compare local with global mean and variance and decide whether the pixel is processed or not

⇒⎩⎨⎧ ≤≤≤⋅

=otherwiseANDif

),(),(

),( 210 ,, GSGGS kkmkmyxfyxfE

yxg yxyxσσσ

Achim J. Lilienthal

→ Contents

Operations Involving Multiple Images

Achim J. Lilienthal

Operations Involving Multiple Images4

Operations Between Two or More Imagesimage subtraction

Angiography

tracking

Achim J. Lilienthal

Operations Involving Multiple Images4

Image SubtractionDSA (Digital Subtraction Angiography)

DSA imagemask image live image

Achim J. Lilienthal

Operations Involving Multiple Images4

Image Subtractiontracking with a stationary camera

background image live image difference image

Achim J. Lilienthal

Operations Involving Multiple Images4

Operations Between Two or More Imagesimage subtraction

Angiography

tracking

image averaging (GW 3.4.2)noise reduction

background modeling

⇒

Achim J. Lilienthal

→ Contents

Spatial Filtering

Achim J. Lilienthal

Spatial Filtering5

Neighbourhood Relations Between Pixelsa pixel has 4 or 8 neighbours in 2D depending on the neighbour definition:

4-neighborhoodeach neighbor must sharean edge with the pixel

8- neighborhoodeach neighbor must share an edge or a corner with the pixel

Achim J. Lilienthal

Spatial Filtering5

Basics of Spatial Filteringthe pixel value in the output image is calculated from a local neighbourhood in the input image

the local neighbourhood is described by a mask with a typical size of 3x3, 5x5, 7x7, … pixels

filtering is performed by moving the mask over the image

the centre pixel in the output image is given a value that depends on the input image and the weights of the mask

Achim J. Lilienthal

Spatial Filtering5

Basics of Spatial Filteringfilter subimage defines coefficients w(s,t)

used to updatepixel at (x,y)

111

11

1

111

Achim J. Lilienthal

Spatial Filtering5

Linear Spatial Filteringfilter subimage definescoefficients w(s,t)

response of the filterat point (x,y) is givenby a sum of products

also called convolution(convolution mask)

∑∑−= −=

++=a

as

b

bttysxftswyxg ),(),(),(

(1,1)(0,1)(-1,1)

(1,0)(0,0)(-1,0)

(1,-1)(0,-1)(-1,-1)

Achim J. Lilienthal

Spatial Filtering5

Linear Spatial Filtering – Implementationgeneric code:

How to Deal With the Border?limit excursion of the centre of the mask → smaller image

set outside pixel value zero → border effects

mirroring border pixel values → border effects

modify filter size along the border → slower

for P(x,y) in imagefor F(u,v) in filter

Q(x,y) += F(u,v) × P(x-u,y-v)end

end

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters (Averaging Filters)for blurring

removal of small (irrelevant) details, bridging smallgaps

for noise reduction

Smoothing Spatial Filters – Mean Filterneed for normalizationto conserve the total “energy”of the image (sum of all greylevels)quickresults in severe edge blurring

111

111

111

x 1/9

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Mean Filter

original Mean 11x11Mean 5x5

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Gaussian Filterweighted average

2D Gaussian kernel

higher weight in the centre to decrease blurring

Why a Gaussian?simple model of blurring in optical systems

smooth

121

242

121

x 1/16

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Median Filtertake the values of the input image corresponding to the desired sub-window (3x3, 5x5,…)

sort them

take the middle value (example: 3x3 → the 5th largest)

forces pixels with distinct grey levels to be more like their neighbours

very good at reduce salt-and-pepper noise

less blurring than linear filters of the same size

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Median Filtertake the median value over the sub-window

Average 3x3 Median 3x3X ray image of a circuit board

Achim J. Lilienthal

Spatial Filtering5

Smoothing Spatial Filters – Median Filtertake the median value over the sub-window

mean 3x3 mean 5x5 mean 11x11original image

Achim J. Lilienthal

Spatial Filtering5

Order Statistics Filters (Fractile Filters)median

min, maxuseful in mathematical morphology

percentilegeneralization of median, min, max

1 1 2 2 3 4 5 7 8

min (0%)

25% percentile

median (50%) max (100%)128

542

731

Achim J. Lilienthal

Spatial Filtering5

Order Statistics Filters (Fractile Filters)median

min, maxuseful in mathematical morphology

percentilegeneralization of median, min, max

order statistics filters are nonlinear filters

order statistics do not have an equivalent in the frequency domain

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filtershighlight fine detail (also noise)

enhance edges

uses image differentiation

Sharpening Spatial Filters – 1Dapproximation to 1st Order Derivation

equivalent to the 1D convolution mask

)()1( xfxfxf

−+≈∂∂

000

1-10

000

Achim J. Lilienthal

Spatial Filtering5

Gradient and Magnitude of the Gradient

Sharpening Spatial Filters – Based on the GradientRoberts

Sobel

Prewitt

…

yf

xf

yf

xff

∂∂

+∂∂

≈⎟⎟⎠

⎞⎜⎜⎝

⎛∂∂

+⎟⎠⎞

⎜⎝⎛∂∂

=∇22

⎟⎟⎠

⎞⎜⎜⎝

⎛∂∂

∂∂

=∇yf

xff ,

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial FiltersRoberts (cross gradient operators)

2 masks approximate | Gx | and | Gy | in yx GGyf

xff +=

∂∂

+∂∂

≈∇

10

0-1

01

-10

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial FiltersSobel Operators

2 masks approximate | Gx | and | Gy | in

detects horizontal and vertical edges

rotations give the other 6 convolution masks, including diagonal edges

yx GGyf

xff +=

∂∂

+∂∂

≈∇

10-1

20-2

10-1

1-2-1

000

121

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Sobel Operatorsweight 2 is supposed to smooth by emphasizing the centre

10-1

20-2

10-1

1-2-1

000

121

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Sobel Operatorsdetection of vertical dark-light edges

10-1

20-2

10-1

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Sobel Operatorscombination of all the directional responses

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial FiltersPrewitt gradient edge detector

2 masks approximate | Gx | and | Gy | in yx GGyf

xff +=

∂∂

+∂∂

≈∇

10-1

10-1

10-1

1-1-1

000

111

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filterscomparison between Sobel and Prewitt operator

Sobel (~ |Gx| + |Gy|) Prewitt (~ |Gx| + |Gy|)

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filtershighlight fine detail (also noise)

enhance edges

uses image differentiation

Sharpening Spatial Filters – 1Dapproximation to 1st order derivation

approximation to 2nd order derivation

equivalent to the 1D convolution mask000

1-21

000)(2)1()1(2

2

xfxfxfxf

−−++=∂∂

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Laplace FilterLaplacian (second order derivative)

thinner edges

not so strong response to a step

better response to fine details

double response to edges

rotation independent ⇒ one mask for all edges

2

2

2

22

yf

xf

∂∂

+∂∂

=∇

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Laplace FilterLaplacian (second order derivative)

filter masks to implement the Laplacianadd the "digital implementation" of the two terms in the Laplacian(90° rotation symmetry)

add also diagonal terms(45° rotation symmetry)

negative values → re-scale

2

2

2

22

yf

xf

∂∂

+∂∂

=∇

010

1-41

010

111

1-81

111

)(2)1()1(2

2

xfxfxfxf

−−++=∂∂

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Laplace Filterdetection of edges independent of direction

isotropic with respect to 90° rotations

0-10

-14-1

0-10

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Laplace FilterLaplace filter + original image ⇒ sharpening

0-10

-15-1

0-10

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial FiltersLaplace filter + original image ⇒ sharpening

Achim J. Lilienthal

Spatial Filtering5

Sharpening Spatial Filters – Unsharp Maskinganalog equivalent used in publishing industry

basic idea: subtract blurred version of an image from original image to generate the edges