geometric transformations

82

Geometric transformationsExample comparing f1, f2, and f3

Shift: x+0.25

à Linear( )( )( ) ( )( ) ( ) ( )( )( )( )

0

1

2

3

4

5

6

0

0

1 0.25 10 7.5

1 0.75 10 1 0.25 20 17.5

0.25 20 0.75 30 27.5

0.25 30 7.5

0

f x

f x

f x

f x

f x

f x

f x

=

=

= - × =

= - × + - × =

= × + × =

= × =

=

83


Shift: x+0.25

à Bi-cubic( )( ) ( )

( ) ( )( )

( ) ( )( )( )

0

2 31

2 32

2 3

2 33

2 3

2 3

0

4 8 1.25 5 1.25 1.25 10

1.41

1 2 0.25 0.25 10

4 8 1.25 5 1.25 1.25 20

6.09

1 2 0.75 0.75 10

1 2 0.25 0.25 20

4 8 1.25 5 1.25 1.25 30

16.56

f x

f x

f x

f x

=

= - × + × - ×

= -

= - × + ×

+ - × + × - ×

=

= - × + ×

+ - × + ×

+ - × + × - ×

=

84


Shift: x+0.25

à Bi-cubic ( ) ( )( )( )

( ) ( )( )

( ) ( )

2 34

2 3

2 3

2 35

2 3

2 36

1 2 0.75 0.75 20

1 2 0.25 0.25 30

4 8 1.75 5 1.75 1.75 10

32.18

1 2 0.75 0.75 30

4 8 1.75 5 1.75 1.75 20

7.97

4 8 1.75 5 1.75 1.75 30

1.41

f x

f x

f x

= - × + ×

+ - × + ×

+ - × + × - ×

=

= - × + ×

+ - × + × - ×

=

= - × + × - ×

= -

85

Geometric transformations

86


Non-affine (non-linear) transformation models!

87

Overview topics today…

Distance measures


Book à Chapter 4 & Chapter 5 (Section 5.3)

Convolution (filtering)

88

Convolution (filtering)

Filtering

Book à Section 5.3

Convolution

Smoothing

Linear operatorsDenoising

Sharpening

Kernel

Window Filter

Point spread function

Impulse response

Pre-processing

Blurring

Super-resolution

Separable

Prewitt

Sobel

Canny

RobinsonKirsch

Scale-space

Fourier

Deconvolution

Anisotropic filter

Image derivative

Laplacian

Edge

Restoration

High and low-pass filter

Wiener

89

FilteringDictionary

90

FilteringIn digital image processing!

“A digital filter is ‘a system’ that performsmathematical operations on a sampled, discretedigital image to reduce or enhance certain aspects ofthat image.”

“Filter”

91

FilteringIntroduction

Ø Single pixel transformations à signal intensity andbrightness transformations (histogram equalization)

Ø Global (all pixels) analyses à geometric (rigid, affine,projective,…) transformations

Ø Neighborhood processing (local region of pixels) àspatial filtering

92

FilteringSpatial filtering of a (2D) digital image

Linear filtering methods

Non-linear filtering methods

93

FilteringDefinition Linear spatial filteringØ Consider filter H on input image f(x,y), producing

output image g(x,y):

( ) ( ), ,H f x y g x y=é ùë û

94

FilteringDefinition Linear spatial filteringØ Consider filter H on input image f(x,y), producing

output image g(x,y):

Ø H is called linear if (for real values ai and aj):

“Additivity” & “Homogeneity”

( ) ( ), ,H f x y g x y=é ùë û

( ) ( ) ( ) ( )( ) ( )

, , , ,

, ,i i j j i i j j

i i j j

H a f x y a f x y a H f x y a H f x y

a g x y a g x y

é ù é ù+ = +é ùë ûë û ë û= +

95

FilteringExample Linear spatial filteringØ Sum operator “S” is linear

( ) ( ) ( ) ( )( ) ( )

( ) ( )

, , , ,

, ,, ,

i i j j i i j j

i i j j

i i j j

a f x y a f x y a f x y a f x y

a f x y a f x ya g x y a g x y

é ù é ù+ = +é ùë ûë û ë û= += +

å å åå å

96

FilteringExample non-linear spatial filteringØ “Maximum” operator is non-linear; consider ‘images’:

1

0 22 3

f é ù= ê úë û

2

6 54 7

f é ù= ê úë û

97

FilteringExample non-linear spatial filteringØ “Maximum” operator is non-linear; consider ‘images’:

Ø Find an example of a1 and a2 where the followingequality does not hold:

1

0 22 3

f é ù= ê úë û

2

6 54 7

f é ù= ê úë û

1 2 1 2

0 2 6 5 0 2 6 5max max max

2 3 4 7 2 3 4 7a a a aì ü ì ü ì üé ù é ù é ù é ù

+ = +í ý í ý í ýê ú ê ú ê ú ê úë û ë û ë û ë ûî þ î þ î þ

98

FilteringExample non-linear spatial filteringØ “Maximum” operator is non-linear:

1 2

1 2 1 2

,0 2 6 5 0 2 6 5max max max

2 3 4 7 2 3 4 7

a aa a a a

"ì ü ì ü ì üé ù é ù é ù é ù+ = +í ý í ý í ýê ú ê ú ê ú ê ú

ë û ë û ë û ë ûî þ î þ î þ

?

99

FilteringAlgorithm for 2D Linear spatial filtering1. Select a ‘center’ pixel (x,y) in the image

2. Perform some operation that involves the ‘neighboringpixels’ of (x,y)

3. The result of this operation is called the “response” ofthis process at point (x,y)

4. Repeat the procedure for each point (x,y) in the image

100

FilteringAlgorithm for 2D Linear spatial filtering1. Select a ‘center’ pixel (x,y) in the image

5 3 1 41 5 6 16 1 5 71 5 1 8

4451

7 5 7 1 6

1 2 3 11 2 1 2

97

2192

1211

1 9

41

21

101

FilteringAlgorithm for 2D Linear spatial filtering2. Perform some operation that involves the ‘neighboring

pixels’ of (x,y), e.g., the 8-connectivity neighborhood:

5 3 1 41 5 6 16 1 5 71 5 1 8

4451

7 5 7 1 6

1 2 3 11 2 1 2

97

2192

1211

1 9

41

21

102

FilteringAlgorithm for 2D Linear spatial filtering2. Perform some operation that involves the ‘neighboring

pixels’ of (x,y), e.g., the 8-connectivity neighborhood:

“Average” operation: 1 à (2+1+2+3+1+4+5+6+1)/9 = 2.8

5 3 1 41 5 6 16 1 5 71 5 1 8

4451

7 5 7 1 6

1 2 3 11 2 1 2

97

2192

1211

1 9

41

21

103

FilteringAlgorithm for 2D Linear spatial filtering3. The result of this operation is called the “response” of

this process at point (x,y)

“Average” operation: 1 à 2.8

5 3 2.8 41 5 6 16 1 5 71 5 1 8

4451

7 5 7 1 6

1 2 3 11 2 1 2

97

2192

1211

1 9

41

21

104

FilteringAlgorithm for 2D Linear spatial filtering4. Repeat the procedure for each point (x,y) in the image

5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3

“Average” operation on 3 x 3 neighborhood

105


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3


0

0 0 00

106


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3


0 0 0

107


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3


0 0 0

108


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3


0 0 0

109


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3


0

0 0 00

110


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3


0

00

111


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3


112


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3


0 0 0

00

113

Filtering2D Linear spatial filteringØ Formal algebraic notation:

5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

11 12 13

21 22 23

31 32 33

2 1 23 46 11

5

f f ff f f f

f f f

é ù é ùê ú ê ú= =ê ú ê úê ú ê úë û ë û

114


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

11 12 13

21 22 23

31 32 33

2 1 23 46 11

5

f f ff f f f

f f f


11 12 13

21 22 23

31 32 33

1 1 11 11 1

1 11

9

w w ww w w w

w w w


Mask / filter mask / window / kernel

115


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

11 12 13

21 22 23

31 32 33

2 1 23 46 11

5

f f ff f f f

f f f


11 12 13

21 22 23

31 32 33

1 1 11 11 1

1 11

9

w w ww w w w

w w w


Mask / filter mask / window / kernel

Response value g à ( )3 3

1 12.8ij ij

i jg w f

= =

= =åå

116

Filtering2D Linear spatial filteringØ Formal algebraic notation at any position (x,y):

5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

( ),f x y

( ),w a b

117


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3

( ),f x y

( ),w a b

Filtered image g(x,y)

118


5 3 1 41 5 6 16 1 5 7

445

1 2 3 11 2 1 2

97

1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

1.9 2.8 2.8 3.3

2.3 3.7 3.7 4.1

1.4 2.7 2.8 3.1

2.4

2.8

1.9

0.7 1.1 1.2 2.6

1.6 2.1 2.1 3.6

2.1

3

( ),f x y

( ),w a b

Filtered image g(x,y)

Neighborhood size: 3 x 3

𝑔 𝑥, 𝑦 = & & 𝑓 𝑥 + 𝑎, 𝑦 + 𝑏 𝑤 𝑎, 𝑏,

-./,

,

0./,

119

2D Linear spatial filteringØ Formal algebraic notation at any position (x,y) and

with general size of mask w being (2r+1) x (2s+1) :

Filtering

fN1 fNM

f11 …

…

f1M

w00

wr-s wrs

w-r-s …

…

w-rs

( ) ( ), ; , with 1,..., ; 1,...,xy xyf f x y g g x y x N y M= = = =

gN1 gNM

g11 …

…

g1M

𝑔 𝑥, 𝑦 = & & 𝑓 𝑥 + 𝑎, 𝑦 + 𝑏 𝑤 𝑎, 𝑏1

-./1

2

0./2

120

Filtering2D Linear spatial filtering (average)

Original image 3 x 3 neighborhood

121



122



123



124



125


Original image 101 x 101 neighborhoodà blurring!

126

Filtering

( ) ( )( ) ( ) ( ), , , ,a b

r s

r sg x y x y f x a yw b w a bf

= =- -

= = - -* å å

Spatial filtering ~= “convolution” (finite)

à “Filtered” image (intensity values)g

à “Original” / “input” image (intensity values)f

à “Convolution” mask / filter / window / kernel (weighting coefficients)

w

127

Spatial filtering ~= “convolution” (infinite)

Filtering

( ) ( )( ) ( ) ( ), , , ,a b

g x y x y f x a y b w a bf w= =

¥ ¥

-¥ -¥

= = - -* å å

à “Filtered” image (intensity values)g

à “Original” / “input” image (intensity values)f

à “Convolution” mask / filter / window / kernel (weighting coefficients)

w

128

5 min break!

129

Filtering

( ) ( )( ) ( ) ( ), , , ,a b

g x y f w x y f x a y b w a b¥ ¥

=-¥ =-¥

= * = - -å å

Linear spatial filtering = convolution

A convolution (f*w) is nothing more than:a weighted sum of image values f

around location (x,y) where the weightsare defined by the coefficients in w

130

Linear spatial filtering = convolution

Flip sign à correlation(correlation = convolution with 180 deg rotation of kernel)

(correlation = convolution if the kernel is symmetric)

Filtering

𝑔 𝑥, 𝑦 = 𝑓°𝑤 𝑥, 𝑦 = & & 𝑓 𝑥 + 𝑎, 𝑦 + 𝑏 𝑤 𝑎, 𝑏5

-./5

5

0./5

( ) ( )( ) ( ) ( ), , , ,a b


=-¥ =-¥

= * = - -å å

131

Filtering

( ) ( )( ) ( ) ( ), , , ,a b


=-¥ =-¥

= * = - -å å

Discrete Û continuous 2D convolution

( ) ( )( ) ( ) ( ) , , , , d dg x y f w x y f x a y b w a b a b¥ ¥

-¥ -¥

= * = - -ò ò

132

Filtering

( ) ( )( ) ( ) ( )a

g x f w x f x a w a¥

=-¥

= * = -å

Discrete Û continuous 1D convolution

( ) ( )( ) ( ) ( ) dg x f w x f x a w a a¥

-¥

= * = -ò

133

Filtering

f g g f* = *

Convolution: properties

( ) ( )f g h f g h* * = * *

( ) ( ) ( )f g h f g f h* + = * + *

( ) ( ) ( ) a f g a f g f a g* = * = *

( )d df dgf g g fdr dr dr

* = * = *

à Commutative

à Associative

à Distributive

à Scaling

à Linearity

134

Filtering

f g g f* = *


( )( ) ( ) ( ) ( )( ) ( ) ( ) & ¥ ¥

=-¥ =-¥

* = - * = -å åa a

f g x f x a g a g f x g x a f a

?

135

Filtering

f g g f* = *


( )( ) ( ) ( ) ( )( ) ( ) ( ) & ¥ ¥

=-¥ =-¥

* = - * = -å åa a

f g x f x a g a g f x g x a f a

( )( ) ( ) ( )

( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( )( )

x b

x a a x

a a

x a b a x b f g x f b g x b

b a f a g x a g x a f a

g x a f a g x a f a g f x

¥

- =-¥

¥ ¥

- =-¥ - =-¥-

¥ ¥

- =-¥ =-¥

- ® Þ = - Þ * = -

® Þ - = -

= - = - = *

å

å å

å å

136

FilteringConvolution: properties (cont’d)

( )( ) ( )f x f xd* = à Delta “sifting”

137

FilteringConvolution: properties (cont’d)

( )( ) ( )f x f xd* =

à Shift property

à Delta “sifting”

( )( ) ( )( )( )( )

( )

a

a

a

f g x a f g x

f g x

with f f x a

* - = *

= *

= -

138

Filtering1D convolution: Example 1

f * g

f g

139


fg

f * g

140


fg

f * g

141


fg

f * g

142


fg

f * g

143


fg

f * g

144


fg

f * g

145


f g

f * g

146


f g

f * g

147


f g

f * g

148


f g

f * g

149


f g

f * g

150


f g

f * g

151


f g

f * g ?

Infinite peak at “0”

152


f g = d

( )( ) ( )( ) ( )x xg x f ff dº * =*

153


fg

aa af * g ?

b

154


f

aa

f * g

b

( ) for x

0 for xb a

f xa

ì <ï= í ³ïî

( )2 2a ag x x xd dæ ö æ ö= - + +ç ÷ ç ÷

è ø è ø

g

a

155


( )( ) ( ) ( ) dg sg ff x x s s¥

-¥

* º -ò

( )2 2a ag x x xd dæ ö æ ö= - + +ç ÷ ç ÷

è ø è ø( )

for x0 for xb a

f xa

ì <ï= í ³ïî

156


( )( ) ( ) ( )

( )

( ) ( )

d

d2 2

d d2 2

2 2

f g x x s s

a af x s s s s

a af x s s s f x s s s

a af

g s

x x

f

f

d d

d d

¥

-¥

¥

-¥

¥ ¥

-¥ -¥

* º -

é ùæ ö æ ö= - - + +ç ÷ ç ÷ê úè ø è øë û

æ ö æ ö= - - + - +ç ÷ ç ÷è ø è ø

æ ö æ ö= - + +ç ÷ ç ÷è ø è ø

ò

ò

ò ò

( )2 2a ag x x xd dæ ö æ ö= - + +ç ÷ ç ÷

è ø è ø( )

for x0 for xb a

f xa

ì <ï= í ³ïî

157


( )( )2 2a af g x f x f xæ ö æ ö* = - + +ç ÷ ç ÷

è ø è ø

b

( ) for x

0 for xb a

f xa

ì <ï= í ³ïî

2af xæ ö-ç ÷

è ø

0 32a

b

2af xæ ö+ç ÷

è ø

032a-

158


b

2af xæ ö-ç ÷

è ø

0 32a

b

2af xæ ö+ç ÷

è ø

032a-

b

b

32a3

2a-

12a1

2a-

159


b

b

32a3

2a-

12a1

2a-

( )g x

( )

2 for x2

3 for x and x2 2

30 for x2

ab

ag x b a

a

ì <ïïï ì ü= ³ <í í ý

î þïï

³ïî

160


f

aa

f * g ? à homework (optional)

g

b

11

b

161

Filtering1D convolution: code snippet

Input = “x”; filter = “h”; output = “y”

162

Filtering2D convolution: code snippet

Input = in;filter = kernel;output = out

geometric transformations

Documents