representation and description - sharifee.sharif.edu/~dip/files/representationforview.pdf ·...

ee.sharif.edu/~dip

E. Fatemizadeh, Sharif University of Technology, 2011

1

Digital Image Processing

Representation and Description

1

• Representation and Description

– Representing regions in 2 ways:

• Based on their external characteristics (its boundary): – Shape characteristics

• Based on their internal characteristics (its region): – Regional properties: color, texture, and …

• Both

ee.sharif.edu/~dip


2



2

• Representation and Description

– Describes the region based on a selected representation:

• Representation boundary or textural features

• Description length, orientation, the number of concavities in the boundary, statistical measures of region.

ee.sharif.edu/~dip


3



3

• Invariant Description:

– Size (Scaling)

– Translation

– Rotation

ee.sharif.edu/~dip


4



4

• Boundary (Border) Following:

– We need the boundary as a ordered sequence of points.

ee.sharif.edu/~dip


5



5

• Moore Boundary Tracking Algorithm:

1. Let the starting point, b0 , be the uppermost, leftmost point in the image that is labeled 1. Denote by c0 the west neighbor of b0 . Clearly c0 always will be a background point. Then, examine the 8 neighbors of b0 , starting at c0 and proceeding in clockwise direction. Let b1 denote the first pixel encountered whose value is 1, and let c1 be the points immediately preceding b1 in the sequence. Store the location of b0 and b1.

2. Let b=b1 and c=c1.

3. Let the 8 neighbors of b , starting at c and proceeding in a clockwise direction be denoted by n1,n2,... , nk . Find the first nk whose value is 1.

4. Let b=nk and c=nk-1.

5. Repeat steps 3 and 4 until b=b0 and the next boundary point found is b1.

ee.sharif.edu/~dip


6



6

• Example:

ee.sharif.edu/~dip


7



7

• Freeman Chain Code:

– Code the 4 or 8 connectivity

ee.sharif.edu/~dip


8



8

• Chain Code Problems:

– Long Code

– Low noise Robustness • Solution: Resampling

– Starting Point: • Solution: Rotary/Circular shift until forms a minimum integer

– 10103322 01033221

– Angle normalization: • First difference (Counterclockwise)

– 10103322 3133030 or 33133030 (transition between last and first)

• Useful for integer multiple of used chain code (45° or 90°)

ee.sharif.edu/~dip


9



9

• Example: – Resampling

– 4 and 8 chain codes

ee.sharif.edu/~dip


10



10

• Example:

ee.sharif.edu/~dip


11



11

• Polygon Approximation:

– Minimum-Perimeter Polygons • Read Pages 801-807.

ee.sharif.edu/~dip


12



12

• Example (Cont.):

ee.sharif.edu/~dip


13



13

• Example:

ee.sharif.edu/~dip


14



14


– Merging: • Start from a seed point

• Continue on a line based on local average error (e.g. linear regression)

• Stop if error exceeds a threshold

• Continue from the last point

• ....

– No guarantee for corner detection

ee.sharif.edu/~dip


15



15


– Splitting: • Segments to two parts based on a criteria (e.g. maximum internal

distance)

• Check each segment for splitting based on another criteria (e.g. linearity error)

• ….

ee.sharif.edu/~dip


16



16

• Signatures:

– A 1-D functional representation of a boundary • Distance vs. Angle (in the polar representation):

– Invariant to translation

– Non-Invariant to rotation (may be achieved by start point selection)

» Farthest point from centroid

» The point on eigen axis

» Use chain code solution for the start point

• Line tangent angle

• Histogram of tangent angle

ee.sharif.edu/~dip


17



17

• Example:

ee.sharif.edu/~dip


18



18

• Example:

ee.sharif.edu/~dip


19



19

• Boundary Segments:

– Convex Hull: Smallest convex set containing the S

– Convex Deficiency: The set difference D=H-S

– Problem of noise: • Smoothing (K-neighbour average)

• Polygon approximation

– Convex Hull of Polygon

ee.sharif.edu/~dip


20



20

• Boundary Segments:

• Skeletonization / Thinning:

– Medial Axis Transform (MAT)

• Point p is belong to medial (Region R and Border B): – Has more than one closest neighbor in B

– Chapter 9 and Page 813

ee.sharif.edu/~dip


21



21

• Example of Thinning:

ee.sharif.edu/~dip


22



22

• Boundary Descriptor:

– Simple one:

• Diameter: – Lead to Major axis and minor axis (Perpendicular to major)

– Basic Rectangle

• Curvature

,

Diam max ,i ji j

B D p q

1.5

2 2

1 x y x yk

R x y

ee.sharif.edu/~dip


23



23

• Shape Number

– Smallest integers of first difference circular chain code.

ee.sharif.edu/~dip


24



24

• Example:

ee.sharif.edu/~dip


25



25

• Fourier Descriptors:

1

1

0

1

0

0

exp 2

1exp 2

1ˆ exp 2

K

k

K

u

u

P

s k x k jy k

uka u s k j

K

uks k a u j

K K

uks k a

KPu j

ee.sharif.edu/~dip


26



26

• Example:

ee.sharif.edu/~dip


27



27

• Problem of invariance:

– Rotation

– Normalization will solve!

1

0

1exp 2

Kj j

r

k

uka u s k e j a u e

K K

2

0

ˆ , , ,....s s s

s

s s is ii

i

a u a u a ua u

a u a u a u

ee.sharif.edu/~dip


28



28


– Translation

– Translation to centroid will solve!

xy x x

t xy x y

t xy

j

s k s k x k j y k

a u u

ee.sharif.edu/~dip


29



29


– Scaling

– Normalization will solve

s ss k s k a u a u

2

0

ˆ , , ,....s s s

s

s s is ii

i

a u a u a ua u

a u a u a u

ee.sharif.edu/~dip


30



30


– Starting Point

– Normalization will solve

0 0 0

0

,

2exp

p

p

s k s k k x k k y k k

j k ua u a u

K

2

0

ˆ , , ,....p p p

p

p p ip ii

i

a u a u a ua u

a u a ua u

ee.sharif.edu/~dip


31



31

• Properties in a single table:

ee.sharif.edu/~dip


32



32

• Statistical Moments:

– Boundary Representation

1

0

1

0

,

v as random variable

A

i i

i

An

n i i

i

r v g r

m v p v

v v m p v

1

0

1

0

,

normalized ( ) as histogram

A

i i

i

An

n i i

i

r v g r

g r

m r g r

v r m g r

ee.sharif.edu/~dip


33



33

• Regional Descriptor:

– The simple one: • Area (Number of pixels)

• Perimeter (Length of boundary)

• Compactness (Perimeter2/Area)

• Circularity: Ratio of the area to the area of a circle with same perimeter

• Mean, median, max, min, ratio pixels above/below … from intensity data.

ee.sharif.edu/~dip


34



34

• Example:

– Normalized area: • Ratio of light pixels to total light pixels

ee.sharif.edu/~dip


35



35

• Topology Descriptor:

– Number of holes (H): • Invariants to several operators.

ee.sharif.edu/~dip


36



36


– Number of connected components (C) : • Invariants to several operators.

ee.sharif.edu/~dip


37



37


– Euler number (E=C-H) : • Invariants to several operators.

0 -1

ee.sharif.edu/~dip


38



38


– Polygonal net: • V: # of vertices (7)

• Q: # of edges (11)

• F: # of faces (2)

• E=C-H=V-Q+F

– C=1 , H=3

– E=-2

ee.sharif.edu/~dip


39



39

Original Threshold

Largest Connected Region Skeleton

• Example:

ee.sharif.edu/~dip


40



40

• Texture:

– No formal definition

Smooth Coarse Regular

ee.sharif.edu/~dip


41



41

• Statistical Approaches – 1st order grey level statistics

• From normalised histogram – One pixel gray level repeat n times

– 2nd order grey level statistics • From GLCM (Grey Level Co-ocurrence Matrix)

– Repeatation of two pixels in a pre-defined neighbourhood

• Needs: – A Positioning Operator, P.

– GLCM(i,j): # of times that points with gray level Zi occure relative to points with gray level Zj

ee.sharif.edu/~dip


42



42

• Texture feature from 1st order statistics:

1 1

0 0

2

3

4

12

0

1

0

,

11 : Gray Level Contrast (Normalized)

1

:Skewness

: Kurtosis, Flatness

: Uniformity

log : Entropy

L Ln

n i i i i

i i

z

L

i

i

L

i i

i

z z m P z m z P z

R z

z

z

U z p z

e z p z p z

ee.sharif.edu/~dip


43



43

• Example: Smooth Coarse Regular

ee.sharif.edu/~dip


44



44

• Problem with 1st order histogram

– Lack of spatial information

ee.sharif.edu/~dip


45



45

• Gray Level Co-Occurence Matrix (GLCM):

1 2 3

0 0 0 1 2

1 1 1 1

, 0 1 2 ,2 2 1 0

1 1 0 2

0 0 1 1

0 0 0

0 1 1 : one pixel to right one below

0 1 0

2

0

0

0

0

1 2 11

2 3 2 2 3 216

0 2 0 0 2 0

4 4

z z z z

G = P

ee.sharif.edu/~dip


46



46


ee.sharif.edu/~dip


47



47


1 1

1 1 1 1

2 22 2

1 1 1 1

: # of gray levels

,

,

g g

g g g g

g g g g

g

N N

ij

ij ij

i j

N N N N

r ij c ij

i j j i

N N N N

r r ij c c ij

i j j i

N

gn g p

n

m i p m j p

i m p j m p

ee.sharif.edu/~dip


48



48

• Texture feature from GLCM

,

2

2

2

( ) : Maximum probability (G1)

: Correlation (G2)

( ) : Contrast (G3)

: Uniformity

: Homogeneity1

log ( ) : Entropy

iji j

r c

i j r c

ij

i i

ij

i j

ij

i i

ij ij

i j

Max p

i m j m

i j p

p

p

i j

p p

ee.sharif.edu/~dip


49



49

• Example:

– One pixel immediately to he right

ee.sharif.edu/~dip


50



50

• Example:

ee.sharif.edu/~dip


51



51

• Effect of Horizontal offset: – Correlation index

– Horizontal distance between neighbors

Noisy Sin Circuit board

ee.sharif.edu/~dip


52



52

• Spectral approaches:

– Peaks and Frequencies related to periodicity:

– S(r,θ), Sr(θ),Sθ(r):

0

1

R

r

r

S r S r

S r S

S S

ee.sharif.edu/~dip


53



53

• Spectral approaches:

ee.sharif.edu/~dip


54



54

• Spectral approach:

ee.sharif.edu/~dip


55



55

• Moments of Image as a 2D pdf:

10 01

00 00

Moments of order of (p+q)

,

Central moments of order of (p+q)

,

;

p q

pq

p q

pq

m x y f x y dxdy

x x y y f x y dxdy

m mx y

m m

ee.sharif.edu/~dip


56



56

• Digital Data:

10 01

00 00

00 00

1 0 1010 10 00

00

0 1 0101 01 00

00

Moments of order of (p+q)

( , )

( ) ( ) ( , ), ;

( , )

( ) ( ) ( , ) 0

( ) ( ) ( , ) 0

p q

pq

x y

p q

pq

x y

x y

x y

x y

m x y f x y

m mx x y y f x y x y

m m

f x y m

mx x y y f x y m m

m

mx x y y f x y m m

m

ee.sharif.edu/~dip


57



57

• Moments:

• A set of invariant moments (7 by Hu)

11 20 02

21 12 30 03

00

2nd ordens centrale momenter 2

, ,

3rd ordens centrale momenter 3

, , ,

Normalized central momens

and 12

pq

pq

p q

p q

p q

ee.sharif.edu/~dip


58



58

• Hu Moments:

1 20 02

2 2

2 20 02 11

2 2

3 30 12 12 03

2 2

4 30 12 21 03

2 2

5 30 12 30 12 30 12 21 03

2 2

21 03 21 03 30 12 21 03

2 2

6 20 02 30 12 21 03 11 30 12 21 0

4

3 3

3 3

3 3

4

3

2 2

7 21 03 30 12 30 12 30 12

2 2

30 12 21 0

2 2

8 11 30 12 0

3 30 12

3 21 20 02 30

2

12 03

1 0

21

3

3 3

3 3

ee.sharif.edu/~dip


59



59

• Example:

ee.sharif.edu/~dip


60



60

• Results of invariant moments:

ee.sharif.edu/~dip


61



61

• Complex Zernike Moments:

• Where Vmn are Zernike polynomials:

2 2 1

0

1, ,

, , ,

mn mn

x y

mA f x y V x y dxdy

m n m n even n m

2

0

2

, , , : over unit disc

1 , , ,

!, , ,

! ! !2 2

jn

mn mn

m n

s

mn

s

m s

V r R r e r

R r F m n s r

m sF m n s r r

m n m ns s s

ee.sharif.edu/~dip


62



62

• Some Zernike Polynomials:

, ,

0,0

1,1

2

2,0

2

2,2

3

3,1

3

3,3

1

2 1

3 2

m n m nR R

R

R r

R r

R r

R r r

R r

ee.sharif.edu/~dip


63



63

• How to Compute Zernike Polynomials:

– Scale and Translation invariance transform: • Shift to center of gravity (m10 = m01= 0)

• Fix m00 at a predetermined value (= b):

– Map Image to the unit disc using polar coordinates.

– The center of the image is the origin of the unit disc

– Those pixels falling outside the unit disc are not used in the computation.

2 2 1, tany

r x yx

00

, , ,x y

g x y f x ym

b

ee.sharif.edu/~dip


64



64

• How to Compute Zernike Polynomials:

, ,

1, ,m n m n

x y

mA g x y V x y

ee.sharif.edu/~dip


65



65

• Legendre Moments:

1 1

, 0

1 1

2

2 1 2 1, ,

4

11

2 !

m n m n

kk

k k k

m nP x P y dxdy m n

dP r x

k dx

ee.sharif.edu/~dip


66



66

• Fourier-Mellin Moments:

– Fourier-Mellin Transform:

– Fourier-Mellin Moments:

2

,

0 0

( , ) , ,m in

m nF r f r e rdrd m n

2

,

0 0

1

0

1( , ) , ,

2

: orthogonal polynomial of degree m over the raneg 0 1

in

m n m

m

m

m l m

f r Q r e rdrd m na

Q r r

Q r Q r rdr a m k

ee.sharif.edu/~dip


67



67

• Principal Component Analysis:

– Consider a multi (value/Channel/Spectral) images:

1 2

1

1

1

1 2 1 2

1, : # of observations

1

: Real Positive Definiter Matrix

: 0, ,

,

T

n

K

x k

k

KT T T

x x x k k x x

k

x

x i i j

T T

n n

x x x

E KK

EK

v v v v i j

v v v A A

x

m x x

C x m x m x x m m

C

C

A

ee.sharif.edu/~dip


68



68

• Karhunen–Loève or Hotelling Transform:

– Analysis:

– Complete Synthesis:

– Optimal Synthesis:

1 2, ndiag T

X y y xy = A x -m m = 0 C = AC A

T X

x = A y m

2

1 1 1

Form Using eigenvectors correspondinf to -largest eigenvalues

ˆ

ˆ

PCA Analysis

k

T

k

n k n

rms j j j

j j j k

k k

e E

X

A

x = A y m

x x

ee.sharif.edu/~dip


69



69

• Example:

ee.sharif.edu/~dip


70



70

• Data Preparation:

ee.sharif.edu/~dip


71



71

• Eigenvalues Analysis:

ee.sharif.edu/~dip


72



72

• PCA Analysis:

– Eigneimages

ee.sharif.edu/~dip


73



73

• Example: – Using 2 components

ee.sharif.edu/~dip


74



74

• Example:

– Difference

ee.sharif.edu/~dip


75



75

• PCA in Boundary description:

ee.sharif.edu/~dip


76



76

• PCA in Boundary description:

representation and description - sharifee.sharif.edu/~dip/files/representationforview.pdf ·...

Documents