digital image processing chapter 11: image description and representation 12 september 2007 digital...
TRANSCRIPT
Digital Image ProcessingChapter 11:
Image Description and Representation
12 September 2007
Digital Image ProcessingChapter 11:
Image Description and Representation
12 September 2007
Image Representation and Description? Image Representation and Description?
Objective:To represent and describe information embedded in
an image in other forms that are more suitable than the image itself.
Benefits:- Easier to understand- Require fewer memory, faster to be processed- More “ready to be used”
What kind of information we can use?- Boundary, shape- Region- Texture- Relation between regions
Shape Representation by Using Chain Codes Shape Representation by Using Chain Codes
Chain codes: represent an object boundary by a connected sequence of straight line segments of specified lengthand direction.
4-directionalchain code
8-directionalchain code
Why we focus on a boundary?The boundary is a good representation of an object shapeand also requires a few memory.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Examples of Chain Codes Examples of Chain Codes
Object boundary
(resampling)
Boundaryvertices
4-directionalchain code
8-directionalchain code
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
The First Difference of a Chain Codes The First Difference of a Chain Codes Problem of a chain code: a chain code sequence depends on a starting point.Solution: treat a chain code as a circular sequence and redefine thestarting point so that the resulting sequence of numbers forms an integer of minimum magnitude.
The first difference of a chain code: counting the number of directionchange (in counterclockwise) between 2 adjacent elements of the code.
Example:
1
2
3
0
Example: - a chain code: 10103322 - The first difference = 3133030 - Treating a chain code as a circular sequence, we get the first difference = 33133030
Chain code : The first difference 0 1 1 0 2 2 0 3 3 2 3 1 2 0 2 2 1 3 The first difference is rotational
invariant.
Polygon Approximation Polygon Approximation
Object boundary Minimum perimeterpolygon
Represent an object boundary by a polygon
Minimum perimeter polygon consists of line segments that minimize distances between boundary pixels.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Polygon Approximation:Splitting Techniques Polygon Approximation:Splitting Techniques
1. Find the line joiningtwo extreme points
0. Object boundary 2. Find the farthest pointsfrom the line
3. Draw a polygon(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Distance-Versus-Angle Signatures Distance-Versus-Angle Signatures
Represent an 2-D object boundary in term of a 1-D function of radial distance with respect to .
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Boundary Segments Boundary Segments
Concept: Partitioning an object boundary by using vertices of a convex hull.
Convex hull (gray color)
Object boundary
Partitioned boundary
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Input : A set of points on a cornea boundaryOutput: A set of points on a boundary of a convex hull of a cornea
1. Sort the points by x-coordinate to get a sequence p1, p2, … ,pnFor the upper side of a convex hull2. Put the points p1 and p2 in a list Lupper with p1 as the first point3. For i = 3 to n4. Do append pi to Lupper
5. While Lupper contains more than 2 points and the last 3 points in Lupper do not make a right turn
6. Do delete the middle point of the last 3 points from Lupper
TurnRightOK!
TurnRightOK!
TurnLeftNOK!
Convex Hull Algorithm Convex Hull Algorithm
For the lower side of a convex hull7. Put the points pn and pn-1 in a list Llower with pn as the first point8. For i = n-2 down to 19. Do append pi to Llower
10. While Llower contains more than 2 points and the last 3 points in Llower do not make a right turn11. Do delete the middle point of the last 3 points from Llower
12. Remove the first and the last points from Llower
13. Append Llower to Lupper resulting in the list L14. Return L
TurnRightOK!
TurnRightOK!
TurnLeftNOK!
Convex Hull Algorithm (cont.) Convex Hull Algorithm (cont.)
Skeletons Skeletons
Obtained from thinning or skeletonizing processes
Medial axes (dash lines)
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Thinning Algorithm Thinning Algorithm
Neighborhoodarrangementfor the thinningalgorithm
Concept: 1. Do not remove end points2. Do not break connectivity3. Do not cause excessive erosion
Apply only to contour pixels: pixels “1” having at least one of its 8neighbor pixels valued “0”
p2p9 p3
p8
p7
p1 p4
p6 p5
Let98321)( pppppN
T(p1) = the number of transition 0-1 in the ordered sequence p2, p3, …
, p8, p9, p2.
Notation:
=Let
00 1
1
1
p1 0
0 1
Example
N(p1) = 4T(p1) = 3
Thinning Algorithm (cont.) Thinning Algorithm (cont.)
Step 1. Mark pixels for deletion if the following conditions are true. a)
b) T(p1) =1c)d)
6)(2 1 pN
0642 ppp0864 ppp
p2p9 p3
p8
p7
p1 p4
p6 p5
Step 3. Mark pixels for deletion if the following conditions are true. a)
b) T(p1) =1c)d)
6)(2 1 pN
0842 ppp0862 ppp
Step 4. Delete marked pixels and repeat Step 1 until no change occurs.
(Apply to all border pixels)
Step 2. Delete marked pixels and go to Step 3.
(Apply to all border pixels)
Example: Skeletons Obtained from the Thinning Alg. Example: Skeletons Obtained from the Thinning Alg.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Skeleton
Boundary Descriptors Boundary Descriptors
1. Simple boundary descriptors: we can use
- Length of the boundary- The size of smallest circle or box that can totally enclosing the object
2. Shape number
3. Fourier descriptor
4. Statistical moments
Shape Number Shape Number
Shape number of the boundary definition: the first difference of smallest magnitude
The order n of the shape number: the number of digits in the sequence
1
2
3
0
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Shape Number (cont.) Shape Number (cont.)
Shape numbers of order 4, 6 and 8
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Example: Shape Number Example: Shape Number
Chain code: 0 0 0 0 3 0 0 3 2 2 3 2 2 2 1 2 1 1
First difference:3 0 0 0 3 1 0 3 3 0 1 3 0 0 3 1 3 0
Shape No.0 0 0 3 1 0 3 3 0 1 3 0 0 3 1 3 0 3
1. Original boundary
2. Find the smallest rectanglethat fits the shape
3. Create grid4. Find the nearest Grid.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Fourier Descriptor Fourier Descriptor Fourier descriptor: view a coordinate (x,y) as a complex number (x = real part and y = imaginary part) then apply the Fourier transform to a sequence of boundary points.
)()()( kjykxks
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
1
0
/2)(1
)(K
k
KukeksK
ua Fourier descriptor :
1
0
/2)(1
)(K
k
KukeuaK
ks
Let s(k) be a coordinate of a boundary point k :
Reconstruction formula
Boundarypoints
Example: Fourier Descriptor Example: Fourier Descriptor
Examples of reconstruction from Fourier descriptors
1
0
/2)(1
)(ˆP
k
KukeuaK
ks
P is the number of Fourier coefficients used to reconstruct the boundary
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Fourier Descriptor Properties Fourier Descriptor Properties
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Some properties of Fourier descriptors
Statistical Moments Statistical Moments
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
1. Convert a boundary segment into 1D graph2. View a 1D graph as a PDF function3. Compute the nth order moment of the graph
1
0
)()()(K
ii
nin rgmrr
Definition: the nth moment
where
1
0
)(K
iii rgrm
Boundarysegment 1D graph
Example of moment:The first moment = meanThe second moment = variance
Regional Descriptors Regional Descriptors
Purpose: to describe regions or “areas”
1. Some simple regional descriptors- area of the region- length of the boundary (perimeter) of the region- Compactness
)(
)(2 RP
RAC
where A(R) and P(R) = area and perimeter of region R
2. Topological Descriptors3. Texture4. Moments of 2D Functions
Example: a circle is the most compact shape with C = 1/4
Example: Regional Descriptors Example: Regional Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
White pixels represent “light of the cities”
% of white pixelsRegion no. compared to the total white pixels
1 20.4%2 64.0%3 4.9%4 10.7%
Infrared image of America at night
Topological Descriptors Topological Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Use to describe holes and connected components of the region
Euler number (E):
HCE
C = the number of connected componentsH = the number of holes
Topological Descriptors (cont.) Topological Descriptors (cont.)
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
E = -1
E = 0 Euler Formula
EHCFQV
V = the number of verticesQ = the number of edgesF = the number of faces
E = -2
Example: Topological Descriptors Example: Topological Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Original image:Infrared imageOf Washington D.C. area
After intensityThresholding(1591 connectedcomponents with 39 holes)Euler no. = 1552
The largestconnected
area (8479 Pixels)
(Hudson river)
After thinning
Texture Descriptors Texture Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Purpose: to describe “texture” of the region.
Examples: optical microscope images:
Superconductor(smooth texture)
Cholesterol(coarse texture)
Microprocessor(regular texture)
A
B C
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Statistical Approaches for Texture Descriptors Statistical Approaches for Texture Descriptors
We can use statistical moments computed from an image histogram:
1
0
)()()(K
ii
nin zpmzz
where
1
0
)(K
iii zpzm
z = intensityp(z) = PDF or histogram of z
Example: The 2nd moment = variance measure “smoothness” The 3rd moment measure “skewness” The 4th moment measure “uniformity” (flatness)
ABC
Divide into areasby angles
Fourier Approach for Texture Descriptor Fourier Approach for Texture Descriptor
Originalimage
Fouriercoefficient
image
FFT2D+FFTSHIFT
Sum all pixelsin each area
Divide into areasby radius
Sum all pixelsin each area
0
)()( rSrS
0
1
)()(R
rrSS
Concept: convert 2D spectrum into 1D graphs
Fourier Approach for Texture Descriptor Fourier Approach for Texture Descriptor
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Originalimage
2D Spectrum(Fourier Tr.)
S(r) S()
Anotherimage
Another S()
Moments of Two-D Functions Moments of Two-D Functions
x y
qppq yxfyxm ),(
00
10
m
mx
00
01
m
my
x y
qppq yxfyyxx ),()()(
0000 m 01001
1011011111 mymmxm
102020 mxm 010202 mym
012
20112121 22 mxmymxm
102
02111212 22 mymxmym
102
203030 23 mxmxm
012
020303 23 mymym
The moment of order p + q
The central moments of order p + q
Invariant Moments of Two-D Functions Invariant Moments of Two-D Functions
00
pqpq where 1
2
qp
The normalized central moments of order p + q
Invariant moments: independent of rotation, translation, scaling, and reflection
02201 211
202202 4
20321
212303 33 2
03212
12304
2
03212
123003210321
20321
21230123012305
33
33
0321123011
20321
2123002206
4
Example: Invariant Moments of Two-D Functions Example: Invariant Moments of Two-D Functions
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
1. Original image 2. Half size 3. Mirrored
4. Rotated 2 degree 5. Rotated 45 degree
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Invariant moments of images in the previous slide
Example: Invariant Moments of Two-D Functions Example: Invariant Moments of Two-D Functions
Invariant moments are independent of rotation, translation,scaling, and reflection
Principal Components for Description Principal Components for Description
Purpose: to reduce dimensionality of a vector image while maintaining information as much as possible.
Let
Mean:
Covariance matrix
K
k
Txx
Tkk
Txxx K
E1
1}))({( mmxxmxmxC
Tnxxx ]...[ 21x
K
kkx K
E1
1}{ xxm
Hotelling transformation Hotelling transformation
1
1
1
......0
............
0...0
0...0
yC
Let )( xmxAy
Then we get
0}{ ym Ey
Txy AACC
and
Then elements of are uncorrelated. The component of y with the largest is called the principal component.
)( xmxAy
Where A is created from eigenvectors of Cx as followsRow 1 contain the 1st eigenvector with the largest eigenvalue.Row 2 contain the 2nd eigenvector with the 2nd largest eigenvalue.
….
Eigenvector and Eigenvalue Eigenvector and Eigenvalue
Eigenvector and eigenvalue of Matrix C are defined as
Let C be a matrix of size NxN and e be a vector of size Nx1.If
eCe
Where is a constant
We call e as an eigenvector and as eigenvalue of C
Example: Principal ComponentsExample: Principal Components
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
6 spectral imagesfrom an airborneScanner.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Example: Principal Components (cont.)Example: Principal Components (cont.)
Component 1 32102 931.43 118.54 83.885 64.006 13.40
Example: Principal Components (cont.)Example: Principal Components (cont.)
Original image After Hotelling transform
Principal Components for Description Principal Components for Description
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Relational Descriptors Relational Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Relational Descriptors Relational Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Relational Descriptors Relational Descriptors
Relational Descriptors Relational Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Relational Descriptors Relational Descriptors
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.
Structural Approach for Texture Descriptor Structural Approach for Texture Descriptor
(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.