chapter 18 image segmentation 18.1 introduction – computer vision; image understanding;...

106

Upload: ursula-tyler

Post on 26-Dec-2015

224 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:
Page 2: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Chapter 18 Image Chapter 18 Image SegmentationSegmentation

18.1 Introduction– Computer vision; Image understanding;

Statistical pattern recognition– The three steps of pattern recognition: image

segmentation (object isolation); feature extraction; classification.

Image segmentation

Feature extraction

ClassificationInput

image

Object

image

Feature

vector

Object

class

Page 3: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

An example of pattern An example of pattern recognitionrecognition

The feature space

Lemons

CherriesApples

Grape

Redness

Diameter

Page 4: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Pattern recognition system Pattern recognition system designdesign

Step Function

Object isolation design Select the scene segmentation algorithm that will isolate the individual objects in the image

Feature selection Decide which properties of the objects best distinguish the object type and how to measure these

Classifier design Establish the classification algorithm and select the type of classifier to be used

Classifier training Fix the various adjustable parameters in the classifier to suit the objects being classified

Performance evaluation Estimate the expected rates of the various possible misclassification errors

Page 5: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.2 The image segmentation 18.2 The image segmentation processprocess

Image segmentation is a process that partitions a digital image into disjoint regions (a connected set of pixels).

Four connectivity and eight connectivity. Image segmentation approaches

– Region approach. Assign each pixel to a particular object or region.

– Boundary approach. Only locate the boundaries that exist between the regions.

– Edge approach. Identify edge pixels and then link them together to form the required boundaries.

Page 6: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3 Image segmentation by 18.3 Image segmentation by thresholdingthresholding

In thresholding segmentation scheme, all pixels with gray level at or above the threshold are assigned to the object, while all pixels with gray level below the threshold fall outside the object.

The boundary is that set of interior points, each of which has at least one neighbor outside the object

Page 7: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Image segmentation by Image segmentation by thresholdingthresholding

Global thresholding– In global thresholding, the value of the

threshold gray level is held constant throughout the image.

– Global thresholding work well for cases where the background fray level is reasonably constant, and the objects all have approximately equal contrast above the background.

Page 8: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Image segmentation by Image segmentation by thresholdingthresholding

Adaptive thresholding– In many cases, the background gray level is not

constant, and the contrast of objects varies within the image.

– The adaptive thresholding scheme makes the threshold gray level slowly varies with position in the image.

Page 9: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Segmentation with different Segmentation with different thresholdsthresholds

Page 10: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.3 Optimal threshold 18.3.3 Optimal threshold selectionselection

The exact value of the threshold gray level can have considerable effect on the boundary position and overall size of the extracted object.Thus the area measurement is sensitive to the threshold gray level.

An optimal threshold selection method is needed to make the measured area of the object is not insensitive to small changes of the threshold.

Page 11: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.3 Optimal threshold 18.3.3 Optimal threshold selectionselection

18.3.3.1 Histogram techniques– An object residing on a contrasting background has a

bimodel gray level histogram.– The area of an object defined by the gray level

threshold T is

– Therefore, placing the threshold at the dip in the histogram minimizes the sensitivity of the area measurement to small errors in threshold selection

T

dDDHA )(

Page 12: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

The bimodal histogramThe bimodal histogram

T TT

T

dDDHA )(

A

)(DH

D

Page 13: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.3 Optimal threshold 18.3.3 Optimal threshold selectionselection

When the threshold is chosen at the dip of the histogram, the sensitivity of the area measurement is minimized.

When the image is contaminated by noise, the dip of histogram may shift, the histogram can be smoothed before thresholding.

A more reliable approach is to choose a fixed position between two peaks as the threshold.

Page 14: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.3 Optimal threshold 18.3.3 Optimal threshold selection selection

Methods to enhance the dip in the histogram – The histogram of pixels having a relative high

gradient magnitude may make the dip more accessible.

– Dividing the histogram by the average gradient of pixels to enhance the dip.

– Laplacian filtering, followed by smoothing and thresholding at a gray level of zero or slightly above.

Page 15: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.4 The analysis of spots18.3.4 The analysis of spots

– Definitions An image contains a single spot

can be expressed in polar coordinates centered upon

by Suppose the spot is a monotone spot, then

If all contours of a monotone spot are circles centered on , then the spot is called a concentric circular spot (CCS).

),( yxB

),( 00 yx

),( 00 yx

),( rBp

1221 if ),,(),( rrrBrB pp

),( 00 yx

Page 16: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.4 The analysis of spots 18.3.4 The analysis of spots

– For a CCS, the function is independent of , and is called the spot profile function.

– The gray level threshold is chosen as the gray level at the inflection point (the point of maximum slope) of the spot profile function

– The gray level threshold is chosen as the gray level at the point with maximum magnitude of the second derivative.

),( rBp

Page 17: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.4 The analysis of spots18.3.4 The analysis of spots

– The threshold area function and the perimeter function are functions of the threshold .

– Two spots are p-equivalent if they have identical perimeter functions and H-equivalent if they have identical histograms.

– H-equivalent spots have identical threshold area functions.

)(TA

)(Tp

T

Page 18: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.4 The analysis of spots 18.3.4 The analysis of spots

18.3.4.2 The histogram and the profile– When thresholding a CCS image at gray levels

and , the area of the annular ring between two concentric circles is approximately

D DD

rrrrrA 2)( 22

rr

A 2 r

r

Page 19: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.4 The analysis of spots18.3.4 The analysis of spots

The histogram is

Where r is the inverse function of ,

The area-derived profile We can compute the area derived profile of a

CCS by integrating the histogram to obtain the area function, taking the square root and then the inverse function.

drrdB

r

rD

rA

D

ADH

pDD

B /)(

2

/

/limlim)(

00

)(rBD p

)()( 1 DBDr p

2/12/1

)(1

)(1

)(

T B dDDHTATR

Page 20: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.4 The analysis of spots18.3.4 The analysis of spots

– The perimeter-derived profile

If the perimeter function is known, the profile may be obtained by the inverse of equation

Noncircular and noise spots– For noncircular and noise spots, its histogram could be

used to obtain the profile of the H-equivalent CCS and select the threshold gray level that maximizes the slope (corresponding to the inflection point) at the boundary.

)(2

1)( TPTR

Page 21: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.4 The analysis of spots 18.3.4 The analysis of spots

– Sometimes, the perimeter function of the noncircular and noisy spots could be measured and used to determine the profile of the p-equivalent CCS.

– The area-derived profile is easier to compute and has superior performance.

– Threshold selection based on the area-derived profile and maximum magnitude of the second derivative is the best one.

Page 22: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.5 Average boundary 18.3.5 Average boundary gradientgradient

For highly noncircular spots, we can compute the average boundary gradient around the boundary as a function of the threshold gray level.

The threshold gray level is chosen to maximize the average boundary gradient function.

r

DD

0lim|| B rDpA )(

)(

)()(lim||

0 DH

DpDp

A

D

BD

B

Page 23: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.6 Objects of general 18.3.6 Objects of general shapeshape

The threshold gray level T can be established in four ways:

1. Select T as a local minimum in the histogram.2. Select T corresponding to the inflection point

in the H-equivalent CCS profile function.3. Select T to maximize the average boundary

gradient.4. Select T corresponding to the inflection point

in the p-equivalent CCS profile function.

Page 24: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.7 The watershed 18.3.7 The watershed algorithmalgorithm

– The image is initially thresholded at a low gray level that segments the image into the proper number of projects but with boundaries lying within the objects.

– Then the threshold is raised gradually one gray level at a time. The objects’ boundaries will expand as the threshold increases.

– The points of first contact become the final boundaries between adjacent objects.

Page 25: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.3.7 The watershed 18.3.7 The watershed algorithmalgorithm

Page 26: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.4 Gradient-based 18.4 Gradient-based segmentation methodssegmentation methods

– 18.4.1 Boundary tracking The pixel with highest gray level in the gradient

image is chosen as the initial boundary point. The subsequent boundary points are determined

iteratively by a procedure shown in the figure

Next candidates

Current boundary point

Last boundary point

Page 27: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.4 Gradient-based 18.4 Gradient-based segmentation methods segmentation methods

– Boundary tracking bug Reducing the effects of noise by local average the

gradient image while boundary tracking

Page 28: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.4.2 Gradient image 18.4.2 Gradient image thresholdingthresholding

Application of the watershed algorithm to the gradient image.

Threshold

Object

T1

T2

Gradient

Page 29: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.4.3 Laplacian Edge 18.4.3 Laplacian Edge DetectionDetection

– The Laplacian is a scalar second-derivative operator defined as

– The Laplacian can be approximated by two convolution kernels

),(),(),(2

2

2

22 yxf

yyxf

xyxf

0 -1 0

-1 4 -1

0 -1 0

-1 -1 -1

-1 8 -1

-1 -1 -1

Page 30: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.4.3 Laplacian Edge 18.4.3 Laplacian Edge DetectionDetection

The Laplacian will produce a abrupt zero-crossing at an edge.

Page 31: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.4.3 Laplacian Edge 18.4.3 Laplacian Edge DetectionDetection

– The binary image that results from thresholding a Laplacian-filtered image at zero gray level will produce closed, connected contours.

– A Gaussian lowpass filter is used to smooth the image before apply Laplacian filter. This can combine them into a so-called Laplacian of Gaussian kernel:

2

22

2

22

22

22

22

22

21

1

2

1

yxyx

eyx

e

Page 32: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.5 Edge detection and 18.5 Edge detection and linkinglinking

Edge image (or edge map). An image in which gray level reflects how strongly each corresponding pixel meets the requirements of an edge pixel is called an edge image or edge map.

Binary edge image; directional edge image. Edge point linking is the process of associating

nearby edge points so as to create a closed, connected boundary.

Page 33: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.5.1 Edge detection18.5.1 Edge detection

There is a gray level transition on an edge. The magnitude and direction of the gradient vector are the slope and direction of such transition.

Edge detection operators are convolution kernels of directional derivative masks.

The Roberts Edge operator 2/122 }])1,(),1([])1,1(),({[),( yxfyxfyxfyxfyxg

Page 34: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.5 Edge Detection and 18.5 Edge Detection and LinkingLinking

The Sobel edge operator– Each point in the image is convolved with both

the following two kernels which corresponds to a vertical edge and a horizontal edge. The maximum value of the two convolutions is taken as the output value for that pixel. The result is an edge magnitude image

-1 -2 -1

0 0 0

1 2 1

-1 0 1

-2 0 2

-1 0 1

Page 35: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

The Prewitt Edge operatorThe Prewitt Edge operator

Convolution kernels as:

-1 -1 -1

0 0 0

1 1 1

1 0 -1

1 0 -1

1 0 -1

Page 36: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.5 Edge detection and 18.5 Edge detection and linkinglinking

– The Kirsch edge operator

5 5 5

-3 0 -3

-3 -3 -3

-3 5 5

-3 0 5

-3 -3 -3

-3 -3 5

-3 0 5

-3 -3 5

-3 -3 -3

-3 0 5

-3 5 5

-3 -3 -3

-3 0 -3

5 5 5

-3 -3 -3

5 0 -3

5 5 -3

5 -3 -3

5 0 -3

5 -3 -3

5 5 -3

5 0 -3

-3 -3 -3

Page 37: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Edge detector performanceEdge detector performance

The Roberts operator, being two by two, responds best on sharp transitions in low-noise images.

The other three operators, being three by three, handle more gradual transitions and noisier images better.

Page 38: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

ExampleExample

Page 39: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

ExampleExample

Page 40: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.5.2 Edge linking18.5.2 Edge linking

Heuristic search Curve fitting Hough transform

– A straight line in x-y plane can be expressed in polar coordinates as

– Any line in the x-y plane plots a point in the space, this is called the Hough transform of the straight line.

bmxy

)sin()cos( yx

,

Page 41: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Hough TransformHough Transform

– Any point in the x-y plane corresponds to a sinusoidal curve in the space.

– A two-dimensional histogram in space can be constructed to obtain the parameters of linear boundary segments.

,

,

Page 42: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.6 Region growing18.6 Region growing

– Region growing is an image segmentation approach that has received considerable attention in the computer vision.

– Strong boundaries are stand while weak boundaries are dissolved and the adjacent regions merged.

– The region-merging process is continued until a point is reached where no boundaries are weak enough to be dissolve, and the image segmentation is complete.

Page 43: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7 Binary image processing18.7 Binary image processing

Binary neighborhood encoding– In a binary image, for any 3-by-3 neighborhood

operation, there are 512 (9 bits) possible configurations of 0’s and 1’s.

– Convolution of a binary image with the following 3-by-3 kernel generates a 9-bit image.

– Thus neighborhood operation can be implemented with a 512-entry look-up table.

Page 44: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7 Binary image processing18.7 Binary image processing

Binary neighborhood encoding– Hit-or-miss transformation can reduce solid

objects to their outlines by eliminating interior points.

– The binary neighborhood encoding kernel

16 8 4

32 1 2

64 128 256

Page 45: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.1 Morphological image 18.7.1 Morphological image processingprocessing

Based on the set theoryAlso called mathematical morphologyPassing a structuring element over the

image in an activity similar to convolutionAt each pixel position, a specified logical

operation is performed between the structuring element and the binary image.

Page 46: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.1 Morphological image 18.7.1 Morphological image processingprocessing

18.7.2 Erosion and Dilation– A boundary point is a pixel that is located

inside an object, but that has at least one neighbor outside the object.

Page 47: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.1 Morphological image 18.7.1 Morphological image processingprocessing

18.7.2.1 Erosion– Simple erosion is the process of eliminating all the

boundary points from an object, leaving the object smaller in area by one pixel all around its perimeter.

– General erosion is defined by

– Erosion is useful for removing from a segmented image objects that are too small to be of interest.

}|,{ BSSBE xyyx

Page 48: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

ErosionErosion

Page 49: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.1 Morphological image 18.7.1 Morphological image processing processing

18.7.2.2 Dilation– Simple dilation is the process of incorporating

into the object all the background points that touch it, leaving it larger in area by that amount.

– Dilation is useful for filling holes in segmented objects.

– General dilation is defined by }|,{ BSSBD xyyx

Page 50: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

DilationDilation

Page 51: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7 Binary Image Processing18.7 Binary Image Processing

18.7.3 Opening and Closing– The processing of erosion followed by dilation

is called opening.– Opening can eliminate small and thin objects,

breaking objects at thin points, and smooth the boundary of objects without significantly changing their area.

SSBSBO )(

Page 52: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7 Binary image processing18.7 Binary image processing

– The process of dilation followed by erosion is called closing.

– Closing can fill small and thin holes in objects, connect nearby objects, and smooth the boundaries of objects without significantly changing their area.

SSBSBC )(

Page 53: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

An exampleAn example

Page 54: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.4 Variants of erosion and 18.7.4 Variants of erosion and dilationdilation

Shrinking– Shrinking is a kind of erosion where single-pixel

objects are left intact.– Shrinking is useful when the total object count must be

preserved. Thinning

– 1.normal erosion but the pixels are marked as candidates for removal, but are not actually eliminated; 2. Those candidates that can be removed without destroying connectivity are eliminated, while those that cannot are retained.

Page 55: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7 18.7 Binary image processingBinary image processing

Skeletonization– skeletonization;(medial axis transform, grass-

fire technique) – Similar to thinning, but the rule of deleting

pixels is slight different.Pruning

– The thinning and skeletonizing may produce spurs on the resulting image, pruning can remove these endpoints

Page 56: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Thinning, Skeletonization and Thinning, Skeletonization and shrinkingshrinking

Page 57: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7 18.7 Binary image processingBinary image processing

Thickening– Dilation not merging nearby objects. – Thickening can be implemented by

complementing the image and using the thinning operation on the back ground.

Page 58: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

ThickeningThickening

Page 59: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.5 The distance 18.7.5 The distance transformationtransformation

– The distance transformation converts a binary image to a gray level image, where the gray level at each pixel is the distance from that pixel to the nearest background pixel.

– Chamfer algorithm. An approximate implementation of the distance transformation by erosion-like operation, where pixels are labeled with the iteration number rather than eliminated from the object.

Page 60: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.5 18.7.5 The distance The distance transformationtransformation

One-dimensional

0 0 0 0 1 1 1 1 1 0 0 0 0

0 0 0 0 1 2 3 4 5 0 0 0 0

0 0 0 0 1 2 3 2 1 0 0 0 0

Page 61: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.5 18.7.5 The distance The distance transformationtransformation

Two-dimensional– Similar to convolution. – The forward pass moves from left to right and from the

top to the bottom. At each position, a set of two-term sums is formed by adding each element in the mask to the underlying pixel value. Where the mask is blank, nothing is done.

– The backward pass is from right to left and bottom to top

– The minimum of two passes is used as the pixel value of the resulting image.

Page 62: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.5 18.7.5 The distance The distance transformationtransformation

Two-dimensional masks

4 3 4

3 0

0 3

4 3 4

Page 63: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.5 18.7.5 The distance The distance transformationtransformation

Two-dimensional masks

11 11

11 7 5 7 11

5 0

0 5

11 7 5 7 11

11 11

Page 64: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.7.6 18.7.6 Boundary Curvature Boundary Curvature analysisanalysis

The curvature at a point on a curve is defined as the rate of change of the tangent angle at that point.

The boundary curvature function can assist in the automatic detection and correction of segmentation errors.

Page 65: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.8 18.8 Segmented image Segmented image structurestructure

18.8.1 The object membership map– A separate image of the same size as the

original is used to store the objects.– The gray level of each pixel encodes the

sequence number of the object to which the corresponding pixel in the original image belongs.

– The object membership map is not a compact way, but it can be significantly compressed.

Page 66: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.8 18.8 Segmented image Segmented image structurestructure

18.8.2 The boundary chain code– The chain code consists of the coordinates of

the starting point, followed by the sequence o direction codes that specify the path around the boundary.

– The chain code requires only one (x,y)-coordinate and 3 bits for each boundary point.

– A boundary direction code

Page 67: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.8 18.8 Segmented image Segmented image structurestructure

18.8.2 The boundary chain code– In chain code, the interior points are discarded,

this makes it require considerably less storage, but sometimes less useful when further processing is required.

3 2 1

4 0

5 6 7

Page 68: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.8 Segmented image 18.8 Segmented image structurestructure

18.8.3 Line Segment Encoding

1-1

1-2 2-1

1-3 2-2

1-4

1-5

1-6 1-7

1-8 1-9

Page 69: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

18.8 Segmented image 18.8 Segmented image structurestructure

18.8.3 Line segment encoding– The object segment information can be

organized for storage on disk as

1 9 1-1 1-2 1-3 1-4

2 2 2-1 2-2

102 210 10

Page 70: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:
Page 71: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Chapter 19 Object Chapter 19 Object MeasurementMeasurement

19.2 Size measurements– 19.2.1 Area and Perimeter

Pixel count area. The number of pixels inside (and including) the boundary. The perimeter is the distance around the outside of all the pixels.

Perimeter of a Polygon.– The perimeter of an object can be obtained in a chain code by

where and are the numbers of even and odd steps in the chain code.

oe NNp 2

eN oN

Page 72: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Chapter 19 Object Chapter 19 Object MeasurementMeasurement

Area of the Polygon– The area of the polygon is

where and are the numbers of pixels in the object (including boundary pixels) and in the boundary, respectively.

– One can correct the pixel-count-derived area measurement approximately by subtracting half the perimeter.

1

2b

o

NNA

oNbN

Page 73: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Chapter 19 Object Chapter 19 Object measurementmeasurement

– Computation of area of the polygon can be implemented by dividing the polygon into triangles.

– If the origin is located within the polygon, the area of a triangle is given by

))((2

1

2

1

2

11212221112 yyxxyxyxyxdA

dA

),( 11 yx

),( 22 yx

1x 2x

2y

1y

Page 74: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Chapter 19 Object Chapter 19 Object measurementmeasurement

The formula can be simplified to

And the total area is

The area could also be computed from the Green’s theorem of the contour integral.

)(2

11221 yxyxdA

bN

iiiii yxyxA

111 )(

2

1

Page 75: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Chapter 19 Object Chapter 19 Object measurementmeasurement

19.2.1.2 Boundary Smoothing– The perimeter measurement is artificially high because

of noise and because the boundary points are restricted to a rectangular sampling grid.

– One way to smooth the boundary is to use only a subset of the boundary pixels as vertices.

– Boundary smoothing can also be effected by representing the boundary in parametric form, and then lowpass filtering the one-dimensional boundary function.

Page 76: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Parametric representation of Parametric representation of boundaryboundary

)](),([ pypx

Page 77: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Chapter 19 Object Chapter 19 Object measurementmeasurement

19.2.3 Length and Width– The Length and width of an object can be

obtain from the minimum and maximum row numbers and column numbers.

– For objects of random orientation, it is necessary to locate the major axis of the object.

– There are three ways to locate the major axis.

Page 78: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

Chapter 19 Object Chapter 19 Object measurementmeasurement

– Compute a best fit straight (or curved ) line through the points in the object;

– Compute from moments;– Use the minimum enclosing rectangle (MER)

around the object.

Page 79: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

19.3.1 Rectangularity– The rectangle fit factor is defined as

where is the object’s area and is the area of the object’s MER.

– The rectangle fit factor is bounded between 0 and 1, and achieves its maximum when the object is a rectangle.

R

o

A

AR

oA RA

Page 80: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

– The aspect ratio is the ratio of width to length of the MER.

– 19.3.2 Circularity 1. The ratio of perimeter squared to area which takes

on a minimum value of for a circular shape.

More complex shapes yield higher values.

L

WA

A

PC

2

4

Page 81: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

– 2. The boundary energy

where P is the perimeter of the object.– For a circle, it has minimum boundary energy

P

dppKP

E0

|)(|1

2

0

1

RE

)(

1)(

prpK )( pr

p

Starting point

Page 82: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

– 3. The shape measure is defined as

where is the average distance from an interior point to the boundary of an object.

– The denominator is the IOD of the distance-transformed image.

2

1

3

2

N

iix

N

d

Ag

N

iix

Nd

1

1

Page 83: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

19.3.3 Invariant Moments– The moments of a bounded two-dimensional function

is defined by

where j and k are nonnegative integers.– The set is unique for the function– Suppose takes on the value 1 inside the object

and 0 elsewhere, then the moments is determined by its shape.

dxdyyxfyxM kj

jk ),(

},1,0,,{ kjM jk

),( yxf

),( yxf

),( yxf

Page 84: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

– The parameter is called the order of the moment. The zero-order moment

– The central moments is

where is the coordinate of the center of gravity of an object defined by

kj

dxdyyxfM ),(00

dxdyyxfyyxx kj

jk ),()()(

),( yx

00

01

00

10

M

My

M

Mx

Page 85: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

19.3.3.2 Principal Axes– The angle of rotation that causes the second-order

central moment to vanish can be obtained by

– The coordinate axes at an angle from the x,y axes are called the principal axes of the object.

– If the object is rotated through the angle before moments are computed, or if the moments are computed relative to the axes, then the moments are rotation invariant.

11

0220

112)2tan(

yx ,

yx ,

Page 86: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

– 19.3.3.3 Invariant moments The area-normalized central moments computed

relative to the principal axis are invariant under magnification, translation, and rotation of the object, and they can be used to the recognition of the shapes of the objects.

A large set of moments may be required to distinguish similar shapes. The resulting high-dimensional classifier may become quite sensitive to noise.

Page 87: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

– Gray-level images. If is a gray level image, the moments can be

computed as before, and the zero-order moment becomes the integral optical density of the image.

For a gray level image, the invariant moments reflect not only the shape of the object, but also the density distribution within it.

A reasonable small number of invariant moments can reliably distinguish among the different objects.

),( yxf

Page 88: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

19.3.4 Shape descriptors– The differential chain code

An example of the boundary chain code and the differential chain code

2

4

6

Object a

b

g

fe

c

d

Boundary chain code

Differential chain code

Page 89: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

The shape of an object can be distinguished by analyzing its boundary chain code or the differential chain code.

– Fourier descriptors The tree type of boundary function: boundary chain

code, the polar boundary function, and the complex boundary function, are all periodic. The Fourier transform is an alternative representation of the object’s shape.

The low-frequency components of the spectrum are required to characterize the basic shape of the object.

Page 90: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.3 Shape analysis19.3 Shape analysis

19.3.4 Shape descriptors– 19.3.4.3 The median axis transform

A point inside the object is on the medial axis if and only if it is the center of a circle that is tangent to the boundary of the object at two nonadjacent points.

Page 91: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.4 Texture analysis19.4 Texture analysis

19.4.1 Definition– Texture is an attribute representing the spatial

arrangement of the gray levels of the pixel in a region.– Random texture. Pattern texture.– A texture feature is a value that quantifies some

characteristics of the gray-level variation within the object.

– A texture feature is independent of the object’s position, orientation,size,shape, and average gray level.

Page 92: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.4 Texture analysis19.4 Texture analysis

19.4.2 Texture segmentation– Sometimes objects differ from the surrounding

background in texture but not in average brightness. For segmentation of such images, texture must be computed to map the texture to gray level.

Page 93: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.4 Texture analysis19.4 Texture analysis

19.4.3 Statistical texture features– The module feature

Where N is the number of gray levels, M is the number of pixels in the object, and is the number of pixels having gray level .

N

i ii

i

NNMMHH

NMHI

1 )/11()/1(

/

iH

i

Page 94: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.4.3 Statistical Texture 19.4.3 Statistical Texture FeaturesFeatures

19.4.3.1 The co-occurrence matrix– For a given direction and a distance, the

elements of the co-occurrence matrix P for an object is the number of times, divided by M, that gray levels I and j occur in two pixels separated by that distance and direction in the object. Where M is the number of pixel contributing to P.

jthi, jthi,

Page 95: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.4.3 Statistical Texture 19.4.3 Statistical Texture FeaturesFeatures

19.4.3.1 The Co-Occurrence Matrix– Some texture features can be computed from

the co-occurrence as Entropy

Inertia

Energy

N

i

N

jijij PPH

1 1

log

N

i

N

jijPjiI

1 1

2)(

N

i

N

jijPE

1 1

2][

Page 96: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.4.4 Other Texture Features19.4.4 Other Texture Features

Spectral Features– As magnitude of texture pattern, one can average the

two-dimensional spectrum in annular rings to produce a one-dimensional function of frequency that ignores directionality.

– One can also obtain the directionality of the texture pattern by averaging the spectrum in radial slices to produce a function of angle.

– Each of these functions can be further reduced to scalar features that offer the desired discrimination ability.

Page 97: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.4.4 Other Texture Features19.4.4 Other Texture Features

– Structural features The texture pattern is composed of a spatial

arrangement of texture primitives that are small objects that constitute, for example, one unit of a repeated pattern.

Feature extraction then becomes the task of locating the primitive and quantifying their spatial arrangement.

Page 98: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.5 Curve and surface fitting19.5 Curve and surface fitting

– Curve and surface fitting are useful for object measurement.

– 19.5.1 Minimum mean square error fitting Given a set of points , The mean square error

fitting to find the function is to minimize

Suppose is a parabola

),( ii yx

N

iii xfy

NMSE

1

2)]([1

)(xf

)(xf

2210)( xcxccxf

Page 99: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.5 Curve and surface fitting19.5 Curve and surface fitting

– And let

then the error is

and the mean square error is given by

2

1

0

2

222

211

2

1

1

1

1

c

c

c

xx

xx

xx

y

y

y

NNN

CBY

BCYE

EET

NMSE

1

Page 100: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.5 Curve and surface fitting19.5 Curve and surface fitting

– Differentiating E with respect to C and setting the derivative to zero leads to the solution

where the matrix is the pseudoinverse of B. If B is a square matrix, then

][][ 1 YBBBC TT TT BBB 1][

YBC 1

Page 101: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.5 Curve and surface fitting19.5 Curve and surface fitting

19.5.3 One-dimensional parabola fit– An example

– The fitting result and the error are respectively

25 5 1

1641

931

84.42.21

81.09.01

2

3

5.2

3

8.1

5

4

3

2.2

9.0

BYX

CBYBB

230.0

415.1

747.0

5.136

7.37

3.12

98622756

2275615

56155T

T07.273.292.275.283.1BC T07.27.42.25.03. E

Page 102: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.5 Curve and surface fitting19.5 Curve and surface fitting

– An example of one-dimensional fitting

x

f(x)

Page 103: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:

19.5 Curve and surface fitting19.5 Curve and surface fitting

19.5.4 Two-dimensional Cubic fit

Page 104: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:
Page 105: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition:
Page 106: Chapter 18 Image Segmentation 18.1 Introduction – Computer vision; Image understanding; Statistical pattern recognition – The three steps of pattern recognition: