image segmentation - lina...
Post on 23-May-2020
11 Views
Preview:
TRANSCRIPT
EEE 508
Image Segmentation
• Objective: to determine (extract) object boundaries. • It is a process of partitioning an image into distinct regions by
grouping together neighboring pixels based on some pre-defined similarity criterion.
• It can be viewed as a classification technique that forms regions of similarities in the image.
• The similarity criterion is determined using specific properties or features of pixels representing objects in the image.
• Much more difficult with non-uniform background as compared to uniform background.
1
EEE 508
Image segmentation
• Broad classification of methods: Edge-based methods
The edge information is used to determine boundaries of objects. The boundaries are then analyzed and modified to form closed regions belonging to the objects in the image.
Pixel-based direct classification methods Heuristics or estimation methods derived from the histogram statistics
of the image or clustering algorithms are used to form closed regions belonging to the objects in the image.
Region-based methods Pixels are analyzed directly for a region growing process based on a
pre-defined similarity criterion to form closed regions belonging to the objects in the image.
2
EEE 508
Edge-based image segmentation
• General procedure: Detect edges based on first-order or second-order gradient
information. Track and link relevant edges based on gradient information.
• Edge detection The Gradient magnitude commonly used to detect edges
3
EEE 508
Edge Detection
• One basic tool: Gradient operator
The Gradient magnitude is commonly used for edge detection
( ) ( )( )
( )( )( ) ;
,,
,
,
,,212
211
2
21
1
21
2121
=
∂∂
∂∂
=∇=nngnng
nnnx
nnnx
nnxnngxx(n1,n2) – input image
n2
n1
gradient: points in direction of greatest change (locally at a pixel)
( ) ( ) ( ) 21
2
2
21
2
1
2121
,,,
∂
∂+
∂
∂=
nnnx
nnnxnng
4
EEE 508
Edge Detection
Question: How do we realize (compute) the gradient in discrete-domain?
Example: Robert’s Gradient
Interpretation: this is like a cross-wise 2D first backward difference.
( ) ( ) ( ) ( )
( ) ( ) ( ) ( )21212
21212
21211
21211
,11,,,
,1,1,,
nnxnnxn
nnxnng
nnxnnxn
nnxnng
+−+=∂
∂=
−++=∂
∂=
( ) ( ) ( ) ( ) ( ) 22121
2212121 ,11,,1,1, nnxnnxnnxnnxnng +−++−++=
⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅
n1
n2
5
EEE 508
Edge Detection
In general:
Robert’s Gradient
( ) ( ) ( )( ) ( ) ( )21221212
21121211
,,,,,,
nnhnnxnngnnhnnxnng
∗∗=∗∗=
( ) ( ) ( )22122
21121 ,,, nngnngnng +=
n1
(1)
(-1)
n2
h1( -n1,-n2)
(1)
(-1)
n2
h2( -n1, -n2)
n1
6
EEE 508
Edge Detection
Sobel’s Gradient
n1
(1)(2)
n2
h1( -n1,-n2)
(1)
(-1)
(-1)
(-2) n1
(-1)
(2)
n2
h2( -n1,-n2)
(1)
(-1)
(1)
(-2)
n1
(-1)(-2)
n2
h1(n1,n2)
(-1)
(1)
(1)
(2) n1
(1)
(-2)
n2
h2(n1,n2)
(-1)
(1)
(-1)
(2)
7
EEE 508
Edge Detection
• Prewitt’s Gradient
n1
(1)
(1)
n2
h1( -n1,-n2)
(1)
(-1)
(-1)
(-1)n1
(-1)
(1)
n2
h2( -n1,-n2)
(1)
(-1)
(1)
(-1)
8
EEE 508
Edge Detection
Note: Edge detection methods based on computing some form of gradient or based on computing a difference are sensitive to noise. Example: Background noise appears as isolated edge points ⇒ noise
smoothing can be applied before edge detection (recommended).
9
EEE 508
Edge Detection • Note: Edge detection methods based on computing some form
of gradient or based on computing a difference are sensitive to noise. Example: Background noise appears as isolated edge points ⇒
noise smoothing can be applied before edge detection (recommended).
• Laplacian operator Edge is detected not only when |g(n1, n2)| is large, but also zero
crossings are considered. Laplacian: 2nd derivative in 1D case. Gradient: 1nd derivative in 1D case. Laplacian:
( ) ( ) ( ) ( )22
212
21
212
212
21,,,,
nnnx
nnnxnnxnnL
∂∂
+∂
∂=∇=
10
EEE 508
Edge Detection How do we compute Lapalcian? Several approximations possible:
approximate and with a forward difference
Similarly:
( )1
21,n
nnx∂
∂
( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( ) ( )2121212112111
21
121
212
21212111
21
,1,2,1,1,,,
,,1,,
nnxnnxnnxnngnngn
nnxnn
nnx
nnxnnxnngn
nnx
−+−+=−−=
∂
∂∂∂
=∂
∂
−+==∂
∂
( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( ) ( )1,,21,1,,,,
,1,,,
2121212122122
21
222
212
21212122
21
−+−+=−−=
∂
∂∂∂
=∂
∂
−+==∂
∂
nnxnnxnnxnngnngn
nnxnn
nnx
nnxnnxnngn
nnx
( )21
212 ,
nnnx
∂∂
11
EEE 508
Edge Detection ⇒
Edges detected at zero-crossing of Lx(n1,n2) But, if there is no checking on magnitude of gradient, a lot of edge
points and false edge contours may be generated.
( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( )2121
2121212121212
21
,,,41,1,,1,1,,
nnhnnxnnxnnxnnxnnxnnxnnxnnL
∗∗=−−+++−++=∇=
n1(1)
n2
h( n1, n2)
(-4)
(1)
(1)
(1)
12
EEE 508
Edge Detection
Solution: If large variance ⇒ edge.
At zero-crossing, check local variance:No edge otherwise.
Note: Edge detection methods are used for detecting object/region boundaries for segmentation and image analysis.
13
EEE 508
Edge Detection First- order gradient operators:
• Sobel• Robert• Prewitt• Canny
First-derivative: GradientEdge is detected at the highest
point
Second-derivative: LaplacianEdge is detected at the zero-crossing point
14
EEE 508
Canny Edge Detector
1.Calculating horizontal gradient Gx and vertical gradient Gy at each pixel location by
convolving with partial derivatives of a 2D Gaussian function, given as
2.Computing the gradient magnitude G and direction θG at each pixel location.
3.Applying Non-maximum suppression (NMS) to thin edges.
4.Computing high and low thresholds based on the histogram of the gradients of the entire
image.
5.Performing hysteresis thresholding to determine the edge map.
−=−=
−−+
− 2
2
2
2
2
22
22
2
1),( σσσ
σσ
yxyx
x exeexyxF xx FIG ∗=
−=−=
−−+
− 2
2
2
2
2
22
22
21),( σσσ
σσ
xyyx
y eyeeyyxF yy FIG ∗=
Horizontal Gradient
Vertical Gradient
GradientMagnitude
AndGradient direction
Non-maximal
Suppression
Compute the high and low
thresholds
Hysteresis Thresholding
edge map
inputimage
15
EEE 508
Canny Edge Detector
• Consider a single row or column of the image– Plotting intensity as a function of position gives a signal
Where is the edge?
Effect of Noise
16
EEE 508
Solution 1: Smooth firstCanny Edge Detector
17
EEE 508
Solution 2: First derivative of the Gaussian function
Canny Edge Detector
18
EEE 508
Edge-based image segmentation
• Boundary tracking Objective: assemble meaningful edges to form closed regions. Usually based on pixel-by-pixel search to find connectivity among
edge segments, based on similarity criterion among edge pixels. Geometrical proximity and topographical properties used to
improve edge linking operations for noisy or occluded pixels. Also, probabilistic approaches, graphs and rule-based methods for
model-based segmentation have been used.
19
EEE 508
Edge-based image segmentation Neighbor search method:
Assume the edge-detection operation produces an edge magnitude e(x,y) and an edge orientation φ(x,y) information.
List of edge pixels can be produced from gradient image obtained from edge detection procedure.
Assume the first edge pixel is a boundary pixel bj. A successor boundary pixel bj+1 can be found in the 4-connected or 8-
connected neighborhood if the following conditions are satisfied:
|e(bj)| >T1 and |e(bj+1)| >T1 and |e(bj)-e(bj+1)| <T2 and |φ(bj)-φ(bj+1)| mod 2π < T3
If there is more than one neighboring pixel satisfying these conditions the pixel that minimizes the difference is selected.
Apply the algorithm recursively until all neighbors are searched. If no neighbor is satisfying the conditions, search is stopped and
procedure starts with new edge pixel. Note: this algorithm might leave many edge pixels and partial
boundaries unconnected.
20
EEE 508
Edge-based image segmentation Graph-based search method:
It attempts to find the path between the start and end nodes in a constructed graph by minimizing a cost function based on a distance measure and transitional probabilities.
– An edge map with magnitude and direction information
– A graph derived from the edge map with a minimum cost path (darker arrows) between the start and end nodes.
21
EEE 508
Edge-based image segmentation
An edge map with magnitude and direction information
A graph derived from the edge map with a minimum cost path (darker arrows) between the start and end nodes.
22
EEE 508
Pixel-based direct classification methods
• Use histogram statistics to define single or multiple thresholds to classify an image pixel-by-pixel.
• A simple approach: Bimodal histogram ⇒ set the threshold to gray value
corresponding to the deepest point in the histogram value. Multimodal histogram ⇒ partition the image using some
heuristics, then determine the threshold for each region using histogram of the region.
Classify every pixel by comparing its gray level to the selected threshold.
( )
≤>
=TyxfTyxf
yxg),( if0),( if1
,
23
EEE 508
Pixel-based direct classification methodsOriginal image Image histogram
Thresholded, T=12 Thresholded, T=166 Thresholded, T=225
24
EEE 508
Pixel-based direct classification methods
• Optimal global thresholding To determine the optimal global threshold, apply parametric
distribution-based methods to the histogram. Assume histogram consists of two Gaussian distributions
belonging to two respective classes such as background and object.
Classification is done according to:
( ) ( ) ( ) 1; 212211 =++= PPzpPzpPzp - class probabilities
( )
>≤
=TyxfTyxf
yxg),( if2 Class),( if1 Class
,
25
EEE 508
Pixel-based direct classification methods
Define error probabilities of misclassifying pixels:
For Gaussian distributed Class 1 and Class 2:
where σi and µi are standard deviation and mean for classes
( ) ∫+∞
=T
dzzpTE )(11 ( ) ∫∞−
=T
dzzpTE )(22and
probability to classify a class 1 pixel to class 2
probability to classify a class 2 pixel to class 1
( ) ( ) ( )TEPTEPTE 2211 += - the overall probability of error
( )( ) ( )
22
22
21
21
2
2
22
1
1
22σµ
σµ
σπσπ
−−
−−
+=zz
ePePzp
26
EEE 508
Pixel-based direct classification methods
Determine optimal T by finding a general solution that minimizes E(T). Using the mixture distribution p(z), solution can be found by solving:
where
If the variances of both classes are equal to σ2:
Note: If both classes are with equal likelihood ⇒ T = (µ1 + µ2)/2.
02 =++ CBTAT
( )
+−=−=−=
21
1222
21
22
21
21
22
212
221
22
21 ln2;2;
PPCBA
σσ
σσσµσµσµσµσσ
−
++
=1
2
21
221 ln
2 PPT
µµσµµ
27
EEE 508
Pixel-based direct classification methods
• Pixel classification through clustering Clustering is the process of grouping data points with similar
feature vectors together. Data points that are close to each other in the feature space are clustered together.
The similarity of feature vectors can be represented by an appropriate distance measure such as Euclidean or Mahalanobis distance.
Each cluster is represented by its mean (centroid) and variance (spread) associated with the distribution of the corresponding feature vectors of the data points in the cluster.
The formation of clusters is optimized with respect to an objective function involving pre-specified distance and similarity measures along with additional constraints such as smoothness.
Clustering may produce disjoint regions with holes ⇒ post-processing algorithm might be needed such as region growing and others
28
EEE 508
Pixel-based direct classification methods
• k-Means clustering (similar to LBG for VQ) It partitions d-dimensional data into k clusters such that an
objective function providing the desired properties of the distribution of the feature vectors in terms of similarity and distance measures is optimized.
Procedure:1. Select the number of clusters k with initial cluster centroids vi ; i=1,…,k. 2. Partition input data points into k clusters by assigning each data point
xj to the closest cluster centroid vi using the selected distance measure, e.g. Euclidean distance:
ijij vxd −= where { }nxxxX ,...,, 21= is the input data set
29
EEE 508
Pixel-based direct classification methods
3. Compute a cluster assignment matrix U representing the partition of the data points with the binary membership value of the j-th data point to the i-th cluster such that:
4. Re-compute the centroids using the membership values as
5. If cluster centroids or the assignment matrix does not change from the previous iteration, stop; otherwise go to Step 2.
∑
∑
=
== n
jij
n
jjij
i
u
xuv
1
1 for all i
ijuU = where { }1,0∈iju for all i, j
11
=∑=
k
iiju for all j and for all inu
n
jij <<∑
=10
30
EEE 508
Pixel-based direct classification methods
The k-Means clustering method optimizes the sum-of-squared-error based objective function:
Note: The k-Means clustering method is quite sensitive to the initial cluster assignment and the choice of the distance measure ⇒additional criterion such as within-cluster and between-cluster variances can be included in the objective function as constraints to force the algorithm to adapt the number of clusters k.
( ) ∑∑= =
−=k
i
n
jijw vxvUJ
1 1
2,
31
EEE 508
Region-based image segmentation
• Region-growing based segmentation Examine pixels in the neighborhood based on a pre-defined
similarity criterion. Neighborhood pixels with similar properties are merged to form closed regions for segmentation.
Merging continues until there is an insufficient number of neighborhood pixels to be added in the current region.
Two criteria needed: A similarity criterion for inclusion of pixels in the region. A stopping criterion for stopping the growth, usually based on the
minimum number of neighborhood pixels required to satisfy the similarity criterion.
32
EEE 508
Region-based image segmentation Example:
Assume the pixel in the center is the origin of the region-growing process. Assume stopping criterion: minimum number of similar neighborhood pixels = 30% of the region to be included.Iteration 1: 8 pixels in 3×3 neighborhood (100%) satisfy similarity criterion, include the 8 pixels.
Iteration 2: 9 pixels in 5×5 neighborhood (56%) satisfy similarity criterion, include the 9 pixels.
Iteration 3: 6 pixels in 7×7 neighborhood (25%) satisfy similarity criterion, include none. Stop region-growing.
Segmented region
33
EEE 508
Region-based image segmentation
Example: Segmented regionOriginal image
34
EEE 508
Region-based image segmentation
• Region-splitting Examine the heterogeneity of a predefined property of the entire
region in terms of its distribution and the mean, variance, minimum and maximum values.
If the region is heterogeneous ⇒ split into two or more regions. Continue splitting until all regions satisfy homogeneity criterion
individually. Generated Ri, i=1, …, n regions satisfy the following conditions:
1. Each region Ri, i=1, …, n is connected.2.
3.4. H(Ri) = TRUE for i=1, …, n5. H(Ri∪Rj) = FALSE for i≠jwhere H(⋅) is a logical predicate for the homogeneity criterion.
RRn
ii =
=
1
jijiRR ji ≠=∩ ;, allfor 0
35
EEE 508
Region-based image segmentation
Region-splitting methods can also be implemented by rule-based systems and quadtrees.
In the quadtree-based method, regions are represented by nodes in the quadtree.
Example: Regions R1 and R3 satisfy homogeneity criterion. Regions R2 and R4 failed homogeneity criterion and are further split.
R21 R22
R24R23
R41 R42
R44R43
R1
R3
R
R1 R2 R3 R4
R41 R42 R43 R44R21 R22 R23 R24
36
top related