image segmentation zhiqiang wang [email protected] some examples
TRANSCRIPT
![Page 2: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/2.jpg)
image segmentations
Cell segmentation
Active contour method
Interactive method (graph cut)
Other examples
![Page 3: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/3.jpg)
Cell Segmentation
![Page 4: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/4.jpg)
1st Step: Image resize
Since original image’s resolution is 3978*3054, its size is very big and may let extracting algorithm be time consuming.
![Page 5: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/5.jpg)
2nd Step: Image smooth
To simplify image’s content, noise and detail texture should be removed.
Gaussian filter or Nonlinear diffusion method
![Page 6: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/6.jpg)
3rd Step: interactive segmentation
Using interacting method to select which cell we want to extract.
Level set : initial contour
Water shed : seed point
Graph cut: label foreground and background
![Page 7: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/7.jpg)
3rd Step: Find centroids of subregion
After segmentation, we can get 59 subregions. For each region, we find centroids for each subregion as a seed point.
![Page 8: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/8.jpg)
3rd Step: Find centroids of subregion
![Page 9: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/9.jpg)
How to find center point
In some cases, centroid is outside of the subregion. As a seed point, it would impede further segmentation.
Possible solution: erode the subregion until it become a point. computing the distance between inside pixels and the contour of subregion, take the
point which have max distance value as the seed point.
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
-60
-40
-20
0
20
40
Distance fieldSkeleton of the subregion
![Page 10: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/10.jpg)
Active Contour Model for Image Segmentation
![Page 11: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/11.jpg)
What’s active contour?
This method can also be understood as a special case of a more general technique of matching a deformable model to an image by energy minimization.
AC = Curve fitted iteratively to an image evolve based on its shape and the image value until it stabile (ideally on an object’s boundary).
![Page 12: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/12.jpg)
Advantages of active contour
Threshold Edge detectionAn image of blood vessel
Nice representation of object boundary: Smooth and closed, good for shape analysis and recognition and other applications.
![Page 13: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/13.jpg)
parametric geometric
Curve:
polygon = parametric AC continuous = geometric AC
![Page 14: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/14.jpg)
Parametric Model: Gradient vector flow (GVF)• GVF field is a non-irrotational external force field that points toward the
boundaries when in their proximity and varies smoothly over homogeneous image regions all the way to image borders.
||/ ff Gradient vector flow
![Page 15: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/15.jpg)
Example: Gradient vector flow
• GVF field is a non-irrotational external force field that points toward the boundaries when in their proximity and varies smoothly over homogeneous image regions all the way to image borders.
![Page 16: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/16.jpg)
General Curve evolution• Let a curve moving in time t be denoted by X[x(s,t), y(s,t) ) , where s is curve
parameterization. Let N be the moving curve’s inward normal, and c curvature. And let the curve develop along its normal direction according to the partial differential equation:
![Page 17: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/17.jpg)
Basic deformation equation• Constant Speed Motion (Area decreasing flow)
• Mean curvature motion (Length shortening flow)
• During the evolution process for image segmentation, curvature deformation and/or constant deformation are used and the speed of curve evolution is locally dependent on the image data.
![Page 18: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/18.jpg)
Its main idea of CV model is to minimize the inter class variances
)(
220)(
210
21,,
,,inf121
CoutsideCinside
CCcc
dxdycIdxdycI
dsCccE
CV model
![Page 19: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/19.jpg)
A basic version of the speed function that combine curvature and constant deformation is CV model(Active contour model without edge) Its main idea is to consider the information inside the regions.
Let be the original image to be segmented and C denote the evolving curve. and are positive weights to control C’s smoothness. is the mean value of inside the C and is the mean outside C.
)(
220)(
210
21,,
,,inf121
CoutsideCinside
CCcc
dxdyuIdxdyuI
dsCuuE
0I 1u
0I 2u
← Smooth term
← data term
Evolution speed control (CV model)
220
210 uIuIN
t
C
To minimize the cost function, Euler-lagrange equation is used:
![Page 20: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/20.jpg)
Evolution speed control (CV model)
Its main idea of CV model is to minimize the inter class variances
220
210 cIcIN
t
C
• Mean curvature motion is the steepest descent
flow (or gradient flow) that minimizes arc length of the contour:
![Page 21: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/21.jpg)
Parametric Deformable Model • The curves can be represented as level sets of higher dimensional functions
yielding seamless treatment of topological changes.
![Page 22: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/22.jpg)
Research Problem-- weakness of region based model
failure
success
![Page 23: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/23.jpg)
Evolution speed control--GAC model
• A basic version of the speed function that combine curvature and constant deformation is GAC model:
Smooth term data term
NN
ggt
C
g is an edge-stopping function defined as follow: 1
g 21 G 0I
0G I The term denotes the gradient of a Gaussian smoothed image, where is a smooth parameter.
• During the evolution process for image segmentation, curvature deformation and/or constant deformation are used and the speed of curve evolution is locally dependent on the image data.
![Page 24: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/24.jpg)
GAC model
![Page 25: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/25.jpg)
Features of edge based model
failure
success
![Page 26: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/26.jpg)
3D Case
![Page 27: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/27.jpg)
Interactive segmentation(graph cut and alpha matting)
Reference: Anat Levin, etc. A Closed Form Solution to Natural Image Matting. 2006
![Page 28: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/28.jpg)
Remove complicate background
![Page 29: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/29.jpg)
Over segmentation with meanshift method
![Page 30: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/30.jpg)
Construct graph and perform graph cut agorithm
Source (Label 0)
Sink (Label 1)
Cost to assign to 0
Cost to assign to 1
Cost to split nodes
![Page 31: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/31.jpg)
Construct graph and perform graph cut agorithm
![Page 32: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/32.jpg)
Gaussian Mixture Model and Graph Cut
Gaussian Mixture Model (typically 5-8 components)
Foreground &Background
Background
Foreground
BackgroundG
R
G
RIterated graph cut
![Page 33: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/33.jpg)
More examples
![Page 34: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/34.jpg)
The problem of hard segmentation
![Page 35: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/35.jpg)
Alpha matting
+
![Page 36: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/36.jpg)
Alpha matting
iiiii BFI )1(
+ xx=
Matting is ill posed problem
![Page 37: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/37.jpg)
iiiii BFI )1(
0
1
Scribbles approach
![Page 38: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/38.jpg)
![Page 39: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/39.jpg)
Color Line:
Color lines
213 )1( CCCRC iiii
R
B
G
![Page 40: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/40.jpg)
Color lines
Color Line: 213 )1( CCCRC iiii
R
B
G
![Page 41: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/41.jpg)
Matting results
+
![Page 42: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/42.jpg)
Combine hard segmentation
![Page 43: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/43.jpg)
More examples
![Page 44: Image Segmentation Zhiqiang wang zwang22@kent.edu some examples](https://reader037.vdocuments.us/reader037/viewer/2022103022/56649d1a5503460f949ef824/html5/thumbnails/44.jpg)
Thanks