![Page 1: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/1.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Islam Beltagy
P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik. IEEE TPAMI 2011.
Contour Detection andHierarchical Image Segmentation
![Page 2: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/2.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Outline
Introduction
Contour Detection
Hierarchical Segmentation
Results and Evaluation
Discussion
![Page 3: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/3.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Contours (Ground Truth BSDS500)
Contour
![Page 4: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/4.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Segmentation (Ground Truth-BSDS500)
![Page 5: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/5.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Goal• Contour Detection• Segmentation from Contours
Original Image SegmentationContour
![Page 6: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/6.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Outline
Introduction
Contour Detection
Hierarchical Segmentation
Results and Evaluation
Discussion
![Page 7: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/7.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Contours Detection
Original Image Contour = Pr(x, y, Ο)
![Page 8: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/8.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
• Circular disk around (x, y) with orientation θ• Histograms of Intensities in both halves• �Chi Square distance between the two histograms • Three scales of r and eight values for θ
Contour Detection-Oriented gradient of histograms
![Page 9: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/9.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
•Extract different channels• Brightness L, Color a, Color b
• Components of CIE Lab colorspace
• Texture
Contour Detection
Convolve
K-mean clustering
Replace pixel with cluster index
http://www.cs.berkeley.edu/~malik/papers/LM-3dtexton.pdf
![Page 10: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/10.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
• Local Cues CombinationContour Detection
Scale: 3 different values for the disc diameterα: learned from training data
max
![Page 11: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/11.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
• GlobalizationContour Detection
Eigen Values
Normalized Cuts
Affinity Matrix Apply filters
then sum
![Page 12: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/12.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Outline
Introduction
Contour Detection
Hierarchical Segmentation
Results and Evaluation
Discussion
![Page 13: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/13.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Uncertainty in Segmentation
Hierarchical Segmentation
![Page 14: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/14.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Hierarchical Segmentation Oriented Watershed Transform
Ultrametric Contour Map
![Page 15: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/15.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Oriented Watershed Transform•Watershed Transform
http://cmm.ensmp.fr/~beucher/wtshed.html
![Page 16: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/16.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Oriented Watershed Transform•Watershed Transform
Artifacts
Weight each arc
![Page 17: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/17.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Oriented Watershed Transform
WTOWT
Fitting lines
Weight arcs according to orientation
![Page 18: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/18.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Hierarchical Segmentation• Ultrametric Contour Map - Iterative Merging
![Page 19: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/19.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Brief Summary
Original Image
Oriented Gradientof histograms
Contour
Oriented Watershed TransformUltrametric Contour Map
Hierarchical Segmentation
Threshold
![Page 20: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/20.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Outline
Introduction
Contour Detection
Hierarchical Segmentation
Results and Evaluation
Discussion
![Page 21: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/21.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Result (BSDS500 dataset)
Left to Right: Image, gPb-owt-ucm, threshold = optimal dataset scale, threshold = optimal picture scale
![Page 22: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/22.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Result
Evaluation of segmentation algorithmsEvaluation of contour detector
• BSDS300 Dataset
![Page 23: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/23.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Region benchmarks(2)Covering Rand Index Variation of Information
-ODS: optimal dataset scale
-OIS: optimal image scale
![Page 24: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/24.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Outline
Introduction
Contour Detection
Hierarchical Segmentation
Results and Evaluation
Discussion
![Page 25: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/25.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Discussion
What enforces closure in OWT-UCM for all
possible thresholds?
![Page 26: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/26.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Discussion How to define ground truth ?
Can we use Hierarchal segmentation ?
![Page 27: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/27.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Discussion OIS and the different possible segmentations
below, what do they suggest ?
![Page 28: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/28.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Interactive Segmentation
1
![Page 29: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/29.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Discussion Contours vs Segments.
![Page 30: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/30.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Reference• P. Arbelaez, M. Maire, C. Fowlkes and J. Malik. Contour Detection
and Hierarchical Image Segmentation. IEEE TPAMI, Vol. 33, No. 5, pp. 898-916, May 2011
• P. Arbelaez, M. Maire, C. Fowlkes and J. Malik. From Contours to Regions: An Empirical Evaluation. In CVPR 2009.
• P. Arbelaez and L. Cohen. Constrained Image Segmentation from Hierarchical Boundaries. In CVPR 2008.
![Page 31: Contour Detection and Hierarchical Image Segmentationcv-fall2012/slides/islam-paper.pdf · Original slides from: Hsin-Min Cheng Islam Beltagy P. Arbelaez, M. Maire, C. Fowlkes, and](https://reader033.vdocuments.us/reader033/viewer/2022050407/5f848e14b000242ca34ad62d/html5/thumbnails/31.jpg)
Original slides from: Hsin-Min Cheng http://archer.ee.nctu.edu.tw/powerpoint/GM_1024.pptx
Thanks