a talk to be presented at the oxford robotics research...
TRANSCRIPT
![Page 1: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/1.jpg)
A talk to be presented at the Oxford Robotics Research Group Seminar
Hongdong Li, PhD
ANU and NICTA, Canberra, Australia
September 2013
![Page 2: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/2.jpg)
Outline
• 1. A quick intro. to Computer Vision research @ ANU– hope to finish in about 3~5 minutes’ time.
• 2. A technical talk on “mid-level vision”– image contour extraction and region segmentation.
![Page 3: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/3.jpg)
Canberra & ANU
![Page 4: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/4.jpg)
ANU: Australian National University
• A member of Go8
• A member of IARU
ANU, Berkeley, Cambridge, Copenhagen, ETH, NUS, Oxford, Peking, Tokyo, Yale.
![Page 5: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/5.jpg)
The ANU/NICTA Computer Vision Group (Lead by Prof. Richard Hartley)
• Multi-View Geometry& Optimization.
• Bionic Eyes
• Medical Image Analysis
• Statistical Machine Learning (kernel methods and etc)
• Embedded Robotic Vision
•
Main Research Themes
![Page 6: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/6.jpg)
Multiple View Geometry• Multi-View Geometry has been a “success story”
in computer vision research.
• Major milestones in MVG research, completed by the VGG group, Oxford
• 1993 Marr Prize:– Rothwell, Forsyth, Zisserman, and Mundy,
Extracting Projective Structure from Single Perspective Views of 3D Point Sets, ICCV 1993
• 1998 Marr Prize: – Phil Torr, A. Fitzgibbon, and A. Zisserman,
The Problem of Degeneracy in Structure and Motion Recovery from Uncalibrated Image Sequences , ICCV 1998.
• 2003 Marr Prize:– Andrew Fitzgibbon, Yonatan Wexler, and
Andrew Zisserman, Image-based Rendering using Image-based Priors
![Page 7: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/7.jpg)
@ ANU, we are continue exploring some new, unconventional directions in MVG…
• Novel camera sensors: – Generalized camera model,– Multiple camera rigs, – Non-central camera, – Light field camera,
• Multiple-body motion segmentation– Subspace clustering.– Multiple model fitting.
• Non-rigid, deformable, articulated SFM.– Template based– Template-free approach
• Large scale and efficient optimization. L-inf norm, etc.
![Page 8: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/8.jpg)
Multiple Camera-Rig Video Odometry
Li and Hartley, CVPR08, Kim and Li, Pollyfeys, CVPR 08, Liu and Hartley: camera and planar mirror, CVPR 2013, Kim, Li et al: Camera with symmetric mirror, ICCV 2013.
![Page 9: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/9.jpg)
Multi-Body Motion Segmentation
Multi-body scenario is more realistic in practice, but is substantially hard.
In collaboration with JHU/Adelaide ( Hartley, Li, CVPR’07, CVPR’08)
![Page 10: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/10.jpg)
Non-rigid structure-from-motion ( Dai, and Li, CVPR 2012)
![Page 11: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/11.jpg)
Deformable surface modelling
• Mathieu Salzmann & Hartley, CVPR’09, CVPR’10.
![Page 12: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/12.jpg)
Australia Bionic Eye
• The first stage: budgeted 42 millions over 4 years since 2011.
• 5 BVA Members and 2 contributing universities.
The above picture is for concept illustration purpose only, it is not our actual system.
![Page 13: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/13.jpg)
Australia Bionic Eye Project: Who will benefit
– Profound blind patients.
– AMD patients (Age- related Macular Degeneration).
– RP patients (Retinitis Pigmentosa).
![Page 14: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/14.jpg)
In today’s technical talk, I will focus on…
• Mid-level vision processing (10%)
1. Perceptual contour grouping or completion2. Joint contour-region figure-ground segmentation
![Page 15: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/15.jpg)
Topic-1: Perceptual Boundary Extraction
Topic-2: Joint contour-region segmentation
Joint work with Yansheng Ming, 2rd-year PhD student,and Dr. Xuming He.
Today’s technical talk
![Page 16: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/16.jpg)
Topic-1:
Perceptual Boundary Extraction
![Page 17: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/17.jpg)
Image Boundary Extraction
• Given an image, we want to extract salient contours (boundaries) inside the image.
• This is an important vision task, because boundaries often carry important shape information for visual recognition, and for other vision tasks as well.
![Page 18: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/18.jpg)
The task
• There is no rigorous definition for “boundary”, but this task is however closely related to:
• Edge Detection• Contour Extraction• Region Segmentation
– In particular, in this work, we will use human observer’s labelling results (in BSDS) as the “ground-truth”,
– and we want to produce boundaries that are “good- looking” and resemble human’s perception.
![Page 19: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/19.jpg)
What do we mean by “good-looking” ?
Original imageCanny edges
Pb boundaries Human labelling in BSDS300Clean, smooth, closed, connected,…
![Page 20: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/20.jpg)
What do we mean by “good-looking” ?
Original image Canny edges
Pb boundaries Human labelling
Clean, smooth, closed, connected,…
![Page 21: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/21.jpg)
Preview of Our Results (by using the method to be described in this talk)
We deliberately use edge information only, and without using region segmentation.
![Page 22: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/22.jpg)
How we achieve this ?• We treat contour-extraction as a middle-level perceptual
grouping task (more specifically, contour completion task). • We develop a new CRF model for the task, which
incorporates various mid-level perceptual grouping rules, a.k.a. Gestalt principles. We especially focus on the closure effect principle.
![Page 23: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/23.jpg)
Why Closure ?
– There are numerous psychological evidences, and,– ecologist justifications (e.g. J.Gibson), plus– cognitive vision experiments (e.g. B.Julesz), all show
that…
![Page 24: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/24.jpg)
Why Closure ?
The contour closure effect plays an important role in human visual perception and figure completion.
• Therefore, we want to develop a new contour extraction algorithm that can mimic this, to incorporate or respect this closure-effect.
![Page 25: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/25.jpg)
Challenges
• However, compared with other relatively local Gestalt principles such as “good-continuation” or “proximity”, the “closure effect” is harder to encode, and harder to compute (efficiently).
• This is because, to represent the concept of that “a contour is closed ” we need to consider long-range inter pixel interactions, or even global topological, high-order properties, …
• Most previous work (on contour completion) have focused on “relatively local” Gestalt principles (e.g.: small clique size, pair-wise potential term: Pott model, not P^N model).
• In contrast, much fewer have been devoted to exploiting the “contour-closure” principle.
![Page 26: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/26.jpg)
Specific goal of this work
• The goal of this work is to develop a new computational model for contour completion,
• --- a new CRF model that can not only encode the contour closure effect, but also allow efficient inference.
![Page 27: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/27.jpg)
A high level description of our method
• To solve the image contour-extraction problem,
– we design a graphical model on bottom-up detected boundary segments.
– define a CRF (Conditional Radom Field) with proper potential energy functions on it, and,
– Formulate it as an energy-minimization problem (i.e. an inference problem).
• Solving the inference problem, and obtain the contour map.
![Page 28: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/28.jpg)
Our new method explained: two key components
• The CRF model– Graphical model construction.– Potential functions design.
• The Inference Algorithm.
![Page 29: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/29.jpg)
Input
Boundary graph
Result
Processing flow
Inference Algorithm
Pb boundary
CRF potential energy function design
![Page 30: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/30.jpg)
Reminiscent of CDT (Ren et al, ICCV’05, IJCV’08)
• Our completion-edge proposal method is inspired by, similar to, yet more flexible than Ren et al’s CDT (Constrained Delaunay Triangulation) method.
• X Ren, C Fowlkes and J Malik, Scale-Invariant Contour Completion using Conditional Random Fields., ICCV '05.
![Page 31: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/31.jpg)
The energy function
The decision variable Y= {yi} indicates whether or not the (ith) edgelet should be kept on the final contour.
yi=1 : on a contour (to keep); yi=0 : otherwise (to remove).
31
Y|XX
1
1 exp Y|X
Y|X ,x Y ,X Y Y,XJ P H
N
D i i S q q q Mi q C q C C
P EZ
E y
Data term, Junction term, Closure term, Complexity term
![Page 32: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/32.jpg)
The decision variable Y
The decision variable Y= {yi} indicates whether or not the (i-th) edgelet should be kept on the final contour.
yi=1 : on a contour (to keep); yi=0 : otherwise (to discard).
32
![Page 33: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/33.jpg)
Data TermData term: weighted Pb (local-evidence)
33
Y|XX
1
1 exp Y|X
Y|X ,x Y ,X Y Y,XJ P H
N
D i i S q q q Mi q C q C C
P EZ
E y
![Page 34: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/34.jpg)
Junction Term
Junction potential measures the likelihood of “good- continuation” and “smoothness” etc local Gestalt principles.
34
Y|XX
1
1 exp Y|X
Y|X ,x Y ,X Y Y,XJ P H
N
D i i S q q q Mi q C q C C
P EZ
E y
![Page 35: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/35.jpg)
We propose two types of “completion junctions” to fill in the gap between edgelets
![Page 36: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/36.jpg)
Junction potentials• L-junction potential measures likelihood of the
smoothness and good continuation Gestalt principles;
• T-junction potential captures the likelihood of occluding/occluded relationships, defined as triplet.
36
![Page 37: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/37.jpg)
Model-complexity Term
Model complexity potential:
37
Y|XX
1
1 exp Y|X
Y|X ,x Y ,X Y Y,XJ P H
N
D i i S q q q Mi q C q C C
P EZ
E y
•Similar to the MDL idea, to control the sparseness.
![Page 38: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/38.jpg)
Summary: the potential terms
38
Y|XX
1
1 exp Y|X
Y|X ,x Y ,X Y Y,XJ P H
N
D i i S q q q Mi q C q C C
P EZ
E y
Data term, Junction term, Closure term, Complexity term
•MDL•sparsity
Pb value
![Page 39: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/39.jpg)
The closure-effect term ?
39
Y|XX
1
1 exp Y|X
Y|X ,x Y ,X Y Y,XJ P H
N
D i i S q q q Mi q C q C C
P EZ
E y
Data term, Junction term, Closure term, Complexity term
•MDL•sparsity
Pb value
?
![Page 40: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/40.jpg)
Design the closure-effect potential
– We said that, being global, the closure-effect is hard to encode.
– Instead, we approximate (relax) it by a series of local connectedness constraints.
– The main intuition is to discourage/penalize any floating, isolated, disconnected boundaries or edge segments (i.e. edgelets) that violate the connectedness condition.
40
![Page 41: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/41.jpg)
• To do so, we enumerate, in the local, all possible types of (spatial) configurations that violate the Gestalt principle of closure.
• We represent such configuration violations by some mathematical conditions (actually, using linear inequalities).
• We are therefore able to quantitatively measure the closure- effect simply by counting the number of violations.
• Eventually the closure-effect potential function is written in pseudo-Boolean form.
![Page 42: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/42.jpg)
42
Two basic types of violations
Completion inequality requires that each of the (virtual) completion edgelets must connect to real, observed edgelets; Extension inequality requires that a real edgelet must connect to virtual completion edgelets.
![Page 43: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/43.jpg)
We simply count the number of #(violated inequalities) as the closure-potential
We use a Pseudo-Boolean potential function to encode the closure effect.
43
Y|XX
1
1 exp Y|X
Y|X ,x Y ,X Y Y,XJ P H
N
D i i S q q q Mi q C q C C
P EZ
E y
![Page 44: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/44.jpg)
Piece them together: factor graph topology
44
![Page 45: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/45.jpg)
Parameter learning scheme
• Logistic regression with piecewise learning strategy is used to learn the parameters.
• Use BSDS300 human labelling results as the ground-truth.
• Training time: in minutes.
![Page 46: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/46.jpg)
Where are we now
• We mentioned that, in contrast to other local Gestalt principles, the “closure effect” is hard to encode, hard to compute.
• Now, we have encoded the “closure effect” in our CRF model.
• Next, we consider how to compute (i.e. do inference on) the CRF, hopefully in an efficient way.
![Page 47: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/47.jpg)
Our new method explained
• The CRF model– Graphical model construction.– Potential functions design.
• The Inference Algorithm
![Page 48: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/48.jpg)
Inference
•Doing direct (conventional) inference on our CRF is very hard and inefficient. ( average clique size is 15~20)
•Reasons: – higher-order nonlinear terms
cubic and higher order terms.
– (very) large, global clique size. not very suitable for LBP, or QPBO,– non-submodularity. not very suitable for max-flow graph-cut.
48
![Page 49: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/49.jpg)
Proof of the non-submodularity
•For detailed proof please see the Suppl. Material of the paper.
![Page 50: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/50.jpg)
Our solution
• Since Graph-cut, BP, QPBO, are not suitable,
• Standard combinatorial optimization is too slow,
• We propose a hand-crafted, tailored inference algorithm, by exploiting the special structure of the solution space of the problem.
![Page 51: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/51.jpg)
Energy Function Reduction
• Key observation:– We notice that, if none of the “closure effect” is
violated, then the “junction potentials” can be reduced to linear terms (from 3rd order (cubic) terms).
– In addition, all the closure-effect terms are represented in the original linear inequality form.
![Page 52: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/52.jpg)
Now that, the inference problem becomes (Integer) linear programming
• Minimizing linear function, subject to a set of linear inequality constraints, over binary variables Y.
• (0-1 Integer) Linear Program (ILP).
52
Y
min Y|XE
![Page 53: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/53.jpg)
Experiments and Results
![Page 54: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/54.jpg)
Result: on synthetic images
54
•Our Contour extraction results
![Page 55: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/55.jpg)
Step-by-step sample result
![Page 56: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/56.jpg)
Sample Results
56
Probability of Boundary
Our ResultHuman labeling
![Page 57: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/57.jpg)
Sample Results
Clean, smooth, closed, connected,…
![Page 58: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/58.jpg)
More results
Clean, smooth, closed, connected, …
![Page 59: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/59.jpg)
Results: change of model-complexity parameter
59
0
1
2
3
![Page 60: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/60.jpg)
More results: natural images
60
Clean, smooth, closed, connected,…
![Page 61: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/61.jpg)
More results
Clean, smooth, closed, connected,…
![Page 62: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/62.jpg)
Pb
CDT
C-cut
Ours62
Qualitative Visual Comparison
![Page 63: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/63.jpg)
Quantitative benchmarking (against BSDS300 human ground-truth labelling)
63
![Page 64: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/64.jpg)
Efficacy of the “closure effect”
64
![Page 65: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/65.jpg)
Our results: while are visually pleasing
Drawbacks? topologically not meaningful !
![Page 66: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/66.jpg)
Drawbacks• Although there are no floating,
isolated contours or boundaries, each of the found boundaries does not necessarily enclose a sensible region.
• Topologically meaningless !
• If region information is used (in conjunction), the result should be much improved.
![Page 67: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/67.jpg)
We want to get a more meaningful region/contour segmentation
• © Picture stolen from OBJCUT
![Page 68: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/68.jpg)
Topic-2: Figure-ground segmentation by
combining both contour-cues and region-cues
Today’s technical talk
![Page 69: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/69.jpg)
Motivation
• Contour extraction is a challenging problem due to noises, low contrast,etc.
• Boundary-cue and region-cue are complementary visual cues that needs to be used jointly in combination.
Left: boundary cue for image segmentation; Right: Regional cue for image segmentation (figure credit: T. Cour).
![Page 70: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/70.jpg)
Formulation: joint energy minimization
• Min E_contour(x) + lambda*E_region(y)
• Contour cues: intensity contrast, colour changes, continuity, smoothness, curvature,..
– Methods: snake, active contour,..
• Region cues: intra-region colour similarity, homogeneity, spatial closeness.
•– Methods: clustering, normalized-cut, graph-cut…
![Page 71: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/71.jpg)
An issue
• Why and how to ensure that region-labels and contour- labels are consistent (compatible) ?
min EnergyFunction region_labels, contour_labels. . region_labels and contour_labels are consistents t
![Page 72: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/72.jpg)
Region-Contour Consistency Condition
• if a contour label is active, its two adjacent regions must have different region labels;
• if two adjacent regions have different labels, the boundary in- between must be active.
![Page 73: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/73.jpg)
What if not consistent ?
• A region without proper boundary.
• A contour that does not enclose any meaningful region.
• Cannot provide meaningful partition of the image.
![Page 74: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/74.jpg)
Our goal
min EnergyFunction region_labels, contour_labels. . region_labels and contour_labels are consistents t
![Page 75: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/75.jpg)
Existing solutions are mostly based on local, microscopic analysis
• E.g.:
![Page 76: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/76.jpg)
Lead to a large number of constraints that need to be handled, and many degenerate cases to be resolved
![Page 77: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/77.jpg)
Another example
![Page 78: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/78.jpg)
Our idea: a global topological solution
![Page 79: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/79.jpg)
How to properly fence a house ?
• Is the house securely fenced ?
![Page 80: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/80.jpg)
Flood-fill the yard ?
![Page 81: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/81.jpg)
A quicker way to test
![Page 82: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/82.jpg)
Works on a directed graph too !
• Interior point test, point-in-polygon test.
![Page 83: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/83.jpg)
Winding number
• By definition, the winding number of a closed curve in the plane around a given point is an integer representing the total number of times that curve travels counter- clockwise around the point.
• The winding number depends on the orientation of the curve, and is negative if the curve travels around the point clockwise.
![Page 84: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/84.jpg)
Winding number explained• Suppose we are given a
closed, oriented curve in the xy plane.
• We can imagine the curve as the path of motion of a point, with the orientation indicating the direction in which the object moves.
• Then the winding number of the curve is equal to the total number of counter-clockwise turns that the object makes around the origin.
![Page 85: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/85.jpg)
Math definition: residue theorem
The winding number of a contour about a point is defined by:
![Page 86: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/86.jpg)
Winding number: fast computation
: Winding number for region i ;
, : count contour crossing.
: contour crossing from right: contour crossing from left
![Page 87: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/87.jpg)
Our key intuition
We observe that, in region-contour segmentation, if we use a region’s winding number to denote its region-label, then the following set of linear constraints automatically ensures the region-label and the incident contour-labels are consistent.
• This is because:
• Now, we have • the dependency.
![Page 88: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/88.jpg)
Recall: Our goal
min EnergyFunction region_labels, contour_labels. . region_labels and contour_labels are consistents t
![Page 89: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/89.jpg)
![Page 90: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/90.jpg)
Formulation: ratio optimization
![Page 91: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/91.jpg)
Solve the ratio optimization by LP
• Optimization of this linear-fractional objective function, subject to linear winding number constraints, can be done as a Linear Program (by 0-1 relaxation, and the Charnes-Cooper transformation).
Let denotes all the variables, and c, d, e, f coefficients.
![Page 92: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/92.jpg)
Alternatively, it may also be solvable via “parametric max-flow algorithm” or “negative cycle algorithm”
• But we have not explored these options further, as our method (a single LP) is rather efficient.
![Page 93: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/93.jpg)
Implementation
• Start from super-pixel over segmentation.
![Page 94: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/94.jpg)
Experiment results comparison
![Page 95: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/95.jpg)
![Page 96: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/96.jpg)
F-index on the horse dataset (1 solution competes with 10 solutions)
![Page 97: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/97.jpg)
![Page 98: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/98.jpg)
F-index on the shape dataset
![Page 99: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/99.jpg)
![Page 100: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/100.jpg)
Add human interaction
![Page 101: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/101.jpg)
Current ongoing/future work
• Extend to multiple-label case (multiple regions).
• Adding high-level semantic (category- specific) information, to the salient contour extraction task.
![Page 102: A talk to be presented at the Oxford Robotics Research ...seminars/seminars/Extra/2013_06_09_Hongd… · September 2013. Outline • 1. ... Multiple Camera-Rig Video Odometry. Li](https://reader036.vdocuments.us/reader036/viewer/2022081600/60478d259e6a573bfa76e32f/html5/thumbnails/102.jpg)
What I have covered in this talk
• Topic-1: Contour extraction w.r.t. closure-effect.
• Topic-2: Joint contour-region segmentation w/ winding number. • Low level Vision
More results can be found in CVPR 2012, CVPR 2013, ICIP2014, CVPR 2014 (in preparation).
• High level Vision
• Mid-level Vision