pushmeet kohli microsoft research - damtp...autocollage robotics darpa urban challenge p(x|z) ∏...
TRANSCRIPT
![Page 1: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/1.jpg)
Pushmeet Kohli Microsoft Research
![Page 2: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/2.jpg)
![Page 3: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/3.jpg)
Infer the values of hidden random variables
Geometry Estimation Object Segmentation
Building
Sky
Tree Grass
Pose Estimation
Orientation Joint angles
![Page 4: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/4.jpg)
Medical Imaging
Image Search
Human Computer Interaction
Minority Report (2002)
Image and
Video Editing
Autocollage
Robotics
DARPA Urban Challenge
![Page 5: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/5.jpg)
![Page 6: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/6.jpg)
P(x|z) P(z|xi) ∏ xi
P(xi,xj) ∏ xi,xj Posterior Probability
Object Segmentation
Building
Sky
Tree Grass
xi ϵ Labels L = {l1, l2, … , lk}
=
Likelihood Term Pairwise Consistency Prior
x1 x2
![Page 7: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/7.jpg)
P(x|z) P(z|xi) ∏ xi
P(xi,xj) ∏ xi,xj Posterior Probability
Object Segmentation
Building
Sky
Tree Grass
xi ϵ Labels L = {l1, l2, … , lk}
=
Likelihood Term Pairwise Consistency Prior
Left image Right image
Disparity map
Depth Estimation
![Page 8: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/8.jpg)
![Page 9: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/9.jpg)
E(X) E: {0,1}n → R 0 → fg
1 → bg
Image (D)
n = number of pixels
[Boykov and Jolly ‘ 01] [Blake et al. ‘04] [Rother, Kolmogorov and Blake `04]
![Page 10: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/10.jpg)
Unary Cost (ci) Dark (negative) Bright (positive)
E: {0,1}n → R 0 → fg
1 → bg
n = number of pixels
[Boykov and Jolly ‘ 01] [Blake et al. ‘04] [Rother, Kolmogorov and Blake `04]
∑ fi (xi)
Pixel Colour
E(X) =
![Page 11: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/11.jpg)
Unary Cost (ci) Dark (negative) Bright (positive)
E: {0,1}n → R 0 → fg
1 → bg
n = number of pixels
[Boykov and Jolly ‘ 01] [Blake et al. ‘04] [Rother, Kolmogorov and Blake `04]
x* = arg min E(x)
∑ fi (xi)
Pixel Colour
E(X) =
![Page 12: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/12.jpg)
Discontinuity Cost (dij)
E: {0,1}n → R 0 → fg
1 → bg
n = number of pixels
[Boykov and Jolly ‘ 01] [Blake et al. ‘04] [Rother, Kolmogorov and Blake `04]
∑ fi (xi)
Pixel Colour
E(X) = + ∑ gij (xi,xj)
Smoothness Prior
![Page 13: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/13.jpg)
Discontinuity Cost (dij)
E: {0,1}n → R 0 → fg
1 → bg
n = number of pixels
[Boykov and Jolly ‘ 01] [Blake et al. ‘04] [Rother, Kolmogorov and Blake `04]
∑ fi (xi)
Pixel Colour
E(X) = + ∑ dij|xi-xj|
Smoothness Prior
![Page 14: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/14.jpg)
Global Minimum (x*)
x* = arg min E(x) x
How to minimize E(x)?
E: {0,1}n → R 0 → fg
1 → bg
n = number of pixels
[Boykov and Jolly ‘ 01] [Blake et al. ‘04] [Rother, Kolmogorov and Blake `04]
∑ fi (xi)
Pixel Colour
E(X) = + ∑ dij|xi-xj|
Smoothness Prior
![Page 15: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/15.jpg)
Space of Problems
Submodular
Functions
CSP
Tree Structured
Pair-wise O(n3)
MAXCUT
O(n6)
n = Number of Variables
Segmentation Energy
NP-Hard
![Page 16: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/16.jpg)
![Page 17: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/17.jpg)
.. So what are the challenges?
![Page 18: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/18.jpg)
Image Segmentation
Semantic Segmentation
Texture Denoising
Penalizes all discontinuities
Pairwise Models are Weak
Does not enforce “group” consistency
Does not enforce consistency
of local structure
![Page 19: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/19.jpg)
![Page 20: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/20.jpg)
Unary Potentials [Shotton et al. ECCV 2006]
+ Pairwise Smoothness
Potentials
Higher Order Energy
Colour, Location & Texture Sky
Building
Tree
Grass
Higher Order Potentials (Defined using multiple
Segmentations)
+
Kohli Ladicky Torr [CVPR 2008]
{ 0 if xi=L, I ϵ p C otherwise
h(Xp) =
p
Pixels belonging to a group should take the
same label
![Page 21: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/21.jpg)
Unary Potentials [Shotton et al. ECCV 2006]
+ Pairwise Smoothness
Potentials
+
Higher Order Energy
Colour, Location & Texture Sky
Building
Tree
Grass
Kohli Ladicky Torr [CVPR 2008]
{ 0 if xi=L, i ϵ p C otherwise h(Xp) =
Higher Order Potentials (Defined using multiple
Segmentations)
![Page 22: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/22.jpg)
Unary Potentials [Shotton et al. ECCV 2006]
+ Pairwise Smoothness
Potentials
Segmentation Solution
+
Higher Order Energy
Energy Minimization
Colour, Location & Texture Sky
Building
Tree
Grass
Kohli Ladicky Torr [CVPR 2008]
Higher Order Potentials (Defined using multiple
Segmentations)
![Page 23: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/23.jpg)
Image
Unary Classifiers Pairwise CRF
Pn Potts
Robust Pn Potts
Clustering 1 Clustering 2
![Page 24: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/24.jpg)
Image (MSRC-21)
Pairwise CRF
Higher order CRF
Ground Truth
Sheep
Grass
[Runner-Up, PASCAL VOC 2008 ]
![Page 25: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/25.jpg)
![Page 26: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/26.jpg)
Test Image Test Image (60% Noise)
Training Image
Result
Pairwise Energy
P(x)
Minimized using st-mincut or max-product
message passing
Higher Order Structure not Preserved
![Page 27: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/27.jpg)
Minimize:
Where:
Higher Order Function (|c| = 10x10 = 100)
Assigns cost to 2100 possible labellings!
Exploit function structure to transform it to a Pairwise function
E(X) = P(X) + ∑ hc (Xc) c
hc: {0,1}|c| → R
p1 p2 p3
![Page 28: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/28.jpg)
Test Image Test Image (60% Noise)
Training Image
Pairwise Result
Higher-Order Result
Learned Patterns
![Page 29: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/29.jpg)
Identified transformable families of higher order function s.t.
1. Constant or polynomial number of auxiliary variables (a) added
2. All pairwise functions (g) are submodular
Pairwise Submodular
Function
Higher Order Function
![Page 30: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/30.jpg)
H (X) = F ( ∑ xi ) Example:
H (X)
∑ xi
concave
[Kohli et al. ‘08]
0
![Page 31: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/31.jpg)
Simple Example using Auxiliary variables
{ 0 if all xi = 0 C1 otherwise f(x) =
min f(x) min C1a + C1 ∑ ā xi
x ϵ L = {0,1}n
x = x,a ϵ {0,1}
Higher Order Submodular Function
Quadratic Submodular Function
∑xi < 1 a=0 (ā=1) f(x) = 0
∑xi ≥ 1 a=1 (ā=0) f(x) = C1
![Page 32: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/32.jpg)
min f(x) min C1a + C1 ∑ ā xi x
= x,a ϵ {0,1}
Higher Order Submodular Function
Quadratic Submodular Function
∑xi
1 2 3
C1
C1∑xi
[Kohli et al. ‘08]
![Page 33: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/33.jpg)
min f(x) min C1a + C1 ∑ ā xi x
= x,a ϵ {0,1}
Higher Order Submodular Function
Quadratic Submodular Function
∑xi
1 2 3
C1
C1∑xi
a=1 a=0 Lower envelop of concave functions is
concave
[Kohli et al. ‘08]
![Page 34: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/34.jpg)
min f(x) min f1 (x)a + f2(x)ā x
= x,a ϵ {0,1}
Higher Order Submodular Function
Quadratic Submodular Function
∑xi
1 2 3
Lower envelop of concave functions is
concave
f2(x)
f1(x)
[Kohli et al. ‘08]
![Page 35: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/35.jpg)
min f(x) min f1 (x)a + f2(x)ā x
= x,a ϵ {0,1}
Higher Order Submodular Function
Quadratic Submodular Function
∑xi
1 2 3
a=1 a=0 Lower envelop of concave functions is
concave
f2(x)
f1(x)
[Kohli et al. ‘08]
![Page 36: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/36.jpg)
min f(x) min max f1 (x)a + f2(x)ā x
= x ϵ {0,1}
Higher Order Submodular Function
Quadratic Submodular Function
∑xi
1 2 3
a=1 a=0 Upper envelope
of linear functions is
convex
f2(x)
f1(x)
a ϵ {0,1}
Very Hard Problem!!!! [Kohli and Kumar, CVPR 2010]
![Page 37: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/37.jpg)
Object Prior on size of object (say n/2)
Background
BINARY SEGMENTATION
SIZE/VOLUME PRIORS
[Kohli and Kumar, CVPR 2010]
SILHOUETTE CONSTRAINTS
Rays must touch silhouette
at least once
3D RECONSTRUCTION
[Sinha et al. ‘05, Cremers et al. ’08] [Woodford et al. 2009]
![Page 38: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/38.jpg)
![Page 39: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/39.jpg)
Original Video
MSR
Edited Video
Alternative Representation
MSR
(Memory Efficient)
[Rav-Acha, Kohli, Rother & Fitzgibbon, SIGGRAPH 2008]
![Page 40: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/40.jpg)
A deforming 3D object
x
y x = (x, y) Note:
![Page 41: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/41.jpg)
u = (u, v) Note:
![Page 42: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/42.jpg)
Visibility Mask Warping
w(u,t) b(u,t)
![Page 43: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/43.jpg)
Texture map
![Page 44: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/44.jpg)
visibility mask warping Image Texture map
+ =
![Page 45: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/45.jpg)
+ =
visibility mask warping Image Texture map
![Page 46: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/46.jpg)
[Rav-Acha, Kohli, Rother & Fitzgibbon, SIGGRAPH 2008]
![Page 47: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/47.jpg)
Speed and Accuracy
![Page 48: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/48.jpg)
3,600,000,000 Pixels Created from about 800 8 MegaPixel Images
[Kopf et al. (MSR Redmond) SIGGRAPH 2007 ]
![Page 49: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/49.jpg)
[Kopf et al. (MSR Redmond) SIGGRAPH 2007 ]
![Page 50: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/50.jpg)
Speed and Scalability
Processing Videos 1 minute video of 1M pixel resolution
3.6 B pixels
3D reconstruction [500 x 500 x 500 = .125B voxels]
![Page 51: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/51.jpg)
Speed and Scalability
Dynamic Algorithms [ICCV2005][CVPR 2008][PAMI 2007] [PAMI 2010]
Reuse computation
Hybrid Algorithms [PAMI 2010]
Use different algorithms to solve easy/hard parts
Adaptive Algorithms [AISTATS 2010] [ICML 2011] [CVPR 2011]
Focus on the unsolved parts of the problem
![Page 52: Pushmeet Kohli Microsoft Research - DAMTP...Autocollage Robotics DARPA Urban Challenge P(x|z) ∏ P(z|x i) x i x ∏ P(x i,x j) Posterior Probability ,x j Object Segmentation Building](https://reader036.vdocuments.us/reader036/viewer/2022063014/5fcfd3faa137f66880658018/html5/thumbnails/52.jpg)