diverse m-best solutions in markov random fieldsdbatra/cvpr13diversity... · cvpr 2013 diversity...
TRANSCRIPT
![Page 1: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/1.jpg)
CVPR 2013 Diversity Tutorial
Diverse M-Best Solutions in Markov Random Fields
Dhruv Batra Virginia Tech
Joint work with: Students: Payman Yadollahpour (TTIC), Abner Guzman-Rivera (UIUC) Colleagues: Chris Dyer (CMU), Greg Shakhnarovich (TTIC),
Pushmeet Kohli (MSRC), Kevin Gimpel (TTIC)
![Page 2: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/2.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 2
![Page 3: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/3.jpg)
CVPR 2013 Diversity Tutorial
Ambiguity Ambiguity Ambiguity
(C) Dhruv Batra 3
?
?
One instance / Two instances?
![Page 4: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/4.jpg)
CVPR 2013 Diversity Tutorial
Problems with MAP
(C) Dhruv Batra 4
Model-Class is Wrong! -- Approximation Error
Not Enough Training Data! -- Estimation Error MAP is NP-Hard
-- Optimization Error Inherent Ambiguity
-- Bayes Error Make Multiple Predictions!
Single Prediction = Uncertainty Mismanagement
![Page 5: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/5.jpg)
CVPR 2013 Diversity Tutorial
Multiple Predictions
(C) Dhruv Batra 5
XMAP
X
P (X )
Porway & Zhu, 2011!TU & Zhu, 2002!Rich History!
Sampling
x x x x x x x x x x x x x
![Page 6: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/6.jpg)
CVPR 2013 Diversity Tutorial
Multiple Predictions
(C) Dhruv Batra 6
Flerova et al., 2011!Fromer et al., 2009!Yanover et al., 2003!
M-Best MAP Ideally:
M-Best Modes ✓!
XMAP
X
P (X )
Porway & Zhu, 2011!TU & Zhu, 2002!Rich History!
Sampling
![Page 7: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/7.jpg)
CVPR 2013 Diversity Tutorial
Multiple Predictions
(C) Dhruv Batra 7
Flerova et al., 2011!Fromer et al., 2009!Yanover et al., 2003!
M-Best MAP Ideally:
M-Best Modes ✓!Porway & Zhu, 2011!
TU & Zhu, 2002!Rich History!
Sampling
XMAP
X
P (X )
Our work: Diverse M-Best in MRFs [ECCV ‘12]
- Don’t hope for diversity. Explicitly encode it.
- Not guaranteed to be modes.
![Page 8: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/8.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 8
CRF
. . .
Diverse Segmentations
Re-ranked List
Top Solution
Re-ranker α�ψ(x,y)
Example Result
![Page 9: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/9.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 9
CRF
. . .
Diverse SegmentationsStage 1
Re-ranker
Re-ranked List
Top Solution
α�ψ(x,y)
. . .
Stage 2
Example Result
Discriminative Re-ranking of Diverse Segmentation
[Yadollahpour et al., CVPR13, Wednesday Poster]
![Page 10: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/10.jpg)
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 10
θi(s)
kx1
µi(s)�
i
maxy
S(y) = θi(yi) +�
(i,j)
θij(yi, yj)
![Page 11: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/11.jpg)
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 11
θi(s)
kx1
1
0
0
0
�
i
maxy
S(y) = +�
(i,j)
θij(yi, yj)
![Page 12: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/12.jpg)
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 12
θi(s)
kx1
0
1
0
0
�
i
maxy
S(y) = +�
(i,j)
θij(yi, yj)
![Page 13: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/13.jpg)
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 13
θi(s)
kx1
0
0
1
0
�
i
maxy
S(y) = +�
(i,j)
θij(yi, yj)
![Page 14: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/14.jpg)
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 14
θi(s)
kx1
0
0
0
1
�
i
maxy
S(y) = +�
(i,j)
θij(yi, yj)
![Page 15: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/15.jpg)
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 15
θi(s)
kx1
0
0
0
1
θij(s, t)�
i
+�
(i,j)
k2x1
µij(s, t)maxy
S(y) =
![Page 16: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/16.jpg)
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 16
θi(s)
kx1
0
0
0
1
�
i
+�
(i,j)
XMAP
X
P (X )
maxµ∈C
�
i
θi · µi +�
(i,j)
θij · µij
s.t. µi(·),µij(·) ∈ {0, 1}
θij(s, t)
k2x1
µij(s, t)maxy
S(y) =
![Page 17: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/17.jpg)
CVPR 2013 Diversity Tutorial
MAP Integer Program
(C) Dhruv Batra 17
Graphcuts, BP, Expansion, etc
XMAP
X
P (X )
maxµ∈C
�
i
θi · µi +�
(i,j)
θij · µij
s.t. µi(·),µij(·) ∈ {0, 1}
![Page 18: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/18.jpg)
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best
(C) Dhruv Batra 18
∆(µ,µ(1)) ≥ k
XMAP
X
P (X )
MAP
∆(µ,µ(1)) ≥ k
Diversity
maxµ∈C
�
i
θi · µi +�
(i,j)
θij · µij
s.t. µi(·),µij(·) ∈ {0, 1}
![Page 19: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/19.jpg)
CVPR 2013 Diversity Tutorial
Diverse M-Best
(C) Dhruv Batra 19
∆(µ,µ(1)) ≥ k
∆(µ,µ(2)) ≥ k
∆(µ,µ(M−1)) ≥ k
XMAP
X
P (X )
∆(µ,µ(M−1)) ≥ k
∆(µ,µ(1)) ≥ k
∆(µ,µ(2)) ≥ k
maxµ∈C
�
i
θi · µi +�
(i,j)
θij · µij
s.t. µi(·),µij(·) ∈ {0, 1}
![Page 20: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/20.jpg)
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best
(C) Dhruv Batra 20
∆(µ,µ(1)) ≥ k
Q1: How do we solve DivMBest?
Q2: What kind of diversity functions are allowed?
Q3: How much diversity?
maxµ∈C
�
i
θi · µi +�
(i,j)
θij · µij
s.t. µi(·),µij(·) ∈ {0, 1}
![Page 21: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/21.jpg)
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best
(C) Dhruv Batra 21
Dualize
Diversity-Augmented Score
Primal
∆(µ,µ(1)) ≥ k
maxµ∈C
�
i
θi · µi +�
(i,j)
θij · µij
s.t. µi(·),µij(·) ∈ {0, 1}
+λ ·�∆(µ,µ(1))− k
�
XMAP
X
P (X )
S(y) + Div(y,y(1))
![Page 22: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/22.jpg)
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best • Lagrangian Relaxation
(C) Dhruv Batra 22
Diversity-Augmented Score
Dual
Concave (Non-smooth)
Upper-Bound on Div2Best Score
f(λ) =
Subgradient Descent
∇f(λ(0))
λ(0)
∇f(λ(1))
λ(1)
maxµ∈C
�
i
θi · µi +�
(i,j)
θij · µij
s.t. µi(·),µij(·) ∈ {0, 1}
+λ ·�∆(µ,µ(1))− k
�
minλ≥0
f(λ)
∇f(λ) =�∆(µλ,µ
(1))− k�
λ
Div2Best score
f(λ)
![Page 23: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/23.jpg)
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best • Lagrangian Relaxation
(C) Dhruv Batra 23
Dualize
Diversity-Augmented Energy
−λ ·�∆(µ,µ(1))− k
�f(λ) = min
µ∈C
�
i
θi · µi +�
(i,j)
θij · µij
s.t. µi(·),µij(·) ∈ {0, 1}∆(µ,µ(1)) ≥ k
f(λ) =
Many ways to solve:
1. Subgradient Ascent. Optimal. Slow.
2. Binary Search. Optimal for M=2. Faster.
3. Grid-search on lambda. Sub-optimal. Fastest.
![Page 24: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/24.jpg)
CVPR 2013 Diversity Tutorial
Theorem Statement • Theorem [Batra et al ’12]: Lagrangian Dual
corresponds to solving the Relaxed Primal: • Based on result from [Geoffrion ‘74]
(C) Dhruv Batra 24
Dual
maxµ
�
i
θi · µi +�
ij
θij · µij
s.t. µ ∈ Co�µi(·),µij(·) ∈ {0, 1} | µ ∈ C
�
∆(µ,µ(1)) ≥ k
Relaxed Primal
minλ≥0
LagrangianDual(λ)
![Page 25: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/25.jpg)
CVPR 2013 Diversity Tutorial
Effect of Lagrangian Relaxation
(C) Dhruv Batra 25
µ(1)
µ(2)
µ(3) µ(4)
µ(5)
![Page 26: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/26.jpg)
CVPR 2013 Diversity Tutorial
Effect of Lagrangian Relaxation
(C) Dhruv Batra 26
µ(1)
µ(2)
µ(3) µ(4)
µ(5)
![Page 27: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/27.jpg)
CVPR 2013 Diversity Tutorial
Effect of Lagrangian Relaxation • [Mezuman et al. UAI13]
(C) Dhruv Batra 27
Pairwise Potential Strength Pairwise Potential Strength
![Page 28: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/28.jpg)
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best
(C) Dhruv Batra 28
∆(µ,µ(1)) ≥ k
Q1: How do we solve DivMBest?
Q2: What kind of diversity functions are allowed?
Q3: How much diversity?
minµ∈C
�
i
θi · µi +�
(i,j)
θij · µij
s.t. µi(·),µij(·) ∈ {0, 1}
![Page 29: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/29.jpg)
CVPR 2013 Diversity Tutorial
Diversity • [Special Case] 0-1 Diversity M-Best MAP
– [Yanover NIPS03; Fromer NIPS09; Flerova Soft11]
• [Special Case] Max Diversity [Park & Ramanan ICCV11]
• Hamming Diversity
• Cardinality Diversity
• Any Diversity
(C) Dhruv Batra 29
=⇒
=⇒
maxµ∈C
S(µ) + λ∆(µ,µ(1))
![Page 30: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/30.jpg)
CVPR 2013 Diversity Tutorial
Hamming Diversity
(C) Dhruv Batra 30
∆(µ,µ(1)) = −�
i∈Vµi · µ(1)
i
0
1
0
0
0 1 0 0 = 1
0
1
0
0
1 0 0 0 = 0
![Page 31: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/31.jpg)
CVPR 2013 Diversity Tutorial
Hamming Diversity
• Diversity Augmented Inference:
(C) Dhruv Batra 31
θ̃i
∆(µ,µ(1)) = −�
i∈Vµi · µ(1)
i
maxµ∈C
�
i
θi · µi +�
(i,j)
θij · µij + λ∆(µ,µ(1))
=�
i
�θi − λµ(1)
i
�· µi +
�
(i,j)
θij · µij
![Page 32: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/32.jpg)
CVPR 2013 Diversity Tutorial
Hamming Diversity
• Diversity Augmented Inference:
(C) Dhruv Batra 32
∆(µ,µ(1)) = −�
i∈Vµi · µ(1)
i
Unchanged. Can still use graph-cuts!
Simply edit node-terms. Reuse MAP machinery!
for i = 1,2,. . .,n
θi[x(1)i ] -= λ
endfor
x(2) = Find MAP(θi, θij)
![Page 33: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/33.jpg)
CVPR 2013 Diversity Tutorial
Diverse 2nd-Best
(C) Dhruv Batra 33
∆(µ,µ(1)) ≥ k
Q1: How do we solve DivMBest?
Q2: What kind of diversity functions are allowed?
Q3: How much diversity?
minµ∈C
�
i
θi · µi +�
(i,j)
θij · µij
s.t. µi(·),µij(·) ∈ {0, 1}
![Page 34: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/34.jpg)
CVPR 2013 Diversity Tutorial
How Much Diversity?
• Empirical Solution: Cross-Val for
• More Efficient: Cross-Val for
(C) Dhruv Batra 34
X
P (X )
λ
k
![Page 35: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/35.jpg)
CVPR 2013 Diversity Tutorial
Experiments • 3 Applications
– Interactive Segmentation: Hamming, Cardinality (in paper) – Pose Estimation: Hamming – Semantic Segmentation: Hamming
• Baselines: – M-Best MAP (No Diversity) – Confidence-Based Perturbation (No Optimization)
(C) Dhruv Batra 35
![Page 36: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/36.jpg)
CVPR 2013 Diversity Tutorial
Interactive Segmentation • Setup
– Model: Color/Texture + Potts Grid CRF – Inference: Graph-cuts – Dataset: 50 train/val/test images
(C) Dhruv Batra 36
Image + Scribbles Diverse 2nd Best 2nd Best MAP MAP
1-2 Nodes Flipped 100-500 Nodes Flipped
![Page 37: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/37.jpg)
CVPR 2013 Diversity Tutorial
Pose Tracking • Setup
– Model: Mixture of Parts from [Park & Ramanan, ICCV ‘11] – Inference: Dynamic Programming – Dataset: 4 videos, 585 frames
(C) Dhruv Batra 37 Image Credit: [Yang & Ramanan, ICCV ‘11]
![Page 38: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/38.jpg)
CVPR 2013 Diversity Tutorial
(C) Dhruv Batra 38
![Page 39: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/39.jpg)
CVPR 2013 Diversity Tutorial
Pose Tracking • Chain CRF with M states at each time
(C) Dhruv Batra 39
M Best Solutions
Image Credit: [Yang & Ramanan, ICCV ‘11]
![Page 40: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/40.jpg)
CVPR 2013 Diversity Tutorial
Pose Tracking
(C) Dhruv Batra 40
DivMBest + Viterbi MAP
![Page 41: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/41.jpg)
CVPR 2013 Diversity Tutorial
Pose Tracking
(C) Dhruv Batra 41
45%
50%
55%
60%
65%
70%
75%
80%
85%
1 51 101 151 201 251 301
DivMBest (Re-ranked)
[Park & Ramanan, ICCV ‘11] (Re-ranked)
Confidence-based Perturbation (Re-ranked)
13% Gain
Same Features Same Model
#Solutions / Frame
PC
P A
ccur
acy
Better
![Page 42: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/42.jpg)
CVPR 2013 Diversity Tutorial
Machine Translation Input: Die Regierung will die Folter von “Hexen” unterbinden und gab eine Broschüre heraus MAP Translation: The government wants the torture of ‘witch’ and gave out a booklet
(C) Dhruv Batra 42
![Page 43: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/43.jpg)
CVPR 2013 Diversity Tutorial
Machine Translation Input: Die Regierung will die Folter von “Hexen” unterbinden und gab eine Broschüre heraus 5-Best Translations: The government wants the torture of ‘witch’ and gave out a booklet The government wants the torture of “witch” and gave out a booklet The government wants the torture of ‘witch’ and gave out a brochure The government wants the torture of ‘witch’ and gave out a leaflet The government wants the torture of “witch” and gave out a brochure
(C) Dhruv Batra 43
![Page 44: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/44.jpg)
CVPR 2013 Diversity Tutorial
Machine Translation Input: Die Regierung will die Folter von “Hexen” unterbinden und gab eine Broschüre heraus Diverse 5-Best Translations: The government wants the torture of ‘witch’ and gave out a booklet The government wants to stop torture of “witch” and issued a leaflet issued The government wants to “stop the torture of” witches and gave out a brochure The government intends to the torture of “witchcraft” and were issued a leaflet The government is the torture of “witches” stamp out and gave a brochure
(C) Dhruv Batra 44
![Page 45: Diverse M-Best Solutions in Markov Random Fieldsdbatra/cvpr13diversity... · CVPR 2013 Diversity Tutorial Machine Translation Input:! Die Regierung will die Folter von “Hexen”](https://reader033.vdocuments.us/reader033/viewer/2022042300/5ecab07492b41c303020996e/html5/thumbnails/45.jpg)
CVPR 2013 Diversity Tutorial
Machine Translation Input: Die Regierung will die Folter von “Hexen” unterbinden und gab eine Broschüre heraus Diverse 5-Best Translations: The government wants the torture of ‘witch’ and gave out a booklet The government wants to stop torture of “witch” and issued a leaflet issued The government wants to “stop the torture of” witches and gave out a brochure The government intends to the torture of “witchcraft” and were issued a leaflet The government is the torture of “witches” stamp out and gave a brochure
Correct Translation: The government wants to limit the torture of “witches,” a brochure was released
(C) Dhruv Batra 45