![Page 1: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/1.jpg)
High-Order Inference, Ranking, andRegularization Path for Structured SVM
Puneet Kumar DokaniaSupervisors: Prof. M. Pawan Kumar & Prof. Nikos Paragios
CentraleSupelec and INRIA Saclay
May 30, 2016
Puneet K. Dokania 1
![Page 2: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/2.jpg)
Thesis Overview
Presentation Outline
1 Thesis Overview
2 Parsimonious Labeling
3 Learning to Rank Using High-Order Information
4 Regularization Path for SSVM
5 Future Work
6 Publications
Puneet K. Dokania 2
![Page 3: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/3.jpg)
Thesis Overview
ick Overview
High-Order Inference: Parsimonious Labeling
E(x, y;w) =∑i∈V
θ(xi, yi;w) +∑c∈C
θc(xc, yc;w)︸ ︷︷ ︸diversity
HOAP-SVM: w very high-dimensional→ exhaustive search
min
w
λ
2
‖w‖2 + L(x, y;w)︸ ︷︷ ︸AP-Based
Regularization path for SSVM: Eiciently explore the entire space of
λ ∈ [0,∞]
Puneet K. Dokania 3
![Page 4: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/4.jpg)
Thesis Overview
ick Overview
High-Order Inference: Parsimonious Labeling
E(x, y;w) =∑i∈V
θ(xi, yi;w) +∑c∈C
θc(xc, yc;w)︸ ︷︷ ︸diversity
HOAP-SVM: w very high-dimensional→ exhaustive search
min
w
λ
2
‖w‖2 + L(x, y;w)︸ ︷︷ ︸AP-Based
Regularization path for SSVM: Eiciently explore the entire space of
λ ∈ [0,∞]
Puneet K. Dokania 3
![Page 5: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/5.jpg)
Thesis Overview
ick Overview
High-Order Inference: Parsimonious Labeling
E(x, y;w) =∑i∈V
θ(xi, yi;w) +∑c∈C
θc(xc, yc;w)︸ ︷︷ ︸diversity
HOAP-SVM: w very high-dimensional→ exhaustive search
min
w
λ
2
‖w‖2 + L(x, y;w)︸ ︷︷ ︸AP-Based
Regularization path for SSVM: Eiciently explore the entire space of
λ ∈ [0,∞]
Puneet K. Dokania 3
![Page 6: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/6.jpg)
Parsimonious Labeling
Presentation Outline
1 Thesis Overview
2 Parsimonious Labeling
3 Learning to Rank Using High-Order Information
4 Regularization Path for SSVM
5 Future Work
6 Publications
Puneet K. Dokania 4
![Page 7: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/7.jpg)
Parsimonious Labeling Background
The Labeling Problem
Input
Laice V = 1, · · · ,N, Random variables y = y1, · · · , yNA discrete label set L = l1, · · · , lHEnergy functional to assess the quality of each labeling y:
E(y) =∑i∈V
θi(yi) +∑c∈C
θc(yc). (1)
Output
Labeling corresponding to the minimum energy
y∗ = argmin
yE(y). (2)
HN possible labelings
Puneet K. Dokania 5
![Page 8: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/8.jpg)
Parsimonious Labeling Background
The Labeling Problem
Input
Laice V = 1, · · · ,N, Random variables y = y1, · · · , yNA discrete label set L = l1, · · · , lHEnergy functional to assess the quality of each labeling y:
E(y) =∑i∈V
θi(yi) +∑c∈C
θc(yc). (1)
Output
Labeling corresponding to the minimum energy
y∗ = argmin
yE(y). (2)
HN possible labelings
Puneet K. Dokania 5
![Page 9: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/9.jpg)
Parsimonious Labeling Background
Special case – Metric Labeling (Pairwise)
Pairwise Potentials θ(yi, yj)→Metric over the labels
Recall, distance function θ(yi, yj) : L × L → R+ is metric if:
Non Negative
Symmetric
Triangular Inequality
α−expansion1
– Very Eicient – Approximate solution
1Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, 2001.
Puneet K. Dokania 6
![Page 10: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/10.jpg)
Parsimonious Labeling Background
Special case – PnPos Model
2(High-Order)
Pn Pos Model
θc(yc) ∝
γk , if yi = lk ,∀i ∈ c,
γmax , otherwise,
Very eicient α-expansion algorithm – Approximate solution
2Kohli et al., P3 & Beyond: Solving Energies with Higher Order Cliques, 2007.
Puneet K. Dokania 7
![Page 11: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/11.jpg)
Parsimonious Labeling Background
Special case – PnPos Model
2(High-Order)
Pn Pos Model
θc(yc) ∝
γk , if yi = lk ,∀i ∈ c,
γmax , otherwise,
Very eicient α-expansion algorithm – Approximate solution
2Kohli et al., P3 & Beyond: Solving Energies with Higher Order Cliques, 2007.
Puneet K. Dokania 7
![Page 12: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/12.jpg)
Parsimonious Labeling Background
Special case – PnPos Model
2(High-Order)
Pn Pos Model
θc(yc) ∝
γk , if yi = lk , ∀i ∈ c,
γmax , otherwise,
Very eicient α-expansion algorithm – Approximate solution
2Kohli et al., P3 & Beyond: Solving Energies with Higher Order Cliques, 2007.
Puneet K. Dokania 7
![Page 13: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/13.jpg)
Parsimonious Labeling The Energy Function
Parsimonious Labeling: Energy Function
E(y) =∑i∈V
θi(yi) +∑c∈C
θc(yc).
Unary potentials: Arbitrary
Clique potentials: Diversity
θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity
where, Γ(yc) is the set of unique
labels δc(l1, l2, l3)Energy function for Parsimonious Labeling
E(y) =∑i∈V
θi(yi) +∑c∈C
wcδ(Γ(yc))︸ ︷︷ ︸Diversity
Puneet K. Dokania 8
![Page 14: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/14.jpg)
Parsimonious Labeling The Energy Function
Parsimonious Labeling: Energy Function
E(y) =∑i∈V
θi(yi) +∑c∈C
θc(yc).
Unary potentials: Arbitrary
Clique potentials: Diversity
θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity
where, Γ(yc) is the set of unique
labels
δc(l1, l2, l3)Energy function for Parsimonious Labeling
E(y) =∑i∈V
θi(yi) +∑c∈C
wcδ(Γ(yc))︸ ︷︷ ︸Diversity
Puneet K. Dokania 8
![Page 15: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/15.jpg)
Parsimonious Labeling The Energy Function
Parsimonious Labeling: Energy Function
E(y) =∑i∈V
θi(yi) +∑c∈C
θc(yc).
Unary potentials: Arbitrary
Clique potentials: Diversity
θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity
where, Γ(yc) is the set of unique
labels δc(l1, l2, l3)
Energy function for Parsimonious Labeling
E(y) =∑i∈V
θi(yi) +∑c∈C
wcδ(Γ(yc))︸ ︷︷ ︸Diversity
Puneet K. Dokania 8
![Page 16: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/16.jpg)
Parsimonious Labeling The Energy Function
Parsimonious Labeling: Energy Function
E(y) =∑i∈V
θi(yi) +∑c∈C
θc(yc).
Unary potentials: Arbitrary
Clique potentials: Diversity
θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity
where, Γ(yc) is the set of unique
labels δc(l1, l2, l3)Energy function for Parsimonious Labeling
E(y) =∑i∈V
θi(yi) +∑c∈C
wcδ(Γ(yc))︸ ︷︷ ︸Diversity
Puneet K. Dokania 8
![Page 17: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/17.jpg)
Parsimonious Labeling The Energy Function
Diversity3: Metric over sets
θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity
Metric over sets δ : L → R,∀L ⊆ L, satisfying
Non Negativity
Triangular Inequality
Monotonicity: L1 ⊆ L2 implies δ(L1) ≤ δ(L2)→ Parsimony
Induced Metric: Every diversity induces a metric:
d(li, lj) = δ(li, lj)
Diameter Diversity: δdia(L) = maxli ,lj∈L d(li, lj)
3Bryant and Tupper, Advances in Mathematics, 2012.
Puneet K. Dokania 9
![Page 18: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/18.jpg)
Parsimonious Labeling The Energy Function
Diversity3: Metric over sets
θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity
Metric over sets δ : L → R,∀L ⊆ L, satisfying
Non Negativity
Triangular Inequality
Monotonicity: L1 ⊆ L2 implies δ(L1) ≤ δ(L2)→ Parsimony
Induced Metric: Every diversity induces a metric:
d(li, lj) = δ(li, lj)
Diameter Diversity: δdia(L) = maxli ,lj∈L d(li, lj)
3Bryant and Tupper, Advances in Mathematics, 2012.
Puneet K. Dokania 9
![Page 19: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/19.jpg)
Parsimonious Labeling The Energy Function
Diversity3: Metric over sets
θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity
Metric over sets δ : L → R,∀L ⊆ L, satisfying
Non Negativity
Triangular Inequality
Monotonicity: L1 ⊆ L2 implies δ(L1) ≤ δ(L2)→ Parsimony
Induced Metric: Every diversity induces a metric:
d(li, lj) = δ(li, lj)
Diameter Diversity: δdia(L) = maxli ,lj∈L d(li, lj)
3Bryant and Tupper, Advances in Mathematics, 2012.
Puneet K. Dokania 9
![Page 20: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/20.jpg)
Parsimonious Labeling The Energy Function
Diversity3: Metric over sets
θc(yc) ∝ δ(Γ(yc))︸ ︷︷ ︸Diversity
Metric over sets δ : L → R,∀L ⊆ L, satisfying
Non Negativity
Triangular Inequality
Monotonicity: L1 ⊆ L2 implies δ(L1) ≤ δ(L2)→ Parsimony
Induced Metric: Every diversity induces a metric:
d(li, lj) = δ(li, lj)
Diameter Diversity: δdia(L) = maxli ,lj∈L d(li, lj)
3Bryant and Tupper, Advances in Mathematics, 2012.
Puneet K. Dokania 9
![Page 21: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/21.jpg)
Parsimonious Labeling Special cases of Parsimonious Labeling
Special Case 1: Metric Labeling
If cliques are of size 2→ diversity→ metric
Parsimonious Labeling→Metric Labeling4
Many applications in low level vision tasks: Stereo matching,
Inpainting, Denoising, Image stitching.
4Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, 2001.
Puneet K. Dokania 10
![Page 22: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/22.jpg)
Parsimonious Labeling Special cases of Parsimonious Labeling
Special Case 1: Metric Labeling
If cliques are of size 2→ diversity→ metric
Parsimonious Labeling→Metric Labeling4
Many applications in low level vision tasks: Stereo matching,
Inpainting, Denoising, Image stitching.
4Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, 2001.
Puneet K. Dokania 10
![Page 23: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/23.jpg)
Parsimonious Labeling Special cases of Parsimonious Labeling
Special Case 2: Pn-Pos Model
5
Uniform Metric
d(li, lj) = min(|li − lj|, 1),∀li, lj ∈ L
Diversity→ Diameter diversity over uniform metric
Parsimonious Labeling→ Pn-Pos Model
Labels l1 l2 l3l1 0 1 1
l2 1 0 1
l3 1 1 0
Table: Uniform Metric
θc(l1, l2, l3) = max(d(l1, l2), d(l1, l3), d(l2, l3))
= 1
θc(yc) ∝
0, if yi = lk ,∀i ∈ c,
1, otherwise,
5Kohli et al., P3 “& Beyond: Solving Energies with Higher Order Cliques, 2007.
Puneet K. Dokania 11
![Page 24: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/24.jpg)
Parsimonious Labeling Special cases of Parsimonious Labeling
Special Case 2: Pn-Pos Model
5
Uniform Metric
d(li, lj) = min(|li − lj|, 1),∀li, lj ∈ L
Diversity→ Diameter diversity over uniform metric
Parsimonious Labeling→ Pn-Pos Model
Labels l1 l2 l3l1 0 1 1
l2 1 0 1
l3 1 1 0
Table: Uniform Metric
θc(l1, l2, l3) = max(d(l1, l2), d(l1, l3), d(l2, l3))
= 1
θc(yc) ∝
0, if yi = lk ,∀i ∈ c,
1, otherwise,
5Kohli et al., P3 “& Beyond: Solving Energies with Higher Order Cliques, 2007.
Puneet K. Dokania 11
![Page 25: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/25.jpg)
Parsimonious Labeling Special cases of Parsimonious Labeling
Special Case 2: Pn-Pos Model
5
Uniform Metric
d(li, lj) = min(|li − lj|, 1),∀li, lj ∈ L
Diversity→ Diameter diversity over uniform metric
Parsimonious Labeling→ Pn-Pos Model
Labels l1 l2 l3l1 0 1 1
l2 1 0 1
l3 1 1 0
Table: Uniform Metric
θc(l1, l2, l3) = max(d(l1, l2), d(l1, l3), d(l2, l3))
= 1
θc(yc) ∝
0, if yi = lk ,∀i ∈ c,
1, otherwise,
5Kohli et al., P3 “& Beyond: Solving Energies with Higher Order Cliques, 2007.
Puneet K. Dokania 11
![Page 26: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/26.jpg)
Parsimonious Labeling Special cases of Parsimonious Labeling
Special Case 2: Pn-Pos Model
5
Uniform Metric
d(li, lj) = min(|li − lj|, 1),∀li, lj ∈ L
Diversity→ Diameter diversity over uniform metric
Parsimonious Labeling→ Pn-Pos Model
Labels l1 l2 l3l1 0 1 1
l2 1 0 1
l3 1 1 0
Table: Uniform Metric
θc(l1, l2, l3) = max(d(l1, l2), d(l1, l3), d(l2, l3))
= 1
θc(yc) ∝
0, if yi = lk , ∀i ∈ c,
1, otherwise,
5Kohli et al., P3 “& Beyond: Solving Energies with Higher Order Cliques, 2007.
Puneet K. Dokania 11
![Page 27: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/27.jpg)
Parsimonious Labeling Special cases of Parsimonious Labeling
So far …
E(y) =∑i∈V
θi(yi) +∑c∈C
wcδ(Γ(yc))︸ ︷︷ ︸Diversity
Puneet K. Dokania 12
![Page 28: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/28.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Hierarchical PnPos Model
Given tree metric
d t(l1, l2) = 14, d t(l1, l3) = 4, d t(l1, l1) = 0
Hierarchical PnPos Model→ diameter diversity over tree metric
Diameter diversity at cluster p is maxli ,lj dt(li, lj) = 14.
Puneet K. Dokania 13
![Page 29: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/29.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Hierarchical PnPos Model
Given tree metric
d t(l1, l2) = 14, d t(l1, l3) = 4, d t(l1, l1) = 0
Hierarchical PnPos Model→ diameter diversity over tree metric
Diameter diversity at cluster p is maxli ,lj dt(li, lj) = 14.
Puneet K. Dokania 13
![Page 30: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/30.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Hierarchical PnPos Model
Given tree metric
d t(l1, l2) = 14, d t(l1, l3) = 4, d t(l1, l1) = 0
Hierarchical PnPos Model→ diameter diversity over tree metric
Diameter diversity at cluster p is maxli ,lj dt(li, lj) = 14.
Puneet K. Dokania 13
![Page 31: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/31.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Hierarchical PnPos Model
Given tree metric
d t(l1, l2) = 14, d t(l1, l3) = 4, d t(l1, l1) = 0
Hierarchical PnPos Model→ diameter diversity over tree metric
Diameter diversity at cluster p is maxli ,lj dt(li, lj) = 14.
Puneet K. Dokania 13
![Page 32: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/32.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Move Making Algorithm for Hierarchical PnPos Model
Optimizing directly at the root node is non-trivial
We propose divide and conquer based boom-up approach
Puneet K. Dokania 14
![Page 33: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/33.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Move Making Algorithm for Hierarchical PnPos Model
Optimizing directly at the root node is non-trivial
We propose divide and conquer based boom-up approach
Puneet K. Dokania 14
![Page 34: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/34.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Move Making Algorithm for Hierarchical PnPos Model
Optimizing directly at the root node is non-trivial
We propose divide and conquer based boom-up approach
Puneet K. Dokania 14
![Page 35: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/35.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Move Making Algorithm for Hierarchical PnPos Model
Optimizing directly at the root node is non-trivial
We propose divide and conquer based boom-up approach
Puneet K. Dokania 14
![Page 36: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/36.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Move Making Algorithm for Hierarchical PnPos Model
Optimizing directly at the root node is non-trivial
We propose divide and conquer based boom-up approach
Puneet K. Dokania 14
![Page 37: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/37.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Move Making Algorithm for Hierarchical PnPos Model
Optimizing directly at the root node is non-trivial
We propose divide and conquer based boom-up approach
Puneet K. Dokania 14
![Page 38: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/38.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Move Making Algorithm for Hierarchical PnPos Model
Optimizing directly at the root node is non-trivial
We propose divide and conquer based boom-up approach
Puneet K. Dokania 14
![Page 39: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/39.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Move Making Algorithm for Hierarchical PnPos Model
Solving the problem at leaf node→ Trivial
Fusing at non-leaf node→ Pn-Pos Model
Puneet K. Dokania 15
![Page 40: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/40.jpg)
Parsimonious Labeling Hierarchical Pn Pos Model
Move Making Algorithm for Hierarchical PnPos Model
Solving the problem at leaf node→ Trivial
Fusing at non-leaf node→ Pn-Pos Model
Puneet K. Dokania 15
![Page 41: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/41.jpg)
Parsimonious Labeling Move Making Algorithm for Parsimonious Labeling
Move Making for Parsimonious Labeling
Given any general diversity→ Get the induced metric
Induced Metric→Mixture of tree metrics (r-hst)6
Hierarchical Pn-Pos model over each tree metric→ diameter diversity
over each tree metric (r-hst)
Optimize each Hierarchical Pn-Pos model using proposed move
making algorithm
Fuse solutions or choose the one with minimum energy
6Fakcharoenphol et al., In STOC 2003.
Puneet K. Dokania 16
![Page 42: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/42.jpg)
Parsimonious Labeling Move Making Algorithm for Parsimonious Labeling
Move Making for Parsimonious Labeling
Given any general diversity→ Get the induced metric
Induced Metric→Mixture of tree metrics (r-hst)6
Hierarchical Pn-Pos model over each tree metric→ diameter diversity
over each tree metric (r-hst)
Optimize each Hierarchical Pn-Pos model using proposed move
making algorithm
Fuse solutions or choose the one with minimum energy
6Fakcharoenphol et al., In STOC 2003.
Puneet K. Dokania 16
![Page 43: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/43.jpg)
Parsimonious Labeling Move Making Algorithm for Parsimonious Labeling
Move Making for Parsimonious Labeling
Given any general diversity→ Get the induced metric
Induced Metric→Mixture of tree metrics (r-hst)6
Hierarchical Pn-Pos model over each tree metric→ diameter diversity
over each tree metric (r-hst)
Optimize each Hierarchical Pn-Pos model using proposed move
making algorithm
Fuse solutions or choose the one with minimum energy
6Fakcharoenphol et al., In STOC 2003.
Puneet K. Dokania 16
![Page 44: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/44.jpg)
Parsimonious Labeling Move Making Algorithm for Parsimonious Labeling
Move Making for Parsimonious Labeling
Given any general diversity→ Get the induced metric
Induced Metric→Mixture of tree metrics (r-hst)6
Hierarchical Pn-Pos model over each tree metric→ diameter diversity
over each tree metric (r-hst)
Optimize each Hierarchical Pn-Pos model using proposed move
making algorithm
Fuse solutions or choose the one with minimum energy
6Fakcharoenphol et al., In STOC 2003.
Puneet K. Dokania 16
![Page 45: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/45.jpg)
Parsimonious Labeling Move Making Algorithm for Parsimonious Labeling
Move Making for Parsimonious Labeling
Given any general diversity→ Get the induced metric
Induced Metric→Mixture of tree metrics (r-hst)6
Hierarchical Pn-Pos model over each tree metric→ diameter diversity
over each tree metric (r-hst)
Optimize each Hierarchical Pn-Pos model using proposed move
making algorithm
Fuse solutions or choose the one with minimum energy
6Fakcharoenphol et al., In STOC 2003.
Puneet K. Dokania 16
![Page 46: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/46.jpg)
Parsimonious Labeling Comparison
Comparison
Co-oc7:
Clique potentials→Monotonic
Very fast optimization algorithm
No theoretical guarantees
SoSPD8:
Clique potentials→ Arbitrary→ Upperbound as SoS functions
Slow. Practically, can not go beyond the clique of size 9
Loose multiplicative bound
Parsimonious Labeling9:
Clique potentials→ Diversities
Very fast. We experimented with cliques of size ≈ 1200.
Can be parallelized over the trees and over the levels.
Very tight multiplicative bound.
7Ladicky, Russell, Kohli, and Torr, ECCV 2010.
8Fix, Wang, and Zabih, CVPR 2014.
9Dokania and Kumar, ICCV 2015.
Puneet K. Dokania 17
![Page 47: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/47.jpg)
Parsimonious Labeling Comparison
Comparison
Co-oc7:
Clique potentials→Monotonic
Very fast optimization algorithm
No theoretical guarantees
SoSPD8:
Clique potentials→ Arbitrary→ Upperbound as SoS functions
Slow. Practically, can not go beyond the clique of size 9
Loose multiplicative bound
Parsimonious Labeling9:
Clique potentials→ Diversities
Very fast. We experimented with cliques of size ≈ 1200.
Can be parallelized over the trees and over the levels.
Very tight multiplicative bound.
7Ladicky, Russell, Kohli, and Torr, ECCV 2010.
8Fix, Wang, and Zabih, CVPR 2014.
9Dokania and Kumar, ICCV 2015.
Puneet K. Dokania 17
![Page 48: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/48.jpg)
Parsimonious Labeling Comparison
Comparison
Co-oc7:
Clique potentials→Monotonic
Very fast optimization algorithm
No theoretical guarantees
SoSPD8:
Clique potentials→ Arbitrary→ Upperbound as SoS functions
Slow. Practically, can not go beyond the clique of size 9
Loose multiplicative bound
Parsimonious Labeling9:
Clique potentials→ Diversities
Very fast. We experimented with cliques of size ≈ 1200.
Can be parallelized over the trees and over the levels.
Very tight multiplicative bound.
7Ladicky, Russell, Kohli, and Torr, ECCV 2010.
8Fix, Wang, and Zabih, CVPR 2014.
9Dokania and Kumar, ICCV 2015.
Puneet K. Dokania 17
![Page 49: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/49.jpg)
Parsimonious Labeling Experimental Results
Experimental Seing
Energy Function:
E(y) =∑i∈V
θi(yi) +∑c∈C
wcδ(Γ(yc))︸ ︷︷ ︸Diversity
(3)
Clique Potential: Diameter diversity over truncated Linear Metric:
θi,j(la, lb) = λmin(|la − lb|,M),∀la, lb ∈ L
Cliques: Superpixels generate using Mean Shi.
Clique Weights:
wc = exp(−ρ(yc)
σ2
)where, ρ(yc) is the variance of intensities of pixels in clique yc .
Puneet K. Dokania 18
![Page 50: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/50.jpg)
Parsimonious Labeling Experimental Results
Stereo Matching Results – Visually
(a) Ground Truth (b) Our
(c) α−Exp (d) Co-oc
Puneet K. Dokania 19
![Page 51: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/51.jpg)
Parsimonious Labeling Experimental Results
Stereo Matching Results – Energy and Time
(a) Our
(E = 1.4× 106, 773 sec)(b) Co-oc
(E = 2.1× 106, 306 sec)
Puneet K. Dokania 20
![Page 52: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/52.jpg)
Parsimonious Labeling Experimental Results
Image denoising and Inpainting Results – Visually
(a) Original (b) Our
(c) α−Exp (d) Co-oc
Puneet K. Dokania 21
![Page 53: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/53.jpg)
Parsimonious Labeling Experimental Results
Image denoising and Inpainting Results – Energy and Time
(a) Our
(E = 1.2× 107, 1964 sec)(b) Co-occ
(E = 1.4× 107, 358 sec)
Puneet K. Dokania 22
![Page 54: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/54.jpg)
Learning to Rank Using High-Order Information
Presentation Outline
1 Thesis Overview
2 Parsimonious Labeling
3 Learning to Rank Using High-Order Information
4 Regularization Path for SSVM
5 Future Work
6 Publications
Puneet K. Dokania 23
![Page 55: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/55.jpg)
Learning to Rank Using High-Order Information AP-SVM
Ranking?
Get the feature vector φ(xi)
Learn wSort using si(w) = w>φ(xi)
SVM→ Optimizes accuracy
Accuracy 6= Average Precision
Puneet K. Dokania 24
![Page 56: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/56.jpg)
Learning to Rank Using High-Order Information AP-SVM
Ranking?
Get the feature vector φ(xi)
Learn wSort using si(w) = w>φ(xi)
SVM→ Optimizes accuracy
Accuracy 6= Average Precision
Puneet K. Dokania 24
![Page 57: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/57.jpg)
Learning to Rank Using High-Order Information AP-SVM
Ranking?
Get the feature vector φ(xi)
Learn wSort using si(w) = w>φ(xi)
SVM→ Optimizes accuracy
Accuracy 6= Average Precision
Puneet K. Dokania 24
![Page 58: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/58.jpg)
Learning to Rank Using High-Order Information AP-SVM
Ranking?
Get the feature vector φ(xi)
Learn wSort using si(w) = w>φ(xi)
SVM→ Optimizes accuracy
Accuracy 6= Average Precision
Puneet K. Dokania 24
![Page 59: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/59.jpg)
Learning to Rank Using High-Order Information AP-SVM
AP-SVM10
: Problem Formulation
Single input x, Positive Set P , Negative Set Nφ(xi),∀i ∈ P , φ(xj),∀j ∈ NRank Matrix
Rij =
+1, if i is beer ranked than j
−1, if j is beer ranked than i
Define Joint Score:
S(x,R;w) =1
|P||N |∑i∈P
∑j∈N
Rij(si(w)− sj(w))
Encodes Ranking
10Yue et al., A support vector method for optimizing average precision, 2007.
Puneet K. Dokania 25
![Page 60: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/60.jpg)
Learning to Rank Using High-Order Information AP-SVM
AP-SVM10
: Problem Formulation
Single input x, Positive Set P , Negative Set Nφ(xi), ∀i ∈ P , φ(xj),∀j ∈ N
Rank Matrix
Rij =
+1, if i is beer ranked than j
−1, if j is beer ranked than i
Define Joint Score:
S(x,R;w) =1
|P||N |∑i∈P
∑j∈N
Rij(si(w)− sj(w))
Encodes Ranking
10Yue et al., A support vector method for optimizing average precision, 2007.
Puneet K. Dokania 25
![Page 61: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/61.jpg)
Learning to Rank Using High-Order Information AP-SVM
AP-SVM10
: Problem Formulation
Single input x, Positive Set P , Negative Set Nφ(xi), ∀i ∈ P , φ(xj),∀j ∈ NRank Matrix
Rij =
+1, if i is beer ranked than j
−1, if j is beer ranked than i
Define Joint Score:
S(x,R;w) =1
|P||N |∑i∈P
∑j∈N
Rij(si(w)− sj(w))
Encodes Ranking
10Yue et al., A support vector method for optimizing average precision, 2007.
Puneet K. Dokania 25
![Page 62: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/62.jpg)
Learning to Rank Using High-Order Information AP-SVM
AP-SVM: Objective Function
Loss function ∆(R,R∗) = 1− AP(R,R∗)Objective Function
min
w, ξ
λ
2
‖w‖2 + ξ (4)
s.t. S(x,R∗;w) ≥ S(x,R;w) + ∆(R,R∗)− ξ, ∀R. (5)
Loss augmented inference: R = argmaxRS(x, R;w) + ∆(R,R∗),greedy algorithm O(|P||N |) by Yue et.al.
Puneet K. Dokania 26
![Page 63: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/63.jpg)
Learning to Rank Using High-Order Information AP-SVM
AP-SVM: Joint Score
Joint Score:
S(x,R;w) =1
|P||N |∑i∈P
∑j∈N
Rij(si(w)− sj(w))
Encodes Ranking
Sample Score:
si(w) = w>φ(xi)
No High-Order Information
Puneet K. Dokania 27
![Page 64: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/64.jpg)
Learning to Rank Using High-Order Information High-Order Information
Why High-Order Information?
Puneet K. Dokania 28
![Page 65: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/65.jpg)
Learning to Rank Using High-Order Information High-Order Information
Why High-Order Information?
Puneet K. Dokania 28
![Page 66: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/66.jpg)
Learning to Rank Using High-Order Information High-Order Information
Why High-Order Information?
Puneet K. Dokania 28
![Page 67: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/67.jpg)
Learning to Rank Using High-Order Information High-Order Information
Encoding High-Order Information
Puneet K. Dokania 29
![Page 68: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/68.jpg)
Learning to Rank Using High-Order Information High-Order Information
Encoding High-Order Information
Puneet K. Dokania 29
![Page 69: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/69.jpg)
Learning to Rank Using High-Order Information High-Order Information
Encoding High-Order Information
Define Joint Feature Map (encodes the
structure)
Φ(x, y) =
( ∑i Φ1(xi, yi)∑
i,j Φ2(xi, yi, xj, yj)
)
Φ1 - first-order information
Φ2 - high-order information
Joint labeling: y ∈ −1,+1n
Define Score S(x, y;w) = w>Φ(x, y)
Puneet K. Dokania 30
![Page 70: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/70.jpg)
Learning to Rank Using High-Order Information High-Order Information
Encoding High-Order Information
Define Joint Feature Map (encodes the
structure)
Φ(x, y) =
( ∑i Φ1(xi, yi)∑
i,j Φ2(xi, yi, xj, yj)
)
Φ1 - first-order information
Φ2 - high-order information
Joint labeling: y ∈ −1,+1n
Define Score S(x, y;w) = w>Φ(x, y)
Puneet K. Dokania 30
![Page 71: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/71.jpg)
Learning to Rank Using High-Order Information High-Order Information
Joint Score: Closer look
w>Φ(x, y) =
(w1
w2
)>( ∑i Φ1(xi, yi)∑
i,j Φ2(xi, yi, xj, yj)
)=
∑i
w>1
Φ1(xi, yi) +∑i,j
w>2
Φ2(xi, yi, xj, yj)︸ ︷︷ ︸Encodes High-Order Information
(6)
Single score for the entire dataset→ Ranking?
Puneet K. Dokania 31
![Page 72: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/72.jpg)
Learning to Rank Using High-Order Information High-Order Information
Joint Score: Closer look
w>Φ(x, y) =
(w1
w2
)>( ∑i Φ1(xi, yi)∑
i,j Φ2(xi, yi, xj, yj)
)=
∑i
w>1
Φ1(xi, yi) +∑i,j
w>2
Φ2(xi, yi, xj, yj)︸ ︷︷ ︸Encodes High-Order Information
(6)
Single score for the entire dataset→ Ranking?
Puneet K. Dokania 31
![Page 73: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/73.jpg)
Learning to Rank Using High-Order Information High-Order Information
Ranking Using Max-Marginals
We propose to use dierence of max-marginals
s(xi;w) = m+i (w)−m−i (w), where, m+
i (w) is the max-marginal score
such that sample xi takes label of +1.
m+i (w) = argmaxy,yi=+1w>Φ(x, y)
Dynamic Graph Cuts11
— Very Eicient
11Kohli et al., In PAMI 2007.
Puneet K. Dokania 32
![Page 74: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/74.jpg)
Learning to Rank Using High-Order Information High-Order Information
Ranking Using Max-Marginals
We propose to use dierence of max-marginals
s(xi;w) = m+i (w)−m−i (w), where, m+
i (w) is the max-marginal score
such that sample xi takes label of +1.
m+i (w) = argmaxy,yi=+1w>Φ(x, y)
Dynamic Graph Cuts11
— Very Eicient
11Kohli et al., In PAMI 2007.
Puneet K. Dokania 32
![Page 75: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/75.jpg)
Learning to Rank Using High-Order Information High-Order AP-SVM
HOAP-SVM: Score
Score that can encode ranking and high-order information
Joint Score for the given ranking
S(x,R;w) =1
|P||N |∑i∈P
∑j∈N
Rij(si(w)− sj(w))
Encodes Ranking
Sample score si as dierence of max-marginals
si(w) = m+i (w)−m−i (w)
Encodes High-Order Information
Puneet K. Dokania 33
![Page 76: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/76.jpg)
Learning to Rank Using High-Order Information High-Order AP-SVM
HOAP-SVM: Score
Score that can encode ranking and high-order information
Joint Score for the given ranking
S(x,R;w) =1
|P||N |∑i∈P
∑j∈N
Rij(si(w)− sj(w))
Encodes Ranking
Sample score si as dierence of max-marginals
si(w) = m+i (w)−m−i (w)
Encodes High-Order Information
Puneet K. Dokania 33
![Page 77: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/77.jpg)
Learning to Rank Using High-Order Information High-Order AP-SVM
HOAP-SVM: Score
Score that can encode ranking and high-order information
Joint Score for the given ranking
S(x,R;w) =1
|P||N |∑i∈P
∑j∈N
Rij(si(w)− sj(w))
Encodes Ranking
Sample score si as dierence of max-marginals
si(w) = m+i (w)−m−i (w)
Encodes High-Order Information
Puneet K. Dokania 33
![Page 78: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/78.jpg)
Learning to Rank Using High-Order Information High-Order AP-SVM
HOAP-SVM: Objective Function
Objective Function
min
w, ξ
λ
2
‖w‖2 + ξ (7)
s.t. S(x,R∗;w) ≥ S(x,R;w) + ∆(R,R∗)− ξ, ∀R, (8)
w2 ≤ 0, ξ ≥ 0.
Each max-marginal is a convex function (max over aine functions)
m+i (w) = argmaxy,yi=+1w>Φ(x, y)
The objective function is a dierence of convex program
Puneet K. Dokania 34
![Page 79: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/79.jpg)
Learning to Rank Using High-Order Information High-Order AP-SVM
HOAP-SVM: Objective Function
Objective Function
min
w, ξ
λ
2
‖w‖2 + ξ (7)
s.t. S(x,R∗;w) ≥ S(x,R;w) + ∆(R,R∗)− ξ, ∀R, (8)
w2 ≤ 0, ξ ≥ 0.
Each max-marginal is a convex function (max over aine functions)
m+i (w) = argmaxy,yi=+1w>Φ(x, y)
The objective function is a dierence of convex program
Puneet K. Dokania 34
![Page 80: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/80.jpg)
Learning to Rank Using High-Order Information High-Order AP-SVM
HOAP-SVM: Objective Function
Objective Function
min
w, ξ
λ
2
‖w‖2 + ξ (7)
s.t. S(x,R∗;w) ≥ S(x,R;w) + ∆(R,R∗)− ξ, ∀R, (8)
w2 ≤ 0, ξ ≥ 0.
Each max-marginal is a convex function (max over aine functions)
m+i (w) = argmaxy,yi=+1w>Φ(x, y)
The objective function is a dierence of convex program
Puneet K. Dokania 34
![Page 81: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/81.jpg)
Learning to Rank Using High-Order Information High-Order AP-SVM
CCCP12
Dierence of convex functions can be optimized using CCCP algorithm
12Yuille et al., The concave-convex procedure, 2003.
Puneet K. Dokania 35
![Page 82: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/82.jpg)
Learning to Rank Using High-Order Information High-Order AP-SVM
CCCP12
Dierence of convex functions can be optimized using CCCP algorithm
12Yuille et al., The concave-convex procedure, 2003.
Puneet K. Dokania 35
![Page 83: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/83.jpg)
Learning to Rank Using High-Order Information High-Order AP-SVM
CCCP12
Dierence of convex functions can be optimized using CCCP algorithm
12Yuille et al., The concave-convex procedure, 2003.
Puneet K. Dokania 35
![Page 84: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/84.jpg)
Learning to Rank Using High-Order Information High-Order AP-SVM
CCCP12
Dierence of convex functions can be optimized using CCCP algorithm
12Yuille et al., The concave-convex procedure, 2003.
Puneet K. Dokania 35
![Page 85: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/85.jpg)
Learning to Rank Using High-Order Information Results
Action Recognition
PASCAL VOC 2011 Dataset
10 Action Classes
Unary Feature - POSELET and GIST concatenated
High-Order Feature -POSELET
High-Order Information
Hypothesis: Persons in the same image are more likely to perform same
action
Connected bounding boxes coming from the same image
Puneet K. Dokania 36
![Page 86: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/86.jpg)
Learning to Rank Using High-Order Information Results
PASCAL VOC Results - Average ap over all 10 action
classes
Method Trainval Test
svm 54.7/+4.2 48.82/+4.93
ap-svm 56.2/+2.7 51.42/+2.33
hoap-svm 58.9 53.75
Puneet K. Dokania 37
![Page 87: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/87.jpg)
Learning to Rank Using High-Order Information Results
Visualization - Reading top 4
Puneet K. Dokania 38
![Page 88: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/88.jpg)
Regularization Path for SSVM
Presentation Outline
1 Thesis Overview
2 Parsimonious Labeling
3 Learning to Rank Using High-Order Information
4 Regularization Path for SSVM
5 Future Work
6 Publications
Puneet K. Dokania 39
![Page 89: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/89.jpg)
Regularization Path for SSVM Definition and Motivation
Regularization Path: What and Why
Optimize SSVM objective function
min
w, ξ
λ
2
‖w‖2 +1
n
n∑i=1
ξi
s.t. set of constraints
λ→ important for good generalization→ cross validate
λ ∈ [0,∞]→ cross validation over subset→ poor generalization
ε-optimal regularization path algorithm
Puneet K. Dokania 40
![Page 90: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/90.jpg)
Regularization Path for SSVM Definition and Motivation
Regularization Path: What and Why
Optimize SSVM objective function
min
w, ξ
λ
2
‖w‖2 +1
n
n∑i=1
ξi
s.t. set of constraints
λ→ important for good generalization→ cross validate
λ ∈ [0,∞]→ cross validation over subset→ poor generalization
ε-optimal regularization path algorithm
Puneet K. Dokania 40
![Page 91: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/91.jpg)
Regularization Path for SSVM Definition and Motivation
Regularization Path: What and Why
Optimize SSVM objective function
min
w, ξ
λ
2
‖w‖2 +1
n
n∑i=1
ξi
s.t. set of constraints
λ→ important for good generalization→ cross validate
λ ∈ [0,∞]→ cross validation over subset→ poor generalization
ε-optimal regularization path algorithm
Puneet K. Dokania 40
![Page 92: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/92.jpg)
Regularization Path for SSVM Definition and Motivation
Regularization Path: What and Why
Optimize SSVM objective function
min
w, ξ
λ
2
‖w‖2 +1
n
n∑i=1
ξi
s.t. set of constraints
λ→ important for good generalization→ cross validate
λ ∈ [0,∞]→ cross validation over subset→ poor generalization
ε-optimal regularization path algorithm
Puneet K. Dokania 40
![Page 93: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/93.jpg)
Regularization Path for SSVM Definition and Motivation
Dual Objective and Duality Gap
SSVM dual objective function
min
αf (α)→ smooth convex
s.t.∑y∈Yi
αi(y) = 1,∀i ∈ [n],
αi(y) ≥ 0, ∀i ∈ [n],∀y ∈ Yi.
where, α = (α1, · · · , αn) ∈ R|Y1| × · · ·R|Yn|.
Duality Gap
g(α;λ) =1
n
∑i
(max
y∈YiHi(y;w)−
∑y∈Yi
αi(y)Hi(y;w))
where, Hi(y;w) is the hinge loss.
Puneet K. Dokania 41
![Page 94: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/94.jpg)
Regularization Path for SSVM Definition and Motivation
Dual Objective and Duality Gap
SSVM dual objective function
min
αf (α)→ smooth convex
s.t.∑y∈Yi
αi(y) = 1,∀i ∈ [n],
αi(y) ≥ 0, ∀i ∈ [n],∀y ∈ Yi.
where, α = (α1, · · · , αn) ∈ R|Y1| × · · ·R|Yn|.Duality Gap
g(α;λ) =1
n
∑i
(max
y∈YiHi(y;w)−
∑y∈Yi
αi(y)Hi(y;w))
where, Hi(y;w) is the hinge loss.
Puneet K. Dokania 41
![Page 95: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/95.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Key Idea: ε-Optimal Regularization Path
Puneet K. Dokania 42
![Page 96: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/96.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Key Idea: ε-Optimal Regularization Path
Puneet K. Dokania 42
![Page 97: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/97.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Key Idea: ε-Optimal Regularization Path
Puneet K. Dokania 42
![Page 98: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/98.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Key Idea: ε-Optimal Regularization Path
Puneet K. Dokania 42
![Page 99: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/99.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Key Idea: ε-Optimal Regularization Path
Puneet K. Dokania 42
![Page 100: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/100.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 1: How do we start?
Let Yi = argmaxy∈Yi ∆(y, yi) be the loss-maximizer and yi ∈ Yi,∀i.Let Ψ = 1
n
∑i Ψi(yi), where Ψi(y) = Φ(xi, yi)− Φ(xi, y).
Then, wk = Ψλ is guaranteed to be ε1 optimal for any λ satisfying the
condition:
λ ≥
∥∥∥Ψ∥∥∥2
+ 1
n
∑i max
y∈Yi
(− Ψ>Ψ(y)
)︸ ︷︷ ︸
Inference
ε1
(9)
Puneet K. Dokania 43
![Page 101: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/101.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 1: How do we start?
Let Yi = argmaxy∈Yi ∆(y, yi) be the loss-maximizer and yi ∈ Yi, ∀i.
Let Ψ = 1
n
∑i Ψi(yi), where Ψi(y) = Φ(xi, yi)− Φ(xi, y).
Then, wk = Ψλ is guaranteed to be ε1 optimal for any λ satisfying the
condition:
λ ≥
∥∥∥Ψ∥∥∥2
+ 1
n
∑i max
y∈Yi
(− Ψ>Ψ(y)
)︸ ︷︷ ︸
Inference
ε1
(9)
Puneet K. Dokania 43
![Page 102: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/102.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 1: How do we start?
Let Yi = argmaxy∈Yi ∆(y, yi) be the loss-maximizer and yi ∈ Yi, ∀i.Let Ψ = 1
n
∑i Ψi(yi), where Ψi(y) = Φ(xi, yi)− Φ(xi, y).
Then, wk = Ψλ is guaranteed to be ε1 optimal for any λ satisfying the
condition:
λ ≥
∥∥∥Ψ∥∥∥2
+ 1
n
∑i max
y∈Yi
(− Ψ>Ψ(y)
)︸ ︷︷ ︸
Inference
ε1
(9)
Puneet K. Dokania 43
![Page 103: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/103.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 1: How do we start?
Let Yi = argmaxy∈Yi ∆(y, yi) be the loss-maximizer and yi ∈ Yi, ∀i.Let Ψ = 1
n
∑i Ψi(yi), where Ψi(y) = Φ(xi, yi)− Φ(xi, y).
Then, wk = Ψλ is guaranteed to be ε1 optimal for any λ satisfying the
condition:
λ ≥
∥∥∥Ψ∥∥∥2
+ 1
n
∑i max
y∈Yi
(− Ψ>Ψ(y)
)︸ ︷︷ ︸
Inference
ε1
(9)
Puneet K. Dokania 43
![Page 104: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/104.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 2: How to find the breakpoints?
Let λk+1 = ηλk , 0 ≤ η ≤ 1.
wk → εopt , for all λk+1 obtained using η satisfying the condition:.
1− ε− g(αk ;λk)
Ω(αk , λk)≤ η ≤ 1 (10)
where, Ω(αk , λk) := `αk − λkw>k wk
Puneet K. Dokania 44
![Page 105: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/105.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 2: How to find the breakpoints?
Let λk+1 = ηλk , 0 ≤ η ≤ 1.
wk → εopt , for all λk+1 obtained using η satisfying the condition:.
1− ε− g(αk ;λk)
Ω(αk , λk)≤ η ≤ 1 (10)
where, Ω(αk , λk) := `αk − λkw>k wk
Puneet K. Dokania 44
![Page 106: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/106.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 2: How to find the breakpoints?
Let λk+1 = ηλk , 0 ≤ η ≤ 1.
wk → εopt , for all λk+1 obtained using η satisfying the condition:.
1− ε− g(αk ;λk)
Ω(αk , λk)≤ η ≤ 1 (10)
where, Ω(αk , λk) := `αk − λkw>k wk
Puneet K. Dokania 44
![Page 107: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/107.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 2: Proof Sketch
Keeping wk constant – from kkt condition
wk =1
n
∑i∈[n], y∈Yi
αki (y)
λkΨ(xi, y).
Therefore, using
αk+1
i (y)
λk+1
=αki (y)
λk, ∀y 6= yi;
∑y∈Yi
αi(y) = 1,∀i ∈ [n]; λk+1 = ηλk
New duality gap
g(αk+1;λk+1) = g(αk ;λk)︸ ︷︷ ︸Old gap
+(1− η
)Ω(αk , λk)
≤ ε
Puneet K. Dokania 45
![Page 108: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/108.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 2: Proof Sketch
Keeping wk constant – from kkt condition
wk =1
n
∑i∈[n], y∈Yi
αki (y)
λkΨ(xi, y).
Therefore, using
αk+1
i (y)
λk+1
=αki (y)
λk, ∀y 6= yi;
∑y∈Yi
αi(y) = 1,∀i ∈ [n]; λk+1 = ηλk
New duality gap
g(αk+1;λk+1) = g(αk ;λk)︸ ︷︷ ︸Old gap
+(1− η
)Ω(αk , λk)
≤ ε
Puneet K. Dokania 45
![Page 109: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/109.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 2: Proof Sketch
Keeping wk constant – from kkt condition
wk =1
n
∑i∈[n], y∈Yi
αki (y)
λkΨ(xi, y).
Therefore, using
αk+1
i (y)
λk+1
=αki (y)
λk, ∀y 6= yi;
∑y∈Yi
αi(y) = 1,∀i ∈ [n]; λk+1 = ηλk
New duality gap
g(αk+1;λk+1) = g(αk ;λk)︸ ︷︷ ︸Old gap
+(1− η
)Ω(αk , λk)
≤ ε
Puneet K. Dokania 45
![Page 110: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/110.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Challenge 3: How to optimize eiciently?
Notice that, wk is already ε-optimal at λk+1
Warm starting with wk requires us to reduce the duality gap only by
(ε− ε1)→ very fast convergence
We use Block-Coordinate Frank-Wolfe algorithm13
for the
optimization.
Lagrange duality gap is the by product
13Lacoste-Julien et al., In ICML 2013.
Puneet K. Dokania 46
![Page 111: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/111.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Eects of ε1
Decrease ε1:
(ε− ε1) increases — More passes through the data to get (ε1)opt solution.
η decreases — big jumps — number of breakpoints decreases (see below)
λk+1 = ηλk ; 1− ε− g(αk ;λk)
Ω(αk , λk)≤ η ≤ 1
Increase ε1 — Similar arguments
Puneet K. Dokania 47
![Page 112: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/112.jpg)
Regularization Path for SSVM Regularization Path: Key Ideas, Challenges and Solutions
Eects of ε1
Decrease ε1:
(ε− ε1) increases — More passes through the data to get (ε1)opt solution.
η decreases — big jumps — number of breakpoints decreases (see below)
λk+1 = ηλk ; 1− ε− g(αk ;λk)
Ω(αk , λk)≤ η ≤ 1
Increase ε1 — Similar arguments
Puneet K. Dokania 47
![Page 113: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/113.jpg)
Regularization Path for SSVM Experiments and Analysis
Dataset and bcfw Variants
OCR dataset14
with 6251 train and 626 test samples.
ε = 0.1
20 dierent values of λ equally spaced between [10−4, 10
3]
BCFW variants
BCFW-HEU-G: Heuristic convergence with gap based sampling
BCFW-STD-G: Exact convergence with gap based sampling
RP-BCFW-HEU-G: Regularization Path with BCFW-HEU-G.
14Taskar et al., Max-margin Markov networks, NIPS 2003.
Puneet K. Dokania 48
![Page 114: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/114.jpg)
Regularization Path for SSVM Experiments and Analysis
Dataset and bcfw Variants
OCR dataset14
with 6251 train and 626 test samples.
ε = 0.1
20 dierent values of λ equally spaced between [10−4, 10
3]
BCFW variants
BCFW-HEU-G: Heuristic convergence with gap based sampling
BCFW-STD-G: Exact convergence with gap based sampling
RP-BCFW-HEU-G: Regularization Path with BCFW-HEU-G.
14Taskar et al., Max-margin Markov networks, NIPS 2003.
Puneet K. Dokania 48
![Page 115: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/115.jpg)
Regularization Path for SSVM Experiments and Analysis
Dataset and bcfw Variants
OCR dataset14
with 6251 train and 626 test samples.
ε = 0.1
20 dierent values of λ equally spaced between [10−4, 10
3]
BCFW variants
BCFW-HEU-G: Heuristic convergence with gap based sampling
BCFW-STD-G: Exact convergence with gap based sampling
RP-BCFW-HEU-G: Regularization Path with BCFW-HEU-G.
14Taskar et al., Max-margin Markov networks, NIPS 2003.
Puneet K. Dokania 48
![Page 116: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/116.jpg)
Regularization Path for SSVM Experiments and Analysis
Eect of ε1 for ε = 0.1
Number of breakpoints in the regularization path
ε1 RP-BCFW-HEU-G RP-BCFW-STD-G0.01 142 133
0.05 225 153
0.09 1060 349
Number of passes through the data for optimization
ε1 RP-BCFW-HEU-G RP-BCFW-STD-G BCFW-STD-G0.01 2711.946 4405.881 1138.872
0.05 1301.869 2120.969 1138.872
0.09 1076.005 2100.304 1138.872
Puneet K. Dokania 49
![Page 117: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/117.jpg)
Regularization Path for SSVM Experiments and Analysis
Eect of ε1 for ε = 0.1
Number of breakpoints in the regularization path
ε1 RP-BCFW-HEU-G RP-BCFW-STD-G0.01 142 133
0.05 225 153
0.09 1060 349
Number of passes through the data for optimization
ε1 RP-BCFW-HEU-G RP-BCFW-STD-G BCFW-STD-G0.01 2711.946 4405.881 1138.872
0.05 1301.869 2120.969 1138.872
0.09 1076.005 2100.304 1138.872
Puneet K. Dokania 49
![Page 118: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/118.jpg)
Regularization Path for SSVM Experiments and Analysis
Duality gap for ε1 = 0.01
Puneet K. Dokania 50
![Page 119: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/119.jpg)
Regularization Path for SSVM Experiments and Analysis
Duality gap for ε1 = 0.09
Puneet K. Dokania 51
![Page 120: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/120.jpg)
Regularization Path for SSVM Experiments and Analysis
Test loss for ε1 = 0.01
Puneet K. Dokania 52
![Page 121: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/121.jpg)
Regularization Path for SSVM Experiments and Analysis
Test loss for ε1 = 0.09
Puneet K. Dokania 53
![Page 122: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/122.jpg)
Future Work
Presentation Outline
1 Thesis Overview
2 Parsimonious Labeling
3 Learning to Rank Using High-Order Information
4 Regularization Path for SSVM
5 Future Work
6 Publications
Puneet K. Dokania 54
![Page 123: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/123.jpg)
Future Work
Possible future directions…
High-Order
Parsimonious labeling for semantic labels
SSVM
Latent HOAP-SVM
Discovering label dependence structure
Latent SSVM: Interaction between latent variables?
Regularization path
min
w
λ
2
‖w‖2 + L(x, y;w)
Puneet K. Dokania 55
![Page 124: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/124.jpg)
Publications
Presentation Outline
1 Thesis Overview
2 Parsimonious Labeling
3 Learning to Rank Using High-Order Information
4 Regularization Path for SSVM
5 Future Work
6 Publications
Puneet K. Dokania 56
![Page 125: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/125.jpg)
Publications
List of publications
1 Discriminative parameter estimation for random walks segmentation, In
MICCAI 2013.
2 Learning to Rank using High-Order Information, In ECCV 2014.
3 Parsimonious Labeling, In ICCV 2015.
4 Minding the Gaps for Block Frank-Wolfe Optimization of StructuredSVM, In ICML 2016.
5 Rounding-based Combinatorial Algorithms for Metric Labeling, In JMLR
2016.
6 Deformable Registration through Learning of Context-Specific MetricAggregation, Under submission, ECCV 2016.
7 Partial Linearization based Optimization for Multi-class SVM, Under
submission, ECCV 2016.
Puneet K. Dokania 57
![Page 126: High-Order Inference, Ranking, and ... - Puneet Dokania€¦ · Puneet K. Dokania 10. Parsimonious Labeling Special cases of Parsimonious Labeling Special Case 1: Metric Labeling](https://reader033.vdocuments.us/reader033/viewer/2022052500/5f8da044da7cdf67500f167a/html5/thumbnails/126.jpg)
Thank you
Puneet K. Dokania 58