multiplicative bounds for metric labeling m. pawan kumar École centrale paris École des ponts...
TRANSCRIPT
Multiplicative Boundsfor Metric Labeling
M. Pawan KumarÉcole Centrale Paris
École des Ponts ParisTechINRIA Saclay, Île-de-France
Joint work with Phil Torr, Daphne Koller
Energy Minimization
Va Vb
Labels L = { l1, l2, …, lh}Variables V = { V1, V2, …, Vn}
Labeling f: { 1, 2, …, n} {1, 2, …, h}
E(f) = Σa θa(f(a)) + Σ(a,b) θab(f(a),f(b))minf
θa(f(a))
θb(f(b))
θab(f(a),f(b))
Metric Labeling
Va Vb
sab is non-negative d(.) is a metric distance function
Low-level vision applications NP hard
E(f)minf = Σa θa(f(a)) + Σ(a,b) sabd(f(a),f(b))
Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001 Murphy et al. Loopy Belief Propagation: An Empirical Study, UAI, 1999 Winn et al. Variational Message Passing, JMLR, 2005 Yedidia et al. Generalized Belief Propagation, NIPS, 2001 Besag. On the Statistical Analysis of Dirty Pictures, JRSS, 1986 Boykov et al. Fast Approximate Energy Minimization via Graph Cuts, PAMI, 2001 Komodakis et al. Fast, Approximately Optimal Solutions for Single and Dynamic MRFs, CVPR, 2007 Lempitsky et al. Fusion Moves for Markov Random Field Optimization, PAMI, 2010 Chekuri et al. Approximation Algorithms for Metric Labeling, SODA, 2001 Goemans et al. Improved Approximate Algorithms for Maximum-Cut, JACM, 1995 Muramatsu et al. A New SOCP Relaxation for Max-Cut, JORJ, 2003 Ravikumar et al. QP Relaxations for Metric Labeling, ICML, 2006 Alahari et al. Dynamic Hybrid Algorithms for MAP Inference, PAMI 2010 Kohli et al. On Partial Optimality in Multilabel MRFs, ICML, 2008 Rother et al. Optimizing Binary MRFs via Extended Roof Duality, CVPR, 2007
.
.
.
Approximate Algorithms
Multiplicative Bounds
f*: Optimal Labeling f: Estimated Labeling
Σa θa(f(a)) + Σ(a,b) sabd(f(a),f(b))
Σa θa(f*(a)) + Σ(a,b) sabd(f*(a),f*(b))
≥
Multiplicative Bounds
f*: Optimal Labeling f: Estimated Labeling
≤
M
Σa θa(f(a)) + Σ(a,b) sabd(f(a),f(b))
Σa θa(f*(a)) + Σ(a,b) sabd(f*(a),f*(b))
Outline
• Convex Relaxations
• Move-Making Algorithms
• Comparison
• Move-Making for Metric Labeling
• Move-Making for Truncated Convex Models
Integer Linear Program
Number of facets grows exponentially in problem size
Minimize a linear function over a set of feasible solutions
Convex Relaxations
TimeLP
1976
SOCP
2003
QP
2006
Tig
htn
ess
Expected
Analyzed
Kumar, Kolmogorov and Torr, NIPS 2007
Outline
• Convex Relaxations
• Move-Making Algorithms
• Comparison
• Move-Making for Metric Labeling
• Move-Making for Truncated Convex Models
Expansion Algorithm
Initialize labeling f = f0 (say f0(a) = 1, for all Va)
For α = 1, 2, … , h
End
fα = argminf’ E(f’)
s.t. f’(a) {f(a)} U {lα}
Update f = fα
Boykov, Veksler and Zabih, 2001
Repeat until
convergence
Expansion Algorithm
Sky
House
Tree
Ground
Initialize with TreeStatus: Expand GroundExpand HouseExpand Sky
Slide courtesy Pushmeet Kohli
Variables take label lα or retain current label
Outline
• Convex Relaxations
• Move-Making Algorithms
• Comparison
• Move-Making for Metric Labeling
• Move-Making for Truncated Convex Models
Multiplicative Bounds
Expansion LP
Potts 2M 2
dmax = maxi≠ kd(i,k) dmin = mini≠ kd(i,k)
M = dmax dmin
Multiplicative Bounds
Expansion LP
Potts 2 2
Metric 2M O(log h)
TruncatedLinear
2M 2 + √2
TruncatedQuadratic
2M O(√M)
Outline
• Convex Relaxations
• Move-Making Algorithms
• Comparison
• Move-Making for Metric Labeling
• Move-Making for Truncated Convex Models
Kumar and Koller, UAI 2009
Expansion Algorithm
Initialize labeling f = f0 (say f0(a) = 1, for all Va)
For α = 1, 2, … , h
End
fα = argminf’ E(f’)
s.t. f’(a) {f(a)} U {α}
Update f = fα
Repeat until
convergence
Boykov, Veksler and Zabih, 2001
Modified Expansion Algorithm
Initialize labeling f = f0 (say f0(a) = 1, for all Va)
For α = 1, 2, … , h
End
fα = argminf’ E(f’)
s.t. f’(a) {f(a)} U {α}
Update f = fα
Repeat until
convergence
Modified Expansion Algorithm
Initialize labeling f = f0 (say f0(a) = 1, for all Va)
For α = 1, 2, … , h
End
fα = argminf’ E(f’)
s.t. f’(a) {f(a)} U {Mα(a)}
Update f = fα
Repeat until
convergence
Any label Mα(a) instead of the same label α
Modified Expansion Algorithm
Multiplicative Bound = 2
dmax = maxi,k d(i,k)
dmax dmin
i = Mα(a), k = Mβ(b), α ≠ β
dmin = mini,k d(i,k) i = Mα(a), k = Mβ(b), α ≠ β
Outline
• Convex Relaxations
• Move-Making Algorithms
• Comparison
• Move-Making for Metric Labeling• Visualizing Metrics• Uniform Metric• Hierarchically Separated Tree (HST) Metrics• General Metrics
• Move-Making for Truncated Convex Models
Visualizing Metrics
l5
l1l2
l4l3
w1w2
w3
w4
w5
w6
w7 w9w8
d( i , j ) : shortest path defined by the graph
Visualizing Metrics
l5
l1l2
l4l3
1 15
1
1
11
1 32
d( i , j ) : shortest path defined by the graph
d(1,4) = 3
Visualizing Metrics
l5
l1l2
l4l3
1 15
1
1
11
1 32
d( i , j ) : shortest path defined by the graph
d(1,2) = 5
Outline
• Convex Relaxations
• Move-Making Algorithms
• Comparison
• Move-Making for Metric Labeling• Visualizing Metrics• Uniform Metric• Hierarchically Separated Tree (HST) Metrics• General Metrics
• Move-Making for Truncated Convex Models
Modified Expansion Algorithm
Initialize labeling f = f0 (say f0(a) = 1, for all Va)
For α = 1, 2, … , h
End
fα = argminf’ E(f’)
s.t. f’(a) {f(a)} U {Mα(a)}
Update f = fα
Repeat until
convergence
Any label Mα(a) instead of the same label α
Uniform Metric
l1 l2 l3
ww
w
Mα(a) = lα for all random variables Va
Multiplicative Bound = 2
dmax = maxi,k d(i,k)
dmax
dmin
i = Mα(a), k = Mβ(b), α ≠ β
dmin = mini,k d(i,k) i = Mα(a), k = Mβ(b), α ≠ β
Uniform Metric
l1 l2 l3
ww
w
Mα(a) = lα for all random variables Va
Multiplicative Bound = 2
dmax = 2w
dmax
dmin
dmin = mini,k d(i,k) i = Mα(a), k = Mβ(b), α ≠ β
Uniform Metric
l1 l2 l3
ww
w
Mα(a) = lα for all random variables Va
Multiplicative Bound = 2
dmax = 2w
dmax
dmin
dmin = 2w
Uniform Metric
l1 l2 l3
ww
w
Mα(a) = lα for all random variables Va
Multiplicative Bound = 2
dmax = 2w
dmin = 2w
Same bound as LP
Outline
• Convex Relaxations
• Move-Making Algorithms
• Comparison
• Move-Making for Metric Labeling• Visualizing Metrics• Uniform Metric• Hierarchically Separated Tree (HST) Metrics• General Metrics
• Move-Making for Truncated Convex Models
HST Metric
w1
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1
w1
w2 w2 w3 w3
w4
w5 w5w6 w6 w7 w7
w8 w8
Graph is a Tree. Labels are leaves
HST Metric
w1
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1
w1
w2 w2 w3 w3
w4
w5 w5w6 w6 w7 w7
w8 w8
Edge lengths for all children are the same
HST Metric
w1
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1
w1
w2 w2 w3 w3
w4
w5 w5w6 w6 w7 w7
w8 w8
Edge lengths decrease from root to leaf by factor r ≥ 2
HST Metric
w1
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1
w1
w2 w2 w3 w3
w4
w5 w5w6 w6 w7 w7
w8 w8
w2 ≤ w1/r w3 ≤ w1/r w4 ≤ w1/r
HST Metric
w1
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1
w1
w2 w2 w3 w3
w4
w5 w5w6 w6 w7 w7
w8 w8
w5 ≤ w2/r w6 ≤ w2/r w7 ≤ w3/r w8 ≤ w3/r
Metric Labeling for 3-level HST
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1 w1 w1
w2 w2 w2 w3 w3 w3 w4 w4 w4
Mα(a) = lα
Metric Labeling for 3-level HST
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1 w1 w1
w2 w2 w2 w3 w3 w3 w4 w4 w4
f1
Multiplicative bound of 2 for (a,b) where f*(a), f*(b) {1,2,3}
Metric Labeling for 3-level HST
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1 w1 w1
w2 w2 w2 w3 w3 w3 w4 w4 w4
f1
Multiplicative bound of 2 for (a,b) where f*(a), f*(b) {4,5,6}
f2
Metric Labeling for 3-level HST
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1 w1 w1
w2 w2 w2 w3 w3 w3 w4 w4 w4
f1
Multiplicative bound of 2 for (a,b) where f*(a), f*(b) {7,8,9}
f2 f3
Metric Labeling for 3-level HST
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1 w1 w1
w2 w2 w2 w3 w3 w3 w4 w4 w4
f1 f2 f3
M1(a) = f1(a) M2(a) = f2(a) M3(a) = f3(a)
Metric Labeling for 3-level HST
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1 w1 w1
w2 w2 w2 w3 w3 w3 w4 w4 w4
f1 f2 f3
Multiplicative bound of at most 2(1+1/r)
Metric Labeling for 4-level HST
Multiplicative bound of at most 2(1+1/r + 1/r2)
w1
l1 l2 l3 l4 l5 l6 l7 l8 l9
w1
w1
w2 w2 w3 w3
w4
w5 w5w6 w6 w7 w7
w8 w8
Metric Labeling for L-level HST
Multiplicative bound = 2(1+1/r + … + 1/rL-2)
< 2r/(r-1)
Constant bound for all HST metrics
Outline
• Convex Relaxations
• Move-Making Algorithms
• Comparison
• Move-Making for Metric Labeling• Visualizing Metrics• Uniform Metric• Hierarchically Separated Tree (HST) Metrics• General Metrics
• Move-Making for Truncated Convex Models
General Metrics
Any metric can be approximated using a mixture ofO(h log h) HST metrics with distortion O(log h).
Fakcharoenphol, Rao and Talwar, 2003
ΣT ρ(T)dT(i,j)
ΣT ρ(T) = 1
d(i,j) ≤ ≤ O(log h) d(i,j)
Metric Labeling for General Metrics
Approximate d using ΣT ρ(T)dT
Solve metric labeling for each HST dT
Pick the labeling with the lowest energy
Multiplicative bound = 2r/(r-1)*O(log h) = O(log h)
Same bound as LP
Outline
• Convex Relaxations
• Move-Making Algorithms
• Comparison
• Move-Making for Metric Labeling
• Move-Making for Truncated Convex Models
Kumar and Torr, NIPS 2008
Truncated Convex Models
Va Vb
sab is non-negative d(.) is a convex function
E(f)minf = Σa θa(f(a)) + Σ(a,b) sab min{d(f(a),f(b)), M}
Truncated Convex Models
sab is non-negative d(.) is a convex function
Low-level vision applications NP hard
E(f)minf = Σa θa(f(a)) + Σ(a,b) sab min{d(f(a),f(b)), M}
Truncated Linear Truncated Quadratic
Modified Expansion Algorithm
Initialize labeling f = f0 (say f0(a) = 1, for all Va)
For α = 1, 2, … , h
End
fα = argminf’ E(f’)
s.t. f’(a) {f(a)} U {Mα(a)}
Update f = fα
Repeat until
convergence
Any label Mα(a) instead of the same label lα
Modified Expansion Algorithm
Initialize labeling f = f0 (say f0(a) = 1, for all Va)
For α = 1, 2, … , h
End
fα = argminf’ E(f’)
s.t. f’(a) {f(a)} U {Mα(a)}
Update f = fα
Repeat until
convergence
Any interval of labels Mα(a) instead of a single label
Modified Expansion Algorithm
Initialize labeling f = f0 (say f0(a) = 1, for all Va)
For α = 1, 2, … , h
End
fα = argminf’ E(f’)
s.t. f’(a) {f(a)} U {α, α+1, …, α+L}
Update f = fα
Repeat until
convergence
Any interval of labels Mα(a) instead of a single label
Length of the Interval
Submodular problem
Exact optimization in polynomial time
Small interval implies bigger ratio of dmax/dmin
Length of the Interval
Submodular problem
Optimization of upper bound in polynomial time
Big interval implies smaller ratio of dmax/dmin
Length of the Interval
Truncated Linear
Bound = 2 + max 2M , L
L M
L = √2M Bound = 2 + √2
Truncated Quadratic
Bound = O(√M)L = √M
Conclusion
MoveMaking
LP
Potts 2 2
Metric O(log h) O(log h)
TruncatedLinear
2 + √2 2 + √2
TruncatedQuadratic
O(√M) O(√M)
Future Work
Move-making and convex relaxations connection
Better measures than multiplicative bounds
Designing hybrid algorithms