inference for learning belief propagation. so far... exact methods for submodular energies...
TRANSCRIPT
So far ...
Exact methods for submodular energies
Approximations for non-submodular energies
Move-making ( N_Variables >> N_Labels)
Inference for Learning
Linear Programming Relaxation
Linear Integer Programming
minx g0Tx
s.t. giTx ≤ 0
hiTx = 0
Linear function
Linear constraints
Linear constraints
x is a vector of integers
For example, x {0,1}N
Hard to solve !!
Linear Programming
minx g0Tx
s.t. giTx ≤ 0
hiTx = 0
Linear function
Linear constraints
Linear constraints
x is a vector of reals
Easy to solve!!
For example, x [0,1]N
Relaxation
Roadmap
Express MAP as an integer program
Relax to a linear program and solve
Round fractional solution to integers
2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Unary Cost
Integer Programming Formulation
Unary Cost Vector u = [ 5
Cost of V1 = 0
2
Cost of V1 = 1
; 2 4 ]
2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Unary Cost
Unary Cost Vector u = [ 5 2 ; 2 4 ]T
Label vector x = [ 0
V1 0
1
V1 = 1
; 1 0 ]T
Integer Programming Formulation
2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Unary Cost
Unary Cost Vector u = [ 5 2 ; 2 4 ]T
Label vector x = [ 0 1 ; 1 0 ]T
Sum of Unary Costs = ∑i ui xi
Integer Programming Formulation
2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Pairwise Cost
Integer Programming Formulation
0Cost of V1 = 0 and V1 = 0
0
00
0Cost of V1 = 0 and V2 = 0
3
Cost of V1 = 0 and V2 = 11 0
00
0 0
10
3 0
Pairwise Cost Matrix P
2
5
4
2
0
1 3
0
V1 V2
Label ‘0’
Label ‘1’Pairwise Cost
Integer Programming Formulation
Pairwise Cost Matrix P
0 0
00
0 3
1 0
00
0 0
10
3 0
Sum of Pairwise Costs
∑i<j Pij xixj
= ∑i<j Pij Xij
X = xxT
Integer Programming Formulation
Constraints
• Uniqueness Constraint
∑ xi = 1i Va
• Integer Constraints
xi {0,1}
X = x xT
Integer Programming Formulation
x* = argmin ∑ ui xi + ∑ Pij Xij
xi {0,1}
X = x xT
∑ xi = 1i Va
Roadmap
Express MAP as an integer program
Relax to a linear program and solve
Round fractional solution to integers
Integer Programming Formulation
x* = argmin ∑ ui xi + ∑ Pij Xij
∑ xi = 1i Va
xi {0,1}
X = x xT
Convex
Non-Convex
Integer Programming Formulation
x* = argmin ∑ ui xi + ∑ Pij Xij
∑ xi = 1i Va
xi [0,1]
X = x xT
Convex
Non-Convex
Integer Programming Formulation
x* = argmin ∑ ui xi + ∑ Pij Xij
∑ xi = 1i Va
xi [0,1]
Xij [0,1]
Convex
∑ Xij = xij Vb
Linear Programming Formulation
x* = argmin ∑ ui xi + ∑ Pij Xij
∑ xi = 1i Va
xi [0,1]
Xij [0,1]
Convex
∑ Xij = xij Vb
Schlesinger, 76; Chekuri et al., 01; Wainwright et al. , 01
Roadmap
Express MAP as an integer program
Relax to a linear program and solve
Round fractional solution to integers
Properties
Dominate many convex relaxations
Best known multiplicative bounds
2 for Potts (uniform) energies
2 + √2 for Truncated linear energies
O(log n) for metric labeling
Matched by move-making
Kumar and Torr, 2008; Kumar and Koller, UAI 2009
Kumar, Kolmogorov and Torr, 2007
Algorithms
Tree-reweighted message passing (TRW)
Max-product linear programming (MPLP)
Dual decomposition
Komodakis and Paragios, ICCV 2007