a faster algorithm for linear programming and the maximum flow
Post on 19-Jan-2017
224 Views
Preview:
TRANSCRIPT
A Faster Algorithm for Linear Programming and the Maximum Flow Problem I
Yin Tat Lee(MIT, Simons)
Joint work with Aaron Sidford
THE PROBLEM
Linear Programming
Consider the linear program (LP)
where is a matrix.
• is the number of constraints.• is the number of variables.
𝑛=2 ,𝑚=6𝑛=2 ,𝑚=∞
Previous Results
• All of them are iterative methods.
• Start with some initial point .• While is not optimal– Improve
• Time =
• This talk focus on #iter.
= # of constraints = # of variables
We call it efficient if• Polynomial time• Doesn’t use LP solver
Previous Results (Selected)
Year Author # of iter Cost per iter Efficient steps
1947 Dantzig Pivot Yes
1979 Khachiyan Update the Ellipsoid Yes
1984 Karmarkar Solve linear systems Yes
1986 Renegar Solve linear systems Yes
1989 Vaidya Matrix inverse Yes
1994Nesterov,
Nemirovskii Compute volume No
2013 Lee, SidfordSolve Linear Systems Yes
Solve Linear Systems Yes
= # of constraints = # of variables( is omitted)
Remark: In 2013, Mądry shows how to obtain iters for certain LPs!
Outline
Year Author # of iter Cost per iter Efficient steps
1947 Dantzig Pivot Yes
1979 Khachiyan Update the Ellipsoid Yes
1984 Karmarkar Solve linear systems Yes
1986 Renegar Solve linear systems Yes
1989 Vaidya Matrix inverse Yes
1994Nesterov,
Nemirovskii Compute volume No
2013 Lee, SidfordSolve Linear Systems Yes
Solve Linear Systems Yes
= # of constraints = # of variables( is omitted)
Remark: In 2013, Mądry shows how to obtain iters for certain LPs!
LP AND CENTER
A general framework
We can solve linear program by maintaining center.
Somehow, get a “center” first Put the cost constraint there and move it.
Say we can move portion closer each time
After steps, we are done.
A general framework
We can solve linear program by maintaining center.
Somehow, get a “center” first Put the cost constraint there and move it.
Say we can move portion closer each time
After steps, we are done.Why center?
What if we don’t try to maintain a center?
• It is just like simplex method.
It is good now. Still good.
Oh, it touches. What to do?…..
What if we don’t try to maintain a center?
• It is just like simplex method.
It is good now. Still good.
Oh, it touches. What to do?…..
Avoid bad decision by using
global information!
A general framework
Formally, we have (say ):• . Find the center of • While is large– for some fixed – Update the center of
This is called interior point method.
The initial point is easy:
A general way to define a center
Let be a smooth convex function on such that• as .
For example,Standard log barrier:
BarrierFunction
Center
QUALITY OF A CENTER
Rounding
• Assume center is induced by some barrier function .• Look at the ellipsoid induced by at the center .• Call is rounding if for some .
Self concordant barrier
• is a -self concordant barrier function for if– is smooth.– gives rounding.
is not smooth enough Bad rounding.
Rounding Algorithm
For general barrier function :• Repeat– Tighten the cost constraint– Maintain the rounding ellipsoid induced by .
Why iterations?
Why iterations?
Think .• Newton Method (Using smoothness)Given , we can find the center in steps.
Why iterations?
Let be the old center. Using the smoothness, we have
Why iterations?
So, we need
It takes iters.
Why iterations?
• We can reduce the gap by .
Roughly Speaking:Smoothness + rounding gives
iterations LP solvers.
Quality of analytic center is arbitrary bad in !
• Recall the standard log barrier function
• The center is called analytic center.
Is it tight?
• In practice, it takes steps.• Mizuno, Todd, Ye showed it is “usually” correct on first step.• In 2014, Mut and Terlaky showed an example really takes
iterations where is exponential in .
UNIVERSAL BARRIER FUNCTION
Universal Barrier Function
Theorem [NN94]: For any convex set ,
is a -self concordant barrier function.
“Smaller” set has larger polar. Hence, as
Note that .
Kannan-Lovasz-Simonovits Lemma: For any convex set , the second moment matrix
gives a rounding of .
The cost of Universal Barrier
• To get second moment matrix, you need sampling.• To get 1 sampling, you need to do iters of Markov chain.• To do 1 iter of Markov chain, you need to implement
separation oracle for .• If , one need to solve an LP.
Hence, one iteration requires solving many LPs.
The problem:Get an efficient self concordant barrier
function.
VOLUMETRIC BARRIER FUNCTION
Volumetric Barrier Function
In 1989, Vaidya showed
where Why it is volumetric?
For example:
It is a barrier.
Log BarrierVolumetric Barrier
Why Volumetric is good?
Around , we have
where
Example: . Then, . , , .In general, , , if the row is repeated, is decreased by .
For [0,1] interval with 0 repeated times:
𝑆−1 𝐴=(11⋮ ) 𝑆−1 𝐴=(𝑘1/3
1⋮ )
OUR BARRIER FUNCTION
Repeated Volumetric Barrier Function
How about ?
Suppose , around , we have
So, we have
We call where satisfiesWhat is that?
What is that weight?
• Let .
If for all , the ellipsoid is inside.
Our Condition (John Ellipsoid): if .
𝑤𝑖(∞ )(𝑥)=𝜎 𝑖 (√𝑊 (∞ )𝑆𝑥
−1 𝐴) .
The represents John ellipsoid of
Repeated Volumetric Barrier Function
• Recall
We get
Symmetrize Find John Ellipsoid
The barrier function is not perfect!
• The path is piecewise smooth because it may not touch every constraint.
Our Barrier Function
• Standard Log Barrier:
• Volumetric Barrier:
• John Ellipsoid Barrier:
• Regularized John Ellipsoid Barrier (1):
• Regularized John Ellipsoid Barrier (2):
Lewis Weight
We call is Lewis weight for if
Thanks to Cohen and Peng, we know• Let be rows sample of accordingly to ,
• is the maximizer of
i.e, the maximum ellipsoid such that it “insides” the polytopes.• For , is the John ellipsoid for .
max𝑤 ≥ 0
lndet (𝐴𝑇𝑆−1𝑊1−log−1 (𝑚𝑛 )
𝑆− 1𝐴) .
Computing Lewis Weight
• Cohen and Peng showed how to compute it when .• The repeated volumetric barrier: ,
After renormalization, gives “ “Lewis” weight:
• Cohen, L., Peng, Sidford shows that in fact a similar algorithm find constant “approximate” Lewis weight for in .
CONCLUSION
Our Barrier
Given any polytope , let
Theorem: The barrier function gives iterations algorithm for LP of the form
Algorithm:• While– Move the cost constraint– Maintain the regularized John ellipsoid
= # of constraints = # of variables
However…
• My goal is to design general LP algo fast enough to
beat the best maxflow algorithm!
• We obtained
= # of constraints = # of variables
Compute ( 𝐴𝑇𝐷𝑘 𝐴 )−1min𝐴𝑥 ≥𝑏
𝑐𝑇 𝑥
top related