![Page 1: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/1.jpg)
Smoothed Analysis of AlgorithmsPart I: Simplex Method and Local Search
Heiko RoglinDepartment of Computer Science
16 January 2013
Heiko Roglin Smoothed Analysis of Algorithms
![Page 2: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/2.jpg)
Discrete Optimization
Many problems and algorithms seem well understood.
Linear Programmingefficient algorithms (ellipsoid, interior point)
Simplex method performs well in practice.
Knapsack Problem (KP)NP-hard, FPTAS exists
very easy problem, solvable in almost linear time
Traveling Salesperson Problem (TSP)NP-hard, even hard to approximate
local search methods yield very good solutions
⇒ big gap between theory and practice
Heiko Roglin Smoothed Analysis of Algorithms
![Page 3: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/3.jpg)
Discrete Optimization
Many problems and algorithms seem well understood.
Linear Programmingefficient algorithms (ellipsoid, interior point)Simplex method performs well in practice.
Knapsack Problem (KP)NP-hard, FPTAS existsvery easy problem, solvable in almost linear time
Traveling Salesperson Problem (TSP)NP-hard, even hard to approximatelocal search methods yield very good solutions
⇒ big gap between theory and practice
Heiko Roglin Smoothed Analysis of Algorithms
![Page 4: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/4.jpg)
Discrete Optimization
Many problems and algorithms seem well understood.
Linear Programmingefficient algorithms (ellipsoid, interior point)Simplex method performs well in practice.
Knapsack Problem (KP)NP-hard, FPTAS existsvery easy problem, solvable in almost linear time
Traveling Salesperson Problem (TSP)NP-hard, even hard to approximatelocal search methods yield very good solutions
⇒ big gap between theory and practice
Heiko Roglin Smoothed Analysis of Algorithms
![Page 5: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/5.jpg)
Outline
Outline1 Linear Programming
Why is the simplex method usually efficient?Smoothed Analysis – analysis of algorithms beyond worst case
2 Traveling Salesperson ProblemWhy is local search successful?
3 Smoothed AnalysisOverview of known results
Heiko Roglin Smoothed Analysis of Algorithms
![Page 6: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/6.jpg)
Outline
Outline1 Linear Programming
Why is the simplex method usually efficient?Smoothed Analysis – analysis of algorithms beyond worst case
2 Traveling Salesperson ProblemWhy is local search successful?
3 Smoothed AnalysisOverview of known results
Heiko Roglin Smoothed Analysis of Algorithms
![Page 7: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/7.jpg)
Linear Programming
Linear Programs (LPs)
variables: x1, . . . , xn ∈ Rlinear objective function:max cT x = c1x1 + . . .+ cnxn
m linear constraints:
a1,1x1 + . . .+ a1,nxn ≤ b1
...
am,1x1 + . . .+ am,nxn ≤ bm
c
x∗
Complexity of LPs
LPs can be solved in polynomial time by the ellipsoid method[Khachiyan 1979] and the interior point method [Karmarkar 1984].
Heiko Roglin Smoothed Analysis of Algorithms
![Page 8: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/8.jpg)
Linear Programming
Linear Programs (LPs)
variables: x1, . . . , xn ∈ Rlinear objective function:max cT x = c1x1 + . . .+ cnxn
m linear constraints:
a1,1x1 + . . .+ a1,nxn ≤ b1
...
am,1x1 + . . .+ am,nxn ≤ bm
c
x∗
Complexity of LPs
LPs can be solved in polynomial time by the ellipsoid method[Khachiyan 1979] and the interior point method [Karmarkar 1984].
Heiko Roglin Smoothed Analysis of Algorithms
![Page 9: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/9.jpg)
Simplex Algorithm
c
Simplex Algorithm
Start at some vertex of thepolytope.
Walk along the edges of thepolytope in the direction of theobjective function cT x .
local optimum = global optimum
c Pivot RulesWhich vertex is chosen if there aremultiple options?
Different pivot rules suggested:random, steepest descent, shadowvertex pivot rule, . . .
Heiko Roglin Smoothed Analysis of Algorithms
![Page 10: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/10.jpg)
Simplex Algorithm
c
Simplex Algorithm
Start at some vertex of thepolytope.
Walk along the edges of thepolytope in the direction of theobjective function cT x .
local optimum = global optimum
c Pivot RulesWhich vertex is chosen if there aremultiple options?
Different pivot rules suggested:random, steepest descent, shadowvertex pivot rule, . . .
Heiko Roglin Smoothed Analysis of Algorithms
![Page 11: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/11.jpg)
Simplex Algorithm
c
Simplex Algorithm
Start at some vertex of thepolytope.
Walk along the edges of thepolytope in the direction of theobjective function cT x .
local optimum = global optimum
c Pivot RulesWhich vertex is chosen if there aremultiple options?
Different pivot rules suggested:random, steepest descent, shadowvertex pivot rule, . . .
Heiko Roglin Smoothed Analysis of Algorithms
![Page 12: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/12.jpg)
Simplex Algorithm
c
Simplex Algorithm
Start at some vertex of thepolytope.
Walk along the edges of thepolytope in the direction of theobjective function cT x .
local optimum = global optimum
c Pivot RulesWhich vertex is chosen if there aremultiple options?
Different pivot rules suggested:random, steepest descent, shadowvertex pivot rule, . . .
Heiko Roglin Smoothed Analysis of Algorithms
![Page 13: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/13.jpg)
Simplex Algorithm
c
Simplex Algorithm
Start at some vertex of thepolytope.
Walk along the edges of thepolytope in the direction of theobjective function cT x .
local optimum = global optimum
c Pivot RulesWhich vertex is chosen if there aremultiple options?
Different pivot rules suggested:random, steepest descent, shadowvertex pivot rule, . . .
Heiko Roglin Smoothed Analysis of Algorithms
![Page 14: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/14.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow.
cu
x0
x∗
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 15: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/15.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow.
cu
x0
x∗
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 16: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/16.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow.
cu
x0
x∗
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 17: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/17.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow.
cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 18: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/18.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow.
cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 19: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/19.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 20: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/20.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 21: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/21.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 22: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/22.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 23: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/23.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 24: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/24.jpg)
Simplex Algorithm – Shadow Vertex Pivot Rule
Shadow Vertex Pivot RuleLet x0 be some vertex of thepolytope.
Compute u ∈ Rd such thatx0 maximizes uT x .
Project the polytope onto theplane spanned by c and u.
Start at x0 and follow the edgesof the shadow. cu
x0
x∗
x∗x0
The shadow is a polygon.
x0 is a vertex of the shadow.
x∗ is a vertex of the shadow.
Edges of the shadow correspond to edges of the polytope.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 25: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/25.jpg)
Simplex Algorithm – Running Time
Theoreticians say. . .
shadow vertex pivot rule requiresexponential number of steps
no pivot rule with sub-exponentialnumber of steps known
ellipsoid and interior point methods areefficient
Engineers say. . .
simplex method usually fastest algorithmin practice
requires usually only Θ(m) steps
clearly outperforms ellipsoid method
Heiko Roglin Smoothed Analysis of Algorithms
![Page 26: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/26.jpg)
Simplex Algorithm – Running Time
Theoreticians say. . .
shadow vertex pivot rule requiresexponential number of steps
no pivot rule with sub-exponentialnumber of steps known
ellipsoid and interior point methods areefficient
Engineers say. . .
simplex method usually fastest algorithmin practice
requires usually only Θ(m) steps
clearly outperforms ellipsoid method
Heiko Roglin Smoothed Analysis of Algorithms
![Page 27: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/27.jpg)
Reason for Gap between Theory and Practice
Reason for Gap between Theory and Practice
Worst-case complexity is too pessimistic!
There are (artificial) worst-case LPs on whichthe simplex method is not efficient. These LPs,however, do not occur in practice.e.g., a1,i = 2i ,
∑i a2,i ≡ 3 mod 5, . . .
This phenomenon occurs not only for thesimplex method, but also for many otherproblems and algorithms.
Adversary
“I will trickyouralgorithm!”
GoalFind a more realistic performance measure that is not just based onthe worst case.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 28: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/28.jpg)
Reason for Gap between Theory and Practice
Reason for Gap between Theory and Practice
Worst-case complexity is too pessimistic!
There are (artificial) worst-case LPs on whichthe simplex method is not efficient. These LPs,however, do not occur in practice.e.g., a1,i = 2i ,
∑i a2,i ≡ 3 mod 5, . . .
This phenomenon occurs not only for thesimplex method, but also for many otherproblems and algorithms.
Adversary
“I will trickyouralgorithm!”
GoalFind a more realistic performance measure that is not just based onthe worst case.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 29: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/29.jpg)
Reason for Gap between Theory and Practice
Reason for Gap between Theory and Practice
Worst-case complexity is too pessimistic!
There are (artificial) worst-case LPs on whichthe simplex method is not efficient. These LPs,however, do not occur in practice.e.g., a1,i = 2i ,
∑i a2,i ≡ 3 mod 5, . . .
This phenomenon occurs not only for thesimplex method, but also for many otherproblems and algorithms.
Adversary
“I will trickyouralgorithm!”
GoalFind a more realistic performance measure that is not just based onthe worst case.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 30: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/30.jpg)
Smoothed Analysis
Observation: In worst-case analysis, the adversary is too powerful.Idea: Let’s weaken him!
Perturbed LPs
Step 1: Adversary specifies arbitrary LP:max cT x subject to aT
1 x ≤ b1 . . . aTn x ≤ bn.
W. l. o. g. ‖(ai , bi)‖ = 1.
Step 2: Add Gaussian random variable with standard deviation σto each coefficient in the constraints.
Smoothed Running Time= worst expected running time the adversary can achieve
Heiko Roglin Smoothed Analysis of Algorithms
![Page 31: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/31.jpg)
Smoothed Analysis
Observation: In worst-case analysis, the adversary is too powerful.Idea: Let’s weaken him!
Perturbed LPs
Step 1: Adversary specifies arbitrary LP:max cT x subject to aT
1 x ≤ b1 . . . aTn x ≤ bn.
W. l. o. g. ‖(ai , bi)‖ = 1.
Step 2: Add Gaussian random variable with standard deviation σto each coefficient in the constraints.
Smoothed Running Time= worst expected running time the adversary can achieve
Heiko Roglin Smoothed Analysis of Algorithms
![Page 32: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/32.jpg)
Smoothed Analysis
Observation: In worst-case analysis, the adversary is too powerful.Idea: Let’s weaken him!
Perturbed LPs
Step 1: Adversary specifies arbitrary LP:max cT x subject to aT
1 x ≤ b1 . . . aTn x ≤ bn.
W. l. o. g. ‖(ai , bi)‖ = 1.
Step 2: Add Gaussian random variable with standard deviation σto each coefficient in the constraints.
Smoothed Running Time= worst expected running time the adversary can achieve
Heiko Roglin Smoothed Analysis of Algorithms
![Page 33: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/33.jpg)
Smoothed Analysis
Observation: In worst-case analysis, the adversary is too powerful.Idea: Let’s weaken him!
Perturbed LPs
Step 1: Adversary specifies arbitrary LP:max cT x subject to aT
1 x ≤ b1 . . . aTn x ≤ bn.
W. l. o. g. ‖(ai , bi)‖ = 1.
Step 2: Add Gaussian random variable with standard deviation σto each coefficient in the constraints.
Smoothed Running Time= worst expected running time the adversary can achieve
Heiko Roglin Smoothed Analysis of Algorithms
![Page 34: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/34.jpg)
Smoothed Analysis
Step 1:Adversarychooses input I
Step 2: RandomperturbationI → perσ(I)
Formal Definition:LP(n,m) = set of LPs with n variables and m constraintsT (I) = number of steps of simplex method on LP I
smoothed run time T smooth(n,m, σ) = maxI∈LP(n,m)E [T (perσ(I))]
Why do we consider this model?
First step models unknown structure of the input.
Second step models random influences, e.g., measurementerrors, numerical imprecision, rounding, . . .
smoothed running time low⇒ bad instances are unlikely to occur
σ determines the amount of randomness
Heiko Roglin Smoothed Analysis of Algorithms
![Page 35: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/35.jpg)
Smoothed Analysis
Step 1:Adversarychooses input I
Step 2: RandomperturbationI → perσ(I)
Formal Definition:LP(n,m) = set of LPs with n variables and m constraintsT (I) = number of steps of simplex method on LP Ismoothed run time T smooth(n,m, σ) = maxI∈LP(n,m)E [T (perσ(I))]
Why do we consider this model?
First step models unknown structure of the input.
Second step models random influences, e.g., measurementerrors, numerical imprecision, rounding, . . .
smoothed running time low⇒ bad instances are unlikely to occur
σ determines the amount of randomness
Heiko Roglin Smoothed Analysis of Algorithms
![Page 36: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/36.jpg)
Smoothed Analysis
Step 1:Adversarychooses input I
Step 2: RandomperturbationI → perσ(I)
Formal Definition:LP(n,m) = set of LPs with n variables and m constraintsT (I) = number of steps of simplex method on LP Ismoothed run time T smooth(n,m, σ) = maxI∈LP(n,m)E [T (perσ(I))]
Why do we consider this model?
First step models unknown structure of the input.
Second step models random influences, e.g., measurementerrors, numerical imprecision, rounding, . . .
smoothed running time low⇒ bad instances are unlikely to occur
σ determines the amount of randomness
Heiko Roglin Smoothed Analysis of Algorithms
![Page 37: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/37.jpg)
Smoothed Analysis of the Simplex Algorithm
Lemma [Spielman and Teng (STOC 2001)]
For every fixed plane and every LP the adversarycan choose, after the perturbation, the expectednumber of edges on the shadow is
O(poly
(n,m, σ−1)) . cu
x0
x∗
Theorem [Spielman and Teng (STOC 2001)]
The smoothed running time of the simplex algorithm with shadowvertex pivot rule is O
(poly
(n,m, σ−1)) .
Already for small perturbations exponential running time is unlikely.
Main Difficulties in Proof of Theorem:
x0 is found in phase I→ no Gaussian distribution of coefficients
In phase II, the plane onto which the polytope is projected is notindependent of the perturbations.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 38: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/38.jpg)
Smoothed Analysis of the Simplex Algorithm
Lemma [Spielman and Teng (STOC 2001)]
For every fixed plane and every LP the adversarycan choose, after the perturbation, the expectednumber of edges on the shadow is
O(poly
(n,m, σ−1)) . cu
x0
x∗
Theorem [Spielman and Teng (STOC 2001)]
The smoothed running time of the simplex algorithm with shadowvertex pivot rule is O
(poly
(n,m, σ−1)) .
Already for small perturbations exponential running time is unlikely.
Main Difficulties in Proof of Theorem:
x0 is found in phase I→ no Gaussian distribution of coefficients
In phase II, the plane onto which the polytope is projected is notindependent of the perturbations.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 39: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/39.jpg)
Smoothed Analysis of the Simplex Algorithm
Lemma [Spielman and Teng (STOC 2001)]
For every fixed plane and every LP the adversarycan choose, after the perturbation, the expectednumber of edges on the shadow is
O(poly
(n,m, σ−1)) . cu
x0
x∗
Theorem [Spielman and Teng (STOC 2001)]
The smoothed running time of the simplex algorithm with shadowvertex pivot rule is O
(poly
(n,m, σ−1)) .
Already for small perturbations exponential running time is unlikely.
Main Difficulties in Proof of Theorem:
x0 is found in phase I→ no Gaussian distribution of coefficients
In phase II, the plane onto which the polytope is projected is notindependent of the perturbations.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 40: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/40.jpg)
Improved Analysis
Theorem [Vershynin (FOCS 2006)]
The smoothed number of steps of the simplex algorithm with shadowvertex pivot rule is
O(poly
(n, log m, σ−1)) .
only polylogarithmic in the number of constraints m
Phase I: add vertex x0 in randomdirection. With constant prob. this doesnot change optimal solution.⇒ The plane is not correlated with theperturbed polytope.
With high prob. no angle betweenconsecutive vertices is too small.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 41: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/41.jpg)
Improved Analysis
Theorem [Vershynin (FOCS 2006)]
The smoothed number of steps of the simplex algorithm with shadowvertex pivot rule is
O(poly
(n, log m, σ−1)) .
only polylogarithmic in the number of constraints m
cu
x0
x∗Phase I: add vertex x0 in randomdirection. With constant prob. this doesnot change optimal solution.⇒ The plane is not correlated with theperturbed polytope.
With high prob. no angle betweenconsecutive vertices is too small.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 42: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/42.jpg)
Improved Analysis
Theorem [Vershynin (FOCS 2006)]
The smoothed number of steps of the simplex algorithm with shadowvertex pivot rule is
O(poly
(n, log m, σ−1)) .
only polylogarithmic in the number of constraints m
cu
x0
x∗
x∗x0
Phase I: add vertex x0 in randomdirection. With constant prob. this doesnot change optimal solution.⇒ The plane is not correlated with theperturbed polytope.
With high prob. no angle betweenconsecutive vertices is too small.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 43: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/43.jpg)
Outline
Outline1 Linear Programming
Why is the simplex method usually efficient?smoothed analysis – analysis of algorithms beyond worst case
2 Traveling Salesperson ProblemWhy is local search successful?
3 Smoothed AnalysisOverview of known results
Heiko Roglin Smoothed Analysis of Algorithms
![Page 44: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/44.jpg)
Traveling Salesperson Problem
Traveling Salesperson Problem (TSP)
Input: weighted (complete) graphG = (V ,E , d) with d : E → R+
Goal: Find Hamiltonian cycle ofminimum length.
One of the most intensively studied problems in optimization– both in theory and practice.
Metric TSP: APX-hard Euclidean TSP: PTAS exists
Heiko Roglin Smoothed Analysis of Algorithms
![Page 45: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/45.jpg)
Traveling Salesperson Problem
Traveling Salesperson Problem (TSP)
Input: weighted (complete) graphG = (V ,E , d) with d : E → R+
Goal: Find Hamiltonian cycle ofminimum length.
One of the most intensively studied problems in optimization– both in theory and practice.
Metric TSP: APX-hard Euclidean TSP: PTAS exists
Heiko Roglin Smoothed Analysis of Algorithms
![Page 46: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/46.jpg)
Traveling Salesperson Problem
Traveling Salesperson Problem (TSP)
Input: weighted (complete) graphG = (V ,E , d) with d : E → R+
Goal: Find Hamiltonian cycle ofminimum length.
One of the most intensively studied problems in optimization– both in theory and practice.
Metric TSP: APX-hard Euclidean TSP: PTAS exists
Heiko Roglin Smoothed Analysis of Algorithms
![Page 47: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/47.jpg)
Traveling Salesperson Problem
Traveling Salesperson Problem (TSP)
Input: weighted (complete) graphG = (V ,E , d) with d : E → R+
Goal: Find Hamiltonian cycle ofminimum length.
One of the most intensively studied problems in optimization– both in theory and practice.
Metric TSP: APX-hard Euclidean TSP: PTAS exists
Heiko Roglin Smoothed Analysis of Algorithms
![Page 48: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/48.jpg)
2-Opt Algorithm
Numerous ExperimentalStudies:(TSPLIB, DIMACSImplementation Challenge)
The PTAS is too slow onlarge instances.
The most successfulalgorithms (w. r. t.quality and runningtime) in practice rely onlocal search.
approximation ratio:≈ 1.05number of steps:≤ n · log n
2-Opt:
1 Start with an arbitrary tour.2 Remove two edges from the
tour.3 Complete the tour by two other
edges.4 Repeat steps 2 and 3 until no
local improvement is possibleanymore.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 49: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/49.jpg)
2-Opt Algorithm
Numerous ExperimentalStudies:(TSPLIB, DIMACSImplementation Challenge)
The PTAS is too slow onlarge instances.
The most successfulalgorithms (w. r. t.quality and runningtime) in practice rely onlocal search.
approximation ratio:≈ 1.05number of steps:≤ n · log n
2-Opt:
1 Start with an arbitrary tour.
2 Remove two edges from thetour.
3 Complete the tour by two otheredges.
4 Repeat steps 2 and 3 until nolocal improvement is possibleanymore.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 50: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/50.jpg)
2-Opt Algorithm
Numerous ExperimentalStudies:(TSPLIB, DIMACSImplementation Challenge)
The PTAS is too slow onlarge instances.
The most successfulalgorithms (w. r. t.quality and runningtime) in practice rely onlocal search.
approximation ratio:≈ 1.05number of steps:≤ n · log n
2-Opt:
1 Start with an arbitrary tour.2 Remove two edges from the
tour.
3 Complete the tour by two otheredges.
4 Repeat steps 2 and 3 until nolocal improvement is possibleanymore.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 51: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/51.jpg)
2-Opt Algorithm
Numerous ExperimentalStudies:(TSPLIB, DIMACSImplementation Challenge)
The PTAS is too slow onlarge instances.
The most successfulalgorithms (w. r. t.quality and runningtime) in practice rely onlocal search.
approximation ratio:≈ 1.05number of steps:≤ n · log n
2-Opt:
1 Start with an arbitrary tour.2 Remove two edges from the
tour.3 Complete the tour by two other
edges.
4 Repeat steps 2 and 3 until nolocal improvement is possibleanymore.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 52: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/52.jpg)
2-Opt Algorithm
Numerous ExperimentalStudies:(TSPLIB, DIMACSImplementation Challenge)
The PTAS is too slow onlarge instances.
The most successfulalgorithms (w. r. t.quality and runningtime) in practice rely onlocal search.
approximation ratio:≈ 1.05number of steps:≤ n · log n
2-Opt:
1 Start with an arbitrary tour.2 Remove two edges from the
tour.3 Complete the tour by two other
edges.4 Repeat steps 2 and 3 until no
local improvement is possibleanymore.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 53: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/53.jpg)
2-Opt Algorithm
Numerous ExperimentalStudies:(TSPLIB, DIMACSImplementation Challenge)
The PTAS is too slow onlarge instances.
The most successfulalgorithms (w. r. t.quality and runningtime) in practice rely onlocal search.
approximation ratio:≈ 1.05number of steps:≤ n · log n
2-Opt:
1 Start with an arbitrary tour.2 Remove two edges from the
tour.3 Complete the tour by two other
edges.4 Repeat steps 2 and 3 until no
local improvement is possibleanymore.
Heiko Roglin Smoothed Analysis of Algorithms
![Page 54: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/54.jpg)
Smoothed Analysis
Worst Case [Englert, R., Vocking (SODA 2007)]
Even for 2-dim. Euclidean instances, the worst-case run time is 2Ω(n).
Smoothed Analysis:Adversary chooses for each point i aprobability density fi : [0, 1]d → [0, φ]according to which it is chosen.
Adversary more powerful than before. Hedetermines also the type of noise. φ ∼ 1/σ
Smoothed Analysis [Englert, R., Vocking (SODA 2007)]
The smoothed number of 2-Opt steps is O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 55: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/55.jpg)
Smoothed Analysis
Worst Case [Englert, R., Vocking (SODA 2007)]
Even for 2-dim. Euclidean instances, the worst-case run time is 2Ω(n).
Smoothed Analysis:Adversary chooses for each point i aprobability density fi : [0, 1]d → [0, φ]according to which it is chosen.
Adversary more powerful than before. Hedetermines also the type of noise. φ ∼ 1/σ
Smoothed Analysis [Englert, R., Vocking (SODA 2007)]
The smoothed number of 2-Opt steps is O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 56: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/56.jpg)
Smoothed Analysis
Worst Case [Englert, R., Vocking (SODA 2007)]
Even for 2-dim. Euclidean instances, the worst-case run time is 2Ω(n).
1/√φ
1/√φ
Smoothed Analysis:Adversary chooses for each point i aprobability density fi : [0, 1]d → [0, φ]according to which it is chosen.
Adversary more powerful than before. Hedetermines also the type of noise. φ ∼ 1/σ
Smoothed Analysis [Englert, R., Vocking (SODA 2007)]
The smoothed number of 2-Opt steps is O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 57: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/57.jpg)
Smoothed Analysis
Worst Case [Englert, R., Vocking (SODA 2007)]
Even for 2-dim. Euclidean instances, the worst-case run time is 2Ω(n).
1/√φ
1/√φ
Smoothed Analysis:Adversary chooses for each point i aprobability density fi : [0, 1]d → [0, φ]according to which it is chosen.
Adversary more powerful than before. Hedetermines also the type of noise. φ ∼ 1/σ
Smoothed Analysis [Englert, R., Vocking (SODA 2007)]
The smoothed number of 2-Opt steps is O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 58: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/58.jpg)
Smoothed Analysis
Worst Case [Englert, R., Vocking (SODA 2007)]
Even for 2-dim. Euclidean instances, the worst-case run time is 2Ω(n).
1/√φ
1/√φ
Smoothed Analysis:Adversary chooses for each point i aprobability density fi : [0, 1]d → [0, φ]according to which it is chosen.
Adversary more powerful than before. Hedetermines also the type of noise. φ ∼ 1/σ
Smoothed Analysis [Englert, R., Vocking (SODA 2007)]
The smoothed number of 2-Opt steps is O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 59: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/59.jpg)
Simple Polynomial Bound
Theorem
The smoothed number of 2-Opt steps is O(n7φ3 log2 n).
Proof.
Consider a 2-Opt step (e1, e2)→ (e3, e4).
∆(e1, e2, e3, e4) = d(e1) + d(e2)− d(e3)− d(e4)
Every step decreases tour length by at least
∆ = mine1,e2,e3,e4∈E
∆(e1,e2,e3,e4)>0
∆(e1, e2, e3, e4).
Initial tour has length at most√
dn. Hence,
# 2-Opt Steps ≤√
dn∆
.
Union bound over O(n4) steps + calculations:Pr[∆ ≤ ε] = O(n4 · φ3 · ε · log(1/ε))
Heiko Roglin Smoothed Analysis of Algorithms
![Page 60: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/60.jpg)
Simple Polynomial Bound
Theorem
The smoothed number of 2-Opt steps is O(n7φ3 log2 n).
Proof.
Consider a 2-Opt step (e1, e2)→ (e3, e4).
∆(e1, e2, e3, e4) = d(e1) + d(e2)− d(e3)− d(e4)
Every step decreases tour length by at least
∆ = mine1,e2,e3,e4∈E
∆(e1,e2,e3,e4)>0
∆(e1, e2, e3, e4).
Initial tour has length at most√
dn. Hence,
# 2-Opt Steps ≤√
dn∆
.
Union bound over O(n4) steps + calculations:Pr[∆ ≤ ε] = O(n4 · φ3 · ε · log(1/ε))
Heiko Roglin Smoothed Analysis of Algorithms
![Page 61: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/61.jpg)
Simple Polynomial Bound
Theorem
The smoothed number of 2-Opt steps is O(n7φ3 log2 n).
Proof.
Consider a 2-Opt step (e1, e2)→ (e3, e4).
∆(e1, e2, e3, e4) = d(e1) + d(e2)− d(e3)− d(e4)
Every step decreases tour length by at least
∆ = mine1,e2,e3,e4∈E
∆(e1,e2,e3,e4)>0
∆(e1, e2, e3, e4).
Initial tour has length at most√
dn. Hence,
# 2-Opt Steps ≤√
dn∆
.
Union bound over O(n4) steps + calculations:Pr[∆ ≤ ε] = O(n4 · φ3 · ε · log(1/ε))
Heiko Roglin Smoothed Analysis of Algorithms
![Page 62: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/62.jpg)
Simple Polynomial Bound
Theorem
The smoothed number of 2-Opt steps is O(n7φ3 log2 n).
Proof.
Consider a 2-Opt step (e1, e2)→ (e3, e4).
∆(e1, e2, e3, e4) = d(e1) + d(e2)− d(e3)− d(e4)
Every step decreases tour length by at least
∆ = mine1,e2,e3,e4∈E
∆(e1,e2,e3,e4)>0
∆(e1, e2, e3, e4).
Initial tour has length at most√
dn. Hence,
# 2-Opt Steps ≤√
dn∆
.
Union bound over O(n4) steps + calculations:Pr[∆ ≤ ε] = O(n4 · φ3 · ε · log(1/ε))
Heiko Roglin Smoothed Analysis of Algorithms
![Page 63: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/63.jpg)
Simple Polynomial Bound
Theorem
The smoothed number of 2-Opt steps is O(n7φ3 log2 n).
Proof.
Consider a 2-Opt step (e1, e2)→ (e3, e4).
∆(e1, e2, e3, e4) = d(e1) + d(e2)− d(e3)− d(e4)
Every step decreases tour length by at least
∆ = mine1,e2,e3,e4∈E
∆(e1,e2,e3,e4)>0
∆(e1, e2, e3, e4).
Initial tour has length at most√
dn. Hence,
# 2-Opt Steps ≤√
dn∆
.
Union bound over O(n4) steps + calculations:Pr[∆ ≤ ε] = O(n4 · φ3 · ε · log(1/ε))
Heiko Roglin Smoothed Analysis of Algorithms
![Page 64: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/64.jpg)
Idea for Improvement
The bound is too pessimistic: Not every step yields the smallestpossible improvement ∆ ≈ 1/(n4 log n).
Consider two consecutive steps: They yield ∆ + ∆2 > 2∆.
Consider linked pair: (e1, e2)→ (e3, e4) and (e3, e5)→ (e6, e7).
Sequence of t consecutive steps, contains Ω(t) linked pairs:
S5S2 S3 S6 S7 S8S4S1 S9
(S1, S4) (S2, S5) (S6, S9)
∆Linked ≈ 1/(n3+1/3 log2/3 n).worst and second worst step are unlikely to form a linked pair
This idea yields O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 65: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/65.jpg)
Idea for Improvement
The bound is too pessimistic: Not every step yields the smallestpossible improvement ∆ ≈ 1/(n4 log n).
Consider two consecutive steps: They yield ∆ + ∆2 > 2∆.
Consider linked pair: (e1, e2)→ (e3, e4) and (e3, e5)→ (e6, e7).
Sequence of t consecutive steps, contains Ω(t) linked pairs:
S5S2 S3 S6 S7 S8S4S1 S9
(S1, S4) (S2, S5) (S6, S9)
∆Linked ≈ 1/(n3+1/3 log2/3 n).worst and second worst step are unlikely to form a linked pair
This idea yields O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 66: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/66.jpg)
Idea for Improvement
The bound is too pessimistic: Not every step yields the smallestpossible improvement ∆ ≈ 1/(n4 log n).
Consider two consecutive steps: They yield ∆ + ∆2 > 2∆.
Consider linked pair: (e1, e2)→ (e3, e4) and (e3, e5)→ (e6, e7).
Sequence of t consecutive steps, contains Ω(t) linked pairs:
S5S2 S3 S6 S7 S8S4S1 S9
(S1, S4) (S2, S5) (S6, S9)
∆Linked ≈ 1/(n3+1/3 log2/3 n).worst and second worst step are unlikely to form a linked pair
This idea yields O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 67: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/67.jpg)
Idea for Improvement
The bound is too pessimistic: Not every step yields the smallestpossible improvement ∆ ≈ 1/(n4 log n).
Consider two consecutive steps: They yield ∆ + ∆2 > 2∆.
Consider linked pair: (e1, e2)→ (e3, e4) and (e3, e5)→ (e6, e7).
Sequence of t consecutive steps, contains Ω(t) linked pairs:
S5S2 S3 S6 S7 S8S4S1 S9
(S1, S4) (S2, S5) (S6, S9)
∆Linked ≈ 1/(n3+1/3 log2/3 n).worst and second worst step are unlikely to form a linked pair
This idea yields O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 68: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/68.jpg)
Idea for Improvement
The bound is too pessimistic: Not every step yields the smallestpossible improvement ∆ ≈ 1/(n4 log n).
Consider two consecutive steps: They yield ∆ + ∆2 > 2∆.
Consider linked pair: (e1, e2)→ (e3, e4) and (e3, e5)→ (e6, e7).
Sequence of t consecutive steps, contains Ω(t) linked pairs:
S5S2 S3 S6 S7 S8S4S1 S9
(S1, S4) (S2, S5) (S6, S9)
∆Linked ≈ 1/(n3+1/3 log2/3 n).worst and second worst step are unlikely to form a linked pair
This idea yields O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 69: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/69.jpg)
Idea for Improvement
The bound is too pessimistic: Not every step yields the smallestpossible improvement ∆ ≈ 1/(n4 log n).
Consider two consecutive steps: They yield ∆ + ∆2 > 2∆.
Consider linked pair: (e1, e2)→ (e3, e4) and (e3, e5)→ (e6, e7).
Sequence of t consecutive steps, contains Ω(t) linked pairs:
S5S2 S3 S6 S7 S8S4S1 S9
(S1, S4) (S2, S5) (S6, S9)
∆Linked ≈ 1/(n3+1/3 log2/3 n).worst and second worst step are unlikely to form a linked pair
This idea yields O(n4.33 · φ2.67).
Heiko Roglin Smoothed Analysis of Algorithms
![Page 70: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/70.jpg)
Outline
Outline1 Linear Programming
Why is the simplex method usually efficient?smoothed analysis – analysis of algorithms beyond worst case
2 Traveling Salesperson ProblemWhy is local search successful?
3 Smoothed AnalysisOverview of known results
Heiko Roglin Smoothed Analysis of Algorithms
![Page 71: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/71.jpg)
Overview of Results on Smoothed Analsyis
Linear ProgrammingSimplex Method [Spielman, Teng (STOC 2001)]→ Godel Prize 2008, Fulkerson Prize 2009
Perceptron Algo [Blum, Dunagan (SODA 2002)]Interior Point Algo [Dunagan, Spielman, Teng(MathProg 2011)]
Combinatorial OptimizationComplexity of Binary Optimization Problems[Beier, Vocking (STOC 2004)]2-Opt Algo for TSP[Englert, R., Vocking (SODA 2007)]SSP Algo for Min-Cost Flow Problem[Brunsch, Cornelissen, Manthey, R. (SODA2013)]
Heiko Roglin Smoothed Analysis of Algorithms
![Page 72: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/72.jpg)
Overview of Results on Smoothed Analsyis
Linear ProgrammingSimplex Method [Spielman, Teng (STOC 2001)]→ Godel Prize 2008, Fulkerson Prize 2009Perceptron Algo [Blum, Dunagan (SODA 2002)]Interior Point Algo [Dunagan, Spielman, Teng(MathProg 2011)]
Combinatorial OptimizationComplexity of Binary Optimization Problems[Beier, Vocking (STOC 2004)]2-Opt Algo for TSP[Englert, R., Vocking (SODA 2007)]SSP Algo for Min-Cost Flow Problem[Brunsch, Cornelissen, Manthey, R. (SODA2013)]
Heiko Roglin Smoothed Analysis of Algorithms
![Page 73: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/73.jpg)
Overview of Results on Smoothed Analsyis
Linear ProgrammingSimplex Method [Spielman, Teng (STOC 2001)]→ Godel Prize 2008, Fulkerson Prize 2009Perceptron Algo [Blum, Dunagan (SODA 2002)]Interior Point Algo [Dunagan, Spielman, Teng(MathProg 2011)]
Combinatorial OptimizationComplexity of Binary Optimization Problems[Beier, Vocking (STOC 2004)]2-Opt Algo for TSP[Englert, R., Vocking (SODA 2007)]SSP Algo for Min-Cost Flow Problem[Brunsch, Cornelissen, Manthey, R. (SODA2013)]
Heiko Roglin Smoothed Analysis of Algorithms
![Page 74: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/74.jpg)
Overview of Results on Smoothed Analsyis
Machine Learningk -Means [Arthur, Manthey, R. (FOCS 2009)]PAC-Learning [Kalai, Samorodnitsky, Teng(FOCS 2009)]Belief Propagation [Brunsch, Cornelissen,Manthey, R. (WALCOM 2013)]→ (more in Kamiel’s talk at 14.00)
SchedulingMultilevel Feedback Algo [Becchetti, Leonardi,Marchetti-Spaccamela, Schafer, Vredeveld(FOCS 2003)]Local Search Algos [Brunsch, R., Rutten,Vredeveld (ESA 2011)]
Heiko Roglin Smoothed Analysis of Algorithms
![Page 75: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/75.jpg)
Overview of Results on Smoothed Analsyis
Machine Learningk -Means [Arthur, Manthey, R. (FOCS 2009)]PAC-Learning [Kalai, Samorodnitsky, Teng(FOCS 2009)]Belief Propagation [Brunsch, Cornelissen,Manthey, R. (WALCOM 2013)]→ (more in Kamiel’s talk at 14.00)
SchedulingMultilevel Feedback Algo [Becchetti, Leonardi,Marchetti-Spaccamela, Schafer, Vredeveld(FOCS 2003)]Local Search Algos [Brunsch, R., Rutten,Vredeveld (ESA 2011)]
Heiko Roglin Smoothed Analysis of Algorithms
![Page 76: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/76.jpg)
Overview of Results on Smoothed Analsyis
Multiobjective OptimizationNumber of Pareto optima[Brunsch, R. (STOC 2012)]Knapsack Problem [Beier, Vocking (STOC 2003)]
Classical Algorithms and Data StructuresQuicksort [Fouz, Kufleitner, Manthey, ZeiniJahromi (COCOON 2009)]Binary Search Trees[Manthey, Tantau (MFCS 2008)]Gaussian Elimination [Sankar, Spielman, Teng(SIAM. J. Matrix Anal. 2006)]
Many more results. . .
Heiko Roglin Smoothed Analysis of Algorithms
![Page 77: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/77.jpg)
Overview of Results on Smoothed Analsyis
Multiobjective OptimizationNumber of Pareto optima[Brunsch, R. (STOC 2012)]Knapsack Problem [Beier, Vocking (STOC 2003)]
Classical Algorithms and Data StructuresQuicksort [Fouz, Kufleitner, Manthey, ZeiniJahromi (COCOON 2009)]Binary Search Trees[Manthey, Tantau (MFCS 2008)]Gaussian Elimination [Sankar, Spielman, Teng(SIAM. J. Matrix Anal. 2006)]
Many more results. . .
Heiko Roglin Smoothed Analysis of Algorithms
![Page 78: Smoothed Analysis of Algorithms - lnmb · efficient algorithms (ellipsoid, interior point) Simplex method performs well in practice. Knapsack Problem (KP) NP-hard, FPTAS exists very](https://reader034.vdocuments.us/reader034/viewer/2022050110/5f4786080bf0b327040e72e8/html5/thumbnails/78.jpg)
Overview of Results on Smoothed Analsyis
Multiobjective OptimizationNumber of Pareto optima[Brunsch, R. (STOC 2012)]Knapsack Problem [Beier, Vocking (STOC 2003)]
Classical Algorithms and Data StructuresQuicksort [Fouz, Kufleitner, Manthey, ZeiniJahromi (COCOON 2009)]Binary Search Trees[Manthey, Tantau (MFCS 2008)]Gaussian Elimination [Sankar, Spielman, Teng(SIAM. J. Matrix Anal. 2006)]
Many more results. . .
Heiko Roglin Smoothed Analysis of Algorithms