isye 6661: topics coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. the simplex method...
TRANSCRIPT
![Page 1: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/1.jpg)
ISyE 6661: Topics Covered
1. Optimization fundamentals: 1.5 lectures
2. LP Geometry (Chpt.2): 5 lectures
3. The Simplex Method (Chpt.3): 4 lectures
4. LP Duality (Chpt.4): 4 lectures
5. Sensitivity Analysis (Chpt.5): 3 lectures
6. Large-scale LP (Chpt.6): 1.5 lectures
7. Computational complexity and the Ellipsoid method
(Chpt. 8): 2 lectures
8. Interior Point Algorithms (Chpt. 9): 5 lectures
1
![Page 2: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/2.jpg)
1. Fundamentals of Optimization
• The generic optimization problem:
(P ) : min{f(x) : x ∈ X}.
• Weirstrass’ Theorem: If f is continuous and X is
compact then problem (P ) has an optimal solu-
tion.
• If f is a convex function and X is a convex set,
then (P ) is a convex program.
• Theorem: If x∗ is a local optimal solution of the
convex program (P ) then it is also a global optimal
solution.
2
![Page 3: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/3.jpg)
2. Linear Programming Geometry
• LP in standard form
(P ) : min{cTx : Ax = b, x ≥ 0}.
• LP involves involves minimizing a linear functionover the polyhderal set X = {x : Ax = b, x ≥ 0}.
• Basic building blocks of a polyhedral set:Extreme points and Extreme rays.
• Theorem: (Algebraic characterization of Extremepts.) A vector x is an extreme point of X iff it isa Basic Feasible Solution, i.e., ∃ a partitioning ofA = [B|N ] (with B square and nonsingular) suchthat xB = B−1b and xN = 0.
• Theorem: (Algebraic characterization of Extremerays.) A vector d 6= 0 is an extreme ray of Xiff if it is a Non-negative Basic Direction, i.e., ∃a partitioning of A = [B|N ] (with B square andnonsingular) s.t.
d = α
[−B−1Aj
ej
]≥ 0
for some Aj ∈ N and α > 0.
3
![Page 4: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/4.jpg)
2. Linear Programming Geometry (contd.)
• The Representation Theorem: Let x1, . . . , xk andd1, . . . , dl be the extreme points and extreme raysof X respectively. Then
X =
x : x =k∑
i=1
λixi +
l∑j=1
µjdj
k∑i=1
λi = 1, λi ≥ 0 ∀i, µj ≥ 0 ∀j
.
• To prove the above result, we used:
The Separation Theorem: Let S be a non-emptyclosed convex set, and x∗ 6∈ S. Then ∃ a vector c
s.t. cTx∗ < cTx ∀ x ∈ S.
• Theorem: (Cor. of Rep. Thm.)
(a) An LP min{cTx : x ∈ X} has an optimal solutioniff cTdj ≥ 0 for all extreme rays dj j = 1, . . . , l.
(b) Extreme point optimality: If an LP has an opti-mal solution then there exists an extreme pointthat is optimal.
4
![Page 5: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/5.jpg)
3. The Simplex Method
• Basic idea: Move from one extreme point (bfs) toanother while improving the objective.
• Given a bfs xk with basis B move along one of thej-th Basic Directions (j ∈ N)
dj =
[−B−1Aj
ej
].
• If x is non-degenerate then dj is a feasible direc-tion, i.e., allows a positive step move. If cTdj < 0then dj is an improving direction. Note cTdj =cj − cT
BB−1Aj = c̄j (the reduced cost).
• If no improving direction exists, i.e. c̄j ≥ 0 for allj ∈ N , the current solution is optimal, Stop.
• Chose an improving basic direction dj from j ∈ N ,and move to xk+1 ← xk + αdj where α ≥ 0 is suchthat xk+1 ≥ 0.
• If dj ≥ 0 then α = +∞ implying that the problemis unbounded, Stop.
5
![Page 6: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/6.jpg)
3. The Simplex Method (contd.)
• Theorem: xk+1 is an adjacent bfs to xk with basis
B̂ = B +{Aj}−{Al} where l is some basic variable
that becomes nonbasic.
• Degeneracy, i.e., when a basic variable has a value
of zero, is a problem.
• If xk is a degenerate, α could be zero, i.e., the
basis changes from B to B̂ but xk+1 = xk and
cause Stalling or Cycling.
• Can be dealt with by properly choosing j and l
(e.g. Lexicographic rule).
• Theorem: The Simplex method (with proper pivot
rules) solves LP in a finite number of iterations.
6
![Page 7: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/7.jpg)
3. The Simplex Method (contd.)
• Revised Simplex and Tableau implementations.
• Initializing the Simplex method
– Two-phase Simplex
– Big-M method
7
![Page 8: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/8.jpg)
4. Duality
• Standard form Primal-dual LP pairs:
vP = min cTx vD = max bTy
s.t. Ax = b s.t. ATy ≤ c.x ≥ 0
• Recipe for writing dual problem for general LPs.
• Weak Duality Theorem: vD ≤ vP .
• Proof of WD: By construction of the dual prob-
lem.
• Strong Duality Theorem: If either problem has a
finite optimal value then vD = vP .
• Proof 1 of SD: From the Simplex Method. (cTBB−1
are the optimal dual variables).
• Proof 2 of SD: From the theorems of alternatives
(Farkaas’ Lemma).
8
![Page 9: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/9.jpg)
4. Duality (Contd.)
• Fakaas’ Lemma: Let A ∈ Rm×n and b ∈ Rm then
exactly one of the following two systems (a or b)
is feasible:
(a) Ax = b (b) ATy ≥ 0x ≥ 0 bTy < 0.
• Proof: Use Separating Hyperplane theorem.
• See different forms of Farkaas’ Lemma.
• From Duality to Polyhedral theory:
– An immediate proof of Farkaas’ Lemma.
– A simple proof of the Representation Thm.
– Converse to Rep. Thm.: Convex hull of a fi-
nite number of points is a polytope.
9
![Page 10: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/10.jpg)
4. Duality (Contd.)
• LP Optimality Conditions (Cor. of SD) A pair
(x∗, y∗) is primal-dual optimal iff
Ax∗ = b, x∗ ≥ 0 Primal FeasibilityATy∗ ≤ c Dual Feasibilityx∗j(cj −AT
j y∗) = 0 ∀ j Complementary Slackness.
• Relation between non-degeneracy and uniqueness
amongst primal and dual optimal solutions.
• The Dual Simplex Algorithm:
– A basis B is primal feasible (PF) if B−1b ≥ 0
and dual feasible (DF) if cT − cTBB−1A ≥ 0.
– Start with a basis that is DF but not PF.
– Select a variable (< 0) to leave the basis (move
towards PF).
– Select an entering variable to maintain DF.
10
![Page 11: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/11.jpg)
4. Duality (Contd.)
• Dual Simplex is not analogous to applying Primal
Simplex to the Dual problem.
• When to use Dual Simplex over Primal Simplex?
• Generalized Duality:
The dual of vP = min{cT : Ax ≥ b, x ∈ X} is
vD = max{L(y) : y ≥ 0} where
L(y) := min{cTx + yT (b−Ax) : x ∈ X}.
11
![Page 12: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/12.jpg)
5. Sensitivity Analysis
Consider the LP
z = min{cTx : Ax = b, x ≥ 0}.
An instance of the LP is given by the data (n, m, c, A, b).
• If the optimal solution x∗ is non-degenerate then
the i-th dual variable represents
y∗i =∂z
∂bi
∣∣∣∣∣x∗
i = 1, . . . , m.
• Local Sensitivity Analysis:
(a) How doe the optimal solution x∗ and the opti-
mal value z behave under small perturbations
of the problem data (n, m, c, A, b)?
(b) How to efficiently recover the new optimal solu-
tion and optimal value after the perturbation?
12
![Page 13: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/13.jpg)
5. Sensitivity Analysis (contd.)
• Adding a new variable: Current basis remains PF.
So check DF (reduced cost of the new variable)
and use Primal Simplex to optimize if needed.
• Adding a new constraint: Current basis remain DF.
Check PF, and use Dual Simplex to optimize if
needed.
• Perturbing b← b + δd: Current basis remain DF
and PF over a computable range of δ. Outside
this range, we have DF but not PF, so use Dual
Simplex to optimize.
• Perturbing c← c + δd: Current basis remain DF
and PF over a computable range of δ. Outside
this range, we have PF but not DF, so use Primal
Simplex to optimize.
13
![Page 14: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/14.jpg)
5. Sensitivity Analysis (contd.)
• Perturbing Aj ← Aj + δd where j ∈ N : Current ba-
sis remain DF and PF over a computable range of
δ. Outside this range, we have PF but not DF, so
use Primal Simplex to optimize.
• Perturbing Aj ← Aj + δd where j ∈ B: Current ba-
sis remain DF and PF over a computable range of
δ. Outside this range, both PF and DF maybe
affected.
• Global behavior of value functions:
(a) F (b) = min{cTx : Ax = b, x ≥ 0} is a con-
vex function of b, and the dual solution y is a
subgradient of F (b) at b.
(b) G(c) = min{cTx : Ax = b, x ≥ 0} is a concave
function of c, and −x is a subgradient of −G(c)
at c.
14
![Page 15: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/15.jpg)
6. Large-Scale LP
• Column Generation:
– The Cutting Stock Problem
– Dantzig-Wolfe decomposition.
• Row Generation:
– Benders decomposition.
15
![Page 16: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/16.jpg)
7. Computational Complexity of LP
• A problem (class) is “easy” if there exists an al-
gorithm whose computational “effort” required to
solve any instance of the problem is bounded by
some polynomial of the “size” of that instance
(i.e. if there exists a polynomial time algorithm
for the problem).
• Is LP “easy”?
• The Simplex method may require an exponential
number (in the number of variables) of iterations!
– Klee-Minty (1972).
• Yudin and Nemirovskii (1977) developed Ellipsoid
method and showed that general convex programs
are “easy” and Khachian (1979) used it show that
LP is indeed “easy.”
16
![Page 17: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/17.jpg)
7. The Ellipsoid Method for LP
• The Ellipsoid method answers the following ques-tion
Is X = {x ∈ Rn|Ax ≥ b} = ∅?
• Assume: if X 6= ∅ then 0 < v ≤ vol(X) ≤ V .
• We have a Separation Oracle S(x, X) which re-turns 0 if x ∈ X, otherwise it returns a vectora 6= 0 such that aTy > aTx for all y ∈ X.
0. Find an ellipsoid E0(x0) ⊇ X. Set k = 0.
1. If S(xk, X) = 0 stop X 6= ∅. If vol(Ek(xk)) ≤ v
Stop X = ∅.
2. If S(xk, X) = ak, then X ⊂ Hk := {x : aTk x ≥
aTk xk}. Find
Ek+1(xk+1) ⊇ Ek(x
k) ∩Hk ⊃ X
such that
vol(Ek+1(xk+1))
vol(Ek(xk))< e−1/2(n+1).
3. Set k ← k + 1 and go to step 1.
17
![Page 18: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/18.jpg)
7. The Ellipsoid Method for LP (contd.)
• The numbers v and V depend on n and U (the
largest number in the data (A, b)).
• Theorem: The Ellipsoid method answers the ques-
tion “Is X = {x ∈ Rn|Ax ≥ b} = ∅?” in O(n6 log(nU))
iterations.
18
![Page 19: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/19.jpg)
7. The Ellipsoid Method for LP (contd.)
• Easily modified for optimization of a linear func-
tion over polyhedra. Polynomial complexity is pre-
served.
• Note the complexity does not depend on the num-
ber of constraints in X.
• Equivalence of Separation and Optimization: The
description of X maybe involve an exponential
number of constraints. However as long as we
have a polynomial time Separation Oracle then
the Ellipsoid algorithm guarantees that optimiza-
tion of a linear function over X is still polynomial
time!
19
![Page 20: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/20.jpg)
8. Interior Point Methods
min{cTx : x ∈ X}
• Basic idea:
– Given xk ∈ int(X),
– find a direction dk and a step size αk s.t. xk +
αkdk =: xk+1 ∈ int(X) and cTxk+1 < cTxk.
– Continue until some termination criteria is met.
• The algorithms differ w.r.t choice of dk, αk and
the termination criteria.
• May need some preprocessing to guarantee that
an optimal solution exists.
• The algorithms are convergent
limk→∞
xk = x∗.
A good criteria for finite termination is needed.
20
![Page 21: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/21.jpg)
8. Interior Point Methods:
The Affine Scaling Method
• Basic idea:
– Given xk ∈ int(X), construct an Ellipsoid Ek(xk) ⊂
int(X).
– Choose xk+1 = argmin{cTx : x ∈ Ek}.
• Based on the fact that the minimizer of a linear
form over an Ellipsoid can be found analytically.
• Not proven to be polynomial time.
21
![Page 22: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/22.jpg)
8. Interior Point Methods:
The Primal path following (Barrier) method
• We want to solve
P : min{cTx : Ax = b, x ≥ 0}.
• Use a penalty function to prevent iterates from
approaching the boundary of the polyhedron.
• Reduce penalty as the iterates approach an opti-
mal solution (on the boundary).
• Given µ > 0, the barrier problem is
P (µ) : min{fµ(x) := cTx− µn∑
j=1
log(xj) : Ax = b}.
22
![Page 23: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/23.jpg)
8. Interior Point Methods:
The Barrier method
• For any µ > 0 the function fµ(x) is strictly convex
⇒ the problem P (µ) has a unique optimal solution
x(µ).
• For any µ > 0, x(µ) ∈ int(X), where X = {x : Ax =
b, x ≥ 0}.
• For µ = +∞, x(µ) is the analytic center of X.
• As µ→ 0, x(µ)→ x∗.
• The set of solutions {x(µ) : µ ∈ (0,∞)} is known
as the Central Path.
• How to find x(µ) (at least approximately)?
23
![Page 24: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/24.jpg)
Aside: NLP Optimality Conditions
NLP : min{f(x) : Ax = b, x ≥ 0}
LP (x∗) : min{∇f(x∗)Tx : Ax = b, x ≥ 0}
• Theorem: If x∗ is an optimal solution of NLP thenx∗ is an optimal solution of LP (x∗).
• Theorem: If f is convex, then x∗ is an optimalsolution of NLP iff x∗ is an optimal solution ofLP (x∗).
• Theorem: If x∗ is an optimal solution of NLP thenx∗ solves the KKT system
Ax = b, x ≥ 0ATy + s = ∇f(x∗), s ≥ 0xjsj = 0 j = 1, . . . , n.
• Theorem: If f is convex, then x∗ is an optimalsolution of NLP iff x∗ solves the KKT system
Ax = b, x ≥ 0ATy + s = ∇f(x∗), s ≥ 0xjsj = 0 j = 1, . . . , n.
24
![Page 25: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/25.jpg)
8. Interior Point Methods:
The Barrier method (contd.)
• x(µ) is a solution of the KKT system for the Bar-
rier problem
Ax = b, x > 0ATy + s = c, s > 0xjsj = µ j = 1, . . . , n.
• The system is nonlinear – difficult to solve.
• We are content with β-approximate solutions (0 <
β < 1)
Ax = b, x > 0ATy + s = c, s > 0∑n
j=1(xjsjµ − 1)2 ≤ β2
• For fixed β, limµ→0 xβ(µ) = limµ→0 x(µ) = x∗.
25
![Page 26: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/26.jpg)
8. Interior Point Methods:
The Barrier method (contd.)
• Let β = 1/2. Start with some µk > 0 and a β-
approximation xk of x(µk).
• Linearize the KKT system around xk and solve it
to get the new solution xk+1.
• It can be shown that xk+1 is a β-approximation of
x(µk+1) with µk+1 = (1− 12+4
√n)µk.
• Continue until the duality gap (xk)T sk ≤ ε.
26
![Page 27: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/27.jpg)
8. Interior Point Methods:
The Barrier Method (contd.)
• Theorem: The barrier algorithm reduces the dual-
ity gap from ε0 to ε in O(√
n log ε0ε ) iterations.
27
![Page 28: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/28.jpg)
Not covered: Network Flow Problems
• A very important class of problems.
• Constraint matrix has a very special structure,
called a Network matrix.
• Specialized Simplex type algorithm is strongly poly-
nomial time.
• E.g. Transportation and Assignment Problems.
28
![Page 29: ISyE 6661: Topics Coveredsahmed/isye6661/6661review.pdf · 2003. 12. 4. · 3. The Simplex Method (contd.) •Theorem: xk+1 is an adjacent bfs to xk with basis Bˆ = B+{Aj}−{Al}where](https://reader036.vdocuments.us/reader036/viewer/2022071422/611c48449439de13600e1076/html5/thumbnails/29.jpg)
What’s next?Optimization Courses in SP’04
• ISyE 6662: Optimization II. Ph.D. level class onInteger Programming and Network Flows. Offeredby Prof. Ergun.
• ISyE 8871: Integer Programming. Advanced Ph.D.level class on Integer Programming. Offered byProf. Nemhauser.
• ISyE 6663: Optimization III. Nonlinear Program-ming programming theory for Ph.D. students. Of-fered by Prof. Nemirovskii.
• ISyE 8813: Advanced Ph.D. class on Interior PointMethods. Offered by Prof. Nemirovskii.
• ISyE 6669: Deterministic optimization (MS level).
• ISyE 6673: Financial optimization models (MSlevel). Offered by Prof. Sokol .
• ISyE 6679: Computational Methods in Optimiza-tion. Offered by Prof. Barnes.
29