chapter 9. interior point methods
DESCRIPTION
Chapter 9. Interior Point Methods. Three major variants Affine scaling algorithm - easy concept, good performance Potential reduction algorithm - poly time Path following algorithm - poly time, good performance, theoretically elegant . 9.4 The primal path following algorithm. - PowerPoint PPT PresentationTRANSCRIPT
Linear Programming 2013 1
Chapter 9. Interior Point Methods
Three major variantsAffine scaling algorithm - easy concept, good performancePotential reduction algorithm - poly timePath following algorithm - poly time, good performance,
theoretically elegant
Linear Programming 2013 2
9.4 The primal path following algorithm min max
Nonnegativity makes the problem difficult, hence use barrier function in the objective and consider unconstrained problem ( in the affine space
Barrier function: if for some
Solve min s.t. (9.15)
is strictly convex, hence has unique min point if min exists.
Linear Programming 2013 3
ex) min , s. t. ,
min at
𝑥
𝐵(𝑥)
0 1
− log 𝑥
Linear Programming 2013 4
min s.t.
Let is optimal solution given . , when varies, is called the central path ( hence the name path following )
It can be shown that optimal solution to LP. When , is called the analytic center.
For dual problem, the barrier problem ismax , s.t. (9.16)
( equivalent to min , minimizing convex function)
Linear Programming 2013 5
Figure 9.4: The central path and the analytic center
𝑥 (10)𝑥 (1)
𝑥 (0.1)𝑥 (0.01)
𝑥∗
analytic center
𝑐
Linear Programming 2013 6
Results from nonlinear programming(NLP) min
s.t. , ,
: , all twice continuously differentiable( gradient is given as a column vector)
Thm (Karush 1939, Kuhn-Tucker 1951, first order necessary opti-mality condition)If is a local minimum for (NLP) and some conditions (called con-straint qualification) hold at , then there exist such that
(1) (2) (3)
Linear Programming 2013 7
Remark:(2) is CS conditions and it implies that for non-active con-
straint .(1) says is a nonnegative linear combination of for active
constraints and (compare to strong duality theorem in p. 173 and its Figure )
CS conditions for LP are KKT conditionsKKT conditions are necessary conditions for optimality, but it
is also sufficient in some situations. One case is when objec-tive function is convex and constraints are linear, which in-cludes our barrier problem.
Linear Programming 2013 8
Deriving KKT for barrier problem: min , s.t.
( is row vector of , expressed as a column vector and , is the vector having 1 in all components.)Using (Lagrangian) multiplier for , we get (ignoring the sign of )Note that and . () If we define , KKT becomes
(),where .
Linear Programming 2013 9
For dual barrier problem,min s.t. ()
( is column vector of and is unit vector.)
Using (Lagrangian) multiplier for , we get
Now , hence we have the conditions,
which is the same conditions we obtained from the primal barrier function.
Linear Programming 2013 10
The conditions are given in the text as
(9.17)
where .
Note that when , they are primal, dual feasibility and comple-mentary slackness conditions.
Lemma 9.5: If and satisfy conditions (9.17), then they are opti-mal solutions to problems (9.15) and (9.16)
Linear Programming 2013 11
Pf) Let and satisfy (9.17), and let be an arbitrary vector that sat-isfies and . Then
attains min at .equality holds iff
Hence for all feasible . In particular, is the unique optimal solu-tion and .Similarly for and for dual barrier problem.
Linear Programming 2013 12
Primal path following algorithm Starting from some and primal and dual feasible find solution of
the barrier problem iteratively while .
To solve the barrier problem, we use quadratic approximation (2nd order Taylor expansion) of the barrier function and use the minimum of the approximate function as the next iterates.
Taylor expansion is
Also need to satisfy
Linear Programming 2013 13
Using KKT, solution to this problem is
The duality gap is Hence stop the algorithm if
Need a scheme to have an initial feasible solution (see text)
Linear Programming 2013 14
The primal path following algorithm
1. (Initialization) Start with some primal and dual feasible and set .2. (Optimality test) If , stop; else go to Step 3.3. Let 4. (Computation of directions) Solve the linear system
, ,
for and .5. (Update of solutions) Let
,,.
6. Let and go to Step 2.
Linear Programming 2013 15
9.5 The primal-dual path following algorithm Find Newton directions both in the primal and dual space.
Instead of finding min of quadratic approximation of barrier func-tion, it finds the solution for KKT system.
(9.26)
System of nonlinear equations because of the last ones. Let . Want such that
We use first order Taylor approximation around
Here is the Jacobian matrix whose -th element is given by
k
ji zzz)z(F
Linear Programming 2013 16
Try to find that satisfies.
We then set . The direction is called a Newton direction.Here F(z) is given by
This is equivalent to
(9.28) (9.29) (9.30)
Linear Programming 2013 17
Solution to the previous system is
where
Also limit the step length to ensure
Linear Programming 2013 18
The primal-dual path following algorithm1. (Initialization) Start with some feasible and set .2. (Optimality test) If , stop; else go to Step 3.3. (Computation of Newton directions) Let
,
Solve the linear system (9.28) – (9.30) for and 4. (Find step lengths) Let
Linear Programming 2013 19
(continued)5. (Solution update) Update the solution vectors according to
6. Let and go to Step 2.
Linear Programming 2013 20
Infeasible primal-dual path following methods
A variation of primal-dual path following.Starts from which is not necessarily feasible for either the pri-mal or the dual, i.e. and/or .Iteration same as the primal-dual path following except feasibility not maintained in each iteration.Excellent performance.
Linear Programming 2013 21
Self-dual method Alternative method to find initial feasible solution w/o using big-M.
Given an initial possibly infeasible point with and , consider the problem
minimize subject to
(9.33)
where .
This LP is self-dual.Note that is a feasible interior solution to (9.33)
Linear Programming 2013 22
Since both the primal and dual are feasible, they have optimal so-lutions and the optimal value is 0.
Primal-dual path following method finds an optimal solution that satisfies
( satisfies strict complementarity )
Can find optimal solution or determine unboundedness depending on the value of . (see Thm 9.8)
Running time : worst case : observed :