geometry of lp: feasible regions, feasible directions, basic feasible solutions ashish goel...
TRANSCRIPT
Geometry of LP: Feasible Regions, Feasible Directions, Basic Feasible Solutions
Ashish Goel
Department of Management Science and Engineering
Stanford University
Stanford, CA 94305, U.S.A.
http://www.stanford.edu/~ashishg
http://www.stanford.edu/class/msande211/
1Lecture #3; Based on slides by Yinyu Ye
LP Feasible Region in the Inequality Form
2
n
jmjmj
Tm
n
jjj
T
n
jjj
T
bxaxa
bxaxa
bxaxa
1
1222
1111
...
x simultaneously satisfy
This is the intersection of the m Half-spaces, and it isa convex (polyhedron) set
Lecture #3; Based on slides by Yinyu Ye
3Lecture #3; Based on slides by Yinyu Ye
4Lecture #3; Based on slides by Yinyu Ye
Corner or Extreme Points
5
Convex Hull:
Extreme Points: A point in a set that does not belong to the hull (convex combination) of the other pointsFor LP in inequality form, an extreme point is the intersection of n hyperplanes associated with the inequality constraints.
Lecture #3; Based on slides by Yinyu Ye
Feasible Direction
6
Direction: A direction is notated by a vector dIt is always associated with a “location” or point xTogether a point and a direction define a ray:
x + ϵd, for all ϵ > 0where d and kd are considered the same direction for all k > 0
Feasible Direction: A direction, d, is said to be “feasible” (relative to a given feasible point x) if x + ϵd is feasible for some ϵ> 0
For LP, all feasible directions at a feasible point form a convex (cone) set
Lecture #3; Based on slides by Yinyu Ye
Extreme Feasible Direction
7
A feasible direction d is extreme if d cannot be written as an convex combination of other feasible directions
Interior Point is a point x where every direction is feasible
Lecture #3; Based on slides by Yinyu Ye
LP Problem in Inequality Form
8
n
jmjmj
Tm
n
jjj
T
n
jjj
T
n
jjj
T
bxaxa
bxaxa
bxaxa
xcxc
1
1222
1111
1
...
s.t.
max
Lecture #3; Based on slides by Yinyu Ye
9
Recall the Production Problem
Objective contour
c
00
10
5.1
10
00 s.t.
2 max
21
21
21
21
21
21
xx
xx
xx
xx
xx
xx
Lecture #3; Based on slides by Yinyu Ye
Basic Theorems of Linear Programming
10
All LP problems fall into one of three cases:•Problem is infeasible: Feasible region is empty.
•Problem is unbounded: Feasible region is unbounded towards the optimizing direction.•Problem is feasible and bounded; and in this case:
– there exists an optimal solution or optimizer.– There may be a unique optimizer or multiple optimizers.– All optimizers form a convex set and they are on a face of
the feasible region.
– There is always at least one corner (extreme) optimizer if the feasible region has a corner point.
Moreover, Local optimality implies global optimality
Lecture #3; Based on slides by Yinyu Ye
Sketch Proof of Local Optimality Implies Global
11
(P) minimize f (x)subject to x ∈ Ω ⊂ Rn,
is a Convex Optimization problem if f (x) is a convex function over a convex feasible region Ω.
Proof by contradiction. Suppose x’ is a local minimizer but not a global minimizer x∗, that is, x’ ∈ Ω and x ∗ ∈ Ω but f (x∗) < f (x’). Now the convex combination point αx’ + (1 − α)x ∗must be feasible (why?), and
f (αx’+ (1 − α)x∗) ≤ αf (x’)+ (1 − α)f (x∗) < f (x’)
for any 0 ≤ α < 1. This contradicts the local optimality as α can be arbitrarily close to 1 so that αx’+ (1 − α)x ∗ can be arbitrarily close to x’.
Lecture #3; Based on slides by Yinyu Ye
12
00
10
5.1
10
00 s.t.
2 max
21
21
21
21
21
21
xx
xx
xx
xx
xx
xx
Optimality Certification of the Production Problem
a4
a5
a1
a2a3
a2
a3
a4
Objective contour
c
Lecture #3; Based on slides by Yinyu Ye
13
Feasible Directions at a Corner
a4
a5
a1
a2a3
a2
a3
a4
Objective contour
c
Lecture #3; Based on slides by Yinyu Ye
How to Certify a Corner being an Optimizer
14
• Every feasible direction at the corner is a worsening (descent in this case) direction, that is, cTd ≤0 in this case.
• Or, equivalently, c is a conic combination of the normal directions at the corner point, that is, there are multipliers α1 ≥0 and α2 ≥0, such as c= α1 ai1+ α2 ai2, in the 2-dimensional case.
• In the n-dimensional case: c= α1 ai1+…+ αn ain
where ai1,…, ain are the normal directions associated with the corner point.
Lecture #3; Based on slides by Yinyu Ye
LP in Standard (Equality) Form
15
0
...
s.t.
min
1
1222
1111
1
x
bxaxa
bxaxa
bxaxa
xcx c
n
jmjmjm
n
jjj
n
jjj
n
jjj
T
0
, s.t.
min
x
bAx
xcT
Lecture #3; Based on slides by Yinyu Ye
Reduction to Standard Form
16
•max cTx to min – cTx
•Eliminating ”free” variables: substitute with the
difference of two nonnegative variables
x := xl − xll, (xl, xll) ≥ 0.
•Eliminating inequalities: add a slack variableaT x ≤ b = ⇒ aT x + s = b, s ≥ 0
aT x ≥ b = ⇒ aT x − s = b, s ≥ 0
Lecture #3; Based on slides by Yinyu Ye
Reduction of the Production Problem
17
min -x1 -2x2
s.t. x1 +x3 = 1
x2 +x4 = 1
x1 +x2 +x5 = 1.5
(x1, x2, x3, x4, x5) ≥ 0
0
0
5.1
1
1 s.t.
2max
2
1
21
2
1
21
x
x
xx
x
x
xx
x3, x4,and x5 are called slack variables
Lecture #3; Based on slides by Yinyu Ye
Basic and Basic Feasible Solution
18
In the LP standard form, select m linearly independent columns, denoted by the variable index set B, from A. Solve
AB xB = b
for the dimension-m vector xB . By setting the variables, xN , of x corresponding to the remaining columns of A equal to zero, we obtain a solution x such that Ax = b.
Then, x is said to be a basic solution to (LP) with respect to the basic variable set B. The variables in xB are called basic variables, those in xN are nonbasic variables, and AB is called a basis.
If a basic solution xB ≥ 0, then x is called a basic feasible solution, or BFS. Note that AB and xB follow the same index order in B.Two BFS are adjacent if they differ by exactly one basic variable.
Lecture #3; Based on slides by Yinyu Ye
BS of the Production Problem in Equality Form
19
x1 +x3 = 1
x2 +x4 = 1
x1 +x2 +x5 = 1.5
(x1, x2, x3, x4, x5) ≥ 0
Basis 3,4,5 1,4,5 3,4,1 3,2,5 3,4,2 1,2,3 1,2,4 1,2,5
Feasible? √ √ √ √ √
x1 , x2 0, 0 1, 0 1.5, 0 0, 1 0, 1.5 .5, 1 1, .5 1, 1
x1
x2
Lecture #3; Based on slides by Yinyu Ye
BFS and Corner Point Equivalence Theorem
20
Theorem Consider the feasible region in the standard
LP form. Then, a basic feasible solution and a corner
(extreme) point are equivalent; the formal is algebraic
and the latter is geometric.
Algorithmically, we need only to look at BFSs for finding
an optimizer. But still too many to look…
Lecture #3; Based on slides by Yinyu Ye