![Page 1: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/1.jpg)
Linear ProgrammingThe Simplex Algorithm
Chapter 5
University of ChicagoBooth School of Business
Kipp Martin
October 17, 2017
1
![Page 2: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/2.jpg)
Outline
List of Files
Key Concepts
Geometry and Theory
The Simplex Algorithm
2
![Page 3: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/3.jpg)
List of Files
I simplexpivot.m (does one pivot)
I basicsimplex.m (executes simplex algorithm)
I lptest1.m
I lptest2.m
File Location:
http://faculty.chicagobooth.edu/kipp.martin/root/
htmls/coursework/36900/matlab.html
Reading: See Chapter 5 of the text.
3
![Page 4: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/4.jpg)
Key Concepts
I standard form
I basis and basic solution
I basic feasible solution
I extreme point
I fundamental theorem of linear programming
I the simplex algorithm
4
![Page 5: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/5.jpg)
DISCLAIMER!
What follows in these notes is a textbook description of thesimplex algorithm!
These notes are designed to explain the simplex algorithm from aconceptual point of view.
We ignore how to actually implement the algorithm. This isdiscussed in part 2 of the notes.
5
![Page 6: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/6.jpg)
Motivation
I Projection and inverse projection are fantastic when it comesto understanding linear programming theory.
I As presented, the projection and inverse projection algorithmsare not practical for solving linear programming problems.
I Not only do projection and inverse projection have worst caseexponential running time, but this is the “expected case” –these algorithms must run to termination before you have theanswer. They do not terminate early.
I We now present an algorithm with worst case exponentialbehavior. But in practice it works very well. It terminateswhen optimality conditions are satisfied.
6
![Page 7: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/7.jpg)
Motivation
I We will return to projection and inverse projection basedalgorithms when we study decomposition approaches.
I We want to solve min{c>x |Ax ≥ b} where A has millions ofnonzero elements.
I Two approaches – launch a direct frontal attack or tryflanking decomposition maneuvers.
I We study the frontal attack first.
Kipp’s lemma – if you can read a linear program into main memoryyou can solve it reasonably fast.
7
![Page 8: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/8.jpg)
Geometry and Theory
Standard Form: for mathematical convenience we are going toassume every linear program we work with is in standard form. Thestandard form is:
min c>x
s.t. Ax = b
x ≥ 0
Any linear program can be put into standard form.
8
![Page 9: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/9.jpg)
Geometry and TheoryThe linear program
max 10x1 + 9x2
.7x1 + x2 ≤ 630
.5x1 + (5/6)x2 ≤ 600
x1 + (2/3)x2 ≤ 708
.1x1 + .25x2 ≤ 135
x1, x2 ≥ 0
in standard form is
min −10x1 −9x2.7x1 +x2 +x3 = 630.5x1 +(5/6)x2 +x4 = 600x1 +(2/3)x2 +x5 = 708
.1x1 +.25x2 +x6 = 135x1, x2, x3, x4, x5, x6 ≥ 0
9
![Page 10: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/10.jpg)
Geometry and Theory
After writing the linear program in standard form assume that A isan m × n matrix – i.e. we have m constraints (rows) and nvariables (columns).
Let B index a set of m variables. If the columns of A indexed by Bare linearly independent then the columns of A indexed by B are abasis and the variables xi , i ∈ B are called basic variables.
10
![Page 11: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/11.jpg)
Geometry and Theory
By xB we mean an m component vector of variables withcomponents xBi
for i = 1, . . . ,m. We often say variable xBiis basic
in row i .
Let AB be the m ×m submatrix of A consisting of the columns ofA indexed by B. The variables not indexed B are called nonbasicvariables. The nonbasic variables are indexed by N. The matrixAN is the m× (n−m) submatrix of A consisting of the columns ofA indexed by N. Similarly, xN is an (n −m) component vector ofnonbasic variables.
11
![Page 12: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/12.jpg)
Geometry and Theory
Let’s look at an example using MATLAB. The constraint matrix ofthe sample linear program is
>> A = [.7 1 1 0 0 0;
.5 5/6 0 1 0 0;
1 2/3 0 0 1 0;
.1 .25 0 0 0 1]
Define a basis by B by
>> B = [2 4 1 6]
The basis matrix corresponding to B expressed in MATLAB is
>> AB = A(:, B)
What is an easy way in MATLAB to test if B is really a basis?
12
![Page 13: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/13.jpg)
Geometry and Theory
Since
>> B = [2 4 1 6]
We would say
I Variable x2 is basic in row 1
I Variable x4 is basic in row 2
I Variable x1 is basic in row 3
I Variable x6 is basic in row 4
13
![Page 14: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/14.jpg)
Geometry and Theory
>> inv(AB)*A
ans =
0 1.0000 1.8750 0 -1.3125 0
0.0000 0.0000 -0.9375 1.0000 0.1563 0
1.0000 0 -1.2500 0 1.8750 0
0 0 -0.3437 0 0.1406 1.0000
Stated another way
I Variable x1 is basic in row 3
I Variable x2 is basic in row 1
I Variable x4 is basic in row 2
I Variable x6 is basic in row 4
14
![Page 15: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/15.jpg)
Geometry and Theory
If we set the non-basic variables to zero, what is the implied valueof the basic variables for a given b.
>> inv(AB)*b
=
252.0000
120.0000
540.0000
18.0000
I Variable x1 is basic in row 3 – value = ?
I Variable x2 is basic in row 1 – value = ?
I Variable x4 is basic in row 2 – value = ?
I Variable x6 is basic in row 4 – value = ?
15
![Page 16: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/16.jpg)
Geometry and TheoryHere is the math. The constraint matrix is
Ax = b
Partitioned into basic and non-basic variables
ABxB + ANxN = b
Multiply by the basis inverse
A−1B ABxB + A−1B ANxN = A−1B b
xB + A−1B ANxN = A−1B b
Since xN = 0,
xB = A−1B b
16
![Page 17: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/17.jpg)
Geometry and Theory
We have
xB = A−1B b
But we also require xB ≥ 0.
If, indeed, xB ≥ 0, then xB is a basic feasible solution.
Now, for some geometry.
17
![Page 18: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/18.jpg)
Geometry and TheoryWe are working with the constraints:
.7x1 +x2 +x3 = 630
.5x1 +(5/6)x2 +x4 = 600x1 +(2/3)x2 +x5 = 708
.1x1 +.25x2 +x6 = 135x1, x2, x3, x4, x5, x6 ≥ 0
Which is equivalent to the following (and plotted on the followingpage).
.7x1 + x2 ≤ 630
.5x1 + (5/6)x2 ≤ 600
x1 + (2/3)x2 ≤ 708
.1x1 + .25x2 ≤ 135
x1, x2 ≥ 0
18
![Page 19: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/19.jpg)
Geometry and Theory
19
![Page 20: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/20.jpg)
Geometry and Theory
If you were to enumerate every basic feasible solution, they wouldcorrespond to (moving clockwise from the origin):
I (0, 0)
I (0, 540)
I (300, 420)
I (540, 252)
I (708, 0)
20
![Page 21: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/21.jpg)
Geometry and Theory
These points are called extreme points of the feasible region. Apoint, x , is an extreme point of the feasible region if and only if itcannot be written as
x = λ1x1 + λ2x
2, λ1 + λ2 = 1, λ1, λ2 ≥ 0
where x1 and x2 are distinct points in the feasible region.
In math jargon we say x is an extreme point if and only if it is nota convex combination of two distinct feasible points.
Another way to state this is that x is an extreme point if and onlyif it is not on a line segment connecting two distinct points in thefeasible region.
21
![Page 22: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/22.jpg)
Geometry and Theory
There are three famous theorem critical to the Simplex algorithm.
Theorem 1: A point x is an extreme point of the LP feasibleregion if and only if it is a basic feasible solution.
Theorem 2 (finite basis theorem): Any point in a bounded LPfeasible region can be written as a convex combination of extremepoints.
Theorem 3 (fundamental theorem of linear programming): Ifthere is an optimal solution to a linear program there is an optimalextreme point solution to the linear program.
Theorem 1 + Theorem 3 imply we can solve a linear by restrictingour search to basic feasible solutions. This is exactly what theSimplex algorithm does.
22
![Page 23: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/23.jpg)
The Simplex Algorithm
The linear program is standard form is:
min c>x
s.t. Ax = b
x ≥ 0
Let z0 = c>x and write this as:
z0 − c>x = 0
Ax = b
x ≥ 0
23
![Page 24: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/24.jpg)
The Simplex AlgorithmLet B index the basic variables and N the nonbasic variables. Then
c> = [c>B , c>N ]
A = [AB ,AN ]
and we rewrite
z0 − c>x = 0
Ax = b
x ≥ 0
as
z0 − c>B xB − c>N xN = 0
ABxB + ANxN = b
xB , xN ≥ 0
24
![Page 25: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/25.jpg)
The Simplex AlgorithmIn the system below multiply the second set of constraints by A−1B
z0 − c>B xB − c>N xN = 0
ABxB + ANxN = b
xB , xN ≥ 0
this multiplication gives:
z0 − c>B xB − c>N xN = 0
A−1B ABxB + A−1B ANxN = A−1B b
xB , xN ≥ 0
which is
z0 − c>B xB − c>N xN = 0
xB + A−1B ANxN = A−1B b
xB , xN ≥ 0
25
![Page 26: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/26.jpg)
The Simplex AlgorithmOkay, we have:
z0 − c>B xB − c>N xN = 0
xB + A−1B ANxN = A−1B b
xB , xN ≥ 0
Now multiply the second set of constraints by c>B to get:
c>B xB + c>B A−1B ANxN = c>B A−1B b
and add to
z0 − c>B xB − c>N xN = 0
The terms c>B xB and −c>B xB cancel when we add.26
![Page 27: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/27.jpg)
The Simplex Algorithm
This gives:
z0 + (c>B A−1B AN − c>N )xN = c>B A−1B b
xB + A−1B ANxN = A−1B b
xB , xN ≥ 0
We can rewrite as
z0 = c>B A−1B b + (c>N − c>B A−1B AN)xN
xB = A−1B b − A−1B ANxN
xB , xN ≥ 0
27
![Page 28: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/28.jpg)
The Simplex Algorithm
Reduced Cost: The reduced costs of the nonbasic variables aredefined by
w>N := c>N − c>B A−1B AN
and the system on the previous slide is
z0 = c>B A−1B b + w>NxN
xB = A−1B b − A−1B ANxN
xB , xN ≥ 0
28
![Page 29: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/29.jpg)
The Simplex Algorithm
Assume the basis B indexes a feasible solution. Then xB ≥ 0 andxN = 0.
z0 = c>B A−1B b + w>NxN
xB = A−1B b − A−1B ANxN
xB , xN ≥ 0
1. If a component has xN has a negative reduced cost, increasingthe value of that variable will lower the value of z0 and wehave an improved feasible solution assuming xB remainsnonnegative.
2. If a component of xN has a negative reduced cost we want toincrease it as much as possible – until a basic variable hitszero.
29
![Page 30: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/30.jpg)
The Simplex Algorithm
Example: Consider the linear program
min 2x1 −3x2x2 ≥ 1
x1 +x2 ≥ 2−.5x1 +x2 ≤ 8−x1 +x2 ≤ 6x1, x2 ≥ 0
In standard form this linear program is
min 2x1 −3x2x2 −x3 = 1
x1 +x2 −x4 = 2−.5x1 +x2 +x5 = 8−x1 +x2 +x6 = 6x1, x2, x3, x4, x5, x6 ≥ 0
30
![Page 31: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/31.jpg)
The Simplex Algorithm
Index a set of basic variables with B = {1, 2, 5, 6}, the objectivefunction row and system the
z0 − w>NxN = c>B A−1B b
xB + A−1B ANxN = A−1B b
xB , xN ≥ 0
is
z0 +5x3 −2x4 = −1x1 +x3 −x4 = 1
x2 −x3 = 1x5 +1.5x3 −.5x4 = 7.5
x6 +2x3 −x4 = 6
Why is this feasible?
31
![Page 32: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/32.jpg)
The Simplex Algorithm
Rewriting gives:
z0 = −1 −5x3 +2x4x1 = 1 −x3 +x4
x2 = 1 +x3x5 = 7.5 −1.5x3 +.5x4
x6 = 6 −2x3 +x4
The current solution implied by the tableau above is:
x1 = 1, x2 = 1, x5 = 7.5, x6 = 6 x3 = 0, x4 = 0
with solution value z0 = −1.
32
![Page 33: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/33.jpg)
The Simplex Algorithm
Variable x3 has a negative reduced cost of −5.
z0 = −1 −5x3 +2x4x1 = 1 −x3 +x4
x2 = 1 +x3x5 = 7.5 −1.5x3 +.5x4
x6 = 6 −2x3 +x4
If x3 is increased from 0 to 1 this one unit increase decreases theobjective function value by 5, the amount of the reduced cost, andthe corresponding solution is
x1 = 0, x2 = 2, x5 = 6, x6 = 4 x3 = 1, x4 = 0
which is feasible!
33
![Page 34: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/34.jpg)
The Simplex Algorithm
Increasing x3 is clearly a good thing. How much can x3 beincreased?
z0 = −1 −5x3 +2x4x1 = 1 −x3 +x4
x2 = 1 +x3x5 = 7.5 −1.5x3 +.5x4
x6 = 6 −2x3 +x4
The negative coefficients in the x3 column limit the increase of x3.In this case the increase in x3 is limited by the minimum ratiobelow (we take absolute values of the negative numbers and donot consider the positive numbers):
1/1 = 1.0, 7.5/1.5 = 5.0, 6/2 = 3.0
The limit on the increase in x3 is the minimum ratio of 1.0.
34
![Page 35: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/35.jpg)
The Simplex Algorithm
Bringing x3 in at a value of 1.0 drives basic variable x1 to zero andout of the basis.
We are making x3 basic and x1 nonbasic. In simplex terminologywe pivot x3 into the basis and x1 out. By convention thetableau is written this way
z0 +5x3 −2x4 = −1x1 +x3 −x4 = 1
x2 −x3 = 1x5 +1.5x3 −.5x4 = 7.5
x6 +2x3 −x4 = 6
35
![Page 36: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/36.jpg)
The Simplex Algorithm
In the tableau below, pivot in x3 and pivot out x1.
z0 +5x3 −2x4 = −1x1 +x3 −x4 = 1
x2 −x3 = 1x5 +1.5x3 −.5x4 = 7.5
x6 +2x3 −x4 = 6
This gives the new tableau with basis B = {3, 2, 5, 6}.
z0 −5x1 +3x4 = −6x3 +x1 −x4 = 1
x2 +x1 −x4 = 2x5 −1.5x1 +x4 = 6
x6 −2x1 +x4 = 4
The objective function value has moved from −1 to −6.
36
![Page 37: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/37.jpg)
The Simplex Algorithm
The updated tableau is:
z0 −5x1 +3x4 = −6x3 +x1 −x4 = 1
x2 +x1 −x4 = 2x5 −1.5x1 +x4 = 6
x6 −2x1 +x4 = 4
Variable x4 has a reduced cost of −3 (why not 3?). This variablecan come in at a maximum of
6/1 = 6, 4/1 = 4
The minimum ratio is 4, and this corresponds to the row where x6is basic. Pivot in variable x4 and pivot out variable x6.
37
![Page 38: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/38.jpg)
The Simplex Algorithm
In the tableau below, pivot in x4 and pivot out x6.
z0 −5x1 +3x4 = −6x3 +x1 −x4 = 1
x2 +x1 −x4 = 2x5 −1.5x1 +x4 = 6
x6 −2x1 +x4 = 4
This gives the new tableau with basis B = {3, 2, 5, 4}.
z0 x1 −3x6 = −18x3 −x1 +x6 = 5
x2 −x1 +x6 = 6x5 +.5x1 −x6 = 2
x4 −2x1 +x6 = 4
The objective function value has moved from −6 to −18.
38
![Page 39: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/39.jpg)
The Simplex Algorithm
The updated tableau is:
z0 x1 −3x6 = −18x3 −x1 +x6 = 5
x2 −x1 +x6 = 6x5 +.5x1 −x6 = 2
x4 −2x1 +x6 = 4
Variable x1 has a reduced cost of −1 (why not 1?). This variablecan come in at a maximum of
2/.5 = 4
The minimum ratio is 4, and this corresponds to the row where x5is basic. Pivot in variable x1 and pivot out variable x5. Note:variable x1 was pivoted out in a previous iteration and is nowcoming back into the basis.
39
![Page 40: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/40.jpg)
The Simplex Algorithm
In the tableau below, pivot in x1 and pivot out x5.
z0 x1 −3x6 = −18x3 −x1 +x6 = 5
x2 −x1 +x6 = 6x5 +.5x1 −x6 = 2
x4 −2x1 +x6 = 4
This gives the new tableau with basis B = {3, 2, 1, 4}.
z0 −2x5 −x6 = −22x3 +2x5 −x6 = 9
x2 +2x5 −x6 = 10x1 +2x5 −2x6 = 4
x4 +4x5 −3x6 = 12
The objective function value has moved from −18 to −22.
40
![Page 41: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/41.jpg)
The Simplex Algorithm
The updated tableau is:
z0 −2x5 −x6 = −22x3 +2x5 −x6 = 9
x2 +2x5 −x6 = 10x1 +2x5 −2x6 = 4
x4 +4x5 −3x6 = 12
I claim we are optimal!
41
![Page 42: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/42.jpg)
The Simplex Algorithm
Oh No! No! No! No! I am worried again.
I am willing to buy into the fact that at this local point there is nofeasible improving direction. But what if I were somewhere else inthe feasible region?
Optimality Conditions to the rescue!!!!
42
![Page 43: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/43.jpg)
The Simplex Algorithm
The linear program of interest (i.e the primal) is:
min c>x
Ax = b
x ≥ 0
The dual is:
min b>u
u>A ≤ c>
Let’s derive the optimality conditions.
43
![Page 44: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/44.jpg)
The Simplex Algorithm
Optimality Conditions: (x , u) are an optimal primal-dual pair ifand only if
Ax = b,
x ≥ 0
u>A ≤ c>,
(u>A− c>)x = 0
So where are we?
44
![Page 45: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/45.jpg)
The Simplex AlgorithmLet u> = c>B A−1B . The dual constraints are u>A ≤ c> so dualfeasibility is:
c>B A−1B A ≤ c>
written another way
c>B A−1B [AB ,AN ] ≤ [c>B , c>N ]
[c>B I , c>B A−1B AN ] ≤ [c>B , c>N ]
[0, c>B A−1B AN − c>N ] ≤ 0
what is
w>N = c>N − c>B A−1B AN?
45
![Page 46: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/46.jpg)
The Simplex Algorithm
The reduced costs are:
w>N = c>N − c>B A−1B AN?
The simplex stopping criterion is positive reduced costs whichcorresponds to dual feasibility.
46
![Page 47: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/47.jpg)
The Simplex Algorithm
But the tableau is:
z0 + (c>B A−1B AN − c>N )xN = c>B A−1B b
xB + A−1B ANxN = A−1B b
xB , xN ≥ 0
We can rewrite as
z0 = c>B A−1B b + (c>N − c>B A−1B AN)xN
xB = A−1B b − A−1B ANxN
xB , xN ≥ 0
so where are we?
47
![Page 48: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/48.jpg)
The Simplex Algorithm
At every step of the simplex algorithm we are primal feasible.
We terminate when we are dual feasible.
What about complementary slackness? The condition
(u>A− c>)x = 0
But in simplex
[0, c>B A−1B AN − c>N ] = (u>A− c>)
Do we have complementary slackness?
48
![Page 49: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/49.jpg)
The Simplex Algorithm
Here is the geometry of what we have done.
2 4 6 8x1
2
4
6
8
10
12x2
Feasible Region
Iteration 1
Iteration 2
Iteration 3
Iteration 4
49
![Page 50: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/50.jpg)
The Simplex Algorithm
Doing simplex pivots by hand is cumbersome and does notcontribute much to learning. However, it is critical to understand:
I Selecting a variable to enter the basis
I Selecting a variable to leave the basis
I Doing a pivot
In order to facilitate this process use simplexpivot.m.
50
![Page 51: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/51.jpg)
The Simplex Algorithm
simplexpivot.m
Inputs:
I A – the constraint matrix
I b – the right-hand-side vector
I c – the objective function vector
I B – an index set of the basic variables
Outputs:
I tableau – the tableau associated with basis B
I B – the updated basis vector after the user makes a pivot
Assumptions: we are working with standard form and solving aminimization problem.
51
![Page 52: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/52.jpg)
The Simplex Algorithm
Let’s attack with simplexpivot.m the following linear program(use lptest2.m to execute this).
min 2x1 −3x2x2 −x3 = 1
x1 +x2 −x4 = 2−.5x1 +x2 +x5 = 8−x1 +x2 +x6 = 6x1, x2, x3, x4, x5, x6 ≥ 0
We use the following basis matrix
B = [1 2 5 6]
52
![Page 53: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/53.jpg)
The Simplex Algorithm
[tableau, B] = simplexpivot(A, b, c, B);
-0.00 -0.00 5.00 -2.00 -0.00 -0.00 -1.00
----------------------------------------------------------
1.00 0.00 1.00 -1.00 0.00 0.00 1.00
0.00 1.00 -1.00 0.00 0.00 0.00 1.00
0.00 0.00 1.50 -0.50 1.00 0.00 7.50
0.00 0.00 2.00 -1.00 0.00 1.00 6.00
Enter the index of a variable to enter the basis 3
Row 1 has ratio 1.00
Row 3 has ratio 5.00
Row 4 has ratio 3.00
Enter the ROW NUMBER (not variable index) of the variable
to leave the basis 1
53
![Page 54: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/54.jpg)
The Simplex Algorithm
-0.00 -0.00 5.00 -2.00 -0.00 -0.00 -1.00
----------------------------------------------------------
1.00 0.00 1.00 -1.00 0.00 0.00 1.00
0.00 1.00 -1.00 0.00 0.00 0.00 1.00
0.00 0.00 1.50 -0.50 1.00 0.00 7.50
0.00 0.00 2.00 -1.00 0.00 1.00 6.00
The result of a pivot on column 3 row 1
-5.00 -0.00 -0.00 3.00 -0.00 -0.00 -6.00
----------------------------------------------------------
1.00 0.00 1.00 -1.00 0.00 0.00 1.00
1.00 1.00 0.00 -1.00 0.00 0.00 2.00
-1.50 0.00 0.00 1.00 1.00 0.00 6.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
B = 3 2 5 6
54
![Page 55: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/55.jpg)
The Simplex Algorithm
>> [tableau, B] = simplexpivot(A, b, c, B);
-5.00 -0.00 -0.00 3.00 -0.00 -0.00 -6.00
----------------------------------------------------------
1.00 0.00 1.00 -1.00 0.00 0.00 1.00
1.00 1.00 0.00 -1.00 0.00 0.00 2.00
-1.50 0.00 0.00 1.00 1.00 0.00 6.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
Enter the index of a variable to enter the basis 4
Row 3 has ratio 6.00
Row 4 has ratio 4.00
Enter the ROW NUMBER (not variable index) of the variable
to leave the basis 4
55
![Page 56: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/56.jpg)
The Simplex Algorithm
-5.00 -0.00 -0.00 3.00 -0.00 -0.00 -6.00
----------------------------------------------------------
1.00 0.00 1.00 -1.00 0.00 0.00 1.00
1.00 1.00 0.00 -1.00 0.00 0.00 2.00
-1.50 0.00 0.00 1.00 1.00 0.00 6.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
The result of the pivot on variable 4 in row 4 is
1.00 -0.00 -0.00 -0.00 -0.00 -3.00 -18.00
----------------------------------------------------------
-1.00 0.00 1.00 0.00 0.00 1.00 5.00
-1.00 1.00 0.00 0.00 0.00 1.00 6.00
0.50 0.00 0.00 0.00 1.00 -1.00 2.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
B = 3 2 5 4
56
![Page 57: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/57.jpg)
The Simplex Algorithm
>> [tableau, B] = simplexpivot(A, b, c, B);
1.00 -0.00 -0.00 -0.00 -0.00 -3.00 -18.00
----------------------------------------------------------
-1.00 0.00 1.00 0.00 0.00 1.00 5.00
-1.00 1.00 0.00 0.00 0.00 1.00 6.00
0.50 0.00 0.00 0.00 1.00 -1.00 2.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
Enter the index of a variable to enter the basis 1
Row 3 has ratio 4.00
Enter the ROW NUMBER (not variable index) of the variable
to leave the basis 3
57
![Page 58: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/58.jpg)
The Simplex Algorithm
1.00 -0.00 -0.00 -0.00 -0.00 -3.00 -18.00
----------------------------------------------------------
-1.00 0.00 1.00 0.00 0.00 1.00 5.00
-1.00 1.00 0.00 0.00 0.00 1.00 6.00
0.50 0.00 0.00 0.00 1.00 -1.00 2.00
-2.00 0.00 0.00 1.00 0.00 1.00 4.00
The result of the pivot on variable 1 in row 3 is
-0.00 -0.00 -0.00 -0.00 -2.00 -1.00 -22.00
----------------------------------------------------------
0.00 0.00 1.00 0.00 2.00 -1.00 9.00
0.00 1.00 0.00 0.00 2.00 -1.00 10.00
1.00 0.00 0.00 0.00 2.00 -2.00 4.00
0.00 0.00 0.00 1.00 4.00 -3.00 12.00
B = 3 2 1 4
58
![Page 59: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/59.jpg)
The Simplex Algorithm
The original decisions variables were x1 and x2.
What is the optimal value of x1?
What is the optimal value of x2?
-0.00 -0.00 -0.00 -0.00 -2.00 -1.00 -22.00
----------------------------------------------------------
0.00 0.00 1.00 0.00 2.00 -1.00 9.00
0.00 1.00 0.00 0.00 2.00 -1.00 10.00
1.00 0.00 0.00 0.00 2.00 -2.00 4.00
0.00 0.00 0.00 1.00 4.00 -3.00 12.00
59
![Page 60: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/60.jpg)
The Simplex Algorithm
The key steps in the Simplex Algorithm (assume minimization):
1. Start with a basic feasible solution and initialize the tableau.
2. Find a variable (pivot column) with a negative reduced cost.
3. Perform the minimum ratio test and find the pivot row.
4. Pivot and update the tableau (including reduced costs andright-hand-side).
5. Update the basic variable (B) and nonbasic variable (N) indexsets.
60
![Page 61: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/61.jpg)
The Simplex AlgorithmThe MATLAB file basicsimplex.m implements the simplexalgorithm as we have described it.Inputs:
I A – the constraint matrix
I b – the right-hand-side vector
I c – the objective function vector
I B – an index set of the basic variables
Outputs:
I X – the optimal solution values of the variables
I objVal – the optimal objective function value.
I B – the basis of the optimal solution.
Assumptions: we are working with standard form and solving aminimization problem.
61
![Page 62: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/62.jpg)
The Simplex Algorithm
Step 1: Start with a basic feasible solution and initialize thetableau.
numRows = size( b);
numVars = size( c);
N = setdiff([1:numVars], B)
AB = A(:, B);
AN = A(:, N);
numNonbasicVars = size( N’);
cN = c(N);
cB = c(B);
negRedCost = -1;
T = inv(AB)*A;
w = c’ - (cB’*inv(AB))*A;
wN = w( N);
bBAR = inv(AB)*b;
objVal = cB’*bBAR;
tableau = [-w , objVal; inv(AB)*A, bBAR]
62
![Page 63: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/63.jpg)
The Simplex Algorithm
Step 2: Find a variable (pivot column) with a negative reducedcost.
for k = 1:numNonbasicVars
% look for a negative reduced cost
if wN( k) < 0
negRedCost = wN( k);
%record the index
negRedCostIdx = k;
break
end
end
63
![Page 64: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/64.jpg)
The Simplex Algorithm
Step 3: Perform the minimum ratio test and find the pivot row.
% do minimum ratio test
% get the minimum ratio index
minRatioVal = inf;
for k = 1:numRows
if T( k, N(negRedCostIdx)) > 0
if bBAR(k) / T( k, N(negRedCostIdx)) < minRatioVal
minRatioVal = bBAR(k) / T( k, N(negRedCostIdx));
%record the index
minRatioIdx = k;
end
end
end
64
![Page 65: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/65.jpg)
The Simplex AlgorithmStep 4: Pivot and update the tableau (including reduced costsand right-hand-side).
pivEl = T(minRatioIdx, N(negRedCostIdx));
% multiply pivot row by (1/pivEl)
T(minRatioIdx,:) = (1/pivEl).*T(minRatioIdx,:);
% also multiply the rhs by the pivot element
bBAR(minRatioIdx) = (1/pivEl)*bBAR(minRatioIdx);
% now pivot on row minRatioIdx, column N(negRedCostIdx)
for k = 1:numRows
if k ~= minRatioIdx
multiplier = T(k, N(negRedCostIdx));
if multiplier ~= 0
T(k,:) = T(k,:) - multiplier .* T(minRatioIdx,:);
bBAR(k) = bBAR(k) - multiplier*bBAR( minRatioIdx);
end
end
end
objVal = objVal + wN( negRedCostIdx)*bBAR( minRatioIdx );
w = w -wN( negRedCostIdx) .* T(minRatioIdx,:);
65
![Page 66: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/66.jpg)
The Simplex Algorithm
Step 5: Update the basic variable (B) and nonbasic variable (N)index sets.
tmpIdx = B( minRatioIdx) ;
B( minRatioIdx) = N( negRedCostIdx);
N( negRedCostIdx) = tmpIdx;
% update Basic and Non-Basic matrices
AB = A(:, B);
AN = A(:, N);
cN = c(N);
cB = c(B);
wN = w(N);
66
![Page 67: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/67.jpg)
The Simplex Algorithm
Here is a formal mathematical statement of the Simplex Algorithm.
Step 1: (Initialization) Initialize with a basis matrix AB such thatA−1B b ≥ 0 and create the initial tableau
z0 − wNxN = c>B A−1B b
xB + A−1B ANxN = A−1B b
Step 2: (Pivot Column Selection) If wN ≥ 0, stop, the currentsolution is optimal; else, select q ∈ N such that wq < 0.
67
![Page 68: Linear Programming The Simplex Algorithm Chapter 5](https://reader035.vdocuments.us/reader035/viewer/2022071602/613d5be3736caf36b75c6440/html5/thumbnails/68.jpg)
The Simplex Algorithm
Step 3: (Minimum Ratio Test and Pivot Row Selection) Findthe maximum allowable increase of variable xq by the minimumratio test
i∗ ← argmin {xBi/aiq | aiq > 0, i = 1, . . .m}
Step 4: (Pivot and Update Tableau) Pivot on element ai∗q andupdate the tableau.
Step 5: (Update the Basic and Non-Basic Index Sets)
B ← (B\{Bi∗}) ∪ {q} N ← (N\{q}) ∪ {Bi∗}
Go to Step 2.
68