solving lp models - isu public homepage serverolafsson/part3.ppt · ppt file · web view ·...
TRANSCRIPT
Part 3 IE 312 1
Solving LP Models Improving Search
Unimodal Convex feasible region Should be successful!
Special Form of Improving Search Simplex method (now) Interior point methods (later)
Part 3 IE 312 2
Simple Example Top Brass Trophy Company Makes trophies for
football wood base, engraved plaque, brass football on top $12 profit and uses 4’ of wood
soccer wood base, engraved plaque, soccer ball on top $9 profit and uses 2’ of wood
Current stock 1000 footballs, 1500 soccer balls, 1750 plaques,
and 4800 feet of wood
Part 3 IE 312 3
Formulation
0,480024175015001000s.t.
912max
21
21
21
2
1
21
xxxxxxxxxx
Part 3 IE 312 4
Graphical Solution2000
1500
1000
500
500 1000 1500 2000
15002 x
15001 x
175021 xx
480024 21 xx
01 x
02 x
Optimal Solution
Part 3 IE 312 5
Feasible Solutions Feasible solution is a
boundary point if at least one inequality constraint that can be strict is active
interior point if no such constraints are active
Extreme points of convex sets do not lie within the line segment of any other points in the set
Part 3 IE 312 6
Example2000
1500
1000
500
500 1000 1500 2000
Part 3 IE 312 7
Optimal Solutions Every optimal solution is a boundary point
We can find an improving direction whenever we are at an interior point
If optimum unique the it must be an extreme point of the feasible region
If optimal solution exist, an optimal extreme point exists
Part 3 IE 312 8
LP Standard Form Easier if we agree on exactly what a
LP should look like Standard form
only equality main constraints only nonnegative variables variables appear at most once in left-
hand-side and objective function all constants appear on right hand side
Part 3 IE 312 9
Converting to Standard Inequality constraints
Add nonnegative, zero-cost slack variables
Add in inequalities Subtract in inequalities
Variables not nonnegative nonpositive - substitute with negatives unrestrictive sign (URS) - substitute
difference of two nonnegative variables
Part 3 IE 312 10
Top Brass Model
0,,,,,480024175015001000s.t.
912max
654321
621
521
42
31
21
xxxxxxxxxxxxxxxx
xx
Part 3 IE 312 11
Why? Feasible directions
Check only if active Keep track of active constraints
Equality constraints Always active
Inequality constraints May or may not be active
Prefer equality constraints!
Part 3 IE 312 12
Standard Notation
ariablesdecision v ofnumber sconstraint ofnumber
constraintth of side-hand-right
constraintth in oft coefficien constraint
oft coefficiencost
ariabledecision vth
nm
ib
ixa
xc
jx
i
jij
jj
j
Part 3 IE 312 13
LP Standard Form
jx
ibxa
xc
j
n
jjjij
m
jjj
,0
,s.t.
maxmin/
1
1
In standard notation
In matrix notation
0xbAx
xc
s.t.
maxmin/
Part 3 IE 312 14
Write in Matrix Form1 2 3
1 2 3 4
1 3 5
2 3
1 2 3 4 5
max 30 120 42 5 55s.t.
02 18 50
, , , , 0
x x xx x x x
x x xx x
x x x x x
bA
c
)(
Part 3 IE 312 15
Extreme Points Know that an extreme point
optimum exists Will search trough extreme points
An extreme point is define by a set of constraints that are active simultaneously
Part 3 IE 312 16
Improving Search Move from one extreme point to a
neighboring extreme point Extreme points are adjacent if they
are defined by sets of active constraints that differ by only one element
An edge is a line segment determined by a set of active constraints
Part 3 IE 312 17
Basic Solutions Extreme points are defined by set of
active nonnegativity constraints
A basic solution is a solution that is obtained by fixing enough variable to be equal to zero, so that the equality constraints have a unique solution
Part 3 IE 312 18
Example
480024175015001000
621
521
42
31
xxxxxxxxxx
48000241750015001000
21
21
42
31
xxxx
xxxx
Choose x1, x2, x3, x4 to be basic
)0,0,400,350,1100,650(x
Part 3 IE 312 19
Where is the Basic Solution?2000
1500
1000
500
500 1000 1500 2000
)0,0,400,350,1100,650(x
Part 3 IE 312 20
Example Compute the basic solution for x1 and x2
basis:
Solve
0,,,822314
321
321
321
xxxxxx
xxx
82314
21
21
xxxx
31
2
1
xx
Part 3 IE 312 21
Existence of Basis Solutions Remember linear algebra?
A basis solution exists if and only if the columns of corresponding equality constraint form a basis(in other words, a largest possible linearly independent collection)
Part 3 IE 312 22
Checking The determinant of a square matrix
D is
A matrix is singular if its determinant = 0 and otherwise nonsingular
Need to check that the matrix is nonsingular
j
jjj d )det()1()det( 1
)1( DD
Part 3 IE 312 23
Example Check whether basic solutions exist for
0,,1021584
321
21
321
xxxxxxxx
Part 3 IE 312 24
Basic Feasible Solutions A basic feasible solution to a LP
is a basic solution that satisfies all the nonnegativity contraints
The basic feasible solutions correspond exactly to the extreme points of the feasible region
Part 3 IE 312 25
Example Problem Suppose we have x3, x4, x5 as slack
variables in the following LP:
Lets plot the original problem, compute the basic solutions and check feasibility
0,...,320
51
52
41
321
xxxxxxxxx
Part 3 IE 312 26
Solution Algorithm Simplex Algorithm
Variant of improving search Standard display:
x1 x2 x3 x4 x5 x6max c 12 9 0 0 0 0 b
1 0 1 0 0 0 10000 1 0 1 0 0 15001 1 0 0 1 0 17504 2 0 0 0 1 4800
A
Part 3 IE 312 27
Simplex Algorithm Starting point
A basic feasible solution (extreme point) Direction
Follow an edge to adjacent extreme point: Increase one nonbasic variable Compute changes needed to preserve
equality constraints One direction for each nonbasic variable
Part 3 IE 312 28
Top Brass Examplex1 x 2 x3 x4 x 5 x 6
max c 12 9 0 0 0 0 b1 0 1 0 0 0 10000 1 0 1 0 0 15001 1 0 0 1 0 17504 2 0 0 0 1 4800N N B B B B
x(0) 0 0 1000 1500 1750 4800
A
Initial solutionBasic variables
Part 3 IE 312 29
Looking in All Directions …x1 x 2 x 3 x 4 x5 x6
max c 12 9 0 0 0 0 b1 0 1 0 0 0 10000 1 0 1 0 0 15001 1 0 0 1 0 17504 2 0 0 0 1 4800N N B B B B
x(0) 0 0 1000 1500 1750 4800Dx1 1 0 -1 0 -1 -4Dx2 0 1 0 -1 -1 -2
A
Must adjust these!Can increase eitherone of those
Part 3 IE 312 30
So Many Choices ... Want to try to improve the objective
The reduced cost of a nonbasic variable:
Want
n
jjj xcf
1
)( xcx
Δxcjc
onminimizati if 0
onmaximizati if 0
j
j
c
c Definesimprovingdirection
Part 3 IE 312 31
Top Brass Example Improving x1 gives
Improving x2 gives
Both directions are improving directions!
012
41010100009121
Δxc
c
09
21101000009122
Δxc
c
Part 3 IE 312 32
Where and How Far? Any improving direction will do If no component is negative Improve forever - unbounded!
Otherwise, compute the minimum ratio
DD
0:min)(
jj
tj xx
x
Part 3 IE 312 33
Computing Minimum Ratiox 1 x 2 x 3 x 4 x 5 x 6N N B B B B
x(0) 0 0 1000 1500 1750 4800Dx 1 0 -1 0 -1 -4
1000 1750 4800-(-1) -(-1) -(-4)
10004
4800,1
1750,1
1000min
0:min)(
DD
jj
tj xx
x
Part 3 IE 312 34
Moving to New Solution
4800175015000001004101011000
480017501500100000
)0()1(
D xxx
Part 3 IE 312 35
Updating Basis New basic variable
Nonbasic variable generating direction
New nonbasic variable(s) Basic variables fixing the step size
Part 3 IE 312 36
What Did We Do?2000
1500
1000
500
500 1000 1500 2000
)0(x )1(x
Part 3 IE 312 37
Where Will We Go?2000
1500
1000
500
500 1000 1500 2000
)0(x )1(x
Why is thisguaranteed?
)2(x
)3(x
Optimum in three steps!
Part 3 IE 312 38
Simplex Algorithm (Simple)
Step 0: Initialization. Choose starting feasible basis, construct basic solution x(0), and set t=0
Step 1: Simplex Directions. Construct directions Dx associated with increasing each nonbasic variable xj and compute the reduced cost cj =c ·Dx.
Step 2: Optimality. If no direction is improving, then stop; otherwise choose any direction Dx(t+1) corresponding to some basic variable xp.
Step 3: Step Size. If no limit on move in direction Dx(t+1) then stop; otherwise choose variable xr such that
Step 4: New Point and Basis. Compute the new solution
and replace xr in the basis with xp. Let t = t+1 and go to Step 1.
)1(
)()1(
)1(
)(
)1(
)(
set and 0:min
D
DD
D t
r
trt
jtj
tj
tr
tr
xxx
xx
xx
)1()()1( D ttt xxx
Part 3 IE 312 39
Stopping The algorithm stop when one of two
criteria is met: In Step 2 if no improving direction
exists, which implies local optimum, which implied global optimum
In Step 3 if no limit on improvement, which implies problem is unbounded
Part 3 IE 312 40
Optimization Software Spreadsheet (e.g, MS Excel with What’s Best!) Optimizers (e.g., LINDO) Combination
Modeling Language Solvers
Either together (e.g., LINGO) or separate (e.g., GAMS with CPLEX)
LINDO and LINGO are in Room 0010 (OR Lab) Also on disk with your book
Part 3 IE 312 41
LINDO The main software that I’ll ask you to use is
called LINDO Solves linear programs (LP), integer
programs (IP), and quadratic programs (QP) We will look at many of its more advanced
features later on, but as of yet we haven’t learned many of the concepts that we need
Part 3 IE 312 42
Example
0,,,,,480024175015001000s.t.
912max
654321
621
521
42
31
21
xxxxxxxxxxxxxxxx
xx
Part 3 IE 312 43
LINDO ProgramMAX 12 x1 + 9 x2
ST
x1 + x2 = 1000
x2 + x4 = 1500
x1 + x2 + x5 = 1750
4x1+ 2x2 + x6 = 4800
x1>=0
x2>=0
x3>=0
x4>=0
x5>=0
x6>=0
END
Part 3 IE 312 44
Part 3 IE 312 45
Part 3 IE 312 46
OutputLP OPTIMUM FOUND AT STEP 4
OBJECTIVE FUNCTION VALUE
1) 12000.00
VARIABLE VALUE REDUCED COST
X1 1000.000000 0.000000
X2 0.000000 3.000000
X4 1500.000000 0.000000
X5 750.000000 0.000000
X6 800.000000 0.000000
X3 0.000000 0.000000
Part 3 IE 312 47
Output (cont.) ROW SLACK OR SURPLUS DUAL PRICES
2) 0.000000 12.000000
3) 0.000000 0.000000
4) 0.000000 0.000000
5) 0.000000 0.000000
6) 1000.000000 0.000000
7) 0.000000 0.000000
8) 0.000000 0.000000
9) 1500.000000 0.000000
10) 750.000000 0.000000
11) 800.000000 0.000000
NO. ITERATIONS= 4
Part 3 IE 312 48
LINDO: Basic Syntax Objective Function Syntax: Start all
models with MAX or MIN Variable Names: Limited to 8
characters Constraint Name: Terminated with a
parenthesis Recognized Operators (+, -, >, <, =) Order of Precedence: Parentheses not
recognized
Part 3 IE 312 49
Syntax (cont.) Adding Comment: Start with an
exclamation mark Splitting lines in a model: Permitted in
LINDO Case Sensitivity: LINDO has none Right-hand Side Syntax: Only constant
values Left-hand Side Syntax: Only variables and
their coefficients
Part 3 IE 312 50
Why Modeling Language? More to learn! More ‘complicated’ to use than
LINDO (at least at first glance) Advantages
Natural representations Similar to mathematical notation Can enter many terms simultaneously Much faster and easier to read
Part 3 IE 312 51
Why Solvers? Best commercial software has modeling
language and solvers separated Advantages:
Select solver that is best for your application Learn one modeling language use any solver Buy 3rd party solvers or write your own!
Part 3 IE 312 52
Example ProblemBisco’s new sugar-free, fat-free chocolate squares are so popular that the companycnnot keep up with demand. Regional demands shown in the following table total2000 cases per week, but Bisco can produce only 60% of that number.
NE SE MW WDemand 620 490 510 380Profit 1.6 1.4 1.9 1.2
The table also shows the different profit levels per case experienced in the regionsdue to competition and consumer tastes. Bisco wants to find a maximum profitplan that fulfills between 50% and 70% of each region’s demand.
Part 3 IE 312 53
Problem Formulation
4,3,2,1,
1200
max
4
1
4
1
iuxl
x
xp
iii
ii
iii
Part 3 IE 312 54
LINDO Solutionmax 1.60 x1 + 1.40 x2 + 1.90 x3 + 1.20 x4
st x1 + x2 + x3 + x4 <=1200 x1 >= 310 x1 <= 434 x2 >= 245 x2 <= 343 x3 >= 255 x3 <= 357 x4 >= 190 x5 <= 266end
Part 3 IE 312 55
LINGO Solution Capacity constraint@SUM(REGIONS(I): CASES(I)) <=1200;
Minimum/maximum cases@FOR(REGIONS(I):CASES(I) <= UBOUND;CASES(I) >= LBOUND);
Part 3 IE 312 56
LINGO Solution Objective function
MAX = @SUM(REGIONS(I): PROFIT*CASES(I));
We also need to define REGIONS, CASES, etc, and type in the data.
Part 3 IE 312 57
LINGO Solution Defining sets
SETS:REGIONS / NE SE MW W/:
LBOUND,UBOUND, PROFIT, CASES;
ENDSETS
Part 3 IE 312 58
LINGO Solution Enter the data
DATA:LBOUND = 310 245 255 190;UBOUND = 434 343 357 266;PROFIT = 1.6 1.4 1.9 1.2;
ENDDATA
Part 3 IE 312 59
Sensitivity Analysis Basic Question: How does our solution
change as the input parameters change? The objective function?
More/less profit or cost The optimal values of decision variables?
Make different decisions! Why?
Only have estimates of input parameters May want to change input parameters
Part 3 IE 312 60
What We Know Qualitative Answers for All Problems Quantitative Answers for Linear
Programs (LP) Dual program Same input parameters Decision variables give sensitivities Dual prices Easy to set up Theory is somewhat complicated
Part 3 IE 312 61
Back to Example ProblemBisco’s new sugar-free, fat-free chocolate squares are so popular that the companycnnot keep up with demand. Regional demands shown in the following table total2000 cases per week, but Bisco can produce only 60% of that number.
NE SE MW WDemand 620 490 510 380Profit 1.6 1.4 1.9 1.2
The table also shows the different profit levels per case experienced in the regionsdue to competition and consumer tastes. Bisco wants to find a maximum profitplan that fulfills between 50% and 70% of each region’s demand.
Part 3 IE 312 62
LINDO Formulationmax 1.60 x1 + 1.40 x2 + 1.90 x3 + 1.20 x4st x1 + x2 + x3 + x4 <=1200 x1 >= 310 x1 <= 434 x2 >= 245 x2 <= 343 x3 >= 255 x3 <= 357 x4 >= 190 x5 <= 266end
Part 3 IE 312 63
LINDO Solution (second half)
ROW SLACK OR SURPLUS DUAL PRICES 2) 0.000000 1.600000
3) 98.000000 0.000000
4) 26.000000 0.000000
5) 0.000000 -0.200000
6) 98.000000 0.000000
7) 102.000000 0.000000
8) 0.000000 0.300000
9) 0.000000 -0.400000
10) 266.000000 0.000000
Part 3 IE 312 64
Dual Prices The Dual is Automatically Formed
Also in LINGO Also in (all) other optimization software
Report dual prices Gives us sensitivities to RHS parameter Know how much objective function will
change When will the optimal solution change? Need to select that we want sensitivity
analysis
Part 3 IE 312 65
LINDO Sensitivity Analysis (part)
RANGES IN WHICH THE BASIS IS UNCHANGED:
OBJ COEFFICIENT RANGES
VARIABLE CURRENT ALLOWABLE ALLOWABLE
COEF INCREASE DECREASE
X1 1.600000 0.300000 0.200000
X2 1.400000 0.200000 INFINITY
X3 1.900000 INFINITY 0.300000
X4 1.200000 0.400000 INFINITY
X5 0.000000 0.000000 INFINITY
Part 3 IE 312 66
Interpretation As long as prices for the NE region
are between $1.4 and $1.9, we want to sell the same quantity to each region, etc.
Part 3 IE 312 67
Example
An insurance company is introducing two new product lines: special risk insurance and mortgages. The expected profit is $5 per unit on special risk insurance and $2 per unit on mortgages. Management wishes to establish a sales target for the new product lines to maximize the expected profit. The work requirements are as follows:
Part 3 IE 312 68
LINDO Formulationmax 5 x1 + 2 x2
st
3 x1 + 2 x2 <= 2400x2 <= 8002 x1 <= 1200x1 >=0x2 >=0
end
Part 3 IE 312 69
Graphical Solution
100 200 300 400 500 600 700 800
800
700
600
500
400
300
200
100
Part 3 IE 312 70
Solution VARIABLE VALUE REDUCED COST
X1 600.000000 0.000000
X2 300.000000 0.000000
ROW SLACK OR SURPLUS DUAL PRICES
2) 0.000000 1.000000
3) 500.000000 0.000000
4) 0.000000 1.000000
5) 600.000000 0.000000
6) 300.000000 0.000000
Part 3 IE 312 71
Sensitivity AnalysisRANGES IN WHICH THE BASIS IS UNCHANGED:
OBJ COEFFICIENT RANGES VARIABLE CURRENT ALLOWABLE ALLOWABLE COEF INCREASE DECREASE X1 5.000000 INFINITY 2.000000 X2 2.000000 1.333333 2.000000
RIGHTHAND SIDE RANGES ROW CURRENT ALLOWABLE ALLOWABLE RHS INCREASE DECREASE 2 2400.000000 1000.000000 600.000000 3 800.000000 INFINITY 500.000000 4 1200.000000 400.000000 666.666687 5 0.000000 600.000000 INFINITY 6 0.000000 300.000000 INFINITY
Part 3 IE 312 72
New Decisions!
100 200 300 400 500 600 700 800
800
700
600
500
400
300
200
100
OptimumMoves!
Part 3 IE 312 73
What-If ? Solve New Problemmax 5 x1 + 2 x2
st
3 x1 + 2 x2 <= 2400x2 <= 2902 x1 <= 1200x1 >=0x2 >=0
end
Part 3 IE 312 74
New Solution VARIABLE VALUE REDUCED COST
X1 600.000000 0.000000
X2 290.000000 0.000000
ROW SLACK OR SURPLUS DUAL PRICES
2) 20.000000 0.000000
3) 0.000000 2.000000
4) 0.000000 2.500000
5) 600.000000 0.000000
6) 290.000000 0.000000
Part 3 IE 312 75
Interior Point Methods Simplex always stays on the boundary Can take short cuts across the interior Interior point methods
More effort in each move More improvement in each move Much faster for large problems
Part 3 IE 312 76
Simple Example Frannie’s Firewood sells up to 3 cords of
firewood to two customers One will pay $90 per half-cord Other will pay $150 per full cord
0,
321s.t
15090max2customer tosold cords ofnumber
1customer tosold cords-half ofnumber
21
21
21
2
1
xx
xx
xxxx
Part 3 IE 312 77
Graphical Solution
1 2 3 4 5 6
4
3
2
1
1x
2x
06* x
Part 3 IE 312 78
Improving Directions Which direction improves the objective
function the most? The gradient Direction:
xccxxccx
cxcx
DD
minif maxif
)(f
Part 3 IE 312 79
Most Improving Direction?
?792max
?194min
321
321
xxx
xxx
Part 3 IE 312 80
Back to Example
1 2 3 4 5 6
4
3
2
1
1x
2x
06* x
211)0(x
2653
2650)1(x
Part 3 IE 312 81
Maintaining Feasibility At the initial point all directions are
feasible because it is an interior point At the new point we have to make sure
that a direction Dx at x(1) satisfies
Interior point algorithms begin inside and move through the interior, reaching the boundary only at an optimal solution
021
21 DD xx
Part 3 IE 312 82
Valid Interior Point Search?2000
1500
1000
500
500 1000 1500 2000
Part 3 IE 312 83
Valid Interior Point Search?2000
1500
1000
500
500 1000 1500 2000
Part 3 IE 312 84
Valid Interior Point Search?2000
1500
1000
500
500 1000 1500 2000
Part 3 IE 312 85
LP Standard Form For Simplex used the form
In Frannie’s Firewood0xbAx
xc
s.t.
maxmin/
0,,
321s.t
15090max
321
321
21
xxx
xxx
xx
Part 3 IE 312 86
Benefits of Standard Form?
In Simplex: Made easy to check which variables are basic,
non-basic, etc. Needed to know which solutions are on boundary
Here quite similar: Know which are not on boundary Check that nonnegativity constraints are strict!
A feasible solution to standard LP is interior point if every component is strictly positive
Part 3 IE 312 87
Interior Points?
0,,,12261032s.t
825min
4321
41
321
431
xxxxxxxxx
xxx
6133611418608 )3()2()1( xxx
Part 3 IE 312 88
Projections Must satisfy main equality constraints
Want direction Dx that satisfies this equation and is as nearly d as possible
The projection of a vector d onto a system of equalities is the vector that satisfies the constraints and minimizes the total squared difference between the components
0xAbAxD
Part 3 IE 312 89
Obtaining Projection The projection of d onto ADx=0 is
where
is the projection matrix.
Pdx D
AAAAIP TT 1)(
Part 3 IE 312 90
Example: Frannie’s Firewood
95
94
92
94
95
92
92
92
98
94
94
92
94
94
92
92
92
91
100010001
94
94
92
94
94
92
92
92
91
1121
94
112
1
49
112
1
1121
1
1
AAAAIP
AAAA
AAAA
TT
TT
TT
Part 3 IE 312 91
Example
D
9269
199
14
053
95
94
92
94
95
92
92
92
98
Pdx The cost vector is c=(90 150 0)
Part 3 IE 312 92
Improvement The projection matrix is design to
make an improving direction feasible with minimum changes
Is it still an improving direction? Yes!
The projection Dx=Pc of c onto Ax=b is an improving direction at every x
Part 3 IE 312 93
Sample Exercise
0,,52
1s.t.max
321
32
31
321
xxxxxxx
xxx
Determine the direction d of most rapid improvement Project it onto the main equality constraints to get Dx Verify that the move direction Dx is feasible Verify that the move direction Dx is improving