1

Optimization

Linear Programming andSimplex Method

2

Optimization MethodsOne-Dimensional Unconstrained Optimization

Golden-Section SearchQuadratic InterpolationNewton's Method

Multi-Dimensional Unconstrained OptimizationNon-gradient or direct methodsGradient methods

Linear Programming (Constrained)Graphical SolutionSimplex Method

3

• Basic linear programming (LP) problem consists of two major parts:– The objective function– A set of constraints

• For maximization problem, the objective function is generally expressed as

nn xcxcxc 2211 ZMaximize

cj= payoff of each unit of the jth activity that is undertaken

xj= magnitude of the jth activity

Z= total payoff due to the total number of activities

Standard Form Linear Programming Problem

4

• The constraints can be represented generally as

where

aij= amount of the ith resource that is consumed for each unit of the jth activity

bi = amount of the ith resource that is available

• The general second type of constraints specifies that all activities must have a positive value. i.e,

ininii bxaxaxa 2211

Standard Form Linear Programming Problem

0ix

5

Problem

Products

Resource Regular Premium Resource Availability

Raw gas 7 m3/tonne 11 m3/tonne 77 m3/week

Production time

10 hr/tonne 8 hr/tonne 80 hr/week

Storage 9 tonne 6 tonne

Profit $150/tonne $175/tonne

How many tonnes of regular and premium gas to produce in order to maximize weekly profit?

6

Developing LP FormulationLet x1 = 1 tonne of regular gas

x2 = 1 tonne of premium gas

Maximize Z = 150x1 + 175x2

subject to

7x1 + 11x2 ≤ 77 (material constraint) (1)

10x1 + 8x2 ≤ 80 (time constraint) (2)

x1 ≤ 9 (regular gas storage) (3)

x2 ≤ 6 (premium gas storage) (4)

x1, x2 ≥ 0 (positivity) (5, 6)

7

Graphical Solution

Feasible solution space (the shaded area) contains all points that obey the constraints or points that represent the feasible solutions.

Points located outside of this area are known as infeasible solution.

8

Graphical Solution

We can adjust the line representing the objective function Z = 150x2 + 175x2

to locate the optimum point.

One of the corner points (points where two or more lines intersect) will be an optimum point.

– Corner points can be feasible or infeasible

9

Graphical Solution

Some insights:• Increasing storages

( constraints (3) and (4)) does not improve profit.

• Raising either materials or production time can improve profit.

10

Possible outcomes that can generally be obtained in a LP problem

1. Unique solution. The maximum objective function intersects a single point.

2. Alternate solutions. Problem has an infinite number of optima corresponding to a line segment.

3. No feasible solution.

4. Unbounded problems. Problem is under-constrained and therefore open-ended.

11

(a)Unique solution – represented by a corner point Alternate solutions – represented by points on one

of the constraint line

(b) No feasible solution

(c) Unbounded problems

12

Key properties of LP that drives the design of Simplex Method (An algorithm for solving LP problems)

1. The optimum point is always at a feasible corner point. (Why?)

– We only need to check the corner points.

2. If a corner point feasible solution has an objective function value that is better that or equal to all adjacent corner point feasible solutions, then it is optimal.

– We don't necessarily need to search all corner points.

13

Key properties of LP that drives the design of Simplex Method

3. There are finite number of corner point feasible solutions.

–Any method that checks only corner points will terminate eventually.

14

Basic idea behind the Simplex method1. Starting at a feasible corner point solution.

2. Repeatedly move to a better adjacent corner point feasible solution until an optimum point is found.

15

Finding Corner Points Algebraically

• How to find corner points algebraically?

• How to locate adjacent corner points?

• How to decide which adjacent point to move to next?

• Simplex method offers a neat solution (involves the use of slack variables) to address these questions.

16

First, constraint equations are reformulated as equalities by introducing slack variables – variables that measures how much of a constrained resource is available.

0,,,,,

6

9

80810

77117

tosubject

175150Maximize

432121

42

31

221

121

21

SSSSxx

Sx

Sx

Sxx

Sxx

xxZ

0,

6

9

80810

77117

tosubject

175150Maximize

21

2

1

21

21

21

xx

x

x

xx

xx

xxZ

Slack variablesSi > 0 means the corresponding resource is not fully consumed.

Si < 0 means the corresponding resource is over-consumed.

17

• Adding slack variables results in an augmented system of linear equations which is under specified (has more unknowns than equations).– 6 unknowns (= 2 original variables + 4 slack variables)– 4 equations

• In general, if a LP problem has n variables and m constraints, the resulting augmented system will have a total of (n original variables + m slack variables) and m equations.

Augmented System

Continue …

18

• To solve an augmented system with– 6 unknowns (= 2 original variables + 4 slack variables)– 4 equations

We need to set 2 of the variables constants before we can solve the system of equations.

e.g., Setting x1 = 0, x2 = 0 yields

Augmented System

Continue …

6

9

80

77

4

3

2

1

S

S

S

S

6

9

80810

77117

42

31

221

121

Sx

Sx

Sxx

Sxx

19

Solving the resulting system yields x1=0, x2=0, S1=77, S2=80,

S3=9, S4=6, which represent one of the augmented solutions –

values of all (original + slack) variables are given.

For this example, the values tell us that if we don't produce any regular (x1) or premium (x2) gas, we would have the following amount of unconsumed resources:

– 77 m3 of raw gas, – 80 production hours– 9 tonne of regular gas storage– 6 tonne of premium gas storage

6

9

80

77

4

3

2

1

S

S

S

S

Augmented System

20

• An augmented solution which corresponds to a corner point is also known as a basic solution.– The augmented solution x1=0, x2=0, S1=77, S2=80, S3=9,

S4=6 is a basic solution. It corresponds to corner point A.

– In addition, it is a basic feasible solution because all variables ≥ 0.

• So far, you know how to find an augmented solution. – How should we set the variables so that the augmented

solution is a basic solution?

Augmented System

Continue …

21

• The equations (or constraints) have a one-to-one relationship with the slack variables. – For example, S1 relates to the 1st equation, S4 relates to

the 4th equation.

• If a point is on the line 7x1 + 11x2 = 77 (line labeled "1"), what's the value of S1?

• If a point is on the line 10x1 + 8x2 = 80 (line labeled "2"), which variable has the value zero?

• At the corner point D, which variables have the value zero?

Characteristics of Slack Variables

Show Graph & Equations

22

• Setting two variables to zero has the effect of "selecting two lines", and solving the resulting system means finding the intersecting point (a corner point) of the selected lines.

Characteristics of Slack Variables

6

9

80810

77117

42

31

21

21

Sx

Sx

xx

xx

The value of x1 and x2 of this system is the coordinate of the corner point C (the point where line "1" and line "2" intersect.)

For example, let S1=0, S2=0. Then the system becomes

23

Finding Corner Points Algebraically

We now know how to find the corner points.

Next

• How to find an adjacent corner point?

• How to decide which adjacent corner point to move to?

24

Basic and Non-Basic VariablesCorner Point

Non-Basic Variables

Basic Variables

Some of the Adjacent Corner Points

A x1, x2 S1, S2, S3, S4 B, E

B x2, S2 x1, S1, S3, S4 A, C

C S1, S2 x1, x2, S3, S4 B, D

D S1, S4 x1, x2, S2, S3 C, E

E x1, S4 x2, S1, S2, S3 A, D

F x1, S2 x2, S1, S3, S4Variables in which their values are set to zero are called non-basic variables.

Remember: Non-basic, variable set to zero, corresponding constraint is active

Do you see a pattern between any pair of adjacent corner points?

25

Basic and Non-Basic Variables

• If two points are adjacent corner points, then– Their basic sets are identical except for one member.– Their non-basic sets are identical except for one

member.

• The reverse of the above statement is not always true. (e.g., F and A)

• Questions: – To find an adjacent corner point, can we just switch one

variable between the basic and non-basic sets?

– How should we decide which corner point to move to next?

26

Basic and Non-Basic Variables

How do we make sure the following conditions are satisfied?

– The corner points are adjacent.

– The corner points are feasible.

– The new corner point gives better value for the objective function than the current corner point.

27

Which variables to switch?All three conditions from the previous slide can be

satisfied if the following steps are taken:

• Selecting an entering basic variable – a non-basic variable that will increase the objective function value the most if allowed to take on a positive value. Move this variable from the non-basic set to the basic set.

• Finding the leaving basic variable – a basic variable that is forced to a value zero first after allowing the entering basic variable to increase. Move the variable that is forced to zero from the basic set to the non-basic set.

28

Which variables to switch?

Suppose we are currently at point A.

Basic set = { S1, S2, S3, S4 }, non-basic set = { x1, x2 }

Which variable should be the entering basic variable?

Which variable should be the leaving basic variable?

0,,,,,

6

9

80810

77117

tosubject

175150Maximize

432121

42

31

221

121

21

SSSSxx

Sx

Sx

Sxx

Sxx

xxZ

29

Which variables to switch?

x2 has a larger coefficient in the objective function, so we should make x2 the entering basic variable. (x1 remains a non-basic variable and has the value zero)

If we keep increasing x2, other basic variables will decrease. Which basic variable will become zero first?

0,,,,,

6

9

808

7711

tosubject

175150Maximize

432121

42

3

22

12

21

SSSSxx

Sx

S

Sx

Sx

xxZ

30

Which variables to switch?

If we take the ratio of the RHS value to the coefficient of the entering basic variable (x2), the smallest non-negative ratio can tell us which basic variable will reach zero first if we keep increasing the value of x2.

So S4 should be made the leaving basic variable.

0,,,,,

)61/6(6

N/A9

)108/80(808

)711/77(7711

tosubject

175150Maximize

432121

42

3

22

12

21

SSSSxx

Sx

S

Sx

Sx

xxZ

Show Graph & Equations

Minimum ratio test

31

Which variables to switch?

So setting x1=0, S4=0 and solve the system of equations would yield an augmented solution correspond to point E.

Solving the system of equations at every corner point is not very efficient. Simplex method offers an efficient way to compute the value of each variable by updating the equations incrementally.

32

1

2

4

35

6

E

A B

C

D

0,,,,,

6

9

80810

77117

tosubject

175150Maximize

432121

42

31

221

121

21

SSSSxx

Sx

Sx

Sxx

Sxx

xxZ

F