Special Cases of

Linear Programming

Carlos A. Santos Silva

April 30th, 2010

REVISITING LINEAR

PROGRAMMING

Linear Programming Summary (Class 2)

• Z = value of overall performance measure.

• xj = level of activity j, for j = 1, 2, …, n. [Decision variables]

• cj = parameters of Z related to xj.

• aij = amount of resource i consumed by each unity of activity j.

• bj = amount of resource i that is available for allocation of activities, for i = 1, 2, …, m.

0 and

subject to

Maximize

x

bax

cxZ

0,,0,0 and

subjet to

maximize

21

2211

22222221

11212111

1111

n

mnmnmm

nn

nn

nn

xxx

bxaxaxa

bxaxaxa

bxaxaxa

xcxcxcZ

Assumptions (Class2)

•Proportionality

• Contribution of each activity to functions is proportional to the level of

activity

•Additivity

• Every function is the sum of individual contributions

•Divisibility

• Decision values are allowed to have any values

• If there are only integers, use integer programming (class 4)•Certainty

• Values of each parameters are assumed to be a known constant

• If there is uncertainty, make sensitivity analysis (class 3)

Topics of Class 4

•Integer Programming Problems

•Mix-Integer Programming Problems

•Binary-Integer Programming Problems

INTEGER PROGRAMMING (CHAPTER 11)

What if decisions variables are not continuous?

•In many problems, decision variables are not continuous, but integers:

• People

• M achines

• Yes or no decisions (connect/disconnect, install/not to install) -> Binary

•In this case, the Divisibility assumption has to be dropped

• We can apply Linear Programming if we introduce additional constraints;

•There is a new class of optimization problems:

• Integer Programming: All decision variables are integers

• Binary Integer Programming: Decision variables are binary

• Mix-Integer Programming: Some decision variables are integers (binary or not) and some

are continuous

Simplified Planning Example (GI)

•Problem:

• M inimize Investment Costs of Renewable Energy Power Plants

•Domain

• Wind Cost: 1.5 m€ / M W

• Geothermal Cost: 3 m€ / M W

•Constraints:

• Wind efficiency: 0,25

• Geothermal efficiency: 0,85

• Production Increase: 12 M W

• Budget: 50 m€

Linear Programming M odel

•However, this solution may not be real:

• New geothermal power plants are in modules of 1 M W

• Optimal wind towers for the site are 1 M W

0,

5035,1

1285,025,0

tsConstrains

35,1:Minimize

21

21

21

21

xx

xx

xx

xxZ

Integer Programming M odel

•14 MW of Geothermal only guarantee 11.9MW

of production:

• We may consider smaller wind turbines, less

efficiency for the site

Investment in Renewable Energy

Wind Geo

Costs 1,5 3

Constraints Used

Installed Production Capacity 0,25 0,85 12,15 >= 12

Available Budget 1,5 3 43,5 <= 50

Solution 1 14 Cost 43,5

M ix Integer Programming

•400 kW of wind guarantee the 12 MW of production

Investment in Renewable Energy

Wind Geo

Costs 1,5 3

Constraints Used

Installed Production Capacity 0,25 0,85 12 >= 12

Available Budget 1,5 3 42,6 <= 50

Solution 0,4 14 Cost 42,6

M ore realistic example (GI)

•Problem:

• Built or not to Built a second geothermal power plant and a storage system in 2013

in order to maximize the NPV of investments

•Domain

• BAU: 10 M W Geothermal Plant

• NPV: 50 M €

• S1: 12 M W Geothermal Plant

• NPV: 60M€

• S2: wind

• NPV: 25M€

• S3: Pump storage

• NPV: 50M€

• Constraints:

• S3 if and only if S1 or S2

• S1 or S2

Bin Integer Binary Programming

•Built:

• second geothermal

• pump-storage

Investment in Renewable Energy

Geo1 Geo2 Wind Storage

Costs 50 60 25 50

Constraints Used

Geo2 or Wind 0 1 1 0 1 <= 1

Geo2 and Storage 1 -1 -1,1E-16 <= 0

Wind and Storage 1 -1 -1 <= 0

Solution 1 1 0 1 Cost 160

SPECIAL CASES OF BIP

Decision variables only with N possible values

•Problem:

• M inimize Investment Costs of Renewable Energy Power Plants

••Domain

• Wind Cost: 1.5 M €/ M W

• Geothermal Cost: 3 M € / M W

••Constraints:

• Available Budget: 50 M €

• Wind capacity factor: 0,25

• Geothermal efficiency: 0,85

• Production Increase: 12 M W

• Geothermal only in blocks of 6 or 10 or 16 M W

M ix Integer Programming M odel

Investment in Renewable Energy

Wind Geo y1 y2 y3

Costs 1,5 3

Constraints Used

Installed Production Capacity 0,25 0,85 13,6>= 12

Available Budget 1,5 3 48<= 50

Geo only in 6 or 10MW 1 -6 -10 -16 -7,3E-11= 0Geo=6y1+10y2+16y3

Only one Geo possibility 1 1 1 1= 1y1+y2+y3=0

Solution 0 16 0 0 1Cost 48

Fixed-Charge Problem

•Problem:

• Built or not to Built a second geothermal power plant and a storage system in 2013

in order to maximize the NPV of investments while considering investment costs

Domain

• BAU: 10 M W Geothermal Plant

• Investment: 30M€

• NPV: 50M€

• S1: 12 MW Geothermal Plant

• Investment:35M€

• NPV: 60M€

• S2: wind

• Investment:15M€

• NPV: 25M€

• S3: Pump storage

• Investment:40M€

• NPV: 50M€

Constraints:

• S3 if and only if S1 or S2

• S1 or S2

M ix Integer Programming M odel

Investment in Renewable Energy

Geo1 Geo2 Wind Storage Geo1Inv Geo2Inv WindInv StorageInv

Costs 50 60 25 50 -30 -35 -15 -40

Constraints Used

Geo2 or Wind 0 1 1 0 1 <= 1

Geo2 and Storage 1 -1 -4,6E-12 <= 0

Wind and Storage 1 -1 -1 <= 0

Geo1 1 1 <= 1

Geo2 1 1 <= 1

Wind 1 4,55E-12 <= 1

Storage 1 1 <= 1

Geo1Invest 1 -1000 -999 <= 0

Geo2Invest 1 -1000 -999 <= 0

WindInvest 1 -1000 -8,1E-28 <= 0

StorageInvest 1 -1000 -999 <= 0

Solution 1 1 4,55E-12 1 1 1 4,55E-15 1 Cost 55

SOLVING IP PROBLEMS

IP Complexity

•Primary determinants of computational complexity:

• number of integer variables,

• these variables are binary or general integer

• Variables,

• any special structure in the problem.

• In LP, number of constraints is more important than

the number of variables.

IP problems are much more difficult than LP

• we could apply LP and round the obtained solution (LP

relaxation)

Simplex cannot be used to optimize IP!!

Branch-and-Bound

“Smart” Enumerative method

• An enumerative method tests all solutions and finds optimal solution

• For BIP with N variables there are 2N possibilities (N=20 → >1M possibilities)

• For IP with N variables and M possible values there are MN possibilities

• A “smart” enumerative method reduces search space

Branch-and-Bound Algorithm

1. Branching – tests possible solutions by creating a tree

2. Bounding – finds optimal bound (using LP relaxation)

3. Fathoming – if bound of a branch is worst than known solution, dismiss branch

Another Simplified Planning Example (GI)

•Problem:

• Built or not to Built a second geothermal power plant and a storage system in 2013

in order to maximize the NPV of investments

Domain

• S1: 5 MW Geothermal Plant

• Investment:17M€

• NPV: 30M€

• S2: wind

• Investment:15M€

• NPV: 25M€

•Constraints:

• Budget 20 M €

• S1 or S2

1 - Branching

2 - Bounding

3 – Fathoming

1- Branching

2- Bounding

3 - Fathoming

PAPER DISCUSSION OPTIMIZATIONENERGYPLANNINGCO2 EMISSIONS -MILP

Nonlinear Programming

Carlos A. Santos Silva

April 30th, 2010

LINEAR PROGRAMMING FLAWS

Some times linear assumptions are not realistic

•Cost function is not linear

• Elastic or inelastic prices: %variation on

demand quantity is different from

%variation in price;• Price P(x)=xp(x)-cx, where p(x) is profit per unit and c

are fixed costs

• Marginal costs: production costs vary

depending on the quantity• Learning curves

• Volume Discounts

n

j

nnnjj cxxpxcxxpxxPxf1

111 )(...)()()(

m

i

n

j

ijij xCxf1 1

)()(

Some times linear assumptions are not realistic

•Cost function and/or constraint is not linear

•Risk securities: cost function includes not only expected return but also risk

• Let μ be the expected return and σ be some risk measurement based in variance

•Revenue

•Risk

•Optimization

n

j

jj xxR1

n

i

n

j

jiij xxxV1 1

L

xV

xR Constraint

Minimize

Graphical Representation of the Problem - I

Optimal solutions can be anywhere (not only on Corner Feasible Solutions)

• The search space is now infinite!

Graphical Representation of the Problem - II

There are local optima and global optimum

• In general it is difficult to know if optimal is local or global

• In general, when a local optimum is found, it is difficult to find another solution

Local optima

Global optimum

Integer Programming using M atlab

•Binary Integer Programming

• Bintprog(f,A,b,Aeq,Beq,x0,options)

• Uses branch and bound

Mix-Integer Programming or Integer Programming

• No Mathworks code

• Lots of functions in Matlab Central

• http://www.mathworks.com/matlabcentral/fileexchange/6990

• Linear programming + branch and bound

OPTIMIZATION SOFTWARE

NONLINEAR PROGRAMMING

Nonlinear optimization problem

•General form

• f and g can be any continuous function

•Characterization of minima/maxima

• Convex and concave functions

• Convex Sets

•Computation by iterative

algorithms

• Unconstrained optimization 1-D

• Bisection method

• Newton method

• Unconstrained optimization n-D

• Gradient search

• Newton method

•Other problems• Quadratic programming

• Convex programming

• Nonconvex

mib

xxxxxf

i

n

,...,2,1,xg subject to

),...,,(, Minimize 21

Concave and convex functions

•Concave (convex downward)

•Convex (convex upward)

02

2

x

f

02

2

x

f

Convex Sets

•The feasible region of all LP problems is a convex set

•In general, the feasible region of a NLP problem is a convex set if all

g(x) are convex functions

NLP problems global optimality conditions

•If objective function is concave and there is no constraints or

•If objective function is concave and constraints are convex set

• Local maxima is global

•If objective function is convex and there is no constraints or

•If objective function is convex and constraints are convex set

• Local minima is global

Verifies!Does not verify!

NLP problems and algorithms

•Unconstrained optimization

• No constraints

• Search procedures•Linearly Constrained optimization

• All constraints are linear

• Simplex method extension

•Quadratic programming

• Quadratic cost function and linear constraints

• Simplex method extension

•Convex Programming

• Assurance of global optimum

• Generalization of unconstrained optimization

•Non convex optimization

• All other problems

• No algorithm to find an optimal solution

UNCONSTRAINED

PROGRAMMING (1-D)

Algorithms

Analytical solution

• Not always feasible

Bisection method

1. Select a point with positive

derivative x’

2. Select a point with negative

derivative x’’

3. Consider medium point x

4. Evaluate derivative of new point1. If positive, x’ =x

2. If negative, x’’=x

5. Go to step 3 until x’-x’’<ε

Newton method

1. Select initial solution x’

2. Calculate x’’=x’-f’/f’’

3. Set x’=x’’

4. Go to 2 until x’’-x’< ε

• Newton is faster than Bisection

• Both methods depend on starting points

• Both methods may be trapped into local solutions

UNCONSTRAINED

PROGRAMMING (N-D)

Gradient Search /Steepest Descent (Hill Climbing)

1. Find an initial solution x’

2. Calculate x=x’+t ∇f(x’), where t is a certain step

3. Set x’=x

4. Go to 2 or stop if x’-x<ε

Quasi-Newton / Conjugate Gradient

•Newton can be extended to

multidimensional space:

• Second derivative corresponds to

Hessian matrix

• M ethod requires the use of the inverse

Hessian, which may not be invertible

• In that case it is necessary to

approximate (quasi) Hessian matrix

•Conjugate combines different

gradient directions

• It is faster than gradient search

PAPER #5“Optimized model for community –based hybrid energy system

Isolated energy supply

•Problem

• Find best energy mix (size) that minimizes total life cycle costs

•Domain• Solar PV

• Wind

• M icro-hydro

• Batteries

• Diesel generator

Constraints

Efficiency coefficients

Costs

NLP M odel – Size optimization

•Capital Cost

•Power Contraint

•Other Constraints

NLP M odel – Operation cost minimization

•Minimize annual operating cost :

•where total annualized capital cost is :

•and unit cost of electricity is

Optimization M odel

Hourly load and renewable resources availability

Final Results

EXAMPLE

From class 2007-2008

•Problem

• Electricity cost minimization in house considering consumer changes

•Domain

• Passive measurements (insulations X1 and X2 or windows X3 and X4)

• Active measurements (change appliances X5 to X18 and microgeneration capacity X20)

Constraints

Efficiency coefficients

Budgets

Consumption patters

Electricity Price

M icrogeneration availability

Subsidy

Domestic Consumption in Portugal

Scenarios

NLP Model

Results

LINEAR OR NONLINEAR?

Comparison

•Linear

• Optimal solution

•Less accurate model

•Analytical sensitivity analysis

•Nonlinear

•Suboptimal

• It is possible to verify if it is

optimal

•M ore accurate

• Much more accurate

• Iterative sensitivity analysis

OPTIMIZATION SOFTWARE

Nonlinear Optimization using Solver

•Generalized Reduced Gradient (GRG2) nonlinear optimization code …

Nonlinear Optimization using M atlab

•fminsearch(fun,x0,options)

• finds the minimum of a scalar function of several variables, starting at an initial

estimate. This is generally referred to as unconstrained nonlinear optimization

fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,options)

• attempts to find a constrained minimum of a scalar function of several variables

starting at an initial estimate.

•fminbnd(fun,X1,X2)

• attempts to find a local minimize X of the function FUN in the interval x1 < X < x2