final examychen/cse543/notes/lecture 10.pdf · 2018. 10. 30. · final exam • basic concepts...

Final Exam

• Basic concepts

– Definitions (uncon/con, local/global min, nec/suff. cond, convexity, regularity, active constraints, exact/inexact, Lagrange, dual, ...)

• Theory

– Understand the assumptions and scopes

– Be able to apply the results to toy examples

• Algorithms

– Understand the assumptions and scopes

– Be able to calculate an iteration on toy examples

– Qualitative comparison (pros and cons)

• Geometry

– Understand the mechanism behind the theory/algorithms

– Can give intuitive rationale using geometrical illustrations

Final Exam (cont’d)

• Take-home exam

• Materials to cover:– Related sections in the textbook

• Definitions, theorems, algorithms that are covered in class

• Sections 1.1, 1.2, 1.4, 1.6, 1.7, 3.1, 3.2, 3.3, 3.4, 4.2, 5.1, 5.5– Lecture notes

• Examples in the lecture notes

– Homework problems

• No need:– Detailed mathematical proof, convergence analysis

– Sections/terms not mentioned in class/lecture notes

• Since duality does not require continuity, it

provides a tool for discrete optimization

Discrete Optimization

• Discrete optimization is very popular

– Task planning and processor scheduling

– Assignment and facility location problems

– Traveling Salesman problems

– Network routing and design

– Set packing, covering, partitioning

– Circuit design

(see Section 5.5.1 for examples)

Discrete Optimization

“Yes or no” decisions encoded with binary variables:

1 if decision is yesxj

0 if decision is no.

Binary Integer Programming (BIP):• Binary variables + linear constraints.

Integer Programming for Decision Making

Problem:

1. Cal wants to expand:• Build new factory in Los Angeles, San Francisco, or both.• Build new warehouse (only one).• Warehouse must be built close to city of a new factory.

2. Available capital: $10,000,000

3. Cal wants to maximize “total net present value” (profitability vs. time value of money)

NPV Price1 Build a factory in L.A.? $9m $6m2 Build a factory in S.F.? $5m $3m3 Build a warehouse in L.A.? $6m $5m4 Build a warehouse in S.F.? $4m $2m

Binary Integer Programming Example:Cal Aircraft Manufacturing Company

What decisions are to be made?

1. Build factory in LA2. Build factory in SFO3. Build warehouse in LA4. Build warehouse in SFO

1 if decision i is yesIntroduce 4 binary variables xi=

0 if decision i is no


1. Cal wants to expand (TBD)2. Available capital: $10,000,0003. Cal wants to maximize “total net present value” (profitability vs. time value of money)

NPV Price1 Build a factory in L.A.? $9m $6m2 Build a factory in S.F.? $5m $3m3 Build a warehouse in L.A.? $6m $5m4 Build a warehouse in S.F.? $4m $2m

What is the objective?• Maximize NPV:

Z = 9x1 + 5x2 + 6x3 + 4x4

Remaining constraints?• Don’t go beyond means:

6x1 + 3x2 + 5x3 + 2x4

LA factory(x1), SFO factory(x2), LA warehouse(x3),SFO warehouse (x4)

• Build new factory in Los Angeles, San Francisco, or both.• Build new warehouse (only one).• Warehouse must be built close to city of a new factory.

What are the constraints between decisions?1. Only one warehouse:

x3 exclusive-or x4 x3 + x4 < 1

2. Warehouse in LA only if Factory is in LA:x3 implies x1x3 – x1 < 0

3. Warehouse in SFO only if Factory is in SFO:x4 implies x2x4 - x2 < 0


• Mutually exclusive choices

• Example: at most 2 decisions in a group can be yes:

LP Encoding:x1 +…+ xk < 2.

Encoding Choices: An Example

LA factory(x1), SFO factory(x2), LA warehouse(x3),SFO warehouse(x4)

• Build new factory in Los Angeles, San Francisco, or both.• Build new warehouse (only one).• Warehouse must be built close to city of a new factory.

What are the constraints between decisions?1. Only one warehouse:

x3 exclusive-or x4x3 + x4 < 1

2. Warehouse in LA only if Factory is in LA:x3 implies x1x3 – x1 < 0

3. Warehouse in SFO only if Factory is in SFO:x4 implies x2x4 - x2 < 0


Complete binary integer program:

Maximize Z = 9x1 + 5x2 + 6x3 + 4x4

Subject to: 6x1 + 3x2 + 5x3 + 2x4 0


Branch and Bound

• An global optimal method that enumerates the variable space by a tree search

Bounding Principle

The B&B Procedure

Lower Bounding Methods

• Getting tight lower bound is the key to theperformance of B&B

• Relaxation methods

– Constraint relaxation: relax the requirement ofsome constraints (thus the problem has largerfeasible region and lower minimum value)

– Lagrangian relaxation: using the weak dualtheorem (solve the dual problem for certainLagrangian multipliers)

Constraint Relaxation

• Most popular: relax the integrality requirements

for integer variables

– E.g. x ∈ {0,1} relaxed to x ∈ [0,1]

• Example problem, solution is -21 at (0,1,1,1) but if

we relax x to [0,1], solution is -22 at (1,1,0.5,0)

Minimize - 8x1 - 11x2 - 6x3 - 4x4

Subject to 5x1 + 7x2 + 4x3 + 3x4

B&B for Binary Integer Programs (BIPs)

Problem: Optimize f(x) st A(x) 0, xi {0,1}, x D

Domain Di encoding:

• partial assignment to x,– {x1 = 1, x2 = 0, …}

Branch Step:1. Find variable xj that is unassigned in Di2. Create two subproblems by splitting Di:

• Di1 Di {xj = 1}

• Di0 Di {xj = 0}

Example: B&B for BIPsSolve:min Z = -9x1 - 5x2 - 6x3 - 4x4Subject to:

– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -x1 + x3 0

– -x2 + x4 0

– xi 1, xi 0, xi integer

Queue: {} Incumbent: none Best cost Z*: inf

• Initialize

{}

Example: B&B for BIPsSolve:min -(9x1 + 5x2 + 6x3 + 4x4)Subject to:

– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -x1 + x3 0

– -x2 + x4 0


Queue: {} Incumbent: none Best cost Z*: inf

• Dequeue {}

{}

Example: B&B for BIPsSolve:min -(9x1 + 5x2 + 6x3 + 4x4) Subject to:

– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -x1 + x3 0

– -x2 + x4 0


Queue: Incumbent: none Best cost Z*: inf

• Bound {}• Constrain xi by {}• Relax to LP• Solve

Z = -16.5, x =

{}


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -x1 + x3 0

– -x2 + x4 0


Queue: Incumbent: none Best cost Z*: inf

• Try to fathom:• infeasible?• worse than incumbent?• integer solution?

Z = -16.5, x =

{}


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -x1 + x3 0

– -x2 + x4 0


Queue:

Incumbent: none Best cost Z*: inf

• Branch:• select unassigned xi

• pick non-integer• Split on xi

Z = -16.5, x =

{x1 = 0}{x1 = 1}

{}

{x1 = 0} {x1 = 1}


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -x1 + x3 0

– -x2 + x4 0


Queue:

Incumbent: none Best cost Z*: inf

• dequeue:• depth first or• best first

Z = 16.5, x =

{x1 = 0}{x1 = 1}

{}

{x1 = 0} {x1 = 1}


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -x1 + x3 0

– -x2 + x4 0


Queue:{x1 = 1} Incumbent: none Best cost Z*: inf

• Bound {x1 = 0}• constrain x by {x1 = 0}• relax to LP• solve

{}

{x1 = 0} {x1 = 1}

Z = 16.5, x =

Example: B&B for BIPsSolve:min -(9 0 + 5x2 + 6x3 + 4x4) Subject to:

– 6 0 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -0 + x3 0

– -x2 + x4 0




{}

{x1 = 0} {x1 = 1}

Z = 16.5, x =

Example: B&B for BIPsSolve:min 9 0 + --5x2 - 6x3 - 4x4Subject to:

– 6 0 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -0 + x3 0

– -x2 + x4 0




{}

{x1 = 0} {x1 = 1}

Z = -9,5, x =

Example: B&B for BIPsSolve:min 9 0 + -5x2 - 6x3 - 4x4Subject to:

– 6 0 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -0 + x3 0

– -x2 + x4 0



{}

{x1 = 0} {x1 = 1}

Z = -9,5, x =


Example: B&B for BIPsSolve:min 9 0 + -5x2 - 6x3 - 4x4Subject to:

– 6 0 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -0 + x3 0

– -x2 + x4 0


Queue:{x1 = 1} Incumbent: x = Best cost Z*: -9

{}

{x1 = 0} {x1 = 1}

Z = 1-9,5, x =



– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -x1 + x3 0

– -x2 + x4 0



{}

{x1 = 0} {x1 = 1}

Z = 19,5, x =

• Dequeue and bound


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -x2 + x4 0


Queue:Incumbent: x = Best cost Z*: -9

{}

{x1 = 0} {x1 = 1}

Z = -16,2, x =

• Dequeue and bound


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -x2 + x4 0



{}

{x1 = 0} {x1 = 1}

Z = -16.2, x =



– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -x2 + x4 0


Queue:{x2 = 1}{x2 = 0} Incumbent: x = Best cost Z*: -9

{}

{x1 = 0} {x1 = 1}

Z = -16.2, x =

• Branch• Dequeue

{x2 = 1} {x2 = 0}


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -x2 + x4 0



{}

{x1 = 0} {x1 = 1}

Z = 16.2, x =

• Bound

{x2 = 1} {x2 = 0}


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -12 + x4 0



{}

{x1 = 0} {x1 = 1}

Z = -16, x =

• Bound

{x2 = 1} {x2 = 0}



– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -12 + x4 0


Queue:

Incumbent: x = Best cost Z*: -9

{x2 = 0}

{}

{x1 = 0} {x1 = 1}

Z = -16, x =

• Branch

{x2 = 1} {x2 = 0}

{x3 = 1} {x3 = 0}

{x3 = 1} {x3 = 0}{x2 = 0}


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -12 + x4 0


Queue:


{}

{x1 = 0} {x1 = 1}

Z = 16, x =

• Bound

{x2 = 1} {x2 = 0}

{x3 = 1} {x3 = 0}

{x3 = 1} {x3 = 0}{x2 = 0}


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -12 + x4 0



{}

{x1 = 0} {x1 = 1}

Z = 16, x =

• Bound

{x2 = 1} {x2 = 0}

{x3 = 1} {x3 = 0}


– 6x1 + 3x2 + 5x3 + 2x4 10

– 13 + x4 1

– -11 + 13 0

– -12 + x4 0



{}

{x1 = 0} {x1 = 1}

• Bound

{x2 = 1} {x2 = 0}

{x3 = 1} {x3 = 0}

• Try to fathom:• infeasible?

Z = 16, x =


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -12 + x4 0



{}

{x1 = 0} {x1 = 1}

{x2 = 1} {x2 = 0}

{x3 = 1} {x3 = 0}

• Dequeue & bound

Z = 16, x =


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -12 + x4 0



{}

{x1 = 0} {x1 = 1}

{x2 = 1} {x2 = 0}

{x3 = 1} {x3 = 0}

• dequeue & bound

Z = -16, x =


Example: B&B for BIPsSolve:min -(9x1 + 5)x2 + 6x3 + 4x4 Subject to:

– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -12 + x4 0


Queue:


{}

{x1 = 0} {x1 = 1}

{x2 = 1} {x2 = 0}

{x3 = 1} {x3 = 0}

{x4 = 0} {x4 = 1}{x2 = 0}• dequeue & bound

Z = -14, x =


{x4 = 0} {x4 = 1}

Example: B&B for BIPsSolve:min -(9x1 + 5)x)2 + 6x3 + 4x4 Subject to:

– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -12 + x4 0


Queue:

Incumbent: x =

{}

{x1 = 0} {x1 = 1}

{x2 = 1} {x2 = 0}

{x3 = 1} {x3 = 0}

• dequeue & bound

Z = -14, x =


{x4 = 0} {x4 = 1}

Incumbent: x = Best cost Z*: Best cost Z* -14

{x4 = 1}{x2 = 0}

Example: B&B for BIPsSolve:min -(9x1 + 5x2 + 6x3 + 4 ) x4 Subject to:

– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + 14 1

– -11 + x3 0

– -12 + 14 0


Queue:

Incumbent: x =

{}

{x1 = 0} {x1 = 1}

{x2 = 1} {x2 = 0}

{x3 = 1} {x3 = 0}

• dequeue & bound

Z = -18, x =


{x4 = 0} {x4 = 1}

Incumbent: x = Best cost Z*: Best cost Z*: -14

{x4 = 1}{x2 = 0}


– 6x1 + 3x2 + 5x3 + 2x4 10

– x3 + x4 1

– -11 + x3 0

– -12 + x4 0


Queue:

Incumbent: x =

{}

{x1 = 0} {x1 = 1}

{x2 = 1} {x2 = 0}

{x3 = 1} {x3 = 0}

• dequeue & bound

Z = -13.8, x =


{x4 = 0} {x4 = 1}

Incumbent: x = Best cost Z*: Best cost Z*:-14

{x2 = 0}

Lagrangian Relaxation

Remarks

• The dual should be considerably easy to solve

– Usually for problems with special structures, such as

separable functions

• Can dualize only a subset of the constraints

instead all of them

• It is not necessary to optimally maximize q(μ)

– But a tighter lower bound is desirable

An Example of Lagrangian Relaxation

• Minimize -20x1- 30x2 + 550y1 + 720y2S.t. x1 – 200y1 ≤ 0, x2 – 75y2 ≤ 0

1.5x1 + 4 x2 ≤ 300

x1, x2 ≥ 0, y1, y2 = 0 or 1 (f* = -3450)

• A Lagrangian relaxation

Minimize g(x,y,μ) = -20x1- 30x2 + 550y1 + 720y2 + μ1(x1 –200y1) + μ2(x2 – 75y2 )

s.t. 1.5x1 + 4 x2 ≤ 300, x1, x2 ≥ 0, y1, y2 = 0 or 1

– When μ1=0, μ2 = 1, g is minimized at (200,0,0,0), g = -4000 (a very loose (bad) lower bound)

– When μ1=3, μ2 = 3, g is minimized at (200,0,1,0), g = -3450 (a tight lower bound)

Comparison of Constraint Relaxation

and Lagrangian Relaxation

• Lagrangian relaxation can eliminate constraints

• For convex-cost linear-constraint problems, it can

be shown that Lagrangian relaxation is always

better/same as constraint relaxation

• However, the minimization of dual may be

difficult

• Lagrangian relaxation may generate very bad

bounds for nonconvex problems

final examychen/cse543/notes/lecture 10.pdf · 2018. 10. 30. · final exam • basic concepts...

Documents