university of massachusetts at amherst 665 computer algorithm mathematical programming professor...
Post on 22-Dec-2015
220 views
TRANSCRIPT
![Page 1: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/1.jpg)
University of Massachusetts at Amherst
665 Computer Algorithm Mathematical Programming
Professor Maciej CiesielskiSpring 2004
(DFG)
![Page 2: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/2.jpg)
OUTLINE
1. Non Linear Programming2. Quadratic Programming3. Linear Programming
A. Geometric SolutionsB. Basic SolutionsC. Simplex MethodD. LP modifications
4. Integer ProgrammingA. UnimodularityB. Network Flow Problems
5. Conclusions
![Page 3: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/3.jpg)
Mathematical Programming Nonlinear Programming
Minimize f(x) subject to:
g(x) = 0h(x) ≥ 0
where x = [x1,...xn]T, and all functions f, g, h, are differentiable.
{
![Page 4: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/4.jpg)
Convexity
To obtain global optimum, it is important that the constraint set
{x | g(x) = 0,h(x) ≥ 0}
and the function f(x) be convex.
If the function is convex over a convex constraint set, then local minimun is also a globa1 minimum
![Page 5: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/5.jpg)
Set convexity
Convex set non-convex set
Function convexity
Convex function non-convex function
![Page 6: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/6.jpg)
Classical optimization deals with unconstrained optimization problems.
Solution: move the constraints to the function and solve the unconstrained optimization problem
For the minimization problem
Min f(x) subject to g(x) = 0
1. Penalty function method: replace f(x) by
F(x) = f(x)+ M( g(x))2, M >> 0
2. Lagrangian multipliers: replace f(x) by
L(x,λ) = f(x) + Σ λi gi(x)
At the optimum point, x*, L(x*, λ) = f(x*)
i
![Page 7: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/7.jpg)
Quadratic Programming (QP)
Minimize xTQx
Subject to:
{Ax ≥ b
Where Q is an n x n symmetric matrix, A is a constraint matrix, and b is a constraint vector
Matrix Q is positive definite, i.e.,
xTQx > 0
For every x 0. Thus f is convex.
![Page 8: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/8.jpg)
Linear Programming (LP)
Minimize cT
Subject to
Ax ≥ b
x ≥ 0
Where cT is a coefficient vector for f, A is a constraint matrix, and b es a constraint vector.
The constraint set
{x | Ax ≥ b}
Is a convex polyhedron, and f is linear, convex.
{
![Page 9: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/9.jpg)
Linear Programming (LP)Linear Programming (LP)
Optimization problem:Optimization problem: • 2 types of products are made by the factory• 2 types of products are made by the factory• 3 machines are needed to make each • 3 machines are needed to make each
productproduct ConstraintsConstraints:: Machines are available for a Machines are available for a limited time during one production session:limited time during one production session: •• time on time on A A ≤≤ 50 50 hourshours • • time on time on B B ≤≤ 35 35 hourshours • • time on time on C C ≤≤ 80 hours 80 hours
![Page 10: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/10.jpg)
To complete the product, each lot must be processed by all three machines for a certain number of hours Required manufacturing time
Machines A B C 1
2
Time Constraint: 50 35 80
10 5 5
5 5 15
![Page 11: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/11.jpg)
Profit• Product 1 can be sold for $100 per lot• Product 2 can be sold for $80 per lot Optimization Problem: Design the production schedule which rnaximizes the profit of both products manufactured during one production session.
![Page 12: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/12.jpg)
Formulation x1 = number of lots of product 1
x2 = number of lots of product 2
max f (x) = 100x1 + 80x2
10x1 + 5x2 ≤≤ 505x1 + 5x2 ≤≤ 355x1 + 15x2 ≤≤ 80
{
![Page 13: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/13.jpg)
Convert the problem to a standard LinearProgram (LP):
Max f(x) = 100x1 + 8Ox2
10x1 + 5x2+ s1 = 50
s. to 5x1 + 5x2+ s2 = 35
5x1 + 15x2+ s3 = 80
Basis
New Matrix A
{
![Page 14: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/14.jpg)
Geometric solutionGeometric solution
Family of curves, constant for Family of curves, constant for xx11,, xx22,,
F(x) = 100 xF(x) = 100 x1,1, ++ 80 x80 x22 = 800 = 800
(const)(const)
(a)(a)10x10x1,1, ++ 5x5x22 50 50
(b)(b) 5x 5x1, 1, + 5x+ 5x22 35 35
(c)(c) 5x 5x1, 1, + 15x+ 15x22 80 80
a bxx11
xx22
15
10
5
15105 c
P2
P13
4
P0
F(x) = 800F(x) = 800
![Page 15: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/15.jpg)
Basic Solution Given a system of equalities
Ax=b where x = n - vector, b = m - vector, A = m x n matrix, select a set of m linearly independent columns such that (m x m) matrix B is nonsingular, i.e. /B / 0. Then one may uniquely solve the equation
Bxb= bwhere Xb is an m - element subvector of x.
![Page 16: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/16.jpg)
By putting x=(xb,0) we obtain a solution to
Ax = b Such a solution (with n - m components of x not associated with columns of B m x m ) to the resulting set of equations is said to be a basic solution with regard to the basis B. The Xb variables, associated with columns of B, are called basic variables.
Basic Feasible SolutionBasic Feasible Solution
If a feasible solution (i.e. the one which satisfies all contraints) is also basic, it is called a basic feasible solution.
![Page 17: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/17.jpg)
Fundamental Theorem of Linear Fundamental Theorem of Linear ProgrammingProgramming
Given an L.P. in standard formGiven an L.P. in standard form
min cmin cTTxx
Ax = bAx = b
x x 0 0
where where A A is is m m xx n n matrix of rank matrix of rank m m (i.e. (i.e. m < n m < n and and the the m m rows are linear independent),rows are linear independent),
s. to
If there is a feasible solution, there is a If there is a feasible solution, there is a basic feasible solutionbasic feasible solution
If there is an optimal feasible solution, there If there is an optimal feasible solution, there is an optimal basic feasible solutionis an optimal basic feasible solution
![Page 18: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/18.jpg)
This theorem shows that it is necessary only to This theorem shows that it is necessary only to consider consider basic feasible solutionsbasic feasible solutions when seeking an when seeking an optimal solution to a linear program, as the optimal solution to a linear program, as the optimal value is always achieved as such a optimal value is always achieved as such a solutionsolution
Ultimate goalUltimate goal::
find a basic feasible solution with a base find a basic feasible solution with a base B B composed of composed of originaloriginal variables only, and which is variables only, and which is optimum.optimum.
![Page 19: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/19.jpg)
Solution: Solution: xx1 1 = 0, x= 0, x2 2 = 0, s= 0, s1 1 = 50, s= 50, s2 2 = 35, s= 35, s3 3 = 80,= 80,
is is a basic feasible solution (point a basic feasible solution (point PP00 (0,0) (0,0)))
Function value:Function value: F(0, 0. 50, 35, 80) = 0F(0, 0. 50, 35, 80) = 0
ƒ (x) = 100 xƒ (x) = 100 x1,1, ++ 80 x80 x22 + + 0 s0 s1 1 ++ 0 s0 s2 2 ++ 0 s0 s3 3
1010 5 5 100100
55 5 5 010010
55 1515 001001
xx11
xx22
ss11
ss2,2,
ss33
5 05 0
3 53 5
8 08 0
=
basis
![Page 20: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/20.jpg)
Simplex MethodSimplex Method
Proceed from one basic feasible solution (extreme Proceed from one basic feasible solution (extreme point) to another, in such a way as to continually point) to another, in such a way as to continually decrease the value of the decrease the value of the f (x) f (x) until a minimum is until a minimum is reached. reached.
General commentsGeneral comments
It is easy to find initial basic feasible solutions It is easy to find initial basic feasible solutions with with slack variablesslack variables..
Finding initial basic solution is part of the Simplex Finding initial basic solution is part of the Simplex method (Lue 84).method (Lue 84).
![Page 21: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/21.jpg)
Simplex AlgorithmSimplex Algorithm
1.1. Select the column, such that the new resulting Select the column, such that the new resulting basic feasible solution will yield a basic feasible solution will yield a lower valuelower value to to f(x) f(x) than the previous one.than the previous one.
2.2. Select the pivot element in that column.Select the pivot element in that column.
By an elementary evaluation determineBy an elementary evaluation determine
• Which vector Which vector aajj should enter the basis (so that should enter the basis (so that f(x) f(x) is reduced), andis reduced), and
• Which vector should leave the basis. Which vector should leave the basis.
![Page 22: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/22.jpg)
Construct a Simplex Tableau, [Construct a Simplex Tableau, [A, b A, b ] appended by ] appended by a row at the top with cost coefficients and current a row at the top with cost coefficients and current cost.cost.
j =j = 1 2 3 4 5 0
rrjj 100 80 0 0 0 0
yy1j1j 10 5 1 0 0 50
yy2j2j 5 5 0 1 0 35
yy3j3j 5 15 0 0 1 80
yi1 yi2 yi3 yi4 yi5 yi0
Point P0 (0,0)
F(0,0,50,35,80) = 0
![Page 23: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/23.jpg)
Step 1Step 1.. Select Select columncolumn q q with with rrqq > 0. > 0.
((xxqq enters the basis)enters the basis)
q : rq : rqq = max r = max rii
Step 2.Step 2. Select Select pivot elementpivot element p p in column in column q q
Which minimizes ratio: Which minimizes ratio:
p :p : = min for positive = min for positive yyiqiq
yiq
yi0
ypq
yp0
yiq
yi0
i
![Page 24: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/24.jpg)
Step 3Step 3.. PivotPivot on element on element yypqpq and update the and update the
tableau (update all rows, including the tableau (update all rows, including the top one)top one)
• In row In row pp divide divide yypjpj by by yypqpq , , j = 0, 1,…, nj = 0, 1,…, n
• Subtract from each row Subtract from each row i i p p
•• yypjpj , , j = 0, 1,…, nj = 0, 1,…, n yp q
yi q
![Page 25: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/25.jpg)
Pivot on Pivot on yy1 11 1
j =j = 1 2 3 4 5 0
rrjj 100 80 0 0 0 0
yy1j1j 10 5 1 0 0 50
yy2j2j 5 5 0 1 0 35
yy3j3j 5 15 0 0 1 80
yi1 yi2 yi3 yi4 yi5 yi0
Update the tableUpdate the tablej =j = 1 2 3 4 5 0
rrjj 0 30 -10 0 0 -500
yy1j1j 1 0.5 0.1 0 0 5
yy2j2j 0 2.5 -0.5 1 0 10
yy3j3j 0 12.5 -0.5 0 1 55
yi1 yi2 yi3 yi4 yi5 yi0
Point P0 (0,0)F(0,0,50,35,80) = 0
Point P1 (5,0)F(5,0,0,10,55) = 500
![Page 26: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/26.jpg)
Pivot on Pivot on yy2 22 2
Update the tableUpdate the tablej =j = 1 2 3 4 5 0
rrjj 0 0 -4 -12 0 -620
yy1j1j 1 0 0.2 -0.2 0 3
yy2j2j 0 1 -0.2 0.4 0 4
yy3j3j 0 0 2 -5 1 5
yi1 yi2 yi3 yi4 yi5 yi0
Point P1 (3,4)F(3,4,0,0,5) = 620
j =j = 1 2 3 4 5 0
rrjj 0 30 -10 0 0 -500
yy1j1j 1 0.5 1 0 0 5
yy2j2j 0 2.5 -0.5 1 0 10
yy3j3j 0 12.5 -0.5 0 1 55
yi1 yi2 yi3 yi4 yi5 yi0
Point P1 (5,0)F(5,0,0,10,55) = 500
![Page 27: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/27.jpg)
LP ModificationsLP Modifications
1. Maximization1. Maximization
2. Unconstrained x2. Unconstrained x
max c max c t t x = min (-c x = min (-c t t x)x)
let x = xlet x = x++ - x - x -- ; x; x++, x , x -- 0) 0)
min (c min (c t t xx++ - c - c t t x x --))
[A, -A] = bxx++
x x --
xx++, x , x -- 0 0
s. tos. to
![Page 28: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/28.jpg)
3. Constraint set:3. Constraint set: Ax Ax b b
Slack variablesSlack variables,, xxss : A : Axx + x + xss = b = b
[A, I ] = bxx
xxss
x , xx , xs s 0 0
min [cT , 0]
xx
xxss
Columns corresponding to Columns corresponding to xxss form a basic form a basic solutionsolution
![Page 29: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/29.jpg)
4. Constraint set:4. Constraint set: Ax Ax b b
Surplus variablesSurplus variables,, xxpp : A : Ax x - x - xpp = b = b
[A, -I ] = bxx
xxpp
x , xx , xp p 0 0
min [cT , 0]
xx
xxpp
![Page 30: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/30.jpg)
Artificial variablesArtificial variables,, xxaa : A: Axx - x - xpp + x + xaa = b = b
(cannot star with negative basis of(cannot star with negative basis of xxp p ))
[A, -I , I ] = b
x , xx , xpp , , xxaa 0 0
min [cT , 0 , k I ]
xxxxpp
xxaa
xxxxpp
xxaaS. to
Columns corresponding to Columns corresponding to xxaa form a basic form a basic solutionsolution
![Page 31: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/31.jpg)
Integer ProgrammingInteger Programming
Integer ProgramInteger Program
max cmax cTTxx
Ax Ax b b
xxii integer integer
Typically, Typically, xxii = 0,1 = 0,1
(0,1 Integer Programming)(0,1 Integer Programming)
s. to
![Page 32: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/32.jpg)
IP ExampleIP Example: : matching problemmatching problem
Given a graph Given a graph G G , find maximal set of edges in , find maximal set of edges in G, G, such that no two edges are adjacent to the same such that no two edges are adjacent to the same vertex.vertex.• Maximum matching: matching of maximum Maximum matching: matching of maximum
cardinality.cardinality.
• Weighted matching: matching with Weighted matching: matching with
)(max iew
1 3
2e1 e2
e3
![Page 33: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/33.jpg)
otherwise
matchinginincludedisif
0
e1x i
i
1 3
2e1 e2
e3
321 xxx max
10x
1xx
1xx
1xx
tos
i
32
31
21
,
.
![Page 34: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/34.jpg)
In matrix form:In matrix form:
10x
bAxtos
i ,.
xcTmax
110
101
011
e
e
e
e.g.
3
2
1
A
Where Where b = b = [ 1,…1][ 1,…1]T T , , A = A = incidence matrix of incidence matrix of GG
1 2 3
![Page 35: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/35.jpg)
max max 1x1x11 + + 1x1x22 + + 1x1x33
1
1
1
110
101
011
s.to
3
2
1
x
x
x
One solution to this IP problem:One solution to this IP problem:
xx11 , , xx22 , , xx33 = 0, 1 = 0, 1
xx11 = 1 , x= 1 , x22 = 0 , x= 0 , x33 = 0= 0
Other possible solutions:Other possible solutions:
xx11 = 0 , x= 0 , x22 = 1 , x= 1 , x33 = 0, or x= 0, or x11 = 0 , x= 0 , x22 = 0 , x= 0 , x33 = = 1 1
![Page 36: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/36.jpg)
QuestionQuestion
Can the solution to this IP problem be obtained Can the solution to this IP problem be obtained by dropping the integrality constraint:by dropping the integrality constraint:
xxii = 0, 1 = 0, 1
And solving the LP problem instead?And solving the LP problem instead?
In our example, solution to the IP is In our example, solution to the IP is notnot obtainable from LP.obtainable from LP.
ReasonReason:: matrix matrix A A does not have certain property does not have certain property (total unimodularity) needed to guarantee integer (total unimodularity) needed to guarantee integer solutions.solutions.
![Page 37: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/37.jpg)
If If xxii = 0, 1 = 0, 1 is relaxed, such that is relaxed, such that xxii 0 , 0 , then then the solution to the associated LP problem is non – the solution to the associated LP problem is non – integer:integer:
2
1xxx 321
Reason: the Reason: the A A matrix matrix
110
101
011
Is not Is not totally unimodulartotally unimodular: |: |A| = - 2A| = - 2
![Page 38: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/38.jpg)
Unimodularity: LP -> IPUnimodularity: LP -> IP
Given a constraint set in standard formGiven a constraint set in standard form
where where A, b A, b are integerare integer
Partition Partition A = A = [[B/N B/N ]; ]; x = x = [[xxBB, x, xN N ]]
B B is nonsingular is nonsingular m m xx m m basis, basis, NN is non-basic is non-basic
0x
bAx
i
bNxBxx
xNBAx NB
N
B
/
![Page 39: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/39.jpg)
Basic solution is Basic solution is
bBx
0x1
B
N
In particular, when In particular, when B = I B = I and and B B -1-1 = = I thenI then
XXBB = = b b
a solution can be obtained by inspection (as in a solution can be obtained by inspection (as in the initial step of Simplex method).the initial step of Simplex method).
![Page 40: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/40.jpg)
Since Since xxBB = B = B –1 –1 bb with with xxNN = 0, b = 0, b integerinteger
A sufficient condition for a basic solution A sufficient condition for a basic solution xxBB to be to be integer is thatinteger is that
B B –1 –1 be an integer matrixbe an integer matrix
![Page 41: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/41.jpg)
UnimodularityUnimodularity
A square matrix A square matrix B B is called is called unimodularunimodular if if
D = |det B| = 1D = |det B| = 1
An integer matrix An integer matrix A A is is totally unimodulartotally unimodular if every if every square, nonsingular submatrix of square, nonsingular submatrix of A A is unimodular.is unimodular.
Equivalently: Equivalently: A A is totally unimodular if every is totally unimodular if every subdeterminant of subdeterminant of A A is 0, +1, or –1.is 0, +1, or –1.
![Page 42: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/42.jpg)
Recall that for Recall that for B B nonsingular: nonsingular: Bted
BB 1
Where Where B B ++ , , adjoint matrix adjoint matrix
= [ = [ i j i j cofactor of element cofactor of element a a i ji j in in det det A A ]]TT
B B ++ and and det B det B are integer if are integer if BB –1 –1 is integer is integer
![Page 43: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/43.jpg)
333231
232221
131211
bbb
bbb
bbb
B
T
211222111321231122132312
311232113113331113323312
312232213123332132233322
bbbbbbbbbbbb
bbbbbbbbbbbb
bbbbbbbbbbbb
B
)()()(
)()()(
)()()(
Cofactor of Cofactor of aai j i j :: Determinant obtained by Determinant obtained by omitting the iomitting the ithth row and the j row and the jthth column of column of AA and and then multiplying by (-1) then multiplying by (-1) i + ji + j . .
![Page 44: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/44.jpg)
For For B B unimodularunimodular, , B B –1–1 integerinteger::
If If A A is is totally unimodulartotally unimodular, every basis matrix , every basis matrix BB is is unimodular and every basic solutionunimodular and every basic solution
( x( xBB , , yyNN) = ( B ) = ( B –1–1 b, 0 ) b, 0 )
Is Is integerinteger..
In particular, the optimal solution is integerIn particular, the optimal solution is integer
![Page 45: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/45.jpg)
Theorem 1Theorem 1::
If If A A is totally unimodular then every basic is totally unimodular then every basic solution of solution of Ax = b Ax = b is is integerinteger..
For LP’s with equality constraints total For LP’s with equality constraints total unimodularity is sufficient but not necessary.unimodularity is sufficient but not necessary.
For LP with inequality constraints, For LP with inequality constraints, Ax Ax b b, , total unimodularity of total unimodularity of AA is both is both necessary necessary and sufficient and sufficient for all extreme points of for all extreme points of s = {x : Ax s = {x : Ax b , x b , x 0 } 0 } to be integer for to be integer for every integer vector every integer vector b.b.
![Page 46: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/46.jpg)
ConclusionConclusion::
Any IP with totally unimodular constraint Any IP with totally unimodular constraint matrix can be solved as an LP.matrix can be solved as an LP.
Totally unimodular matrix : Totally unimodular matrix : a a i j i j = 0 , +1, -1 = 0 , +1, -1
Also, every determinant of Also, every determinant of A A must be must be 0, +1 or 0, +1 or -1-1
![Page 47: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/47.jpg)
Special Linear Programs with Integer Special Linear Programs with Integer SolutionsSolutions
are LP with the property that they possess are LP with the property that they possess optimal solutions in integers.optimal solutions in integers.
• max flowmax flow
• min – cost flowmin – cost flow
• assignment problemassignment problem
• shortest pathshortest path
• transportation problemtransportation problem
Network flow problems:Network flow problems:
![Page 48: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/48.jpg)
Some Totally Unimodular Linear ProgramsSome Totally Unimodular Linear Programs
Assignment problemAssignment problem(special case of min – cost capacitated flow (special case of min – cost capacitated flow problems)problems)
[ [ m m jobs x jobs x m m men ]men ]
otherwise
jobtoassignedismanif
0
ji1x ji
c i j = cost of assigning man i to job j
![Page 49: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/49.jpg)
ijij
ij xcMax
s. to.s. to.
10,
)1(1,1
)1(1,1
1
1
ij
ij
m
j
ij
m
i
x
manperjobmix
jobpermanmjx
![Page 50: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/50.jpg)
1
1
1
1
1
1
X
111
1
1
1
111
1
1
1
111
1
1
1
X11 X12 X13 X21 X22 X23 X31 X32 X33
![Page 51: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/51.jpg)
xcTmaxIn matrix notation:In matrix notation:
Ax = 1 Ax = 1 wherewhere
11
0
1
1
0
11
0
1
1
0
11
1
1
A
m 2m m 2...
Exactly Exactly m m 1’s in each row1’s in each row
Exactly 2 1’s in each columnExactly 2 1’s in each column
![Page 52: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/52.jpg)
Network Flow ProblemsNetwork Flow Problems::
• a a set of originsset of origins VV11 ; each origin i ; each origin i V V11 ; ; supplies asupplies a11 of of
commodity.commodity.
• a a set of destinationsset of destinations VV22 ; ; each destinationeach destination j j V V22 ; ; has a has a
demand demand bbii of commodity.of commodity.
• costcost per unit commodity; per unit commodity; ccijij associated with sending associated with sending
commodity through (commodity through (i, j i, j ).).
Given:Given:
![Page 53: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/53.jpg)
Constraint SetConstraint Set: : (Totally unimodular)(Totally unimodular)
jiji
2i
1
1i
iΓij
ji
iΓji
ij
cx0
iΓVib
Vi0
iΓVia
xx
))((for
for
))((for
)(),()(),(
![Page 54: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/54.jpg)
nodendestinatiofor
nodesernalintfor
nodesourcefor
)(),()(),( f
0
f
xxiΓij
ji
iΓji
ij
Special case: for single source, single Special case: for single source, single destination, max flow problemdestination, max flow problem
in any case:in any case: Ax Ax b b
![Page 55: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/55.jpg)
No matter which problem it No matter which problem it isis
cutmin
flowmax
flowtcosmin
the general format is the general format is A’x A’x b b
It can be shown thatIt can be shown that• If If A A is totally unimodular then [is totally unimodular then [A/I A/I ] is also totally ] is also totally
unimodularunimodular• The transpose of totally unimodular matrix is also The transpose of totally unimodular matrix is also
totally unimodulartotally unimodular
• ..
I
AA'
where where A = A = incidence matrix of the corresponding incidence matrix of the corresponding digraph, totally unimodular. digraph, totally unimodular. I = I = identity matrix, identity matrix, A’ A’ is is totally unimodulartotally unimodular..
![Page 56: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/56.jpg)
• Exploit a Exploit a special structurespecial structure of the problem of the problem (total unimodularity, etc.) to obtain integer (total unimodularity, etc.) to obtain integer solutions by solving simpler problems.solutions by solving simpler problems.
• TransformTransform the problem to another problem the problem to another problem for which an approximate solution is easier for which an approximate solution is easier to find.to find.
• Once the structure of the problem is well Once the structure of the problem is well understood, understood, use heuristicuse heuristic, but… stay away , but… stay away from brute force approach.from brute force approach.
No efficient algorithms exist for general Integer No efficient algorithms exist for general Integer Programming problemsProgramming problems
![Page 57: University of Massachusetts at Amherst 665 Computer Algorithm Mathematical Programming Professor Maciej Ciesielski Spring 2004 (DFG)](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649d785503460f94a5b3cb/html5/thumbnails/57.jpg)
• Graph TheoryGraph Theory
• Mathematical OptimizationMathematical Optimization
A large number of CAD problems can be cast in A large number of CAD problems can be cast in analytical form:analytical form:
For some problems – efficient algorithms exist.For some problems – efficient algorithms exist.
For others – need to resort to For others – need to resort to heuristicheuristic, , suboptimal solutions.suboptimal solutions.
Some known successful heuristic approachesSome known successful heuristic approaches
• Simulated annealingSimulated annealing• LP roundingLP rounding
ConclusionsConclusions