c&o 355 lecture 9
DESCRIPTION
C&O 355 Lecture 9. N. Harvey http://www.math.uwaterloo.ca/~harvey/. TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box .: A A A A A A A A A A. Outline. Complementary Slackness “Crash Course” in Computational Complexity Review of Geometry & Linear Algebra - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/1.jpg)
C&O 355Lecture 9
N. Harveyhttp://www.math.uwaterloo.ca/~harvey/
![Page 2: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/2.jpg)
Outline
• Complementary Slackness• “Crash Course” in Computational Complexity• Review of Geometry & Linear Algebra• Ellipsoids
![Page 3: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/3.jpg)
Duality: Geometric View• We can “generate” a new constraint aligned with c by
taking a conic combination (non-negative linear combination)
of constraints tight at x.• What if we use constraints not tight at x?
x1
x2
x1 + 6x2 · 15
Objective Function c
x-x1+x2· 1
![Page 4: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/4.jpg)
Duality: Geometric View• We can “generate” a new constraint aligned with c by
taking a conic combination (non-negative linear combination)
of constraints tight at x.• What if we use constraints not tight at x?
x1
x2-x1+x2· 1
x1 + 6x2 · 15
Objective Function c
x Doesn’t prove x is optimal!
![Page 5: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/5.jpg)
Duality: Geometric View• What if we use constraints not tight at x?• This linear combination is a feasible dual solution,
but not an optimal dual solution• Complementary Slackness: To get an optimal dual
solution, must only use constraints tight at x.
x1
x2-x1+x2· 1
x1 + 6x2 · 15
Objective Function c
x Doesn’t prove x is optimal!
![Page 6: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/6.jpg)
Weak DualityPrimal LP Dual LP
Theorem: “Weak Duality Theorem”If x feasible for Primal and y feasible for Dual then cTx · bTy.Proof: cT x = (AT y)T x = yT A x · yT b. ¥
Since y¸0 and Ax·b
![Page 7: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/7.jpg)
Weak DualityPrimal LP Dual LP
Corollary:If x and y both feasible and cTx=bTy then x and y are both optimal.
Theorem: “Weak Duality Theorem”If x feasible for Primal and y feasible for Dual then cTx·bTy.Proof:
When does equality hold here?
![Page 8: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/8.jpg)
Weak DualityPrimal LP Dual LP
Theorem: “Weak Duality Theorem”If x feasible for Primal and y feasible for Dual then cTx·bTy.Proof:
Equality holds for ith term if either yi=0 or
When does equality hold here?
![Page 9: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/9.jpg)
Weak DualityPrimal LP Dual LP
Theorem: “Weak Duality Theorem”If x feasible for Primal and y feasible for Dual then cTx·bTy.Proof:
Theorem: “Complementary Slackness”Suppose x feasible for Primal, y feasible for dual, and for every i, either yi=0 or .Then x and y are both optimal.Proof: Equality holds here. ¥
![Page 10: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/10.jpg)
General ComplementarySlackness Conditions
Primal Dual
Objective max cTx min bTyVariables x1, …, xn y1,…, ym
Constraint matrix A AT
Right-hand vector b cConstraintsversusVariables
ith constraint: · ith constraint: ¸ith constraint: =
xj ¸ 0xj · 0xj unrestricted
yi ¸ 0yi · 0yi unrestricted
jth constraint: ¸jth constraint: ·jth constraint: =
for all i,equality holds either
for primal or dual
for all j,equality holds either
for primal or dual
Let x be feasible for primal and y be feasible for dual.
and
,x and y are
both optimal
![Page 11: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/11.jpg)
Example• Primal LP
• Challenge: What is the dual?• What are CS conditions?
• Claim: Optimal primal solution is (3,0,5/3).Can you prove it?
![Page 12: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/12.jpg)
Example
• CS conditions:– Either x1+2x2+3x3=8 or y2=0– Either 4x2+5x3=2 or y3=0– Either y1+2y+2+4y3=6 or x2=0– Either 3y2+5y3=-1 or x3=0
• x=(3,0,5/3) ) y must satisfy:• y1+y2=5 y3=0 y2+5y3=-1) y = (16/3, -1/3, 0)
Primal LP Dual LP
![Page 13: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/13.jpg)
Complementary Slackness Summary
• Gives “optimality conditions” that must be satisfied by optimal primal and dual solutions
• (Sometimes) gives useful way to compute optimum dual from optimum primal– More about this in Assignment 3
• Extremely useful in “primal-dual algorithms”.Much more of this in– C&O 351: Network Flows– C&O 450/650: Combinatorial Optimization– C&O 754: Approximation Algorithms
![Page 14: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/14.jpg)
We’ve now finished C&O 350!
• Actually, they will cover 2 topics that we won’t– Revised Simplex Method: A faster implementation
of the algorithm we described– Sensitivity Analysis: If we change c or b, how does
optimum solution change?
![Page 15: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/15.jpg)
Computational Complexity• Field that seeks to understand how efficiently
computational problems can be solved– See CS 360, 365, 466, 764…
• What does “efficiently” mean?– If problem input has size n, how much time to
compute solution? (as a function of n)– Problem can be solved “efficiently” if it can be
solved in time ·nc, for some constant c.– P = class of problems that can be solved efficiently.
![Page 16: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/16.jpg)
Computational Complexity• P = class of problems that can be solved efficiently
i.e., solved in time ·nc, for some constant c, where n=input size.• Related topic: certificates• Instead of studying efficiency, study how easily can
you certify the answer?• NP = class of problems for which you can efficiently
certify that “answer is yes”• coNP = class of problems for which you can efficiently
certify that “answer is no”– Linear Programs• Can certify that optimal value is large• Can certify that optimal value is small
(by giving primal solution x)
(by giving dual solution y)
![Page 17: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/17.jpg)
Computational Complexity
• Open Problem: Is P=NP?• Probably not• One of the 7 most important problems in mathematics• You win $1,000,000 if you solve it.
NP coNPCan graph be coloredwith ¸ k colors? NPÅcoNP
Does every coloringof graph use · k colors?
PSorting, string matching, breadth-first search, …
Is LP value ¸ k?Is m a prime number?
Manyinterestingproblems
Manyinterestingproblems
![Page 18: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/18.jpg)
Computational Complexity
• Open Problem: Is P=NPÅcoNP?• Maybe… for most problems in NPÅcoNP, they are also in P.• “Is m a prime number?”. Proven in P in 2002. “Primes in P”• “Is LP value ¸ k?”. Proven in P in 1979. “Ellipsoid method”
P
NP coNP
Sorting, string matching, breadth-first search, …
Can graph be coloredwith ¸ k colors? NPÅcoNP
Is LP value ¸ k?Is m a prime number?
Manyinterestingproblems
Manyinterestingproblems
Leonid Khachiyan
Does every coloringof graph use · k colors?
![Page 19: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/19.jpg)
Review
• Basics of Euclidean geometry– Euclidean norm, unit ball, affine maps, volume
• Positive Semi-Definite Matrices– Square roots
• Ellipsoids• Rank-1 Updates• Covering Hemispheres by Ellipsoids
![Page 20: C&O 355 Lecture 9](https://reader035.vdocuments.us/reader035/viewer/2022081420/5681621e550346895dd24639/html5/thumbnails/20.jpg)
2D Example
Unit ball B
Ellipsoid T(B)
De neT(x) = Ax+bwhereA =µ3 20 1
¶and b=
µ11¶.