lecture 5: conjugate duality...0. (ii) strong duality. 0 = f(x*) + h(y*) where is the duality...
TRANSCRIPT
LECTURE 5: CONJUGATE DUALITY 1. Primal problem and its conjugate dual 2. Duality theory and optimality conditions 3. Relation to other type of dual problems 4. Linear conic programming problems
Motivation of conjugate duality Min f(x) (over R) = Max g(y) (over R) and h(y) = - g(y) • f(x) + h(y) = f(x) – g(y) can be viewed as “duality gap” • Would like to have (i) weak duality 0 (ii) strong duality 0 = f(x*) + h(y*)
Where is the duality information • Recall the fundamental result of Fenchel’s conjugate inequality
• Need a structure such that in general
and at optimal solutions 0 = <x*, y*> = f(x*) + h(y*)
Concept of dual cone • Let X be a cone in
• Define its dual cone • Properties: (i) Y is a cone. (ii) Y is a convex set. (iii) Y is a closed set.
Observations
Conjugate (Geometric) duality
Dual side information • Conjugate dual function
• Dual cone
Properties: 1. Y is a cone in 2. Y is closed and convex 3. both are closed and convex.
Conjugate (Geometric) dual problem
Observations
Conjugate duality theory
Conjugate duality theory
Proof
Conjugate duality theory
Example – Standard form LP
Conjugate dual problem
Dual LP
Example – Karmarkar form LP
Example – Karmarkar form LP
Example – Karmarkar form LP • Conjugate dual problem becomes
which is an unconstrained convex programming problem.
Illustration
Example – Posynomial programming • Nonconvex programming problem
• Transformation
Posynomial programming • Primal problem: Conjugate dual problem:
Dual Posynomial Programming
h(y) , supx∈En[< x , y > −f (x)] < +∞
Let g(x) =∑n
j=1 xjyj − log∑n
j=1 cjexj
∂g∂xj
= 0 ⇒ yj =cje
x∗j∑nj=1 cje
x∗j
⇒ yj > 0 and∑n
j=1 yj = 1.⇒ Ω is closed.⇒ Ω = y ∈ En|yj > 0 and
∑nj=1 yj = 1.
log(yj) = log(cjex∗
j )− log∑n
j=1 cjex∗
j
= log(cj) + x∗j − log∑n
j=1 cjex∗
j
⇒ log(yjcj
) + log∑n
j=1 cjex∗
j = x∗j
h(y) = supx∈En [< x , y > −f (x)] = g(x∗)=
∑nj=1 x∗j yj − log
∑nj=1 cje
x∗j
=∑n
j=1 yj [log(yjcj
) + log∑n
j=1 cjex∗
j ]− log∑n
j=1 cjex∗
j
=∑n
j=1 yj log(yjcj
) +∑n
j=1 yj log∑n
j=1 cjex∗
j − log∑n
j=1 cjex∗
j
=∑n
j=1 yj log(yjcj
) + (∑n
j=1 yj)log∑n
j=1 cjex∗
j − log∑n
j=1 cjex∗
j
=∑n
j=1 yj log(yjcj
)
Conjugate dual problem
Degree of difficulties • When degree of difficulty = 0, we have a system of linear
equations:
• When degree of difficulty = k, we have
Duality gap • Definition:
• Observation:
Extremality conditions • Definition:
• Corollary:
Proof of Corollary
Necessary and sufficient conditions • Corollary
• Observation
When will the duality gap vanish?
Nonlinear complementarity problem
Lagrangian Function and Saddle Point
I Let f (x) : S⋂
X and h(y) : Ω⋂
Y be a known conjugatepair with X being closed and convex and f ∈ C′(S).
I The lagrangian function is defined as
L(x , y) , f (x)− < x , y > .
I A point pair (x , y) ∈ S × Y is called a saddle point ofL(x , y) if
L(x , y) > L(x , y) > L(x , y), ∀x ∈ S, y ∈ Y ,
or
infx∈S
L(x , y) = L(x , y) = supy∈Y
L(x , y).
Saddle Point and Optimality
Theorem: (saddle point⇒ Optimality)If (x , y) ∈ S × Y is a saddle point of L(x , y), then x ∈ S ∗ andy ∈ T ∗.Proof: By definition,
infx∈S
L(x , y) = L(x , y) = supy∈Y
L(x , y)
1L(x , y) = infx∈S L(x , y)
= infx∈S[f (x)− < x , y >]= − supx∈S[< x , y > −f (x)]
Hence y ∈ Ω and L(x , y) = −h(y), (now, y ∈ Y⋂
Ω = T )
2L(x , y) = supy∈Y L(x , y)
= supy∈Y [f (x)− < x , y >]
= f (x) + supy∈Y [− < x , y >]
= f (x)− infy∈Y [< x , y >]
Since Y is a cone, x ∈ dual(Y ) = X and infy∈Y [< x , y >] = 0.Hence L(x , y) = f (x). (Now, x ∈ S
⋂X = S )
3 Putting 1 and 2 together, we have
f (x) = L(x , y) = −h(y).
Hence f (x) + h(y) = 0, and x ∈ S ∗, y ∈ T ∗.
Saddle Point and Optimality
Theorem (Convex Optimality with No Gap⇒ Saddle Point)Let f : S
⋂X and h : Ω
⋂Y be a closed convex conjugate dual
pair with no duality gap. Then (x , y) is a saddle point of L(x , y)if and only if x ∈ S ∗ and y ∈ T ∗.Proof: We need only to prove that “If x ∈ S ∗ and y ∈ T ∗, then(x , y) is a saddle point of L(x , y).” When x ∈ S ∗ and y ∈ T ∗,we have < x , y >= 0. Since there is no duality gap,L(x , y) = f (x)− < x , y >= −h(y). Hence,
−h(y)︸ ︷︷ ︸infx∈S L(x ,y)
= L(x , y) = f (x)︸︷︷︸supy∈Y L(x ,y)
Observations
1. If x ∈ S ∗ is known, then y ∈ T ∗ can be found by solving
max L(x , y) = f (x)− < x , y >s. t. y ∈ Y
When Y is linearly structures, then it is a linearly programmingproblem.2. If y ∈ T ∗ is known, then x ∈ S ∗ can be found by solving
min L(x , y) = f (x)− < x , y >s. t. x ∈ S
Linear conic programming problems • Linear Conic Programming (LCoP) A general conic optimization problem is as follows:
( )
where is a closed and convex co
minimize subject to
" " is a linear operator line an
ke "inner product."d
P c xA x b
x∈=
Dual linear conic dual problem
Duality theorems for Linear CP
Duality theorems for linear CP • Theorem 3 (Conic Duality Theorem): (i) If problems (CP) and (CD) are both feasible, then they have optimal solutions. (ii) If one of the two problems has an interior feasible solution with a finite optimal objective value, then the other one is feasible and has the same optimal objective value. (iii) If one of the two problems is unbounded, then the other has no feasible solution. (iv) If (CP) and (CD) both have interior feasible solutions, then they have optimal solutions with zero duality gap.
Examples of conic programs
Example of conic programs
Example of conic programs
Semi-definite Programming (SDP)
Duality theorems for SDP
Quadratic programming problem
Conjugate dual QP
Conjugate dual QP
Conjugate dual QP