convex duaity and kantorovich duality theoremhomepages.wmich.edu/~ledyaev/zhu-talk2-sp2016.pdf ·...

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Convex sets and functionsConvex duality

Kantorovich Duality theorem

Convex Duaity and Kantorovich Duality Theorem

Qiji Zhu

Analysis Seminar

Feburary 12, 2016

Qiji Zhu Convex Duaity and Kantorovich Duality Theorem

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.



Introduction

In the last talk I mentioned that the Kantorovich duality for masstransport problem is, in fact, a special case of the convex duality.Today we go into some of the details by

• First introduce the basic concepts of convex analysis: convexsets and functions, subdifferentials, and convex conjugate;

• next we discuss the Fenchel duality theorem and outline theproof.

• finally culminating in a proof of the Kantorovich dualitytheorem by converting the Kantorovich mass transportproblem into a Fenchel problem.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.



Convex setsConvex functionsSubdifferential

Convex sets

Convex sets

We say C ⊂ X is convex if for any x, y ∈ C and λ ∈ [0, 1],

λx+ (1− λ)y ∈ C.

Note that the sum and difference of convex sets are convex and sois the intersection of any class of convex sets. However, the unionof two convex sets may not be convex.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Convex functions

Convex functions

We say f : X 7→ R ∪ {+∞} is convex if, for any x, y ∈ X andλ ∈ [0, 1],

f(λx+ (1− λ)y) ≤ λf(x) + (1− λ)f(y).


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Convex functions: properties

A convex function is always below any secant line in between thetwo intersection points.

x

y


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Convex functions: properties

Also, a convex function is always above its tangent lines.Moreover, a differentiable convex function has an increasingderivative function.

x

y


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Epigraph characterizations

Epigraph characterizations

Function f : X 7→ R ∪ {+∞} is convex iff epi f is a convex set inX × R.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Subdifferential

Subdifferential is a substitute for the derivative.

Subdifferential

The subdifferential of a lower semi-continuous convex function ϕat x ∈ dom ϕ is defined by

∂ϕ(x) = {x∗ ∈ X∗ : ϕ(y)− ϕ(x) ≥ ⟨x∗, y − x⟩}.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Nonemptyness of subdifferential

The most useful property of a convex function related to Lagrangemultipliers is

Nonemptyness of subdifferential

Let f : X 7→ R ∪ {+∞} be a convex function. Then for anyx ∈ int dom f ,

∂f(x) = ∅.

This result follows directly from the Hahn-Banach separationtheorem.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Nonemptyness of subdifferential: graph

Nonemptyness of subdifferential follows from separation theorem

x

y


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Subdifferential characterizations

Subdifferential characterizations

Function f : X 7→ R ∪ {+∞} is convex iff, for anyx∗ ∈ ∂f(x), y∗ ∈ ∂f(y),

⟨y∗ − x∗, y − x⟩ ≥ 0.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Corollary

Derivative characterizations

Function f : R 7→ R ∪ {+∞} is convex iff f ′ is increasing orf ′′ ≥ 0.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.



Fenchel-Young inequalityFenchel formulationWeak and strong duality

Fenchel-Legendre transform

Fenchel-Legendre transform

Let f : X → R ∪+∞ be a lsc function. The Fenchel-Legendretransform f∗ : X∗ → R ∪+∞ is defined by

f∗(x∗) = supx∈X

[⟨x∗, x⟩ − f(x)]

.

Note that f is not necessarily convex but f∗ is always convex.When f ′−1 exists we have

f∗(x∗) = x∗f ′−1(x∗)− f(f ′−1(x∗)).


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Fenchel-Legendre transform: examples

f(x) dom f f∗(y) dom f∗

0 R 0 {0}

0 R+ 0 −R+

0 [−1, 1] |y| R

0 [0, 1] y+ R

Table: Conjugate pairs of convex functions on R.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Fenchel-Legendre transform: examples

|x|p/p, p > 1 R |y|q/q (1p +1q = 1) R

|x|p/p, p > 1 R+ |y+|q/q (1p +1q = 1) R

−xp/p, 0<p<1 R+ −(−y)q/q (1p +1q = 1) −int R+

− log x int R+ −1− log(−y) −int R+

ex R{y log y − y (y > 0)0 (y = 0)

R+

Table: Conjugate pairs of convex functions on R.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Fenchel-Young inequality

Fenchel-Young inequality follows directly from definition.


f(x) + f∗(x∗) ≥ ⟨x∗, x⟩.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Holder inequality

Let f(x) = |x|p/p in the Fenchel -Young inequality we get

Holder inequality

For 1/p+ 1/q = 1,|x|p

p+

|y|q

q≥ |xy|.

When p = q = 2 we get

Cauchy inequality

|x|2

2+

|y|2

2≥ |xy|.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Fenchel-Young equality

In general,


f(x) + f∗(x∗) = ⟨x∗, x⟩.

iffx∗ ∈ ∂f(x).

This also follows from the definition.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Fenchel-Young inequality: graphic illustration

For increasing function ϕ, ϕ(0) = 0, f(x) =∫ x0 ϕ(s)ds is convex

and f∗(x∗) =∫ x∗0 ϕ−1(t)dt.

s

t

O x

x∗

ϕϕ−1



.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Fenchel-Young inequality: graphic illustration

s

t

O x

x∗

ϕϕ−1



.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.





s

t

O x

x∗

ϕϕ−1


We see x∗ = ϕ(x), x = ϕ−1(x∗).


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Fenchel duality

Let v(y) = infx[f(x) + g(x+ y)]. The Fenchel primal problem is

p = v(0) = infx[f(x) + g(x)]. (1)

The dual problem is

d = v∗∗(0) = supy∗

[−f∗(y∗)− g∗(−y∗)]. (2)


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Verify the dual

We calculate v∗(−y∗) = supx,y[⟨−y∗, y⟩ − f(x)− g(x+ y)].Letting u = x+ y we have

v∗(−y∗) = supx,u

⟨−y∗, u− x⟩ − f(x)− g(u)

= supx[⟨y∗, x⟩ − f(x)] + sup

u[⟨−y∗, u⟩ − g(u)]

= f∗(y∗) + g∗(−y∗).

Thus,

d = v∗∗(0) = sup−y∗

[0− v∗(−y∗)] = sup−y∗

[−f∗(y∗)− g∗(−y∗)].


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Weak duality

Weak duality p ≥ d follows directly from the definition.Strong duality asserting p = d needs additional condition which wediscuss below.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Strong duality

• If both f and g are convex functions it is easy to see that so is

v(y) = infx[f(x) + g(x+ y)].

• We can directly check that dom v = dom g − dom f .

• The sufficient condition for ∂v(0) = ∅, is

0 ∈ int dom v = int[dom g − dom f ]. (3)

A condition (3) is often referred to as a constraint qualification.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Strong duality

Duality and constraint qualification

If l.s.c. convex functions f , g satisfy the constraint qualificationconditions

0 ∈ int dom v = int[dom g − dom f ]

then p = d, and the dual problem has a solution when finite.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Strong duality: Proof

The condition

0 ∈ int dom v = int[dom g − dom f ]

implies that ∂v(0) = ∅. Let −y∗ ∈ ∂v(0) we have

f(x) + g(x+ y) ≥ v(y) ≥ p− ⟨y∗, y⟩

Letting u = x+ y we have

p ≤ f(x)− ⟨y∗, x⟩+ g(u) + ⟨y∗, u⟩

Taking inf on x, u we have

p ≤ −f∗(y∗)− g∗(−y∗) ≤ d ≤ p.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.



Kantorovich DualityKantorovich DualityProof of Kantorovich DualityChecking constraint qualification condition

Setting

We will prove the Kantorovich duality theorem under the followingsetting:

• Assume X and Y are compact sets.

• Consider Banach space E = C(X × Y ), continuous functionson X × Y endowed with the sup norm.

• Then E∗ =M(X × Y ) is the space of regular Radonmeasures endowed with the total variation norm.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Kantorovich Duality

Kantorovich Duality

The function c satisfies the constraint qualification conditions thereexists a(x), b(y) integrable such that

c(x, y) ≥ a(x) + b(y).

Then

infπ∈Π(µ,ν)

∫X×Y

c(x, y)dπ(x, y)

= sup(φ,ψ)∈Φc

{∫Xφ(x)dµ(x) +

∫Yψ(y)dν(y)

}.

where Φc := {(φ,ψ) ∈ C(X)× C(Y ) : φ(x) + ψ(y) ≤ c(x, y)}.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Converting to Fenchel problem

For u ∈ E = C(X × Y ) define

g(u) :=

{0 u(x, y) ≥ −c(x, y)+∞ otherwise.

f(u) :=

{∫X φ(x)dµ+

∫Y ψ(y)dν u(x, y) = φ(x) + ψ(y)

+∞ otherwise.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Converting to Fenchel problem

We have

infu∈E

[f(u) + g(u)]

= inf{∫Xφ1(x)dµ+

∫Yψ1(y)dν : φ1(x) + ψ1(y) ≥ −c(x, y)}

= − sup(φ,ψ)∈Φc

{∫Xφ(x)dµ+

∫Yψ(y)dν}.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Calculating the conjugate

We have

g∗(−π) = supu∈E

[−∫X×Y

udπ − g(u)]

= supu∈E

[−∫X×Y

udπ : −u(x, y) ≤ c(x, y)]

=

{∫X×Y c(x, y)dπ π ∈M+(X × Y )

+∞ otherwise.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.





f∗(π) = supu∈E

[

∫X×Y

u(x, y)dπ − f(u)]

= supu∈E

[

∫X×Y

φ(x) + ψ(y)dπ −∫Xφ(x)dµ−

∫Yψ(y)dν]

=

{0 π ∈ Π(µ, ν)

+∞ otherwise.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.





Thus,

supπ∈E∗

[−f∗(π)− g∗(−π)]

= supπ∈Π(µ,ν)

−∫X×Y

c(x, y)dπ

= − infπ∈Π(µ,ν)

∫X×Y

c(x, y)dπ.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




Constraint qualification condition

We observe

[a(x)− 1] + [b(y)− 1] ∈ int[dom g ∩ dom f ].

Thus

0 ∈ intdom f − dom g.


.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.




References

G. Monge, Memoire sur la theo rie des deeblais et des remblais, InHistorie de l’Academie Royale des Sciences de Paris (1781) 666-704.A. Galichon, Optimal Transport Methods in Economics, preprint 2015.L. V. Kantorovhich, Mathematical methods in the organization andplanning of production. Leningrad Univ. 1939.L. V. Kantorovhich, On the translocation of masses. Dokl. Akad. Nauk.USSR 37 (1942) 199-201.C. Villani, Optimal Transport, Old and New, Springer 2006

C. Villani, Topics in Optimal Transportation, AMS 2003.


convex duaity and kantorovich duality theoremhomepages.wmich.edu/~ledyaev/zhu-talk2-sp2016.pdf ·...

Documents