the moment-lp and moment-sos approaches in optimization€¦ · ii. let k ˆrn and s ˆk be given,...
Post on 18-Jul-2020
2 Views
Preview:
TRANSCRIPT
The moment-LP and moment-SOSapproaches in optimization
Jean B. Lasserre
LAAS-CNRS and Institute of Mathematics, Toulouse, France
Master 2 Optimization: Paris-Orsay, 2016
Jean B. Lasserre semidefinite characterization
SOS-based certificate
K = {x : gj(x) ≥ 0, j = 1, . . . ,m }
Theorem (Putinar’s Positivstellensatz)If K is compact (+ a technical Archimedean assumption) andf > 0 on K then:
† f (x) = σ0(x) +m∑
j=1
σj(x) gj(x), ∀x ∈ Rn,
for some SOS polynomials (σj) ⊂ R[x].
Jean B. Lasserre semidefinite characterization
LP-based certificate
K = {x : gj(x) ≥ 0; (1− gj(x)) ≥ 0, j = 1, . . . ,m}
Theorem (Krivine-Vasilescu-Handelman’s Positivstellensatz)
Let K be compact and the family {1,gj} generate R[x]. If f > 0on K then:
(?) f (x) =∑α,β
cαβm∏
j=1
gj(x)αj (1− gj(x))βj , , ∀x ∈ Rn,
for some NONNEGATIVE scalars (cαβ).
Jean B. Lasserre semidefinite characterization
Dual side of Putinar’s theorem: The K -momentproblem
Given a real sequence y = (yα), α ∈ Nn, determine wherethere exists some finite Borel measure µ on K such that
† yα =
∫K
xα dµ, ∀α ∈ Nn.
TheoremIf K = {x : gj(x) ≥ 0, j = 1, . . . ,m} is compact (+Archimedeanassumption) then † holds if and only if
(?) Ly
(h2 gj)
)≥ 0, ∀h ∈ R[x]d .
The condition (?) is equivalent to countably many LINEARMATRIX INEQUALITIES on the yα’s
Jean B. Lasserre semidefinite characterization
Dual side of Putinar’s theorem: The K -momentproblem
Given a real sequence y = (yα), α ∈ Nn, determine wherethere exists some finite Borel measure µ on K such that
† yα =
∫K
xα dµ, ∀α ∈ Nn.
TheoremIf K = {x : gj(x) ≥ 0, j = 1, . . . ,m} is compact (+Archimedeanassumption) then † holds if and only if
(?) Ly
(h2 gj)
)≥ 0, ∀h ∈ R[x]d .
The condition (?) is equivalent to countably many LINEARMATRIX INEQUALITIES on the yα’s
Jean B. Lasserre semidefinite characterization
Dual side of Putinar’s theorem: The K -momentproblem
Given a real sequence y = (yα), α ∈ Nn, determine wherethere exists some finite Borel measure µ on K such that
† yα =
∫K
xα dµ, ∀α ∈ Nn.
TheoremIf K = {x : gj(x) ≥ 0, j = 1, . . . ,m} is compact (+Archimedeanassumption) then † holds if and only if
(?) Ly
(h2 gj)
)≥ 0, ∀h ∈ R[x]d .
The condition (?) is equivalent to countably many LINEARMATRIX INEQUALITIES on the yα’s
Jean B. Lasserre semidefinite characterization
Dual side of Krivine’s theorem: The K -momentproblem
TheoremIf K = {x : gj(x) ≥ 0, j = 1, . . . ,m} is compact, 0 ≤ gj ≤ 1 on K,and {1,gj} generates R[x], then † holds if and only if
(??) Ly
m∏j=1
gjαj (1− gj
βj
≥ 0, ∀α, β ∈ Nm.
The condition (??) is equivalent to countably many LINEARINEQUALITIES on the yα’s
Jean B. Lasserre semidefinite characterization
Dual side of Krivine’s theorem: The K -momentproblem
TheoremIf K = {x : gj(x) ≥ 0, j = 1, . . . ,m} is compact, 0 ≤ gj ≤ 1 on K,and {1,gj} generates R[x], then † holds if and only if
(??) Ly
m∏j=1
gjαj (1− gj
βj
≥ 0, ∀α, β ∈ Nm.
The condition (??) is equivalent to countably many LINEARINEQUALITIES on the yα’s
Jean B. Lasserre semidefinite characterization
The Generalized problem of Moments
• Polynomials NONNEGATIVE ON A SET K ⊂ Rn areubiquitous. They also appear in many important applications(outside optimization),
. . . modeled asparticular instances of the so called
Generalized Moment Problem, among which:Probability, Optimal and Robust Control, Game theory, Signal
processing, multivariate integration, etc.
Jean B. Lasserre semidefinite characterization
The Generalized Moment Problem (GMP) is theinfinite-dimensional linear program
(GMP) : infµi∈M(Ki )
{s∑
i=1
∫Ki
fi dµi :s∑
i=1
∫Ki
hij dµi≥= bj , j ∈ J}
with M(Ki) space of Borel measures on Ki ⊂ Rni , i = 1, . . . , s.
Jean B. Lasserre semidefinite characterization
The DUAL of the GMP is the linear program GMP∗:
supλj
{s∑
j∈J
λj bj : fi −∑j∈J
λj hij ≥ 0 on Ki , i = 1, . . . , s }
And one can see that ...the constraints of GMP∗ state that
some functions fi −∑
j∈J λj hij
must be nonnegative on a certain set Ki , i = 1, . . . , s.
Jean B. Lasserre semidefinite characterization
� The GPM has great modelling power, in various fields.Global Optimization (continuous, discrete), Control (Robust andoptimal control), Nonlinear Equations, Probability and Statistics,Performance Evaluation (in e.g. Mathematical finance, Markovchains), Inverse Problems (cristallography, tomography),Numerical multivariate Integration, etc ...
Jean B. Lasserre semidefinite characterization
A couple of examples
I: Global OPTIM → f ∗ = infx{ f (x) : x ∈ K }
is the SIMPLEST example of the GMP
because ...
f ∗ = infµ∈M(K)
{∫
Kf dµ :
∫K
1 dµ = 1}
• Indeed if f (x) ≥ f ∗ for all x ∈ K and µ is a probability measureon K, then
∫K f dµ ≥
∫f ∗ dµ = f ∗.
• On the other hand, for every x ∈ K the probability measureµ := δx is such that
∫f dµ = f (x).
Jean B. Lasserre semidefinite characterization
A couple of examples
I: Global OPTIM → f ∗ = infx{ f (x) : x ∈ K }
is the SIMPLEST example of the GMP
because ...
f ∗ = infµ∈M(K)
{∫
Kf dµ :
∫K
1 dµ = 1}
• Indeed if f (x) ≥ f ∗ for all x ∈ K and µ is a probability measureon K, then
∫K f dµ ≥
∫f ∗ dµ = f ∗.
• On the other hand, for every x ∈ K the probability measureµ := δx is such that
∫f dµ = f (x).
Jean B. Lasserre semidefinite characterization
A couple of examples
I: Global OPTIM → f ∗ = infx{ f (x) : x ∈ K }
is the SIMPLEST example of the GMP
because ...
f ∗ = infµ∈M(K)
{∫
Kf dµ :
∫K
1 dµ = 1}
• Indeed if f (x) ≥ f ∗ for all x ∈ K and µ is a probability measureon K, then
∫K f dµ ≥
∫f ∗ dµ = f ∗.
• On the other hand, for every x ∈ K the probability measureµ := δx is such that
∫f dµ = f (x).
Jean B. Lasserre semidefinite characterization
A couple of examples
I: Global OPTIM → f ∗ = infx{ f (x) : x ∈ K }
is the SIMPLEST example of the GMP
because ...
f ∗ = infµ∈M(K)
{∫
Kf dµ :
∫K
1 dµ = 1}
• Indeed if f (x) ≥ f ∗ for all x ∈ K and µ is a probability measureon K, then
∫K f dµ ≥
∫f ∗ dµ = f ∗.
• On the other hand, for every x ∈ K the probability measureµ := δx is such that
∫f dµ = f (x).
Jean B. Lasserre semidefinite characterization
II. Let K ⊂ Rn and S ⊂ K be given, and let Γ ⊂ Nn be alsogiven.
BOUNDS on measures with moment conditions
maxµ∈M(K)
{ 〈1S, µ〉 :
∫K
xα dµ = mα, α ∈ Γ }
to compute an upper bound on µ(S) over all distributionsµ ∈ M(K) with a certain fixed number of moments mα.
• If Γ = Nn then one may use this to compute the Lebesguevolume of a compact basic semi-algebraic setS ⊂ K := [−1,1]n.
Take mα :=
∫[−1,1]n
xα dx, α ∈ Nn.
Jean B. Lasserre semidefinite characterization
II. Let K ⊂ Rn and S ⊂ K be given, and let Γ ⊂ Nn be alsogiven.
BOUNDS on measures with moment conditions
maxµ∈M(K)
{ 〈1S, µ〉 :
∫K
xα dµ = mα, α ∈ Γ }
to compute an upper bound on µ(S) over all distributionsµ ∈ M(K) with a certain fixed number of moments mα.
• If Γ = Nn then one may use this to compute the Lebesguevolume of a compact basic semi-algebraic setS ⊂ K := [−1,1]n.
Take mα :=
∫[−1,1]n
xα dx, α ∈ Nn.
Jean B. Lasserre semidefinite characterization
III. For instance, one may also want:• To approximate sets defined with QUANTIFIERS, like .e.g.,
Rf := {x ∈ B : f (x , y) ≤ 0 for all y such that (x , y) ∈ K}
Df := {x ∈ B : f (x , y) ≤ 0 for some y such that (x , y) ∈ K}
where f ∈ R[x , y ], B is a simple set (box, ellipsoid).
• To compute convex polynomial underestimators p ≤ f of apolynomial f on a box B ⊂ Rn. (Very useful in MINLP.)
Jean B. Lasserre semidefinite characterization
III. For instance, one may also want:• To approximate sets defined with QUANTIFIERS, like .e.g.,
Rf := {x ∈ B : f (x , y) ≤ 0 for all y such that (x , y) ∈ K}
Df := {x ∈ B : f (x , y) ≤ 0 for some y such that (x , y) ∈ K}
where f ∈ R[x , y ], B is a simple set (box, ellipsoid).
• To compute convex polynomial underestimators p ≤ f of apolynomial f on a box B ⊂ Rn. (Very useful in MINLP.)
Jean B. Lasserre semidefinite characterization
The moment-LP and moment-SOS approachesconsist of using a certain type of positivity certificate(Krivine-Vasilescu-Handelman’s or Putinar’s certificate) inpotentially any application where such a characterization isneeded. (Global optimization is only one example.)
In many situations this amounts tosolving a HIERARCHY of :
LINEAR PROGRAMS, orSEMIDEFINITE PROGRAMS
... of increasing size!.
Jean B. Lasserre semidefinite characterization
The moment-LP and moment-SOS approachesconsist of using a certain type of positivity certificate(Krivine-Vasilescu-Handelman’s or Putinar’s certificate) inpotentially any application where such a characterization isneeded. (Global optimization is only one example.)
In many situations this amounts tosolving a HIERARCHY of :
LINEAR PROGRAMS, orSEMIDEFINITE PROGRAMS
... of increasing size!.
Jean B. Lasserre semidefinite characterization
The moment-LP and moment-SOS approachesconsist of using a certain type of positivity certificate(Krivine-Vasilescu-Handelman’s or Putinar’s certificate) inpotentially any application where such a characterization isneeded. (Global optimization is only one example.)
In many situations this amounts tosolving a HIERARCHY of :
LINEAR PROGRAMS, orSEMIDEFINITE PROGRAMS
... of increasing size!.
Jean B. Lasserre semidefinite characterization
The moment-LP and moment-SOS approachesconsist of using a certain type of positivity certificate(Krivine-Vasilescu-Handelman’s or Putinar’s certificate) inpotentially any application where such a characterization isneeded. (Global optimization is only one example.)
In many situations this amounts tosolving a HIERARCHY of :
LINEAR PROGRAMS, orSEMIDEFINITE PROGRAMS
... of increasing size!.
Jean B. Lasserre semidefinite characterization
LP- and SDP-hierarchies for optimization
Replace f ∗ = supλ {λ : f (x)− λ ≥ 0 ∀x ∈ K} with:
The SDP-hierarchy indexed by d ∈ N:
f ∗d = sup {λ : f − λ = σ0︸︷︷︸SOS
+m∑
j=1
σj︸︷︷︸SOS
gj ; deg (σj gj) ≤ 2d }
or, the LP-hierarchy indexed by d ∈ N:
θd = sup {λ : f −λ =∑α,β
cαβ︸︷︷︸≥0
m∏j=1
gjαj (1−gj)
βj ; |α+β| ≤ 2d}
Jean B. Lasserre semidefinite characterization
LP- and SDP-hierarchies for optimization
Replace f ∗ = supλ {λ : f (x)− λ ≥ 0 ∀x ∈ K} with:
The SDP-hierarchy indexed by d ∈ N:
f ∗d = sup {λ : f − λ = σ0︸︷︷︸SOS
+m∑
j=1
σj︸︷︷︸SOS
gj ; deg (σj gj) ≤ 2d }
or, the LP-hierarchy indexed by d ∈ N:
θd = sup {λ : f −λ =∑α,β
cαβ︸︷︷︸≥0
m∏j=1
gjαj (1−gj)
βj ; |α+β| ≤ 2d}
Jean B. Lasserre semidefinite characterization
TheoremBoth sequence (f ∗d ), and (θd ), d ∈ N, are MONOTONE NONDECREASING and when K is compact (and satisfies atechnical Archimedean assumption) then:
f ∗ = limd→∞
f ∗d = limd→∞
θd .
Jean B. Lasserre semidefinite characterization
•What makes this approach exciting is that it is at thecrossroads of several disciplines/applications:
Commutative, Non-commutative, and Non-linearALGEBRAReal algebraic geometry, and Functional AnalysisOptimization, Convex AnalysisComputational Complexity in Computer Science,
which BENEFIT from interactions!
• As mentioned ... potential applications are ENDLESS!
Jean B. Lasserre semidefinite characterization
•What makes this approach exciting is that it is at thecrossroads of several disciplines/applications:
Commutative, Non-commutative, and Non-linearALGEBRAReal algebraic geometry, and Functional AnalysisOptimization, Convex AnalysisComputational Complexity in Computer Science,
which BENEFIT from interactions!
• As mentioned ... potential applications are ENDLESS!
Jean B. Lasserre semidefinite characterization
• Has already been proved useful and successful inapplications with modest problem size, notably in optimization,control, robust control, optimal control, estimation, computervision, etc. (If sparsity then problems of larger size can beaddressed)
• HAS initiated and stimulated new research issues:in Convex Algebraic Geometry (e.g. semidefiniterepresentation of convex sets, algebraic degree ofsemidefinite programming and polynomial optimization)in Computational algebra (e.g., for solving polynomialequations via SDP and Border bases)Computational Complexity where LP- andSDP-HIERARCHIES have become an important tool toanalyze Hardness of Approximation for 0/1 combinatorialproblems (→ links with quantum computing)
Jean B. Lasserre semidefinite characterization
• Has already been proved useful and successful inapplications with modest problem size, notably in optimization,control, robust control, optimal control, estimation, computervision, etc. (If sparsity then problems of larger size can beaddressed)
• HAS initiated and stimulated new research issues:in Convex Algebraic Geometry (e.g. semidefiniterepresentation of convex sets, algebraic degree ofsemidefinite programming and polynomial optimization)in Computational algebra (e.g., for solving polynomialequations via SDP and Border bases)Computational Complexity where LP- andSDP-HIERARCHIES have become an important tool toanalyze Hardness of Approximation for 0/1 combinatorialproblems (→ links with quantum computing)
Jean B. Lasserre semidefinite characterization
There has been also recents attempts to use other types ofalgebraic certificates of positivity that try to avoid the size
explosion due to the semidefinite matrices associated with theSOS weights in Putinar’s positivity certificate
Recent work by Ahmadi et al. and Lasserre, Toh and Zhang
Jean B. Lasserre semidefinite characterization
top related