complexity of testing existence of solutions in polynomial...
TRANSCRIPT
Complexity of Testing Existence of Solutions in Polynomial Optimization
+A New Positivstellensatz
Amir Ali AhmadiPrinceton, ORFE
Affiliated member of PACM, COS, MAE, CSML
1ICERM Meeting on Nonlinear Algebra: Core Computational Methods, Septβ18
Jeffrey Zhang Princeton, ORFE
Georgina HallPrincetonINSEAD
Joint work with
Existence of solutions
2
Consider a polynomial optimization problem (POP):
Suppose the optimal value is finite (i.e., POP is feasible and bounded below). We would like to test if there exists an optimal solution, i.e.,
a feasible point π₯β such that π π₯β β€ π π¦ , βπ¦ feasible.
Informally: βCan we replace the `infβ with a `minβ?β
infπ₯ββπ
π(π₯)
s.t. ππ π₯ β₯ 0, π = 1,β¦ ,π.
Remarks:β’ If feasible set is bounded, a solution always exists.β’ If π = 1, a solution always exists.β’ Finiteness of optimal value comes as a βpromiseβ.
3
Motivation
[Nie, Demmel, Sturmfels, βMinimizing polynomials via sum of squares over the gradient idealβ, Math. Prog. 2005]:
βThis assumption [existence of minimizers] is nontrivial, and we do not address the (important and difficult) question of how to verify that a given polynomial π π₯ has this property.β
- An exact algorithm cannot return a solution if there is none!- Existence of solutions essential for algorithms that exploit optimality conditions.
There are algothims that check existence of solutions:β’ Greuet, Safey El Din, βDeciding reachability of the infimum of a multivariate polynomialβ, International
Symposium on Symbolic and Algebraic Computation, 2011β’ Boucero, Mourrain, βBorder basis relaxation for polynomial optimizationβ, Journal of Symbolic
Computation, 2016β’ Greuet, Safey El Din, βProbabilistic algorithm for polynomial optimization over a real algebraic setβ,
SIAM J. on Optimization, 2014β’ Quantifier eliminationβ’ β¦
All have running time at least exponential in dimensionβ¦Can there be a faster algorithm?
4
Existence of a solution guaranteed?
0 1 2
1 Yes Yes
Linear ProgrammingNP-hard to test
(This work)
2 Yes
Linear Algebra
Yes
Frank, Wolfe (1956)
3 Yes Yes
Andronov, Belousov, Shironin (1982)
4 NP-hard to test (This work)
Degree of constraints
Deg
ree
of
ob
ject
ive
minπ₯ββπ
π(π₯)
s.t. ππ π₯ β₯ 0, π = 1,β¦ ,π
Outline
1) NP-hardness of testing existence of solutions
2) Sufficient conditions for existence of solutions
a. Review of SOS and Positivstellensatze
b. An SOS hierarchy for coercivity
3) An optimization-free Positivstellensatz(brief and independent)
5
Outline
1) NP-hardness of testing existence of solutions
2) Sufficient conditions for existence of solutions
a. Review of SOS and Positivstellensatze
b. An SOS hierarchy for coercivity
3) An optimization-free Positivstellensatz(brief and independent)
6
Main hardness results
7
Theorem (AAA, Zhang)
Testing whether a degree-4 polynomial attains its unconstrainedinfimum is strongly NP-hard.
Theorem (AAA, Zhang)
Testing whether a degree-1 polynomial attains its infimum on a set defined by degree-2 inequalities is strongly NP-hard.
Proof: Reduction from 1-in-3 3SAT.
6 (for this presentation)/
1-in-3 3SAT
- Input: A CNF formula with three literals per clause.- Goal: Find a Boolean assignment so that each clause has exactly one true literal.
Not satisfiable.
This problem is NP-hard.
8
π₯1 β¨ Β¬π₯2 β¨ π₯3 β§ (Β¬π₯1 β¨ π₯2 β¨ π₯3) β§ (Β¬π₯1 β¨ Β¬π₯2 β¨ Β¬π₯3)
π₯1 = 1, π₯2 = 1, π₯3 = β1
1 β 1 β 1 β1 1 β 1 β1 β 1 1
π₯1 β¨ Β¬π₯2 β¨ π₯3 β§ (Β¬π₯1 β¨ π₯2 β¨ π₯3) β§ (Β¬π₯1 β¨ Β¬π₯2 β¨ π₯3)
NP-hardness of checking attainment (1/2)
9
Goal: Given any instance of 1-in-3 3SAT, construct a polynomial that attains its infimum if and only if the instance is satisfiable
π = π₯1 β¨ x2 β¨ Β¬π₯3 β§ (Β¬π₯1 β¨ Β¬π₯2 β¨ π₯3)
ππ π₯ β π=1π 1 β π₯π
2 2+ π₯1 + π₯2 β π₯3 + 1
2 + βπ₯1 β π₯2 + π₯3 + 12
Important Property: ππ π₯ has a zero if and only if π is satisfiable
π = π₯1 β¨ x2 β¨ Β¬π₯3 β§ (Β¬π₯1 β¨ Β¬π₯2 β¨ π₯3)
Step 1:
But ππ π always attains its infimum (independent of whether π is
satisfiable) as its highest order component is π=1π π₯π
4.
NP-hardness of checking attainment (2/2)
10
ππ π₯ππ+π¦2 + 1 β π¦π§ 2π β π π( )
Step 2:
ππ ππ, β¦ , ππ, π, π, π =
β’ Infimum is always zero (ππ is a sum of squares; take π = 0, π¦ β 0, π§ =1
π¦).
β’ If π satisfiable, take π = 1, and π₯ the satisfying assignment Infimum attained.β’ If π not satisfiable, ππ does not vanish Infimum not attained.
Outline
1) NP-hardness of testing existence of solutions
2) Sufficient conditions for existence of solutions
a. Review of SOS and Positivstellensatze
b. An SOS hierarchy for coercivity
3) An optimization-free Positivstellensatz(brief and independent)
11
Sufficient conditions for existence of solutions
12
Existence of solutions
Compactness of the feasible set
Coercivity of the objective function
Convexity of the of the objective function and concavity of the constraints
Archimedean Condition
Stable Compactness
SDP hierarchy
SDP hierarchy
Theorem
- Testing whether a polynomial optimization problem satisfies any of these conditions is strongly NP-hard.- Our results are minimal in the degree.
minπ₯ββπ
π(π₯)
s.t. ππ π₯ β₯ 0, π = 1,β¦ ,π
13
Review of sum of squaresand Positivstellensatze
How to prove positivity?
β’ (Tight) lower bounds for polynomial minimization problems
Is π π₯ > 0 on {π1 π₯ β₯ 0,β¦ , ππ π₯ β₯ 0}?
Why prove positivity?
β’ Infeasibility certificates for systems of polynomial inequalities
β’ Dynamics and control (Lyapunov functions)
β’ Stats/ML (shape-constrained regression),β¦
{π1 π₯ β₯ 0, π2 π₯ β₯ 0,β¦ , ππ π₯ β₯ 0}
βπ1 π₯ > 0 on {π2 π₯ β₯ 0,β¦ , ππ π₯ β₯ 0}
empty
β
14
15
Sum of squares and SDP
Ex:
Optimizing over set of sos polynomials is an SDP!
β’ A polynomial π is a sum of squares (sos) if it can be written as
β’ A polynomial π of degree 2π is sos if and only if βπ β½ 0 such that
16
20th century
π π₯ > 0, βπ₯ β βπ
π π₯ > 0,βπ₯ β π = π₯ ππ π₯ β₯ 0}
1927 1993
Putinar
If π π₯ > 0, βπ₯ β π,then π π₯ = π0 π₯ + π ππ π₯ ππ(π₯) ,
where π0, ππ are sos
1974 1991
Schmudgen
If π π₯ > 0, βπ₯ β π,then π π₯ = π0 π₯ + π ππ π₯ ππ π₯ +
ππ πππ π₯ ππ π₯ ππ π₯ + β―, where π0, ππ , β¦ sos
Requires compactness
Artin
If π π₯ β₯ 0, βπ₯ β βπ,then β sos π s.t. π β π sos.
Stengle
PositivstellensΓ€tze
Search for these sos polynomials (when degree is fixed) --->SDP.
Requires the Archimedean
property
Infeasibility proofs for polynomial (in)equalities
17
Stengleβs Positivstellensatz
ππ π₯ β₯ 0, π = 1,β¦ ,π, βπ π₯ = 0, π = 1,β¦ , π‘infeasible
if and only if
there exist polynomials ππ and sos polynomials π such that
β1 = ππβπ + π0 + ππππ + πππππππ + ππππππππππ+β―+ π1β¦πΞ π1β¦ππ.
Search for these sos certificates of infeasibility (when deg. is fixed) ---> SDP.
18
Back to coercivity
Coercivity
19
Definition: A function π is coercive if for every sequence {π₯π} such that ||π₯π|| β β, we have π π₯ β β.β’ A coercive function attains its infimumβ’ Checking whether a polynomial is coercive is NP-hard
We provide a condition which is (i) both necessary and sufficient for a polynomial to be coercive and (ii) amenable to an SDP hierarchy
Past work:β’ Jeyakumar, Lasserre (2014)
β’ SDP hierarchyβ’ Bajbar, Stein (2015)β’ Bajbar, Behrends (2017)
An sos hierarchy for testing coercivity (1/2)
20
Theorem (AAA, Zhang)
A polynomial π of degree π is coercive if and only if for some integer π β₯ 1 the following SDP is feasible
An sos hierarchy for testing coercivity (2/2)
21
Equivalently, p is coercive if and only if there exist an even integer πβ² and a
scalar πβ² for which the set {(π₯, πΎ)|π π₯ β€ πΎ, π₯π2 β₯ πΎπ
β²+ πβ²} is empty.
Theorem (AAA, Zhang)
A polynomial p is coercive if and only if there exist an even integer π > 0and a scalar π β₯ 0 such that for all πΎ β β, the πΎ-sublevel set of p is contained within a ball of radius πΎπ + π.
A function is coercive if and only if all its sublevel sets are compact.
π π₯ = π₯4 + π¦4 β 2π₯3 + π¦2 + 3π₯ + π¦
Coercivity hierarchy revisited
22
Theorem (AAA, Zhang)
A polynomial π of degree π is coercive if and only if for some integer π β₯ 1 the following SDP is feasible
Toy example
23
The polynomial π π₯ = π₯14 + π₯2
2 is coercive as certified by the following algebraic identity:
β1 =2
3π₯12 β
1
2
2
+2
3πΎ β
1
2
2
+2
3πΎ β π₯1
4 β π₯22 +
2
3π₯12 + π₯2
2 β πΎ2 β 2
Outline
1) NP-hardness of testing existence of solutions
2) Sufficient conditions for existence of solutions
a. Review of SOS and Positivstellensatze
b. An SOS hierarchy for coercivity
3) An optimization-free Positivstellensatz(brief and independent)
24
PutinarPutinar
If π π₯ > 0, βπ₯ β π,then π π₯ = π0 π₯ + π ππ π₯ ππ(π₯) ,
where π0, ππ are sos
25
Recall what a Positivstellensatz establishes
π π₯ > 0, βπ₯ β π₯ β βπ ππ π₯ β₯ 0, π = 1, β¦ ,π}
Under the Archimedean
property
Search for these sos polynomials (when degree is fixed) --->SDP.Similar situation for Psatze of Stengle and Schmudgen.
26
An optimization-free Positivstellensatz (1/2)
[AAA, Hall, Math of OR β18] (2018 INFORMS Young Researchers Prize)
π π₯ > 0, βπ₯ β π₯ β βπ ππ π₯ β₯ 0, π = 1, β¦ ,π}
2π =maximum degree of π, ππ
β
β π β β such that
π π£2 β π€2 β1
π π π£π
2 β π€π2 2 π
+1
2π π π£π
4 + π€π4 π
β π π£π2 + ππ€π
2 π2
has nonnegative coefficients,
where π is a form in π +π + 3 variables and of degree 4π, which can be explicitly written from π, ππ and π .
π π₯ > 0 on π₯ ππ π₯ β₯ 0} β
βπ β β s. t. π π£2 β π€2 β1
π π π£π
2 β π€π2 2 π
+1
2π π π£π
4 + π€π4 π
β π π£π2 + ππ€π
2 π2
has β₯ 0 coefficients
Proof sketch:
27
An optimization-free Positivstellensatz (2/2)
π π₯ > 0 on π₯ ππ π₯ β₯ 0} β
βπ β β s. t. π ππ β ππ β1
π π π£π
2 β π€π2 2 π
+1
2π π π£π
4 + π€π4 π
β π π£π2 + ππ€π
2 π2
has β₯ 0 coefficients
π π₯ > 0 on π₯ ππ π₯ β₯ 0} β
βπ β β s. t. π ππ β ππ β1
π π π£π
2 β π€π2 2 π
+π
ππ π ππ
π + πππ π
β ππππ + πππ
π ππ
has β₯ π coefficients
π π₯ > 0 on π₯ ππ π₯ β₯ 0} β
βπ β β s. t. π ππ βππ βπ
π π ππ
π β πππ π π
+π
ππ π ππ
π + πππ π
β ππππ + πππ
π ππ
has β₯ π coefficients
β’ π π₯ > 0 on π₯ ππ π₯ β₯ 0} β π is pd
β’ Result by Polya (1928):
π even and pdβ βπ β β such that π π§ β π π§π2 π
has nonnegative coefficients.
β’ Make π(π§) even by considering π ππ βππ .We lose positive definiteness of
π with this transformation.
β’ Add the positive definite term π
ππ π ππ
π +πππ π
to π(π£2 β π€2) to make it
positive definite. Apply Polyaβs result.
β’ The term βπ
π π ππ
π βπππ π π
ensures that the converse holds as well.
Want to know more? aaa.princeton.edu
You are cordially invitedβ¦
Princeton Day of OptimizationSeptember 28, 2018
http://orfe.princeton.edu/pdo/
Thank you.