![Page 1: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/1.jpg)
On Convexity of Polynomials over a Box
Georgina HallDecision Sciences, INSEAD
Joint work withAmir Ali Ahmadi
ORFE, Princeton University
1
![Page 2: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/2.jpg)
Convexity over a boxโข A box ๐ฉ is a set of the form:
๐ต = ๐ฅ โ โ๐ ๐๐ โค ๐ฅ๐ โค ๐ข๐ , ๐ = 1, โฆ , ๐}
where ๐1, โฆ , ๐๐, ๐ข1, โฆ , ๐ข๐ โ โ with ๐๐ โค ๐ข๐ .
โข A function ๐ is convex over ๐ฉ if ๐ ๐๐ฅ + 1 โ ๐ ๐ฆ โค ๐๐ ๐ฅ + 1 โ ๐ ๐(๐ฆ)
for any ๐ฅ, ๐ฆ โ ๐ต and ๐ โ [0,1].
โข If ๐ฉ is full dimensional (i.e., ๐๐ < ๐ข๐, ๐ = 1,โฆ , ๐), this is equivalent to
๐ป2๐ ๐ฅ โฝ 0, โ๐ฅ โ ๐ต.
2
![Page 3: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/3.jpg)
Complexity questions
โข Restrict ourselves to polynomial functions.
โข Related work:
Theorem [Ahmadi, Olshevsky, Parrilo, Tsitsiklis]It is strongly NP-hard to test (global) convexity of polynomials of degree 4.
โข One may hope that adding the restriction to a box could make things easier.
3
Goal: study the complexity of testing convexity of a function over a box
![Page 4: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/4.jpg)
Our theorem
Why are we interested in convexity over a box?
4
Theorem [Ahmadi, H.]It is strongly NP-hard to test convexity of polynomials of degree 3 over a box.
โข Nonconvex optimization: branch-and-bound
โข Prior work: โข Sufficient conditions for convexity [Orban et
al.], [Grant et al.]โข In practice, BARON, CVX, Gurobi check
convexity of quadratics and computationally tractable sufficient conditions for convexity
Detecting Imposing
โข Control theory: convex Lyapunov functions
โข Statistics: convex regression
[Ahmadi and Jungers][Chesi and Hung]
![Page 5: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/5.jpg)
Question: What to do a reduction
from?
Idea: A cubic polynomial ๐ is convexover a (full-dimensional) box ๐ต if and
only if ๐ป2๐ ๐ฅ โฝ 0, โ๐ฅ โ ๐ต
๐ป2๐(๐ฅ) is a matrix with entries affine
in ๐
Proof of the theorem
5
Theorem [Ahmadi, H.]It is strongly NP-hard to test convexity of polynomials of degree 3 over a box.
How to prove this?
In general:
Theorem [Nemirovski]:Let ๐ฟ(๐ฅ) be a matrix with entries affine in ๐ฅ.
It is NP-hard to test whether ๐ฟ ๐ฅ โฝ 0 for all ๐ฅ in a full-dimensional box ๐ต.
Generic instance I of a known
NP-hard problem
Instance J of problem we are
interested in
Construct J from I
Reduction
![Page 6: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/6.jpg)
Are we done?No!
6
Issue 1: We want to show strong NP-hardness. Nemirovskiโs result shows weak NP-hardness.
Issue 2: Not every affine polynomial matrix is a valid Hessian!
Example: ๐ฟ ๐ฅ1, ๐ฅ2 =10 2๐ฅ1 + 1
2๐ฅ1 + 1 10. We have
๐๐ฟ11(๐ฅ)
๐๐ฅ2โ
๐๐ฟ12(๐ฅ)
๐๐ฅ1.
![Page 7: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/7.jpg)
Dealing with Issue 1 (1/5)Reminder: weak vs strong NP-hardness
โข Distinction only concerns problems where input is numerical
โข Max(I): largest number in magnitude that appears in the input of instance I (numerator or denominator)
โข Length(I): number of bits it takes to write down input of instance I
7
Strong Weak
โข There are instances ๐ผ that are hard with Max(๐ผ)โค ๐(Length(๐ผ)) (๐ is a polynomial)
โข No pseudo-polynomial algorithm possible
โข Examples:
Max-Cut
Sat
โข The instances that are hard may contain numbers of large magnitude (e.g., 2๐).
โข Pseudo-polynomial algorithms possible
โข Examples:
Partition
Knapsack
![Page 8: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/8.jpg)
Dealing with Issue 1 (2/5)
Why weakly NP-hard?
8
Theorem [Nemirovski]: INTERVAL-PSDNESSLet ๐ฟ(๐ฅ) be a matrix with entries affine in ๐ฅ.
It is (weakly) NP-hard to test whether ๐ฟ ๐ฅ โฝ 0 for all ๐ฅ in a full-dimensional box ๐ต.
PARTITION:
Input: ๐ โ โ๐ such that ๐ 2 โค 0.1
Test: does there exist ๐ก โ โ1,1 ๐
such that ฯ๐ ๐๐๐ก๐ = 0?
INTERVAL PSDNESS
Construct: ๐ถ = ๐ผ๐ โ ๐๐๐โ1,
๐ = ๐ โ ๐โ2 ๐ , where ๐ ๐ = smallest cd of ๐.
Take: ๐ต = โ1,1 ๐ and ๐ฟ ๐ฅ =๐ถ ๐ฅ๐ฅ๐ ๐
.
Test: Is ๐ฟ ๐ฅ โฝ 0 โ๐ฅ โ ๐ต?Show: No to PARTITION โ Yes to INTERVAL PSDNESS
REDUCTION
Weakly NP-hardOperation that can make the numbers in the instance blow up
Example: ๐ด =
1 0 0 0โ1 1 0 0โฎโ1
โฑโ1
โฑโ1
01
but one of the entries of ๐ดโ1 is 2๐โ2!=๐1๐3๐10 ๐2 ๐4๐8
![Page 9: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/9.jpg)
Dealing with Issue 1 (3/5)
9
Theorem [Ahmadi, H.]: INTERVAL-PSDNESSLet ๐ฟ(๐ฅ) be a matrix with entries affine in ๐ฅ.
It is strongly NP-hard to test whether ๐ฟ ๐ฅ โฝ 0 for all ๐ฅ in a full-dimensional box ๐ต.
MAX-CUT:
Input: simple graph G=(V,E) with ๐ = ๐ and adj. matrix A, and a
positive integer ๐ โค ๐2
Test: does there exist a cut in the graph of size greater or equal to ๐?
INTERVAL PSDNESS
Construct: ๐ผ =1
๐+1 3 , ๐ถ = 4๐ผ(๐ผ๐ + ๐ผ๐ด)
๐ =๐
4๐ผ+ ๐ โ 1 โ
1
4๐๐๐ด๐
Take: ๐ต = โ1,1 ๐ and ๐ฟ ๐ฅ =๐ถ ๐ฅ๐ฅ๐ ๐
.
Test: Is ๐ฟ ๐ฅ โฝ 0 โ๐ฅ โ ๐ต?Show: No to MAX-CUT โ Yes to INTERVAL PSDNESS
REDUCTION
Strongly NP-hard Taylor series of 4๐ผ ๐ผ โ ๐ผ๐ด โ1 truncated at the first term
Scaling needed so that ๐ผ๐ โ ๐ผ๐ด โ1 โ ๐ผ๐ + ๐ผ๐ด
Preserves strong NP-hardness
![Page 10: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/10.jpg)
Dealing with Issue 1 (4/5)In more detail: No to MAX-CUT โ Yes to INTERVAL PSDNESS
10
No cut in ๐บ of size โฅ ๐ [max๐ฅโ โ1,1 ๐
1
4ฯ๐,๐ ๐ด๐๐(1 โ ๐ฅ๐๐ฅ๐)] โค ๐ โ 1
Size of largest cut in ๐บ
โ
โ
[ max๐ฅโ โ1,1 ๐
โ1
4๐ฅ๐๐ด๐ฅ] โค โ
1
4๐๐๐ด๐ + ๐ โ 1โ[ max
๐ฅโ โ1,1 ๐
1
4๐ฅ๐ ๐ + 1 3๐ผ๐ โ ๐ด ๐ฅ] โค
๐ ๐+1 3
4โ
1
4๐๐๐ด๐ + ๐ โ 1 โ ๐
๐ผ = ๐ + 1 3
Convex
โ
[ max๐ฅโ[โ1,1]๐
1
4๐ฅ๐ ๐ผ๐ผ๐ โ ๐ด ๐ฅ] โค ๐ โ
1
4๐ฅ๐ ๐ผ๐ผ๐ โ ๐ด ๐ฅ โค ๐, โ๐ฅ โ โ1,1 ๐
โ
๐ฅ๐๐ถโ1๐ฅ โค ๐ +1
4, โ๐ฅ โ โ1,1 ๐
Approximation ๐ถโ1 โ1
4(๐ผ๐ผ โ ๐ด)
Approximation error
โ
Schurcomplement
๐ฟ ๐ฅ =๐ถ ๐ฅ
๐ฅ๐ ๐ +1
4
โฝ 0, โ๐ฅ โ โ1,1 ๐
![Page 11: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/11.jpg)
Dealing with Issue 1 (5/5)For converse: Yes to MAX-CUT โ No to INTERVAL PSDNESS
โข Initial problem studied by Nemirovski
โข Of independent interest in robust control
11
There is a cut of size โฅ ๐:
Let เท๐ฅ๐ = แ1 if node ๐ on one side of cut
โ1 if node ๐ on other side of cutโ Similar steps
to previously โ เท๐ฅ๐๐ถโ1 เท๐ฅ โฅ ๐ +3
4> ๐ +
1
4
โ เท๐ฅ โ โ1,1 ๐ s.t. ๐ฟ เท๐ฅ 0โ
Corollary [Ahmadi, H.]: Let ๐ be an integer and let เท๐๐๐ , เดค๐๐๐ be rational numbers
with เท๐๐๐ โค เดค๐๐๐ and เท๐๐๐ = เท๐๐๐ and เดค๐๐๐ = เดค๐๐๐ for all ๐ = 1,โฆ , ๐ and ๐ = 1,โฆ , ๐.
It is strongly NP-hard to test whetherall symmetric matrices with entries in [ เท๐๐๐; เดค๐๐๐] are positive semidefinite.
![Page 12: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/12.jpg)
Dealing with Issue 2 (1/3)
Proof: Reduction from INTERVAL PSDNESS
12
Theorem [Ahmadi, H.] CONV3BOXIt is strongly NP-hard to test convexity of polynomials of degree 3 over a box.
INTERVAL PSDNESSInput: ๐ฟ ๐ฅ , ๐ต
Test: Is ๐ฟ ๐ฅ โฝ 0, โ๐ฅ โ ๐ต?
Problem: How to construct a cubic polynomial ๐ from ๐ฟ(๐ฅ)?Idea: Want ๐ป2๐ ๐ฅ = ๐ฟ ๐ฅ .Issue: Not all ๐ฟ(๐ฅ) are valid Hessians!
Key ideas for the construction of ๐:
โข Start with ๐ ๐, ๐ =๐
๐๐๐ป๐ณ ๐ ๐
โข For ๐ป2๐ ๐ฅ, ๐ฆ to be able to be psd when ๐ฟ ๐ฅ โฝ 0 , we need to have
a nonzero diagonal: add ๐ถ
๐๐๐ป๐ to ๐ ๐ฅ, ๐ฆ .
โข ๐ฟ ๐ฅ and ๐ป(๐ฆ) do not depend on the same variable: what if โ(๐ฅ, ๐ฆ) s.t. ๐ฟ ๐ฅ = 0 but ๐ป ๐ฆ is not? The matrix cannot be psd: add ๐
2๐ฆ๐๐ฆ to ๐ ๐ฅ, ๐ฆ .
๐ป2๐ ๐ฅ, ๐ฆ =0
1
2๐ป(๐ฆ)
1
2๐ป ๐ฆ ๐ ๐ฟ ๐ฅ
๐ป2๐ ๐ฅ, ๐ฆ =๐ถ๐ฐ๐
1
2๐ป(๐ฆ)
1
2๐ป ๐ฆ ๐ ๐ฟ ๐ฅ
๐ป2๐ ๐ฅ, ๐ฆ =๐ถ๐ฐ๐
1
2๐ป(๐ฆ)
1
2๐ป ๐ฆ ๐ ๐ฟ ๐ฅ + ๐๐ผ๐+1
โ ๐ ๐ฅ =1
2๐ฆ๐๐ฟ ๐ฅ ๐ฆ +
๐ผ
2๐ฅ๐๐ฅ +
๐
2๐ฆ๐๐ฆ, ๐ต = โ1,1 2๐+1
![Page 13: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/13.jpg)
Dealing with Issue 2 (2/3)Show NO to INTERVAL PSDNESS โ NO to CONV3BOX.
This is equivalent to:
Need to leverage extra structure of ๐ฟ ๐ฅ : ๐ฟ ๐ฅ =๐ถ ๐ฅ
๐ฅ๐ ๐ +1
4
13
โ าง๐ฅ โ โ1,1 ๐ s.t. ๐ฟ าง๐ฅ โฝ 0 โ โ เท๐ฅ, เท๐ฆ โ โ1,1 2๐+1 , ๐ง s.t. ๐ง๐๐ป2๐ เท๐ฅ, เท๐ฆ ๐ง < 0
๐ป2๐ ๐ฅ, ๐ฆ =
๐ผ๐ผ๐ ๐ป(๐ฆ)
๐ป ๐ฆ ๐๐ถ + ๐๐ผ๐ ๐ฅ
๐ฅ๐ ๐ +1
4+ ๐
๐ป2๐ เท๐ฅ, เท๐ฆ =
๐ผ๐ผ๐ ๐ ๐๐ ๐ช + ๐๐ผ๐ เดฅ๐
๐ เดฅ๐๐ป ๐ +๐
๐+ ๐
๐ง๐๐ป2๐ เท๐ฅ, เท๐ฆ ๐ง =0
โ๐ถโ1 าง๐ฅ1
๐ ๐ผ๐ผ๐ ๐ ๐๐ ๐ช + ๐๐ผ๐ เดฅ๐
๐ เดฅ๐๐ป ๐ +๐
๐+ ๐
0โ๐ถโ1 าง๐ฅ
1= ๐ +
1
4โ าง๐ฅ๐๐ถโ1 าง๐ฅ + ๐(1 + ๐ถโ1 าง๐ฅ
2
2)
< ๐ as ๐ณ เดฅ๐ โฝ ๐Appropriately scaled so that ๐ง๐๐ป2๐ เท๐ฅ, เท๐ฆ ๐ง remains <0.
Candidates: เท๐ฅ = าง๐ฅ, เท๐ฆ = 0, ๐ง =0
โ๐ถโ1 าง๐ฅ1
Candidates: เท๐ = เดฅ๐, เท๐ = ๐, ๐ง =0
โ๐ถโ1 าง๐ฅ1
Candidates: เท๐ฅ = าง๐ฅ, เท๐ฆ = 0, ๐ =๐
โ๐ชโ๐เดฅ๐๐
![Page 14: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/14.jpg)
Dealing with Issue 2 (3/3)Show YES to INTERVAL PSDNESS โ YES to CONV3BOX.
This is equivalent to:
Butโฆ
14
๐ฟ ๐ฅ โฝ 0 โ๐ฅ โ โ1,1 ๐ โ ๐ป2๐ ๐ฅ, ๐ฆ =๐ผ๐ผ๐
1
2๐ป(๐ฆ)
1
2๐ป ๐ฆ ๐ ๐ฟ ๐ฅ + ๐๐ผ๐+1
โฝ 0, โ ๐ฅ, ๐ฆ โ โ1,1 2๐+1
โ
โฝ ๐โ๐ โ โ๐, ๐ ๐
(Assumption)
๐ณ ๐ + ๐ผ๐ฐ๐+๐ โ๐
๐๐ถ๐ฏ ๐ ๐ป๐ฏ ๐ โฝ 0, โ ๐ฅ, ๐ฆ โ โ1,1 2๐+1
๐ถ chosen large enough so thatโฝ ๐ โ๐ โ โ๐, ๐ ๐+๐
๐ป2๐ ๐ฅ, ๐ฆ โฝ 0, โ ๐ฅ, ๐ฆ โ โ1,1 2๐+1Schur
![Page 15: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/15.jpg)
CorollaryCompletely classifies the complexity of testing convexity of a polynomial ๐ of degree ๐ over a box for any integer ๐ โฅ 1.
15
๐ ๐ = 1
๐ is alwaysconvex
๐ = 2
๐ป2๐(๐ฅ)constant
๐ = 3
Previous theorem (strongly NP-hard)
๐ = 4 and above
Strongly NP-hardProof sketch: โข ๐ ๐ฅ1, โฆ , ๐ฅ๐ =cubic polynomial for which testing
convexity over a box ๐ต is hard
โข ๐ ๐ฅ1, โฆ , ๐ฅ๐, ๐ฅ๐+1 = ๐ ๐ฅ1, โฆ , ๐ฅ๐ + ๐ฅ๐+1๐
โข เทจ๐ต = ๐ต ร [0,1]
We have ๐ป2๐ ๐ฅ, ๐ฅ๐+1 =๐ป2๐(๐ฅ) 0
0 ๐ ๐ โ 1 ๐ฅ๐+1๐โ2
โ ๐ป2๐ ๐ฅ, ๐ฅ๐+1 โฝ 0 on เทจ๐ต โ ๐ป2๐ ๐ฅ โฝ 0 on ๐ต
![Page 16: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/16.jpg)
Summaryโข Interested in testing convexity of a polynomial over a box.
โข Showed that strongly NP-hard to test convexity of cubics over a box.
โข Gave a complete characterization of the complexity of testing convexity over a box depending on the degree of the polynomial.
โข In the process, strengthened a result on the complexity of testing positive semidefiniteness of symmetric matrices with entries belonging to intervals.
16
![Page 17: On Convexity of Polynomials over a Box...of Polynomials over a Box Georgina Hall Decision Sciences, INSEAD Joint work with Amir Ali Ahmadi ORFE, Princeton University 1 Convexity over](https://reader036.vdocuments.us/reader036/viewer/2022062603/5f1e750127847e71d52b76ea/html5/thumbnails/17.jpg)
Thank you for listeningQuestions?
Want to learn more?
https://scholar.princeton.edu/ghall
17