lower bounds for depth three circuits with small bottom fanin neeraj kayal chandan saha indian...
TRANSCRIPT
Lower Bounds for Depth Three Circuits with small bottom fanin
Neeraj Kayal Chandan SahaIndian Institute of Science
A lower bound
Theorem: Consider representations of a degree d polynomial of the form
If the ’s have degree one and at most variables each then there is an explicit (family) of polynomials on variables such that s is at least .
A lower boundTheorem: Consider representations of a degree d polynomial of the form
If the ’s have degree one and at most variables each then there is an explicit (family) of polynomials on variables such that s is at least .
Remark: For a generic , s must be at least
Any asymptotic improvement in the exponent will imply .
A lower boundCorollary (via Kumar-Saraf): Consider representations of a degree d polynomial of the form
If the ’s have degree one and at most variables each then there is an explicit (family) of polynomials on variables such for s is at least .
Remark: Bad news.
Good news.
Background/Motivation
Arithmetic Circuits
…𝑥2𝑥1𝑥𝑛
…
…
-1
𝑥1+𝑥2 𝑥1−𝑥2
𝑥2𝑥1𝑥𝑛
Arithmetic Circuits
…
…
…
-1
𝑥1+𝑥2 𝑥1−𝑥2
𝑥2𝑥1𝑥𝑛
Arithmetic Circuits
…
…
…
-1
𝑥2𝑥1𝑥𝑛
Arithmetic Circuits
…
…
…
-1
𝑥2𝑥1𝑥𝑛
Arithmetic Circuits
…
…
…
-1
)(xHn
𝑥2𝑥1𝑥𝑛
…
…
…
-1
)(xHn
𝑥2𝑥1𝑥𝑛
Size = Number of Edges
…
…
…
-1
)(xHn
𝑥2𝑥1𝑥𝑛
Depth
…
…
…
-1
)(xHn
𝑥2𝑥1𝑥𝑛
This talk.
> Field is .
> Gates have unbounded fanin
>
Two Fundamental Questions
Can explicit polynomials be efficiently computed?
Can computation be efficiently parallelized?
Two Fundamental Questions
Can explicit polynomials be efficiently computed? Does VP equal VNP?
Can computation be efficiently parallelized? Can every efficient computation be also done by small, shallow circuits?
Can computation be efficiently parallelized?
Question: How efficiently can we simulate circuits of
size by circuits of depth ?
Can computation be efficiently parallelized?
Theorem (Hyafil79, VSBR83, AV-08, Koiran-12,
GKKS13, Tavenas-13, Wigderson/Tavenas): Any
circuit of size s and degree can be simulated
by a -depth circuit of size
Can computation be efficiently parallelized?
Theorem (Hyafil79, VSBR83, AV-08, Koiran-12,
GKKS13, Tavenas-13, Wigderson/Tavenas): Any
circuit of size s and degree can be simulated
by a -depth circuit of size … also by a regular,
homogeneous -depth circuit of size
Can computation be efficiently parallelized?
Theorem (Hyafil79, VSBR83, AV-08, Koiran-12,
GKKS13, Tavenas-13, Wigderson/Tavenas): Any
circuit of size s and degree can be simulated
by a -depth circuit of size … also by a regular,
homogeneous -depth circuit of size
Question: Is this optimal?
Can computation be efficiently parallelized?
Theorem (Hyafil79, VSBR83, AV-08, Koiran-12,
GKKS13, Tavenas-13, Wigderson/Tavenas): Any
circuit of size s and degree can be simulated
by a -depth circuit of size … also by a regular,
homogeneous -depth circuit of size
Question: Is this optimal?
(KS15 + Ramprasad): For yes, but with caveats.
Can computation be efficiently parallelized?
Theorem (Hyafil79, VSBR83, AV-08, Koiran-12,
GKKS13, Tavenas-13, Wigderson/Tavenas): Any
circuit of size s and degree can be simulated
by a -depth circuit of size … also by a regular,
homogeneous -depth circuit of size
Corollary: Strong enough lower bounds for low-depth circuits imply VP VNP.
A possible way to approach VP vs VNP
• Strong enough lower bounds for low-depth circuits imply VP VNP.
Low Depth Circuit (’s simple)
Low-depth circuits are easy to analyze..
A possible way to approach VP vs VNP
• Strong enough lower bounds for low-depth circuits imply VP VNP.
Low Depth Circuit (’s simple)
Low-depth circuits are easy to analyze..
Lots of work on lower bounds for low depth
arithmetic circuits in recent years – hope to
discover general patterns and technical
ingredients
’s are: Lower Bound
has degree ~ in VNP GKKS13+KSS14
has degree ~ IMM FLMS14
is sparse ~ in VNP KLSS14
is sparse ~ IMM KS14
has degree one and arity IMM This work
have homogeneous depth three circuits of bottom fanin
in VNP This work
have homogeneous depth five circuits of bottom fanin
IMM Next talk
A possible way to approach VP vs VNP
Prove strong enough lower bounds for low depth
circuits.
Low Depth Circuit (’s simple)
Low-depth circuits are easy to analyze..
Lots of work on low depth arithmetic circuits recently
This talk:
• common pattern/proof strategy
• Technical ingredients
A common Proof Strategy and some technical
ingredients
Proof Strategy (’s simple ). Let .
shallow circuit C
Find a geometric property GP of the ’s.
Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”
Show that rank(M()) is “relatively large”.
Proof Strategy (’s simple ). Let .
shallow circuit C
Find a geometric property GP of the ’s.
Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”
Show that rank(M()) is “relatively large”.
Lower Bounding rank of large matrices
• If a matrix M(f) has a large upper triangular submatrix, then it has large rank
• (Alon): If the columns of M(f) are almost orthogonal then M(f) has large rank.
When the ’s have low degree (’s have low degree). Let .
shallow circuit C
Find a geometric property GP of the ’s.
Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”
Show that rank(M()) is “relatively large”.
Finding a geometric property GP of T
is a product of low degree polynomials
V(T) is a union of low-degree hypersurfaces
V(T) has lots of high-order singularities
Finding a geometric property GP of T
𝑇 (𝑥 , 𝑦 )=𝑔(𝑥 , 𝑦) ∙ h(𝑥 , 𝑦)
Finding a geometric property GP of T
is a product of low degree polynomials
V(T) is a union of low-degree hypersurfaces
V(T) has lots of high-order singularities
V( T) has lots of points
When the ’s have low degree (’s have low degree). Let .
shallow circuit C
Find a geometric property GP of the ’s.
Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”
Show that rank(M()) is “relatively large”.
When the ’s have low degree (’s have low degree). Let .
shallow circuit C
Find a geometric property GP of the ’s.
Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”
Show that rank(M()) is “relatively large”.
Expressing largeness of a variety in terms of rank
is a variety.
Let = set of degree- polynomials.
Expressing largeness of a variety in terms of rank
is a variety.
Let = set of degree- polynomials. Let = set of degree- polynomials which vanish at every point of V.
Hilbert’s Theorem (Informal): If V is “large” then has small dimension.
Expressing largeness of a variety in terms of rank
is a variety.
Let = set of degree- polynomials. Let = set of degree- polynomials which vanish at every point of V.
Hilbert’s Theorem (Informal): If V is “large” then has small dimension.
Let = { () of deg } .
Hilbert’s Theorem (Formal): If V has dimension r then has asymptotic dimension .
When the ’s have low degree (’s have low degree). Let .
shallow circuit C
Find a geometric property GP of the ’s.
Express the property GP in terms of rank of a “big” matrix M: if T has the property than rank(M(T)) is “relatively small”
Show that rank(M()) is “relatively large”.
Restrictions
Example: is a function. f(0, 1, z) is a restriction of f. Geometrically, f(0, 1, z) is same as restricting f to the axis-parallel line W = V(x, y-1). Algebraically, f(0, 1, z) is same as mod
More generally: Let be any ideal and a polynomial. Call mod as an algebraic restriction of f.
Obs: Hilbert’s theorem can be generalized for algebraic restrictions of polynomials.
Employing restrictions
Lemma (KLSS14): If Q is a sparse polynomial then for a suitable random algebraic restriction mod is a low degree polynomial.
Yields lower bounds for homogeneous depth four (KLSS14 and KS14).
Lemma (KLSS14): If T is a sparse polynomial a sum of product of low arity polynomials then for a suitable random algebraic restriction mod is a low degree polynomial.
Employing Restrictions
Lemma: If T is a sparse polynomial a sum of product of low arity polynomials then for a suitable random algebraic restriction mod is a low degree polynomial.
Employing Restrictions
Yields lower bounds for homogeneous depth five with low bottom fanin (KS15 and BC15).
Lemma (Shpilka-Wigderson): A depth three circuit C of size s can be converted to a homogeneous depth circuit C’ of size . Further if C has bottom fanin t then C’ also has bottom fanin t.
A lemma by Shpilka and Wigderson
Yields lower bounds mentioned earlier.
Conclusion
Proving lower bounds for low depth circuits is a potential way to prove lower bounds for more general circuits.
There is a meta-strategy common to many recent (and older) lower bounds. We don’t understand the power or limitations of this meta-strategy.
Open: lower bounds for homogeneous depth three circuits for polynomials of degree .