Time-Space Tradeoffs in Resolution:

Lower Bounds for Superlinear Space

Chris BeckPrinceton University

Joint work with Paul Beame & Russell Impagliazzo

Resolution• Lines are clauses, one simple proof step

• Three basic flavors: Tree-like, Regular, and General Resolution

Proof DAG

“Regular”:On every root to leaf path,no variable resolved more than once.

SAT Solvers• Well-known connection between SAT solvers

based on Backtracking and Resolution. • These algorithms are very powerful – historically

very successful in SAT competitions, sometimes can quickly handle CNF’s with millions of variables.

• On negative SAT instances, computation history yields a Resolution proof. – Tree-like Resolution DPLL– General Resolution DPLL + Clause Learning, e.g.

Chaff & descendants

SAT Solvers

• But while a useful tool, not a panacea for complexity. In many practical situations, Memory becomes the limiting factor. Frequently run into problems with Memory consumption leading to unacceptable runtimes.

• Question: Is this inherent? Or can the right heuristics avoid the memory bottleneck?

Connection with Resolution

• Proof Size Time for Ideal SAT Solver• Proof Space Memory for Ideal SAT Solver• In the past, much success in finding explicit

hard examples, with exponential lower bounds for Resolution Proof Size.

• Question: Can we repeat this success for Proof Space?

Connection with Resolution

• Question: Can we get strong lower bounds for Proof Space?

• Known Results: – Much success with lower bounds for space for

explicit tautologies [ET’99, ABRW’00, T’01, AD’03, N’06, NH’08, BN’08]

– Every tautology on variables has tree-like refutation of space . [Esteban, Torán ‘99] • However, these tree-like refutations are generally

impractical, because of their large size.

Size-Space Tradeoffs• [Ben-Sasson, ‘01] Results for Tree-like resolution, tradeoffs essentially

aren’t possible in this model.

• [Nordström, ‘09] Formulas with min-variable space refutation exponentially large, but with O(1) more space a linear size refutation.

• [Ben-Sasson, Nordström ‘10] Formula which can be refuted in Size O(n) with Space O(n), but Space O(n/log n) Size exp(n(1)). Uses graph pebbling tautologies, and variations.

But, these are all for Space , and SAT solvers generally can afford to store the input formula in memory.

Size-Space Tradeoffs

• Informal Question: Can we find formulas such that small space proofs are much larger, so that it is impractical to SAT solve with small memory?

• Formal Question (Ben-Sasson): “Does there exist a such that any CNF with a refutation of length also has a refutation of length in space O?”

• Theorem (this work): Certain Tseitin Tautologies of size n have refutations in Time and Space nlog n, but all refutations have TS > n1.16 log n, so > 1.16.

Tradeoffs for Regular Resolution

• Theorem (this work): For any k, certain Tseitin Tautologies of size n such that– Have regular refutations in Time nk+1, Space nk.– But with Space only nk-, for any > 0, any regular

refutation has Time at least n log log n / log log log n.

• So, for Regular Resolution, we can give a negative answer to Ben-Sasson’s question.

Space in Resolution

• Clause space . [Esteban, Torán ‘99]

• More conservative than variable space, bit space.

𝐶1

…

⊥𝐶2 x ˅𝑦 𝑥˅𝑧 𝑦˅𝑧

Time step Must be in memory

Tseitin Tautologies

Given an undirected graph and , define a CSP:

Boolean variables:Parity constraints:

When has odd total parity, CSP is unsat.

Tseitin Tautologies• When odd, G connected, corresponding CNF is a

Tseitin Tautology. [Tseitin ‘68]

• Only total parity of matters

• Hard when G is a constant degree expander: [Urqhart 87]: Resolution size .[Torán 99]: Resolution space

• This work: Tradeoffs on grid, , and similar graphs, using isoperimetry.

Graph

• Take as our graph and form the Tseitin tautology,

• We’ll take but it’s only important that it is • Formula size .

A Refutation• Tseitin tautologies can be viewed as a system of

inconsistent -linear equations. If we add them all together, get 1=0, contradiction.

• If we order the vertices (equations) intelligently (column-wise), then we never talk about more than variables at any one time in this derivation.

• Implicational completeness means Resolution can simulate this – yields Size, Space

• Can also do a “binary search” refutation. Idea is to repeatedly bisect the graph and case out on values of edges in the cut.

• Once we split the CNF into two pieces, can discard one of them based on parity of cut.

• After queries, we’ve found a violated input clause – idea yields tree-like proof with Space , Size (Savitch-like savings)

A Different Refutation

Complexity Measure

• Say an assignment to an (unsat) CNF is a critical if it violates only one constraint.

• For to Tseitin formula , “’s vertex”:

• For any Clause , define the “critical vertex set”:

vertex of that constraint

Critical Set Examples Blue = 0Red = 1

function: one odd vertex in corner graph = Grid

ln

For the empty assignment, Critical set is everything.

Critical Set Examples Blue = 0Red = 1

ln

For an assignment that doesn’t cut the graph, Critical set is … still everything.

Critical Set Examples Blue = 0Red = 1

ln

For this assignment, several components.Assignment is all zeros, Upper left is critical.

Critical Set Examples Blue = 0Red = 1

ln

Complexity Measure

• Define . Then is a subadditive complexity measure: , , , when

• Very Useful: Every edge on the boundary of is assigned by . If graph has an isoperimetric inequality, then “medium complexity” clauses are wide.

Complexity vs. Time

• Consider the time ordering of any proof, and plot complexity of clauses in memory vs. time

• Only constraints – start low, end high, and because of subadditivity, cannot skip over any [t, 2t] window of μ-values on the way up.

Time

μ

Input clauses

⊥

Complexity vs. Time

• Consider the time ordering of any proof, and divide time into equal epochs (fix later)

Time

Hi

Med

Low

Two Possibilities

• Consider the time ordering of any proof, and divide time into equal epochs (fix later)

• Either, a medium complexity clause appears in memory at at least one of the breakpoints between epochs,

Time

Hi

Med

Low

Two Possibilities

• Consider the time ordering of any proof, and divide time into equal epochs (fix later)

• Or, all breakpoints only have Hi and Low. Must have an epoch which starts Low ends Hi, and therefore has clauses of every Medium level.

Time

Hi

Med

Low

Medium Sets have large BoundaryClaim: For any .

Proof:If has at least partially full columns, each gives edges as desired. Without loss there is at least one full column, and at least one empty, since at most half are full, only a few are partial.

Medium Sets have large BoundaryClaim: For any .

Proof:Suppose S has a full column and an empty column. For any two columns, has edge disjoint paths between them. QED.

Intuition

• Bottleneck Counting intuition: Since medium clauses are wide, they don’t do much work.

• A random assignment has chance to falsify one, so if first scenario is not very significant.

Time

Hi

Med

Low

Idea: Random Restrictions

• A restriction is a partial assignment of truth values to variables, simplifying formulas.

• a CNF, yields restricted formula

• a proof of , yields restricted refutation of size, space don’t increase.

• Typical to choose a restriction randomly, esp. when using bottleneck counting intuition.

Intuition

• Time divided into epochs

• If we apply a random restriction, and is small, then typically first scenario will not occur in restricted proof.

Time

Hi

Med

Low

Intuition

• Time divided into epochs

• But if is also small, then this scenario is also unlikely in the restricted proof. This part is one of our primary technical contributions.

Time

Hi

Med

Low

Extended Isoperimetric InequalityFor both medium,

have

Two medium sets of different sizes boundary guarantee doubles.

Extends: medium sets of superincreasing sizes.

RestrictionsWe show, for a random restriction w that for any clauses ,

2nd scenario is unlikely if epochs are small

Analysis

• Restricted proof, time divided into epochs

• Bound probability of first asTime

Hi

Med

Low

𝑚 ∙𝑆 ∙( 23 )(1−𝑜 (1))𝑛2

# epochs

# clauses in memory Pr for clause to become medium

Analysis

• Restricted proof, time divided into epochs

• Bound second as:( # medium levels)

Time

Hi

Med

Low

𝑚 ∙( 𝑇𝑚 )𝑘∙( 23 )

𝑘(1−𝑜 (1))𝑛2

# k-tuples per epochPr for k clauses to become mediumAnd have sizes a factor of 2 different

Time Space Tradeoff

• Taking

• So since ,

1≤𝑚 ( 𝑇𝑚 )𝑘

( 23 )𝑘 (1−𝑜 (1 ))𝑛2

+𝑚𝑆( 23 )(1−𝑜 (1) )𝑛2

𝑇 𝑘𝑆𝑘−1≥ 12 ( 32 )

(2𝑘−1) (1−𝑜 (1 ) )𝑛2

𝑇𝑆≥( 94 )(1−𝑜 ( 1) )𝑛2

=21.1699(1−𝑜 (1 ))𝑛2

Regular Resolution • Can define partial information precisely

• Complexity is monotonic wrt proof DAG edges.

• Random Adversary selects random assignments based on proof– No random restrictions

• Precise information implies can apply division into epochs recursively

Yields superpolynomial lower bound

Open Questions

• Extend subdivision technique to General Resolution? This would settle Ben-Sasson’s question.

• Better bounds on his constant? A lower bound greater than 2 for General Resolution will require significant new technical insight.

• Other proof systems?• Other cases for separating search paradigms:

“dynamic programming” vs. “binary search”?

Thanks!

Regular Resolution

• Equivalence between Regular Resolution refutations and Read-Once Branching Program for the Clause Search Problem. [Krajicek]

• Permits nice top-down analysis. Definition:

Common Information: For a clause in proof , := largest partial assignment consistent with every assignment reaching .

Regular Resolution

• Consistency Lemma: If a partial assignment leads to , agrees with on .

• Corollary: If a partial assignment leads to , .

• Idea: Build with a probabilistic strategy, try to keep large but also make many random decisions when not affected.

• Hope: Probability to reach complex is small.

Definition: Adversary Strategy

A Probabilistic Adversary, following a path from the root to the leaves of the proof.• Remembers answers already given, thought of

as a partial assignment .• If new edge is queried, (resolved on)– Doesn’t cut choose randomly– Does cut choose to maximize bad set

Adversary Examples Blue = 0Red = 1

nW

Adversary flips coins for non-cut edges.

?

Adversary Examples Blue = 0Red = 1

nW

For this cut edge, he chooses blue to maximizethe critical set.

Adversary Examples Blue = 0Red = 1

nW

This edge cuts the critical set, he keeps bigger half.

?

Adversary Examples Blue = 0Red = 1

nW

This edge cuts a “good” set. If assigned wrong,crit set becomes empty, b/c multiple odd comps.

?

Adversary therefore chooses to keep the crit set.

Observations

• Adversary’s critical set: –Decreases monotonically.– Is never empty.–Never decreases by more than a factor of

two in one step.

• But since we are flipping coins for every non-cut edge, can prove a crucial lemma:

Main Lemma• For each clause ,

where is the size of largest subset of variables assigned by which doesn’t cut G. (rank in the cut matroid of ).

• Further, for any path ,

(Conditional rank of corresponding edge sets).• Proof: Careful induction. Result is tight.

Consequences of Main Lemma

• Most clauses do not have much information:

• Typically, we have information which is helpful towards only a few complexity levels. Suppose are from distinct medium complexity levels, and our graph is grid, .

• Previous isoperimetric arguments tell us

Consequences of Main LemmaClaim: If , then , in fact as well. Proof: First for any ,These hold for any rank function.When , , the assumption implies RHS and we argued before . This bounds , but all these edges are on critical set, so also.

Consequences of Main Lemma

How to use this? Suppose . Our grid will give us medium levels. But if , we conclude that for every typical clause in a proof, all but levels have conditional probability for any of their clauses to be reached from .

Subdivision idea• Suppose have path to a typical clause s.t.

Conditional Probability, having followed , to– Reach an atypical clause, or to,– Fail to advance levels in the next time steps

is at most .• Because is typical, we can find a similar so

that we do not reach an atypical clause, or fail to advance levels in the next time steps, except with Pr .– (For of our choice)

Main Idea for Harder Tradeoff

These guys each have small Pr to be reached. Mark them, bound to hit marked is Δ < 1.By averaging, can pick a short path s.t. conditionally, bound is still < Δ.

Main Idea for Harder Tradeoff

Would like to repeat and divide again, however…

Main Idea for Harder Tradeoff

Select largest consecutive stretch of bounded levels, throw away breakpoint clauses in those.

These nodes each have .At most “breaks” in the walls, because we are at a typical location.

New∆ ≤Old   ∆+𝑚𝑆𝑛− (1−𝜀 )𝑐

Main Idea for Harder Tradeoff

• Again select a path in between walls, by avg.

By the end of that epoch, adv must have come out below the wall , Pr at least ∆

?

?

Main Idea for Harder Tradeoff

Main Idea for Harder Tradeoff• Height of walls is

Rinse and repeat! times

?

𝑇 ≥ (𝑛(1−𝑜 (1) )𝑐

𝑆 )Ω ( log log𝑛log log log𝑛)