backdoors in the context of learning (short paper)
DESCRIPTION
Backdoors in the Context of Learning (short paper). Bistra Dilkina, Carla P. Gomes, Ashish Sabharwal Cornell University SAT-09 Conference Swansea, U.K., June 30, 2009. SAT: Gap between theory & practice. Boolean Satisfiability or SAT : - PowerPoint PPT PresentationTRANSCRIPT
SAT 2009 Ashish Sabharwal
Backdoors in the Context of Learning(short paper)
Bistra Dilkina, Carla P. Gomes, Ashish Sabharwal
Cornell University
SAT-09 Conference
Swansea, U.K., June 30, 2009
1
SAT 2009 Ashish Sabharwal
• Boolean Satisfiability or SAT :– Given a Boolean formula F in conjunctive normal form
e.g. F = (a or b) and (¬a or ¬c or d) and (b or c)determine whether F is satisfiable
– NP-complete [note: “worst-case” notion]
– widely used in practice, e.g. in hardware & software verification, design automation, AI planning, …
• Large industrial benchmarks (10K+ vars) are solved within seconds by state-of-the-art complete/systematic SAT solvers
• Even 100K or 1M not completely out of question• Good scaling behavior seems to defy “NP-completeness”!
Real-world problems have tractable sub-structure
“Backdoors” help explain how solvers canget “smart” and solve very large instances
SAT: Gap between theory & practice
2
not quite Horn-SATor 2-SAT…
SAT 2009 Ashish Sabharwal
(~500 vars)
Informally:
A backdoor to a given problem is a subset of its variables such that, once assigned values, the remaining instance simplifies to a tractable class.
Formally:define a notion of a poly-time “sub-solver” handles tractable substructure of problem instance e.g. unit prop., pure literal elimination, CP filtering, LP solver, …
• Weak backdoors for finding feasible solutions
• Strong backdoors for finding feasible solutions or proving unsatisfiability
Backdoors to TractabilityBackdoors to Tractability
3
A notion to capture “hidden structure”
SAT 2009 Ashish Sabharwal
The notion of backdoors has provided powerful insights, leading totechniques like randomization, restarts, and algorithm portfolios for SAT
Domain Instance Vars Clause %Vars in Bgraph coloring gcp 1500 187556 0.43planning map_50_97 38364 438840 0.25game theory pne 5000 98930.79 0.64car configuration C210_FS_RZ 1755 5764.333 0.70car configuration C210_FW_UT 2024 9720 0.74verification ssa0432-003 435 1027 3.91verification bf2670-001 1393 3434 2.80verification bf1355-638 2177 6768 10.66
Are backdoors small in practice?
4
Enough to branch on backdoor variables to “solve” the formula heuristics need to be good on only a few vars
SAT 2009 Ashish Sabharwal
• “Traditional” backdoors are defined for a basic tree-search procedure, such as pure DPLL– Oblivious to the now-standard (and essential) feature of
learning during search, i.e, clause learning for DPLL
• Note: state-of-the-art SAT solvers rely heavily on clause learning, especially for industrial and crafted instances– provably leads to shorter proofs for many unsatisfiable formulas
– significant speed-up on satisfiable formulas as well
Does clause learning allow for smaller backdoorswhen capturing hidden structure in SAT instances?
This Talk: Motivation
5
SAT 2009 Ashish Sabharwal
Affirmative answer:
1. First, must extend the notion of backdoors to clause learning SAT solvers: take ‘order-sensitivity’ into account
2. Theoretically, learning-sensitive backdoors for SAT solvers with clause learning (“CDCL solvers”) can be exponentially smaller than traditional strong backdoors
3. Initial empirical results suggesting that in practice,– More learning-sensitive backdoors than traditional (of a given size)– SAT solvers often find much smaller learning-sensitive backdoors
than traditional ones
This Talk: Contribution
6
SAT 2009 Ashish Sabharwal
Input: CNF formula FAt every search node:
– branch by setting a variable to True or False;current partial variable assignment:
– consider simplified sub-formula F|
– apply a poly-time inference procedure to F|(e.g. unit prop., pure literal test, failed literal test / “probing”) Contradiction learn a conflict clause Solution declare satisfiable and exit
Not solved continue branching
“sub
-sol
ver”
fo
r S
AT
DPLL Search with Clause Learning
7
SAT 2009 Ashish Sabharwal
Traditional Backdoor
Bac
kdoo
r
{Sub-solver
infers solution
x
y
z
w
=1
=0
=1
=1
{B
ackd
oor?
Search Tree to Solution
Contradiction:Conflict clause
learnedEarly contradictiondue to previouslylearned clause
Sub-solver infers solutionwith help from
learned clauses
x
y y
=0 =1
=1=0 =0
Search order matters!Search order matters!
Backdoors and Search with Learning
8
SAT 2009 Ashish Sabharwal
Definition [Williams, Gomes, Selman ’03]:
A subset B of variables is a strong backdoor(for F w.r.t. a sub-solver S) if for every truth assignment to variables in B,
S “solves” F|.
Issue: oblivious to “previously” learned clauses; sub-solver must infer contradiction on F| for every from scratch.
“Traditional” Backdoors
9
either finds a satisfying assignment for For proves that F is unsatisfiable
SAT 2009 Ashish Sabharwal
Definition:
A subset B of variables is a learning-sensitive backdoor(for F w.r.t. a sub-solver S) if there exists a search order s.t. a clause learning solver
– branching only on the variables in B– in this search order– with S as the sub-solver at each leaf
“solves” F.
New: Learning-Sensitive Backdoors
10
either finds a satisfying assignment for For proves that F is unsatisfiable
Theoretical Results
SAT 2009 Ashish Sabharwal
Setup:
• Sub-solver: unit propagation
• Clause learning scheme: 1-UIP
• Comparison w.r.t. traditional strong backdoors
Theorem 1: There are unsatisfiable SAT instances for which learning-sensitive backdoors are exponentially smaller than the smallest traditional strong backdoors.
Theorem 2: There are satisfiable SAT instances for which learning-sensitive backdoors are smaller than the smallest traditional strong backdoors.
Learning-Sensitive Backdoors Can Provably be Much Smaller
12
used Rsat for experiments
SAT 2009 Ashish Sabharwal
Proof Idea: Simple Example
13
{x} is a learning-sensitive backdoor (of size 1) :
x=0
p1
p2
qa b
contradiction
Learn 1-UIP clause:(q)
x=1a b
contradictionq
r
With clause learning, branching on xin the right order suffices to prove unsatisfiability(x appears only
in a “long” clause)
SAT 2009 Ashish Sabharwal
Proof Idea: Simple Example
14
In contrast, without clause learning, must branch onat least 2 variables in every proof of unsatisfiability! every “traditional” strong backdoor is of size ≥ 2
Why?•every variable, in at least one polarity, only in “long” clausese.g., p1, q, r, a do not appear in any 2-clauses
•therefore, no unit prop. or empty clause generation by fixing this variable to this value•therefore, this variable by itself cannot be a strong backdoor
SAT 2009 Ashish Sabharwal
Construct an unsatisfiable formula F on n vars. such that
1. certain long clauses must be used in every refutation(i.e., removing a long clause makes F satisfiable)
2. many variables in at least one polarity appear only in such long clauses with (n) variables Controlled unit propagation / empty clause generation Must branch on essentially all variables of the long clauses to
derive a contradiction Such variables must be part of every traditional backdoor set
3. With learning: conflict clauses from previous branches on O(log n) “key variables” enable unit prop. in long clauses
Proof Idea: Exponential Separation
15
SAT 2009 Ashish Sabharwal
Corollary (follows from the proof of Theorem 1) :
There are unsatisfiable SAT instances for which learning-sensitive backdoors w.r.t. one value ordering are exponentially smaller than the smallest learning-sensitive backdoors w.r.t. another value ordering.
Order-Sensitivity of Backdoors
16
Experimental evaluation
SAT 2009 Ashish Sabharwal
Learning-Sensitive Backdoors in Practice
18
Preliminary evaluation of smallest backdoor size Reporting “best found” backdoors over 5000 runs of Rsat (with clause learning) or Satz-rand (no learning) :
•up to 10x smaller than traditional on satisfiable instances•often 2x or less smaller than traditional on unsatisfiable instances
SAT 2009 Ashish Sabharwal
• Considering only the size of the smallest backdoor does not provide much insight into this question
• One way to assess this difficulty:– How many backdoors are there of a given cardinality?
• Experimental setup:– For each possible backdoor size k, sample uniformly at random
subsets of cardinality k from the (discrete) variables of the problem
– For each subset, evaluate whether it is a backdoor or not
How hard is it to find small backdoor sets with learning?
19
Recently reported in a paper at CPAIOR-09(backdoors in the context of optimization problems)
SAT 2009 Ashish Sabharwal
Backdoor Size Distribution
20
E.g., for a Mixed Integer Programming (MIP)optimization instance:
SAT 2009 Ashish Sabharwal
Added Power of Learning
21
E.g., for a Mixed Integer Programming (MIP)optimization instance:
SAT 2009 Ashish Sabharwal
• Defined backdoors in the context of learning during search (in particular, clause learning for SAT solvers)
• Proved that learning-sensitive backdoors can be smaller than traditional strong backdoors– Exponentially smaller on unsatisfiable instances– Somewhat smaller on satisfiable instances (open?)
• Branching order affects backdoor size as well
Future work: stronger separation for satisfiable instances; detailed empirical study
Summary
22