sampling techniques for boolean satisfiability

1

Sampling Techniques for Boolean Satisfiability

Kuldeep S Meel1

(Joint work with Supratik Chakraborty2, Moshe Y Vardi1)

1Department of Computer Science, Rice University2Indian Institute of Technology Bombay, India

Sept 9, 2013 COMP 600

2

Life in The 21st Century!

Was my “I Love You” message/email to my girlfriend delivered to her or her roommate?

3

Life in The 21st Century!

How do we guarantee that the systems work correctly ?

4

Motivating Example

a b

c

64 bit 64 bit

64 bit

c = ab

How do we verify that this circuit works ?• Try for all values of a and b• 2128 possibilities (1022 years)• Not scalable

• Randomly sample some a’s and b’s• Wait! None of the circuits in

the past faulted when 10 < b < 40

• Finite resources! • Let’s sample from regions where

it is likely to fault

5

Constraints DesignDesigning Constraints

• Designers: 1. 100 < b < 2002. 300 < a < 4513. 40 < a < 50 and 30 < b <

40• Past Experience:

1. 400 < a < 20002. 120 < b < 230

• Users:1. 1000<a < 11002. 20000 < b < a < 22000

Problem: How can we uniformly sample the values of a and b satisfying the above constraints?

a b

c

64 bit

64 bit

c = ab

64 bit

6

Set of Constraints

SAT Formula

Uniform Generation of SAT-Witnesses

{(0,1), (1,0), (1,1)}

SAT is NP-complete (Cook 1971)

(a V b)

7

Set of Constraints

Given a SAT formula, can one uniformly sample solutions without enumerating all solutions

SAT Formula



8

Set of Constraints

Given a SAT formula, can one uniformly sample solutions without enumerating all solutions while scaling to real world problems?

SAT Formula

Scalable Uniform Generation of SAT-Witnesses


9

Overview

Uniform Generation of SAT-witnesses

Approximate Model Counting

Future Directions

10

Overview



Future Directions

11

Prior Work

Heuristic WorkGuarantees: weakPerformance: strong

BGP Algorithm XORSample’

Theoretical WorkGuarantees: strongPerformance: weak

BDD-basedGuarantees: strongPerformance: weak

SAT-based heuristics

Guarantees: weakPerformance: strong

INDUSTRY

ACADEMIA

12

Our Contribution

Heuristic WorkGuarantees: weakPerformance: strong

BGP Algorithm XORSample’

Theoretical WorkGuarantees: strongPerformance: weak

BDD-basedGuarantees: strongPerformance: weak

SAT-based heuristics

Guarantees: weakPerformance: strong

INDUSTRY

ACADEMIA

UniWitGuarantees : strongPerformance: strong

13

Central Idea

14

Partitioning into equal “small” cells

15

How to Partition?

How to partition into roughly equal small cells of solutions without knowing the distribution of solutions?

Universal Hashing[Carter-Wegman 1979, Sipser 1983]

16

Universal Hashing Hash functions from mapping {0,1}n to {0,1}m

(2n elements to 2m cells)

Random data => All cells are roughly small

Universal hash functions: Adversarial data => All cells are roughly small

Need stronger bounds on range of the size of cells

17

Lower Universality Lower Complexity

H(n,m,r): Family of r-universal hash functions mapping {0,1}n to {0,1}m (2n

elements to 2m cells)

Higher the r => Stricter guarantees on range of size of cells

r-wise universality => Polynomials of degree r-1

Lower universality => lower complexity

18

Hashing-Based Approaches

n-universal hashing

Uniform Generation

All cells should be small

BGP Algorithm

Solution space

19

Scaling to Thousands of Variables

n-universal hashing 2-universal hashing

Uniform Generation

Random


Only a randomly chosen cells needs to be “small”

BGP Algorithm

Near Uniform Generation

UniWit

Solution space

20

n-universal hashing 2-independent hashing

Uniform Generation

Random


Only a randomly chosen cells needs to be “small”

BGP Algorithm

Near Uniform Generation

UniWit

Solution space

From tens of variables to thousands of variables!

Scaling to Thousands of Variables

21

Highlights Employs XOR-based hash functions

instead of computationally infeasible algebraic hash functions

Uses off-the-shelf SAT solver CryptoMiniSAT (MiniSAT+XOR support)

22

Strong Theoretical Guarantees Uniformity

For every solution y of RF Pr [y is output] = 1/|RF|

23

Strong Theoretical Guarantees Near Uniformity

Success Probability

Polynomial calls to SAT Solver

For every solution y of RF Pr [y is output] >= 1/8 x 1/|RF|

Algorithm UniWit succeeds with probability at least 1/8

24

Experimental Methodology Benchmarks (over 200)

Bit-blasted versions of word-level constraints from VHDL designs

Bit-blasted versions from SMTLib version and ISCAS’85

Objectives Comparison with algorithms BGP &

XORSample’

Uniformity Performance

25

Better Uniformity than State-of-art Generators

XORSample’ UniWit

• Benchmark: case110.cnf; #var: 287; #clauses: 1263

• Total Runs: 1.08x108; Total Solutions : 16384• XORSample’ could not find 772 solutions and

more than 250 solutions were generated only once

26

Results : Performance

case

47

case

105

case

203

case

61

case

15

case

_2_b

14_1

squa

ring1

4

case

_2_p

tb_1

case

_2_b

14_2

0.11

10100

100010000

100000

UniWitXORSample'

Benchmarks

Time(s)

27

Results : Performance

case

47

case

105

case

203

case

61

case

15

case

_2_b

14_1

squa

ring1

4

case

_2_p

tb_1

case

_2_b

14_2

0.11

10100

100010000

100000

UniWitXORSample'

Benchmarks

Time(s)

• UniWit is is 2-3 orders of magnitude faster than XORSample’

• Observed success probability = 0.6 ( >> theoretical guarantee of 0.125)

28

The Story So Far Theoretical guarantees of near

uniformity Major improvements in running time and

uniformity compared to existing generators

But………. How many samples should I test my system to achieve desired coverage? Are 105 samples enough?

Case A: Total solutions -106

Case B: Total solutions - 1060

29

The missing link

What is the total number of satisfying assignments to system of constraints?

30

Overview



Future Directions

31

What is Model Counting? Given a SAT formula F RF: Set of all solutions of F Problem (#SAT): Estimate the number of

solutions of F (#F) i.e., what is the cardinality of RF?

E.g., F = (a v b) RF = {(0,1), (1,0), (1,1)} The number of solutions (#F) = 3#P: The class of counting problems

for decision problems in NP!

32

Practical ApplicationsExciting range of applications!

Probabilistic reasoning/Bayesian inference

Planning with uncertainty

Multi-agent/ adversarial reasoning [Roth 96, Sang 04, Bacchus 04, Domshlak 07]

33

But it is hard! #SAT is #P-complete

Even for counting solutions of 2-CNF SAT

#P is really hard! Believed to be much harder than NP-

complete problems PH P#P

34

The Hardness of Model Counting

35

The Hardness of Model Counting

Can we do better?Approximate counting (with guarantees) suffices for most of the applications

36

Prior Work

Counters Guarantee Confidence

Remarks

Exact counter (e.g. sharpSAT, Cachet)

C = #F 1 Poor Scalability

Lower bound counters (e.g. MBound, SampleCount)

C ≤ #F d Very weak guarantees

Upper bound counters(e.g. MiniCount)

C ≥ #F d Very weak guarantees

Input Formula: F; Total Solutions: #F; Return Value: C

37

Approximate Model CountingDesign an approximate model counter G: inputs:

CNF formula F tolerance e confidence d

the count returned by it is within e of the #F with confidence at least d


38

Approximate Model CountingDesign an approximate model counter G: inputs:

CNF formula F tolerance e confidence d

the count returned by it is within e of the #F with confidence at least d and scales to real world problems

Scalable Approximate Model CountingLies in the 2nd level of Polynomial hierarchy: S2

P

39

Our ContributionInput Formula: F; Total Solutions: #FCounters Guarantee Confidenc

eRemarks

Exact counter (e.g. sharpSAT, Cachet)

C = #F 1 Poor Scalability

ApproxMC #F/(1+e)≤ C ≤ (1+ e) #F

d Scalability + Strong guarantees

Lower bound counters (e.g. MBound, SampleCount)

C ≤ #F d Very weak guarantees

Upper bound counters(e.g. MiniCount

C ≥ #F d Very weak guarantees

The First Scalable Approximate Model

Counter

40

How do we count?

41

Naïve Enumeration: Not Scalable

Not Scalable! (Think of enumerating 2100 solutions)

• Enumerate all solutions

• Exact Counting!• Cachet, Relsat,

sharpSAT

42

Counting through Partitioning

43

Counting through Partitioning

Pick a random cell

Total # of solutions= #solutions in the cell* total # of cells

44

Algorithm in Action

690 710 730 730 731 831 834………….…

t

Algorithm

45

Algorithm in Action

Algorithm

690 710 730 730 731 831 834………….…

t

Median

46

Partitioning

Linear hash functions (3-wise independent)

How to partition into roughly equal cells of solutions without knowing the distribution of solutions?

47

Theoretical ResultsApproxMC (CNF: F, tolerance: e, confidence:d)Suppose ApproxMC(F,e,d) returns C. Then,

Pr [ #F/(1+e)≤ C ≤ (1+ e) #F ] ≥ d

ApproxMC runs in time polynomial in log (1-d)-1,|F|, e-1 relative to SAT oracle

48

Experimental Methodology Benchmarks (over 200)

Grid networks, DQMR networks, Bayesian networks

Plan recognition, logistics problems Circuit synthesis

Tolerance: e= 0.75, Confidence: d = 0.9 Objectives

Comparison with exact counters (Cachet) & bounding counters (MiniCount, Hybrid-MBound, SampleCount) Performance Quality of bounds

49

Results: Performance Comparison

0 6 12 18 24 30 36 42 48 54 60 66 72 78 84 90 96 1021081141201261321381441501561621681741801860

10000

20000

30000

40000

50000

60000

70000

ApproxMCCachet

50

Results: Performance Comparison

0 6 12 18 24 30 36 42 48 54 60 66 72 78 84 90 96 1021081141201261321381441501561621681741801860

10000

20000

30000

40000

50000

60000

70000

ApproxMCCachet

51

Can Solve a Large Class of Problems

Large class of problems that lie beyond the exact counters but can be computed by ApproxMC

0 6 12 18 24 30 36 42 48 54 60 66 72 78 84 90 96 1021081141201261321381441501561621681741801860

10000

20000

30000

40000

50000

60000

70000

ApproxMCCachet

52

Mean Error: Only 4% (allowed: 75%)

Mean error: 4% – much smaller than the theoretical guarantee of 75%

0 10 20 30 40 50 60 70 80 903.2E+01

1.0E+03

3.3E+04

1.0E+06

3.4E+07

1.1E+09

3.4E+10

1.1E+12

3.5E+13

1.1E+15

3.6E+16

Cachet*1.75Cachet/1.75ApproxMC

53

Results: Bounding Counters Range of count from bounding counters = C2-C1

C1: From lower bound counters(MBound/SampleSAT) C2: From upper bound counters (MiniCount)

Range from ApproxMC: [C/(1+e), (1+e)C]

Smaller the range, better the algorithm!

54

Better Bounds Than Existing Counters

ApproxMC improved the upper bounds significantly while also improving the lower bounds

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 622.6E+02

8.2E+03

2.6E+05

8.4E+06

2.7E+08

8.6E+09

2.7E+11

8.8E+12

2.8E+14

9.0E+15

2.9E+17

ApproMCMBound/SampleCount/Mini-Count

55

Overview



Future Directions

56

Future Directions Extension to more expressive constraints (SMT)

Design of independent hash functions SMT solver – efficiently handles input + hash

function

Strong guarantees for uniform generation

Handling user bias (weighted uniform generation and weighted counting)

Scaling beyond thousands of variables

57

Collaboration Opportunities MOOC: Automatic Problem Generation

Use of templates to design questions Uniform generation of template parameters

Applications in Program Synthesis: e.g., Autotuning

Probabilistic reasoning/Bayesian inference Planning with uncertainty/adversarial

reasoning Extension to general set of constraints

Counting points in polyhedra (Loop optimization) Discrete Integration

58

Key Takeaways Uniform sampling and approximate counting are

important problems Prior work didn’t scale or offered weak guarantees UniWit: Scales and theoretical guarantees of near

uniformity ApproxMC: The first scalable approximate model counter Uses easy-to-implement linear hash functions Only a randomly chosen cell has to be small Tools are available online! Go and Try them out!

59

Publications S. Chakraborty, K.S. Meel, M.Y. Vardi “A

Scalable and Nearly-Uniform Generator of SAT-Witnesses” In Proc. of CAV 2013

S. Chakraborty, K.S. Meel, M.Y. Vardi “A Scalable Approximate Model Counter” In Proc. of CP 2013

60

Discussion

Thank You for your attention!

Acknowledgments• NSF• ExCAPE• Intel• BRNS, India• Sun Microsystems• Sigma Solutions,Inc

61

Key Takeaways Uniform sampling and approximate counting

are important problems Prior work either didn’t scale or offered weak

guarantees UniWit: Scales and provides theoretical

gurantees of near uniformity ApproxMC: The first scalable approximate

model counter Uses easy-to-implement linear hash functions Only a randomly chosen cell has to be small Tools are available online! Go and Try them

out!

62

UniWitRF

63

UniWit

IsSmall?

RF

NO

64

UniWit

?

65

UniWit

? NOIsSmall?

66

UniWit

?

??

???

67

UniWit

?

??

???

IsSmall?

YES

68

UniWit

IsSmall?

YES Select a solution randomly with probability “c” from the partition. If no solution is picked, return Failure

sampling techniques for boolean satisfiability

Documents