applications of craig interpolation to model checking k. l. mcmillan cadence berkeley labs

Applications of Craig Interpolation to Model

Checking

K. L. McMillanCadence Berkeley Labs

Copyright 2002 Cadence Design Systems. Permission is granted to reproduce without modification.

Proof!

Debugging!

Don’t have acow, man…


Interpolation

• If A B = false, there exists an interpolant A' for (A,B) such that:

A A'A' B = false

A' refers only to common variables of A,B

• Example: – A = p q, B = q r, A' = q

• Interpolants from proofs– given a resolution refutation of A B,

A' can be derived in linear time.

(Craig,57)

(Pudlak,Krajicek,97)


Applications to Model Checking

• Think of interpolation as a weak form of quantifier elimination (i.e., image comp.).– Let W = sup(A) \ sup(B)

– Then W. A is the strongest interpolant for (A,B)

– An interpolant is thus a weak approximation of the projection W. A, strong enough to refute B.

– Image computation is the most costly aspect of symbolic model checking and predicate abstraction. Interpolation allows us to weaken the image relative to a property we want to prove.


Applications

• Propositional case– Finite-state model checking using a SAT solver– Very robust method for hardware verification

• First-order case– Infinite-state model checking using a FO prover.– Verify, for example, parameterized protocols

• Predicate abstraction– Discover useful predicates for predicate

abstraction– Computation of the abstract transition relation


DPLL SAT solvers

• Input: propositional formula in clause form• Output:

– A model of the formula, or– A refutation using resolution steps.

• Highly efficient– Can handle millions of clauses.– Very effective at reducing proofs to relevant

clauses.


Resolution and interpolation

• Resolution rule:

• Given a resolution refutation for clause sets (A,B), we can derive an interpolant for (A,B) in linear time. (Pudlak,Krajicek,97)

• Interpolant is a Boolean circuit whose structure mirrors that of the proof.

(A p) (p B)

(A B)


Interpolation rules

• Interpolant is a circuit that follows structure of the proof.

A = (p)(p q) B = (q r)(r)

(p) (p q)

(q) (q r)

(r)(r)

q

=q


Interpolation-based MC

• Classic symbolic model checking iterates the strongest post-condition operator to generate the strongest inductive invariant.– Requires quantifier elimination

• We can exploit interpolation to compute an weakened post-condition operator that is strong enough to prove a given property.– Allows SAT-only symbolic model checking– Procedure is complete for finite-state systems


Modeling

System modeled by a transition constraint

ab cp

g

Notation: Q<n> means "add n primes to the symbols in Q"

g = a b

p = g c

c' = p

Model:

C = { g = a b, p = g c, c' = p }


Bounded model checking

• Unfold the model k times:

U = C<0> C<1> ... C<k-1>

ab

cp

g ab

cp

g ab

cp

g

...I<0> F<k>

• Use SAT solver to check satisfiability of I<0> U F<k>

• If unsatisfiable:• property has no Cex of length k• can produce a refutation proof P

Biere,et al. TACAS99


Reachability

• Is there a path (of any length) from I to F satisfying transition constraint C?

• Reachability fixed point:R0 = I

Ri+1 = Ri Img(Ri,C)

R = Ri

• Image operator:Img(P,C) = V'. V. (P(V) C(V,V’))

• F is reachable iff R F false


Reachability

I FR1

R2...

R

= I Img(I,C)= R1 Img(R1,C)

Strongest invariant may be very expensive


Overapproximation

• An overapproximate image op. is Img' s.t.for all P, Img(P,C) implies Img'(P,C)

• Overapprimate reachability:R'0 = I

R'i+1 = R'i Img'(R'i,C)

R' = R'i

• Img' is adequate (w.r.t.) F, when– if P cannot reach F, Img’(P,C) cannot reach F

• If Img' is adequate, then– F is reachable iff R' F false


Adequate image

P F

Img(P,C)

Reached from P Can reach F

Img’(P,C)

But how do you get an adequate Img'?


k-adequate image operator

• Img' is k-adequate (w.r.t.) F, when– if P cannot reach F, Img’(P,C) cannot reach F within k steps

• Note, if k > diameter, then k-adequate is equivalent to adequate.


Interpolation-based image

• Idea -- use unfolding to enforce k-adequacyA = P<-1> C<-1>

B = C<0> C<1> C<k-1> F<k>

P FC C C C C C C

A B

t=0 t=k

Let Img'(P)0= A',

where A' is an interpolant for (A,B)...

Img' is k-adequate!


Huh?

• A A'– Img(P,C) Img'(P,C)

• A' B = false– Img'(P,C) cannot reach F in k steps

• Hence Img' is k-adequate overapprox.

P FC C C C C C C

A B

t=0 t=k

A'

Note: if A,B are consistent, then let Img’(P,C) = T.


Intuition

• A' tells is everything the prover deduced about the image of P in proving it can't reach F in k steps.

• Hence, A' is in some sense an abstraction of the image relative to the property.

P FC C C C C C C

A B

t=0 t=k

A'


Reachability algorithm

let k = 0repeat if I can reach F within k steps, answer reachable R = I while Img'(R,C) F = false

R' = Img'(R,C) R if R' = R answer unreachable R = R' end while increase kend repeat


Termination

• Since k increases at every iteration, eventually k > d, the diameter, in which case Img' is adequate, and hence we terminate.

Notes:– don't need to know when k > d in order to

terminate– often termination occurs with k << d– depth bound for earlier method (Sheeran et al

'00) is "longest simple path", which can be exponentially longer than diameter


Performance v. Localization

time, interpolation method

tim

e,

pro

of-

base

d a

bst

ract

ion

Source: Nina Amla


k-bound comparisonp

roof-

base

d a

bst

ract

ion

, la

st k

interpolation last k


Interpolation v K-inductionR

un

tim

e,

k-in

du

ctio

n

Run time, interpolation


From finite- to infinite-state

• Up to now…– Model checking finite state systems – Interpolants for propositional formulas

• Now we extend to…– Model checking infinite-state systems– Interpolants for first-order formulas

• Craig 1957 gives a straightforward method that eliminates individual variables by adding quantifiers...

...but our goal is to eliminate quantifiers!


Quantifier-free interpolants

• In LIUF, we can obtain quantifier-free interpolants from proofs.

• Linear inequalities (over rationals)– handle index and pointer arithmetic

• Uninterpreted functions– model memory and register file contents

Here, we will just give the "intuition".For precise rules, see references in the paper.


Linear inequalities

• Naturally generated by Simplex procedure, Fourier-Motzkin.

• Interpolants can be computed in linear time

(0 x) (0 y)

(0 c1x + c2y)0 c1,c2

Pudlak,1997


Interpolation example

A = (0 x-y) (0 y-z-1) B = (0 z-x)

(0 y-z-1) (0 z-x)

(0 y-x-1) (0 x-y)

(0 -1)

(0 y-z-1) (0 0)

(0 y-z-1) (0 x-y)

(0 x-z-1)

Just sum the inequalities from A, and you get an interpolant.


Adding EUF

• Rules for equality and uninterpreted functions– Transitivity

– Equality contradiction

– Congruence

a = b b = c

a = c

a = b a b

a=b

f(a)=f(b)


Interpolants and transitivity

• Prove c = m by transitivity chain• Interpolant is [c = e, h = k]

– summarizes "subchains" on A side– solves all A variables in terms of common vars

A B

c d ef

ghi

jk m


Transitivity rule

• Apply transitivity to eliminate center variable from interpolant.

A B

c d ef

ghi

jk m

c = i i = mc = m

[c = e, i = h] [i = k]

[c = e, h = k]


Equality contradiction rule

• Use solns for c,i to rewrite disequality.• Obtain disequality over common vars.

A B

c d ef

ghi

c = i

[c = e, i = h]

(c i) in A

[e h]


Congruence rule

• Use congruence to derive a soln for f(c).• Doesn't work for arity > 1

– can solve this problem, but it's complicated!

A B

c d eq

ghi

jk m

c = mf(c) = f(m)

[c = e, h = k]

f(c) f(e)

[f(c) = f(e), h = k]


Putting it together

• Also have rules for:– combining equalities and inequalities.– connecting to Boolean reasoning.

• This allows us to derive quantifier-free interpolants from proofs in LIUF.

• Can also extend to...– restricted integer arithmetic– restricted use of select/update

• Note: quantifiers in A,B still yield quantifiers in the interpolant.


Infinite-state model checking

• Same as finite-state case, but express transition constraints in LIUF.– now, no termination guarantee

• Some simple examples...– Safety of N-process "bakery" algorithm– Safety of simplified N-process cache protocol– Fisher's timed mutual exclusion

But note, for systems with function symbols inthe state, we do sometimes need quantifiers in the invariant...


Quantified interpolants

• Parameterized systems require universal quantifiers in the transition constraints.– example: array update

• Idea (Craig, 1957):– Instantiating a quantifier in A yields a quantifier in interpolant A'

a[i] = x j. a'(j) = if i=j then x else a(j)


2) interpolate...

a(z) = 0 a(z) = 1a'(z) = if z=x then 1 else a(z)

a'(z) = if z=y then 1 else a(z)

1) instantiate...

Quantifier example

• That is, instantiation of an A-side quantifier with a B-side variable leads to a quantifier in the interpolant.

j. a'(j) = 0

a[x] = 1;

a[y] = 1;

a(z) = 2

A

B

3) quantify...

z. a(z) = 0 a(z) = 1


Example: N-process "bakery"

• N processes p(i) of the following form:

j. ticket’(i) > ticket(j)

j. ticket(j) > ticket(i) state(i) = NC

NC

C

• Property: ij. (C(i) C(j))

• Invariant derived (roughly):ij. (ticket(i) ticket(j) C(j))

(C(i) C(j))


Predicate Abstraction

• Given a set of state predicates P, compute the strongest invariant of a program expressible as a Boolean combination of the predicates in P.

• Approach: – Abstract state to predicate valuation– Compute reachable states of abstraction

• Used successfully in software model checking– SLAM, BLAST, etc…

(Graf and Saidi)


Example• A simple transition relation: x’ = x + 1, y’ = y• Predicates: x=y, x 0

x’ = x + 1y’ = y

vx=y, v x 0 v’x=y, v’ x 0?

x=y x 0

x’y’ x’ 0

• Two key problems in predicate abstraction– Choosing the predicates– Computing the abstract transition relation

We can apply interpolation to both

yes v’x=y, v’ x

0

x’=y’ x’ 0

no!


Predicate selection in BLAST

• Use atomic predicates occurring in interpolants• Can always express invariant strong enough to

exclude this trace.

x1 = ctr0

ctr1 = ctr0+1

y1 = ctr1

x1 = m0

y1 m0+1

A

B

[y1=x1+1]

x:=ctr;

ctr := ctr+1;

y := ctr;

assume x=m;

assume ym+1;

Program trace we want to refute

x1 = ctr0

ctr1 = ctr0+1

y1 = ctr1

x1 = m0

y1 m0+1

Translate into transition constraintsCompute interpolants

[HJMM,POPL04]


Experiments

Program LOC* PreviousTime

New Time

Predicates Total Average

kbfiltr 12k 1m12s 3m48s 72 6.5

floppy 17k 7m10s 25m20s 240 7.7

diskperf 14k 5m36s 13m32s 140 10

cdaudio 18k 20m18s 23m51s 256 7.8

parport 61k DNF 74m58s 753 8.1

parclass 138k DNF 77m40s 382 7.2

Windows DDK

IRP

22 state

* Pre-processed Source: R. Jhala


Transition relation refinement

• Abstract transition relation:– Let W = sup(C), and let P be the predicates– Then the abstract transition relation is:

– Best known approaches to computing C' uses an exponential number of decision procedure calls.

– We can use interpolants to approximate this image computation...


Transition interpolants

• An abstract bounded model checking instanceA = C'<i>

B = I<0> C'<0> C'<i-1> C'<i+1> C'<k-1> F<k>

I FC' C' C' C' C' C' C'

A B

t=0 t=k

The ith transition invariant is Ti = A'<-i>, where A' is an interpolant for (A,B) derived from refutation P.

B

* Q means hide the quantified variables in Q by renaming


Using transition interpolants

• From the interpolant properties, we know:– Each Ti is only over the state variables vp

– The conjunction of the Ti's rules out all bad paths of length k.

• Abstraction refinement loop:– Start with "true" as abstract transition relation– While abstraction has a bad path (of length k):

• Do BMC at depth k, abd strengthen the abstract transition relation using the resulting Ti's

• Eventually, the property is proved, or BMC finds a "real" abstract counterexample.


Comparison to CEX-based method

• Properties of arrays -- no benchmarks can be handled by Cartesian abstraction


Comparison by # of refinements

• Interpolation method is choosing more relevant refinements.


Summary

• Interpolants can be derived from proofs in certain first-order theories, including LIUF.

• Provides a weak replacement for quantifier elimination in various applications.– Finite-state model-checking with a SAT solver– Infinite-state model checking with decision proc.– Predicate selection– Transition relation approximation

• Exploit the prover's ability to focus proofs on relevant facts, to construct approximations tailored to properties.


Future work

• Potential to improve the efficiency of hardware model checking by not modeling at the bit level.

• Potential to move from control-oriented to data-oriented properties in software model checking.

• Modify prover to control the atomic predicates in the interpolants.– Relatively compete method of predicate

selection


Infinite-state verification

• Predicate abstraction approach (Graf,Saïdi,97)– Choose a set of predicates to represent state

• I.e., for bakery: ticket1 > ticket0 and ticket0 > ticket1

– Transform C into a predicate-state transducer– Interpolants are now strictly Boolean

• Convergence guaranteed, but may have false negatives

• Advantages of interpolation approach:– Avoid conversion to a Boolean formula– Avoid building BDD’s!– Strong ability to ignore irrelevant predicates


Software model checking

• Predicate abstraction– requires a judicious choice of predicates– the ability to filter out irrelevant predicates

could allow a much larger set to be used

• Cartesian abstraction– loses correlation between predicates– sometimes too coarse (false negatives)– interpolation-based method will use correlation

only if needed for proof -- no need for Cartesian abstraction


Conclusion

• SAT solvers have the ability:– to generate refutations for bounded reachability– to filter out irrelevant facts.

• These abilities can be exploited to generate an abstract image operator, using Craig interpolation.

• This yields a reachability procedure that– operates directly on infinite-state systems– is robust w.r.t. irrelevant facts

• For these reasons, may be useful for software model checking.


SAT solvers

• Find a satisfying assignment to a set of clauses, or prove unsatisfiability.

• Very effective at finding temporal counterexamples in "bounded model checking”.

• Also some application to proving temporal properties (Sheeran et al '00) but limited capacity.


Refutations

• Conflict clause generation in SAT solvers is really sequence of resolution steps

• Given an unsatisfiable problem, solver can produce a refutation– proof of empty clause by resolution steps

• This talk:– two methods to exploit refutations in

unbounded model checking– these have characteristics that may be useful

for software model checking


Proof-based abstraction

BMCat depth k

Cex?done

No Cex?

Use refutation to choose abstraction

MC abstraction doneTrue?

False?

Incr

ease

k


Abstraction (cont)

• Property: G (c X c)

ab cp

g

Model:

C = { g = a b, p = g c, c' = p }

'

free variable

C'property, C C' C property


Advantage

• C' may refer to fewer state variables than C– reduction in the state explosion problem

But how do we choose the set of constraints in C'?



• Let C' be set of constraints used in proof P:C' = { c C | some ci occurs in P }

• C' admits no counterexample of length k– let U' = C'0 C'1 ... C'k-1

– P is a refutation of I0 U' Fk

• Model check property on C'– property true for C' implies true for C– else Cex of length k' > k (why?)

• restart for k = k'


Algorithm

BMCC at depth k

Cex?done

No Cex?

Refutation P inducesabstraction C'

Model check C' doneTrue?

Cex of depth k'?

let

k =

k'

Notice: MC counterexample is thrown away!


Termination

• Depth k increases at each iteration• Eventually k > d, diameter of C'• If k > d, no counterexample is possible

In practice, termination uses occurs when k d/2

Usually, diameter C' << diameter of C


Abstraction results

solid = original, gray = manual, open = proof-based abstraction

345

305 306 306 305

104

307

73

97

52 54

292

312

285

126

354

289

212

151

51

0

50

100

150

200

250

300

350

400S

tate

va

ria

ble

s


Inference

• SAT solver seems to be very effective at narrowing down the proof to relevant facts.

In most cases, it did better than manual abstraction.


Run times

0.01

0.1

1

10

100

1000

To

tal r

un

tim

e (

s)


Run time breakdown

0%

50%

100%

solid = BMC time, open = MC time


IBM GP benchmarks

Thanks to Jason Baumgartner

0.01

0.1

1

10

100

1000

0.01 0.1 1 10 100 1000


Ba

um

ga

rtn

er

et a

l.


A (fuzzy) hypothesis

• Parameterized models allowing no abstraction

SAT-based BMC "succeeds" when number of relevant variables is small, and fails otherwise.

"success" is BMC for k = diameter of relevant logic

Model Max state vars

German protocol 42

"swap" 21


Industrial benchmarks

0

100

200

300

400

500

600

700

0 100 200 300 400 500 600 700

Original state variables

Ab

stra

ctio

n s

tate

var

iab

les


Implications

• Most of the time if bounded model checking succeeds, unbounded model checking also succeeds

• No need to settle for time bounded result• Bounded model checking may be applicable only to localizable

properties


An interpolating decision proc.

• Convert to CNF form• Boolean reasoning done by SAT solver

– Generates “blocking clauses”• Clauses necessary to refute satisfying Boolean

assignments

– Produces refutations using resolution

• Ground decision procedure– Linear inequalities, equality, uninterpreted

functions– Discharges blocking clauses


Cutting planes proofs

• Also supports equality• Interpolants can be computed in linear

time• Can be extended to handle congruence

w.r.t. uninterpreted functions.

(0 a) (0 b)

(0 c1a + c2b)0 c1,c2


Interpolation example

A = (0 a-b) (0 b-c-1) B = (0 c-a)

(0 b-c-1) (0 c-a)

(0 b-a-1) (0 a-b)

(0 -1)

(0 b-c-1) (0 0)

(0 b-c-1) (0 a-b)

(0 a-c-1)


An interpolating decision proc

SATsolver

GroundDecision

Procedure

Blockingclauses

proofs

(A,B) in CNF

Interpolation

proof

A’

SAT!

applications of craig interpolation to model checking k. l. mcmillan cadence berkeley labs

Documents

b slide

cadence design systems

p model

q slide

r i imgr i

q r r p p q qq q r rr

r i image operator

p p qb