Counterexample-Guided Focus

Thomas WiesInstitute of Science and Technology (IST)

Austria

joint work with

Andreas PodelskiUniversity of Freiburg

Motivation

public void filter(Predicate p) /*: requires "p null" modifies content ensures "content = old content Å (pred p)" */ { Node e = root; while (e != null) { Node c = e;

e = e.next; if (!p.contains(c.data)) { if (c.prev == null) { e.prev = null; root = e; } else { c.prev.next = e; e.prev = c; } } }

next

next

prev

next

prev

prev

root

Verify complex properties of heap-manipulating programs

:p

p

p

p

e

c

Quantified properties• data structure invariants

8 x. next(prev(x)) = x• functional correctness

8 x. next*(root,x) $ (old next)*(root, x)Æ x 2 pred(p)

Verification of Safety Properties

reachable states

state space

erro

r st

ates

safeinvariant

Existing tools: SLAM, BLAST, ARMC, MAGIC, …

Software Model Checking

P1 ´ x·0 P2 ´ y>0 …

P1ÆP2Æ…

reachable states

erro

r st

ates

state space

• generic approach• offers high degree of automation

(through use of automated reasoning techniques)

The Eternal Quest for the Right Precision/Efficiency Tradeoff

reachable states

error states error states

reachable states

Crucial problem in the verification of heap programs.

Goal: Adapted AbstractionFine-tune precision to the specific verification task.

error states

reachable states

Boolean Heaps [Podelski, Wies SAS’05]

Use idea of [Sagiv, Reps, Wilhelm 2002]:Partition heap according to a finite set of predicates.

0

7

3

Use idea of [Sagiv, Reps, Wilhelm 2002]:Partition heap according to a finite set of predicates.

Boolean Heaps

Abstract state

0

7

3

5Abstract domaindisjunctions of abstract states

Abstr. transformer for loop

Most Precise Abstract Transformer

Abstr. transformer for loopInductive invariant for

Verification succeeds!

Most Precise Abstract Transformer

reachable states

error states

Precision-Efficiency TradeoffNumber of abstract states is doubly-exponential innumber of predicates

Most precise abstract transformer is impractical• expensive to construct• keeps track of irrelevant information

Solution: apply additional abstraction

Cartesian Abstraction

x

y

S

Sx £ Sy

Sx

Sy

..., [Cousot, Cousot PPCA’95], [Ball, Podelski, Rajamani TACAS’01],…

for abstracting sets of vectors

Cartesian Abstraction

• abstract states are sets of bit-vectors• Cartesian abstraction applies• abstr. transformer w/ Cartesian abstraction is

efficiently implementable:– check entailments between QF formulas– number of entailment checks polynomial in number of

predicates

• precise enough for many practical examples• not precise enough for many practical examples

Abstract Transformer with Cartesian Abstraction

Inductive invariant for

Verification succeeds!

3 7

Inductive invariant for

Verification fails!

0 7

,

FocusCommon recipe in shape analysis

– start from coarse but efficient abstract transformer– adapt precision to each individual program statement

and individual data structures(partial concretization / materialization / focus)

ProblemFine-tuning precision uniformly makes analysis again too precise (i.e., often inefficient)

Exciting research directionParameterized focus that adapts abstract transformer to the individual verification tasks e.g. [Manevich et al., 2004, 2007, 2009]

Counterexample-Guided Focus

Idea: take this direction to its logical extreme

Fine-tune focus to the individual steps of the analysis of the individual verification task

This fine-tuning must be automated.

We use counterexamples for this purpose.

x

y

S

Sx £ Sy

Sx

Sy

Loss of Precision under Cartesian abstraction

splitting is guidedby counterexamples

Effect of Counterexample-Guided Focus

Inductive invariant for

Verification succeeds!

3 7

Inductive invariant for

Verification fails!

0 7

Nested Lazy CEGAR Loop

• outer loop refines abstract domain by inferring new predicates

• inner loop fine-tunes abstract transformer using counterexample-guided focus

Progress theorem: every spurious counterexample is eventually eliminated

Implementation in the Tool Bohne

• (doubly-linked) lists• lists with iterators• sorted lists• skip lists• search trees• trees w/ parent pointers• threaded trees

6

3 9

1 5

4

first

rootVerified data structure implementations:

Implementation in the Tool Bohne

• absence of runtime errors• shape invariants

-acyclic-sharing-free-doubly-linked-parent-linked-threaded-sorted …

• partial correctness

6

3 9

1 5

4

first

rootVerified properties:

Summary of Experiments• no manual adaptation of abstract domain / abstract

transformer required• many examples fail without counterexample-guided

focus• number of explored abstract states is drastically

reduced

Further Related WorkShape analysis

• three-valued shape analysis [Sagiv, Reps, Wilhelm 2002]

– decision procedures in TVLA [Yorsh et al. 2004, …, Lev-Ami et al. 2006]

– parameterized focus for concurrent programs [Manevich et al., 2004, 2007, 2009]

• …

Predicate abstraction

• CE-guided refinement of abstract transformers [Das, Dill 2002]

• nested refinement for predicate abstraction [Ball et al. 2004]

• indexed predicate abstraction [Lahiri, Bryant 2004]

• lazy abstraction [Henzinger et al. 2002]

• lazy shape analysis [Beyer et al. 2006]

Interpolants

• quantified Craig interpolants [McMillan 2008, Kovács, Voronkov 2009]

• abstractions from proofs [Henzinger et al. 2004]

Template-based techniques [Gulwani et al. 2008, Srivastava, Gulwani 2009]

Conclusion

• Focus can be made effective in a CEGAR setting– CEGAR lazily applies focus– CEGAR drives fine-tuning of focus to the extreme

• CEGAR can be made effective for inferring quantified invariants because– focus provides progress of CEGAR and– focus provides precision needed for verifying practical

examples

Focus and CEGAR can be fruitfully integrated to enhance one another

Counterexample-Guided Focus

• analysis of abstract program produces spurious counterexamples

• spuriousness results from imprecise abstract transformer

• construct fine-tuned focus operator that locally adapts precision of abstract transformer– locally refine the abstract domain of the pre-image of

the abstract transformer– locally refine the pre-image itself by splitting disjuncts

below and above the universal quantifier– both refinements are guided by the spurious

counterexample

Costs and Gains of Automation

benchmark

Bohne Bohne w/o VC TVLA Bohne TVLA Bohne TVLA

List.traverse 0.12 0.003 0.179 4 12 (14) 1.5 8

List.create 1.05 0.041 0.133 9 12 (14) 1.67 6

List.getLast 1.13 0.033 0.214 9 14 (16) 2.5 10

List.insertBefore 4.44 0.108 0.503 10 18 (20) 11.5 15

List.append 7.01 0.188 0.462 16 18 (20) 6.5 23

List.filter 2.08 0.052 0.600 10 18 (20) 3 19

List.reverse 7.84 0.083 0.331 13 14 (16) 2.5 12

List.partition 40.71 0.525 1.508 15 18 (20) 3 72

run. time (in s) # predicates avrg. # abstr. states/loc.

Comparison between TVLA and Bohne for variouslist-manipulating programs

Checked properties: absence of runtime errors preservation of list structure (acyclicity, sharing freeness)