predicate abstraction for software verification shaz qadeer compaq systems research center (joint...

Predicate Abstraction for Software Verification

Shaz QadeerCompaq Systems Research Center

(joint work with Cormac Flanagan)

Systems software

• OS components– file system, buffer cache manager

• Abstract from low-level to high-level operations– tedious low-level programming detail

• Expected to work – with multiple concurrent clients– in the presence of crashes

Outline

• Background– verification condition– strongest postcondition– loop invariants

• Inferring loop invariants– predicate abstraction– universally-quantified invariants

• Frangipani • Related work

Verification condition

x := a;if (x < b) { x := b;}assert x = max(a,b);

sp(P,true) x a x b

Program P

Check for validity:

Statement S

x := e

A ; B

assume e

A B�

while {I} e do B end

Guarded command language (Dijkstra 76)

Variables have arbitrary values in program’s initial state

if e then A else B end

(assume e ; A)

(� assume e ; B)

Strongest postcondition semantics

sp(S,Q)

y.(Q[xy] x=e[xy])

sp(B,sp(A,Q))

e Q

sp(A,Q) sp(B,Q)

Statement S

x := e

A ; B

assume e

A B�

Denote sp(S, true) by sp(S)

Checking loop invariants

• Given loop

• Need to check– Invariant holds on entry to loop

• sp( C ) I

– Invariant holds at end of every iteration• sp( C; H; assume I e; B) I where H = havoc variables modified in B

C;

{I} while e do B end

Desugaring loops

S = “while {I} e do B end”

H = havoc variables modified in Bdesugar(S) = assert I ; H ; assume I ; ( (assume e ; B; assert I; assume false) � assume e)

Example

assume 0 < a.length; i := 0;

while (i < a.length) { a[i] := 0; i := i + 1;}

assert a[0] = 0;

{ int j; 0 <= j j < i a[j] = 0, 0<=i}

Translationassume 0 < a.length; i := 0;assert int j; 0 <= j j < i a[j] = 0;assert 0 <= i; havoc a[*], i;assume int j; 0 <= j j < i a[j] = 0;assume 0 <= i; ( assume (i < a.length); a[i] := 0; i := i + 1; assert int j; 0 <= j j < i a[j] = 0; assert 0 <= i; assume false;)� assume (i < a.length)assert a[0] = 0;

ESC/Java with loop invariants/*@ loop_invariant i >= 0; loop_invariant 0 <= spot; loop_invariant spot <= MAXDIRENTRY; loop_invariant (\forall int j; 0 <= j && j < i && bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED ==>

bdisk[addr].dirEntries[j].name != name); loop_invariant (\forall int j; spot == MAXDIRENTRY && 0 <= j && j < i ==> bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED);

loop_invariant spot == MAXDIRENTRY || bdisk[addr].dirEntries[spot].inum == DIRENTRY_UNUSED;

loop_invariant (\forall DirEntry t; t != de ==> t.name == \old(t.name)); loop_invariant (\forall DirEntry t; t != de ==> t.inum == \old(t.inum)); loop_invariant (\forall DirEntry t; t.inum == FS.DIRENTRY_UNUSED ||

(0 <= t.inum && t.inum < FS.IMAX));*/for (i = 0; i < cwd.inode.length; i++) { GetDirEntry(de, addr, i); if (de.inum != DIRENTRY_UNUSED && de.name == name) { return ERROR; } if (de.inum == DIRENTRY_UNUSED && spot == MAXDIRENTRY) { spot = i; }}

Outline

• Background– verification condition– strongest postcondition– loop invariants

• Inferring loop invariants– predicate abstraction– universally-quantified invariants

• Frangipani• Related work

Inferring loop invariants

• Could try

– I0 = sp( C )

– In+1 = In sp( C; H; assume In e; B)

•In is an invariant for first n iterations

• Problem: May not converge

C;

{I} while e do B end

Divergence

State Space

I0 I1 I2 In... ...

Predicate abstraction

• Invariant is boolean combination of predicates

• Split problem into two parts

1 Find a suitable set of predicates

2 Find the boolean combination of these predicates

Predicate abstraction (contd.)

• Predicates over program variables– a = expr1, b = expr2, c = expr3, d = expr4

: boolean expression over {a,b,c,d} formula over program variables

:formula over program variables boolean expression over {a, b, c, d}

(F) = least boolean function G over {a,b,c,d} such that F (G)

Abstract state space• Predicates {a, b, c, d}• They generate an abstract space of size 24 = 16

ab

ab

ab

ab

cd

cd

cd

cd

F

(F)

State Space

Inferring loop invariants

• Compute

– I0 = (sp( C ))

– In+1 = In (sp( C; H; assume E (In); B))

• Converges yielding valid loop invariant

– Best loop invariant for the given predicates

Method 1 (slow!)• Is F a b c d satisfiable? No!

• Can compute (F) by asking 2n such queries

ab

ab

ab

ab

cd

cd

cd

cd

F

(F)X X X X

X

X

XX

X

X X

Method 2 (Das-Dill-Park 99)Order the variables: a < b < c < d a

b

c

Fa

Fab

Fabc

Fab

(F) ab

ab

ab

ab

cd

cd

cd

cd

F

X X X X

X

X

XX

X

X X

•O(2n+1) queries•Sensitive to ordering

Method 3 (Shankar-Saidi 99)

•O(3n) queries•No ordering required

Queries of size 1: Fa, F a, Fb, etc.Number = nC121

Queries of size 2: Fab, Fab, etc.Number = nC222

Queries of size n: . . .Number = nCn2n

.

.

New method• F a b c d ? No!

ab

ab

ab

ab

cd

cd

cd

cd

F

(F)X X X X

X

X

XX

X

X X

• F a c d ? No!• F c d ? No! • Removed 1/4 of state space in 3 queries!

= (c d) (a c) (a b) ( c d)

Universally-quantified loop invariants

/*@ loop_invariant i >= 0; loop_invariant 0 <= spot; loop_invariant spot <= MAXDIRENTRY; loop_invariant (\forall int j; 0 <= j && j < i && bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED ==>

bdisk[addr].dirEntries[j].name != name); loop_invariant (\forall int j; spot == MAXDIRENTRY && 0 <= j && j < i ==> bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED);

loop_invariant spot == MAXDIRENTRY || bdisk[addr].dirEntries[spot].inum == DIRENTRY_UNUSED;

loop_invariant (\forall DirEntry t; t != de ==> t.name == \old(t.name)); loop_invariant (\forall DirEntry t; t != de ==> t.inum == \old(t.inum)); loop_invariant (\forall DirEntry t; t.inum == FS.DIRENTRY_UNUSED ||

(0 <= t.inum && t.inum < FS.IMAX));*/for (i = 0; i < cwd.inode.length; i++) { GetDirEntry(de, addr, i); if (de.inum != DIRENTRY_UNUSED && de.name == name) { return ERROR; } if (de.inum == DIRENTRY_UNUSED && spot == MAXDIRENTRY) { spot = i; }}

Inferring -quantified loop invariants

assume 0 < a.length; i := 0;

/*@ loop_invariant 0 <= i, int j; 0 <= j j < i a[j] = 0 */while (i < a.length) { a[i] := 0; i := i + 1;}

assert a[0] = 0;

Inferring -quantified loop invariants

assume 0 < a.length; i := 0;

/*@ loop_predicate 0 <= i, int j; 0 <= j j < i a[j] = 0 */while (i < a.length) { a[i] := 0; i := i + 1;}

assert a[0] = 0;

Inferring -quantified loop invariants

assume 0 < a.length; i := 0;

/*@ skolem_constant int j *//*@ loop_predicate 0 <= i, 0 <= j, j < i, a[j] = 0 */while (i < a.length) { a[i] := 0; i := i + 1;}

assert a[0] = 0;

Inferring -quantified loop invariants

I0 = (sp(i := 0))In+1 = In (sp(i := 0; havoc i, a[*]; assume In i < a.length; a[i] := 0; i := i+1))

/*@ loop_predicate 0 <= i, 0 <= j, j < i, a[j] = 0 */

TFTF TFTT TTFF TTFT I0

TTTT

I1

Inferring -quantified loop invariants

/*@ loop_predicate 0 <= i, 0 <= j, j < i, a[j] = 0 */

TFTF TFTT TTFF TTFT

TTTT

I0

I1

0 i (0 j) (j < i) a[j] = 0 0 j j < i

Inferring -quantified loop invariants

/*@ loop_predicate 0 <= i, 0 <= j, j < i, a[j] = 0 */

TFTF TFTT TTFF TTFT

TTTT

I0

I1

0 i int j; 0 j j < i a[j] = 0 int j; 0 j j < i

Outline

• Background– verification condition– strongest postcondition– loop invariants

• Inferring loop invariants– predicate abstraction– universally-quantified invariants

• Frangipani• Related work

Frangipani

• Distributed file system (Thekkath, Mann, Lee 1997)

• Built on top of Petal virtual disk (Lee, Thekkath 1996)

• Implements the file abstraction from the virtual disk block abstraction

• Exports file and directory operations - create, delete, rename etc. - to clients

Verification

• Loop invariant inference built on top of ESC/Java (Detlefs, Leino, Nelson, Saxe 1998)

• ESC/Java uses automatic theorem prover Simplify (Nelson 81)

• Frangipani data structures and “create” modeled abstractly in Java

• Assume – single thread of execution – no crashes

. . .

. . .

. . .

. . .ibusy

idisk

bdisk

bbusy

F T T

T T

F F F

FFFF

Inode { int inum; int addr; int mode; int length;}

Block { int addr; Entry[] entries;}

Entry { String name; int inum;}

addr addr

Data structures on disk:

Data structures in memory:

. . .

. . .

vArray

vbusy T T FFTF

• vArray contains inodes• distinct entries must contain distinct inodes

itov: int int

/*@ invariant int i, j: vbusy[i] ((vArray[i].inum = j) (itov[j] = i));invariant int i: ibusy[i] bbusy[idisk[i].addr];invariant idisk bdisk;invariant int i: idisk[i] null idisk[i].inum = i;..*/

/*@ requires vbusy[num] vArray[num].mode = DIR *//*@ ensures \result = ERROR there is an entry with name s in directory vArray[num] */int create(int num, String s) { . . .}

Main loop in “create”/*@ loop_invariant i >= 0; loop_invariant 0 <= spot; loop_invariant spot <= MAXDIRENTRY; loop_invariant (\forall int j; 0 <= j && j < i && bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED ==>

bdisk[addr].dirEntries[j].name != name); loop_invariant (\forall int j; spot == MAXDIRENTRY && 0 <= j && j < i ==> bdisk[addr].dirEntries[j].inum != DIRENTRY_UNUSED);

loop_invariant spot == MAXDIRENTRY || bdisk[addr].dirEntries[spot].inum == DIRENTRY_UNUSED;

loop_invariant (\forall DirEntry t; t != de ==> t.name == \old(t.name)); loop_invariant (\forall DirEntry t; t != de ==> t.inum == \old(t.inum)); loop_invariant (\forall DirEntry t; t.inum == FS.DIRENTRY_UNUSED ||

(0 <= t.inum && t.inum < FS.IMAX));*/for (i = 0; i < cwd.inode.length; i++) { GetDirEntry(de, addr, i); if (de.inum != DIRENTRY_UNUSED && de.name == name) { return ERROR; } if (de.inum == DIRENTRY_UNUSED && spot == MAXDIRENTRY) { spot = i; }}

Performance

Algorithm # queries(worst case)

# queries(on benchmark)

Naive 2n 32768

Das-Dill-Park 2n+1 4026

Saidi-Shankar 3n 2252

New n.2n 473

Related work

• Inferring/computing loop invariants• Predicate abstraction

– Graf-Saidi 97– Bensalem-Lakhnech-Owre 98, Colon-Uribe 98– Saidi-Shankar 99, Das-Dill-Park 99– Ball-Majumdar-Millstein-Rajamani 2001

Future Work

• Heuristics for generating predicates

• Multiple threads

– locks, semaphores, ...

– thread-modular reasoning

• Linked lists, reachability

• Target C