Specifying and Checking Properties of

Programs

Thomas BallSriram K. Rajamani

Software Productivity ToolsMicrosoft Research

Software Validation

• Large scale reliable software is hard to build and test.

• Different groups of programmers write different components.

• Integration testing is a nightmare.

Property Checking

• Programmer provides redundant partial specifications

• Code is automatically checked for consistency

• Different from proving whole program correctness – Specifications are not complete

Interface Usage Rules•Rules in documentation

–Incomplete, unenforced, wordy–Order of operations & data

access–Resource management

•Disobeying rules causes bad behavior

–System crash or deadlock–Unexpected exceptions–Failed runtime checks

Example: Sockets

the "communication domain" in which communication is to takeplace; see protocols(5).

Sockets of type SOCK_STREAM are full-duplex byte streams,similar to pipes. A stream socket must be in a connectedstate before any data may be sent or received on it. A con-nection to another socket is created with a connect(2) call.Once connected, data may be transferred using read(2V) andwrite(2V) calls or some variant of the send(2) and recv(2)calls. When a session has been completed a close(2V), maybe performed. Out-of-band data may also be transmitted asdescribed in send(2) and received as described in recv(2).

The communications protocols used to implement a SOCK_STREAMinsure that data is not lost or duplicated. If a piece of

Does a given usage rule hold?

• Checking this is computationally impossible!

• Equivalent to solving Turing’s halting problem (undecidable)

• Even restricted computable versions of the problem (finite state programs) are prohibitively expensive

Why bother?

Just because a problem is undecidable, it doesn’t go away!

Scientific curiosity

Undecidability and complexity theory are the most significant contributions of theoretical computer science.

Software property checking, a very practical and pressing problem is undecidable.

Automatic property checking =

Study of tradeoffs• Soundness vs completeness

– Missing errors vs reporting false alarms

• Annotation burden on the programmer• Complexity of the analysis

– Local vs Global– Precision vs Efficiency– Space vs Time

Broad classification

• Underapproximations– Testing

• After passing testing, a program may still violate a given property

• Overapproximations– Type checking

• Even if a program satisfies a property, the type checker for the property could still reject it

Current trend

• Confluence of techniques from different fields:– Model checking– Automatic theorem proving– Program analysis

• Significant emphasis on practicality

• Several new projects in academia and industry

Outline

• Property checking of software• Undecidable• Important

• Techniques– Model checking– Automatic theorem proving– Program analysis

• Current projects

Model Checking• Algorithmic exploration of state space of the system

• Started by [Clarke-Emerson][Quille-Sifakis] in the early 80s

• Had a breakthrough with Ken McMillan’s thesis in 92 (symbolic model checking with Binary Decision Diagrams)

• Several advances: symmetry reductions, partial order reductions, compositional model checking

• Most hardware companies use a model checker in the validation cycle

enum {N, T, C} state[1..2]

int turn

init state[1] = N; state[2] = N

turn = 0

trans state[i]= N & turn = 0 -> state[i] = T; turn = i

state[i] = N & turn !=0 -> state[i] = T state[i] = T & turn = i -> state[i] = C state[i] = C & state[2-i] = N -> state[i] = N state[i] = C & state[2-i] != N -> state[i] = N; turn = 2-i

N1,N2turn=0

T1,N2turn=1

T1,T2turn=1

C1,N2turn=1

C1,T2turn=1

N1,T2turn=2

T1,T2turn=2

N1,C2turn=2

T1,C2turn=2

N = noncritical, T = trying, C = critical

Model Checking

• Strengths– Fully automatic (when it works)– Computes inductive invariants

• I such that F(I) I

– Provides error traces

• Weaknesses– Scale– Operates only on models

• How do you get from the program to the model?

Theorem proving– Early theorem provers were proof checkers

• They were built to support asssertional reasoning in the Hoare-Dijkstra style

• Cumbersome and hard to use

– Greg Nelson’s thesis in early 80s paved the way for automatic theorem provers• Theory of equality with uninterpreted functions• Theory of lists• Theory of linear arithmetic• Combination of the above !

– Automatic theorem provers based on Nelson’s work are widely used• ESC• Proof Carrying Code

Theory of Equality. • Symbols: =, , f, g, …• Axiomatically defined:

E = E

E2 = E1

E1 = E2

E1 = E2 E2 = E3

E1 = E3

E1 = E2

f(E1) = f(E2)

• Example of a satisfiability problem: g(g(g(x)) = x g(g(g(g(g(x))))) = x g(x) x

• Satisfiability problem decidable in O(n log n)

a : array [1..len] of int;

int max := -MAXINT;i := 1;{ 1 j i. a[j] max}while (i len)

if( a[i] > max) max := a[i];

i := i+1;endwhile{ 1 j len. a[j]

max}

( 1 j i. a[j] max) ( i > len)

( 1 j len. a[j]

max}

Automatic theorem proving• Strengths

– Handles unbounded domains naturally– Good implementations for

• equality with uninterpreted functions• linear inequalities• combination of theories

• Weaknesses– Hard to compute fixpoints– Requires inductive invariants

• Pre and post conditions• Loop invariants

Program analysis

– Originated in optimizing compilers• constant propagation• live variable analysis• dead code elimination• loop index optimization

– Type systems use similar analysis• Are the type annotations consistent?

Program analysis• Strengths

– Works on code – Pointer aware– Integrated into compilers– Precision efficiency tradeoffs well studied

• flow (in)sensitive• context (in)sensitive

• Weakenesses– Abstraction is hardwired and done by the

designer of the analysis– Not targeted at property checking (traditionally)

An optimizing compiler doubles performance every 18 years

-Todd Proebsting

Computing power doubles every 18 months

-Gordon Moore

Model Checking, Theorem Proving and Program

Analysis– Very related to each other

– Different histories• different emphasis• different tradeoffs

– Complementary, in some ways

Outline

• Property checking of software• Undecidable• Important

• Techniques– Model checking– Automatic theorem proving– Program Analysis

• Current projects

Industrial successes with software validation

• PREfix– Statically simulate a small number of paths

symbolically inside each function • theorem proving with under-approximations

– Scale by “summarizing” at function boundaries– Works on Windows NT (several million lines of

code)

• Other projects– ESC project at Compqq SRC– SPIN model checker and the Feaver project at

Bell Labs– Verisoft model checker at Bell labs

Current trend..

Code

Intent

Model

• Use program analysis and automated theorem proving for model extraction

• User annotations simplify the model extraction problem

• Use model checking on the model

Some current projects- Software productivity tools (MSR)

- SLAM, Vault, ESP, Behave!

- Stanford MC project- Berkeley Open Source Quality- KSU’s Bandera- IBM’s CANVAS- SRI Symbolic analysis laboratory (SAL )- Verimag IF language and Tool Set- NASA Ames’ JavaPathfinder- Compaq SRC’s ESC-Java- Bell Labs’ Verisoft and FeaVer- UMass Amherst LASER

When I use a model checker, it runs and runs for ever and never comes back… when I use a static analysis tool, it comes back immediately and says “I don’t know”

- Patrick Cousot

Predictions– The holy grail of full program

verification has been abandoned. It will probably remain abandoned

– Less ambitious tools like powerful type checkers will emerge and become more widely used

– These tools will exploit ideas from various analysis disciplines

– Tools will alleviate the “chicken-and-egg” problem of writing specifications