Carnegie Mellon University

Java PathFinder and

Model Checking of Programs

Guillaume Brat, Dimitra Giannakopoulou, Klaus Havelund, Mike Lowry, Phil Oh, Corina Pasareanu, Charles Pecheur, John Penix, Willem Visser

NASA Ames Research Center

Automated Software Engineering Group

Alex Groce, Flavio Lerda

Carnegie Mellon University

School of Computer Science

Matt Dwyer, John Hatcliff

Kansas State University

Department of Computing and Information Sciences

Carnegie Mellon University

Outline

• Motivation

• Model Checking and Testing

• Java PathFinder

• Program Model Checking

Carnegie Mellon University

Motivation

• Software errors are expensive– Mars Polar Lander– Ariane 501

Software bugs in space do not fly

Carnegie Mellon University

Model Checking

• Verification and Validation are crucial– Model checking has been shown effective

OK

Error traceorFinite-state model

Temporal logic formula

Model Checker

Line 5: …Line 12: …Line 15:…Line 21:…Line 25:…Line 27:… …Line 41:…Line 47:…

Carnegie Mellon University

The dream

• Model Check Programs

OK

Error traceorProgram

Temporal logic formula

Model Checker

Line 5: …Line 12: …Line 15:…Line 21:…Line 25:…Line 27:… …Line 41:…Line 47:…

void add(Object o) { buffer[head] = o; head = (head+1)%size;}

Object take() { … tail=(tail+1)%size; return buffer[tail];}

Carnegie Mellon University

Some of the Issues

• Semantics Gap– Programming Languages

vs.

Modeling Languages

• Complexity

• Not Automated

void add(Object o) { buffer[head] = o; head = (head+1)%size;}

Object take() { … tail=(tail+1)%size; return buffer[tail];}

Gap

Carnegie Mellon University

Outline

• Motivation

• Model Checking and Testing

• Java PathFinder

• Program Model Checking

Carnegie Mellon University

Model Checking and Testing

• Software complexity is too high

• Some of the presented methods are not sound

• This is not model checking anymore

• It is “automated” testing

Carnegie Mellon University

The assumption

• Programs have bugs– Knowing that there are doesn’t mean knowing

where they are

• Testing is not always effective– Requires a lot of knowledge of the system

• Model checking can be used to find bugs systematically– If no bug is found we have a non-result

Carnegie Mellon University

Coverage Metrics

• Testing has coverage metrics– They tell you how good your testing is

– They can be used to measure confidence

• Testing is not very effective for concurrent systems– You don’t just have to guess the inputs but also the

timing of the inputs and the scheduling

• Model checking can address these issues– We are still missing metrics for concurrent programs

Carnegie Mellon University

Bug hunting

• Bug hunting instead of trying to prove something correct– We can accept unsound methods– We may be able to handle real world examples– If we allow for modeling we are still not

checking the correctness of the system itself

Carnegie Mellon University

Outline

• Motivation

• Model Checking and Testing

• Java PathFinder

• Program Model Checking

Carnegie Mellon University

Model Checking for Java

• Explicit State Model Checker

• Java Bytecode as Input Language

• Assertions, Deadlock Freedom,

LTL Properties

• Source Level Error Trace

• Special JVM– Allows guided execution

SpecialJVM

SpecialJVM

ModelChecker

ModelChecker

StateSpace

StateSpace

ClassesBytecode

ClassesBytecode

Carnegie Mellon University

Architecture

Generic Verification EnvironmentGeneric Verification Environment

GenericGeneric C++C++

CC

JavaJavaSearch Algorithms(model checking,

testing)

Search Algorithms(model checking,

testing)

StorageSubsystem

(hash table, bitstate hashing)

StorageSubsystem

(hash table, bitstate hashing)

SpecialJVM

SpecialJVM

ClassLoader

ClassLoader

Expression EvaluatorExpression Evaluator

Carnegie Mellon University

Outline

• Motivation

• Model Checking and Testing

• Java PathFinder

• Program Model Checking

Carnegie Mellon University

Programs are complex

• Enabling Technologies– Slicing– Abstractions– State Compression– Partial Order Reduction– Heuristic Search

Carnegie Mellon University

Property-directed Slicing

• Slicing criterion automatically generated

• Backwards slicing automatically finds dependencies

Resultingslice

Slice

Source programmentionedin property

indirectlyrelevant

Carnegie Mellon University

Abstractions

• Remove behaviors but preserve errors– manual or partially automated

• Over-approximation– Preserve correctness– Type-based abstractions– Predicate abstraction– Semi-automated

Carnegie Mellon University

JPF Predicate Abstraction

• Annotation used to indicate abstractions

• Source-to-source translation

• Java PathFinder can find abstract error traces

…Abstract.remove(x);Abstract.remove(y);Abstract.addBoolean(“EQ”, x==y);…

…Abstract.remove(x);Abstract.remove(y);Abstract.addBoolean(“EQ”, x==y);…

Carnegie Mellon University

Choice-bounded Search

• An abstract trace that does not contain any non-deterministic choice correspond to at least one concrete trace

• Bias the model checker to look only choice-free traces

Carnegie Mellon University

Storing the States

• States are complex objects– Classes, Instances, Threads, Stack Frames

ClassesClasses

ObjectsObjects

ThreadsThreadsThread

Stack Frame (Locals, Stack)

Stack Frame (Locals, Stack)

Stack Frame (Locals, Stack)

Thread

Stack Frame (Locals, Stack)

Stack Frame (Locals, Stack)

ClassFields/Methods

ObjectFields/Methods

ObjectFields/Methods

ClassFields/Methods

Carnegie Mellon University

State Compression• Instructions modify only part of a state

• Different states share common subparts

X0 X1X = X + 1

X11Y27Z75T45W11

X11Y27Z75T45W11

Carnegie Mellon University

State CompressionClassFields

ObjectFields

ClassMonitors

ObjectMonitors

ThreadData

StackFrames

State

Pools

Array

Compression is very effective: up to 94%!

Carnegie Mellon University

Partial Order Reduction

• Do not explore “equivalent” traces

• Requires analysis before model checking

X=11Y=28

X=12Y=27

X=11Y=27

X=12Y=28

X++

Y++X++

Y++

Access to local variable is perfect candidate for partial order reduction.

Java does not provide enough information.

Assume that every access to a shared object is made in mutual exclusion.

Massive use of partial order reduction.

Use lockset algorithm to check that mutual exclusion is actually present.

Carnegie Mellon University

Heuristic Search• Depth first search leads to very long counter examples• Reactive system often exhibit periodic behavior• It is possible to discover errors at a shorter depth

• Heuristic Search– Breadth first like state generation– Priority queue for the states based on some heuristic

• The challenge– Find good heuristics:

• Based on the property being checked• Based on the program structure• JPF offers an API for user-defined heuristics

Carnegie Mellon University

An example

• DEOS– Real time OS from Honeywell– 1500 lines of code– Subtle concurrency error

• Testing did not reveal it• We (re)discovered the bug!

– Dependency analysis– Type abstraction– Choice-free heuristic

Carnegie Mellon University

Conclusion

• Model check programs poses some specific issues– Some we can deal with

– Some we looked for a way around

• Model checking can be used for systematic testing– Can be automated

– Can handle concurrent systems

• This is still work in progress!

Carnegie Mellon University

Future directions

• Apply the same techniques to C/C++– Next summer internship proposal

• Combine property and heuristic specification– Allow the model checker to direct the search

• Combine coverage, model checking and runtime analysis– Develop metrics– Check the system under certain assumptions