an overview of jml tools and applicationsleavens/jml/oldreleases/sttt04.pdf · this article appears...

This article appears in Software Tools for Technology Transfer.The original version is available at www.springerlink.com

An overview of JML tools and applications

Lilian Burdy1, Yoonsik Cheon2, David R. Cok3, Michael D. Ernst4, Joseph R. Kiniry5, Gary T.Leavens6?, K. Rustan M. Leino7, Erik Poll5

1 INRIA, Sophia-Antipolis, France2 Dept. of Computer Science, University of Texas at El Paso, El Paso, Texas, USA3 Eastman Kodak Company, R&D Laboratories, Rochester, New York, USA4 Computer Science & Artificial Intelligence Lab, MIT, Cambridge, Massachusetts, USA5 Dept. of Computer Science, University of Nijmegen, Nijmegen, the Netherlands6 Dept. of Computer Science, Iowa State University, Ames, Iowa, USA7 Microsoft Research, Redmond, Washington, USA

Received: date / Revised version: date

Abstract. The Java Modeling Language (JML) can beused to specify the detailed design of Java classes andinterfaces by adding annotations to Java source files. Theaim of JML is to provide a specification language that iseasy to use for Java programmers and that is supportedby a wide range of tools for specification type-checking,runtime debugging, static analysis, and verification.

This paper gives an overview of the main ideas be-hind JML, details about JML’s wide range of tools, anda glimpse into existing applications of JML.

1 Introduction

JML [57,58], the Java Modeling Language, is useful forspecifying detailed designs of Java classes and interfaces.JML is a behavioral interface specification language forJava; that is, it specifies both the behavior and the syn-tactic interface of Java code. The syntactic interface of aJava class or interface consists of its method signatures,the names and types of its fields, etc. This is what iscommonly meant by an application programming inter-face (API). The behavior of such an API can be pre-cisely documented in JML annotations; these describethe intended way that programmers should use the API.In terms of behavior, JML can detail, for example, thepreconditions and postconditions for methods as well asclass invariants, in the Design by Contract style [73].

An important goal for the design of JML is that itshould be easily understandable by Java programmers.This is achieved by staying as close as possible to Javasyntax and semantics. Another important design goal isthat JML not impose any particular design methodologyon users; instead, JML should be able to document Javaprograms designed in any manner.

? Supported in part by US NSF grants CCR-0097907 and CCR-0113181

The work on JML was started by Gary Leavens andhis colleagues and students at Iowa State University. Ithas since grown into a cooperative, open effort. Severalgroups worldwide are now building tools that supportthe JML notation and are involved with the ongoingdesign of JML. For an up-to-date list, see the JML web-site, www.jmlspecs.org. The open, cooperative natureof the JML effort is important both for tool developersand users, and we welcome participation by others. Forpotential users, the fact that there are several tools sup-porting the same notation is clearly an advantage. Fortool developers, using a common syntax and semanticscan make it much easier to get users interested. After all,one of the biggest hurdles to using a new specification-centric tool is often the lack of familiarity with the as-sociated specification language.

The next section introduces theJML notation. Sections 3 through 7then discuss the tools currently avail-able for JML in more detail. Section 8discusses the applications of JML inthe domain of Java Card, the Javadialect for programming smartcards.Section 9 discusses some related lan-guages and tools, and Section 10 con-cludes.

2 The JML Notation

JML blends Eiffel’s Design by Contract approach [73]with the Larch tradition [41,20,56] (both of which sharefeatures and ideas with VDM [52]).1 Because JML sup-ports quantifiers such as \forall and \exists, and be-cause JML allows model (i.e., specification-only) fields

1 JML also has takes some features from the refinement calculus[75], which we do not discuss in this paper.

2 Burdy et al.: An overview of JML tools and applications

and methods, specifications can easily be made moreprecise and complete than is typical for Eiffel software.However, following Eiffel’s use of its expression syntax inassertions, JML uses Java’s expression syntax in asser-tions; this makes JML’s notation easier for programmersto learn than notations based on a language-independentspecification language, such as the Larch Shared Lan-guage [58,59] or OCL [91].

Figure 1 gives an example of a JML specification thatillustrates its main features. JML assertions are writtenas special annotation comments in Java code, either after//@ or between /*@ ... @*/, so that they are ignoredby Java compilers but can be used by tools that sup-port JML. Within annotation comments, JML extendsthe Java syntax with several keywords—in the examplein Figure 1, the JML keywords invariant, requires,assignable, ensures, and signals are used. It also ex-tends Java’s expression syntax with several operators—in the example \forall, \old, and \result are used;these begin with a backslash so they do not clash withexisting Java identifiers.

The central ingredients of a JML specification arepreconditions (given in requires clauses), postcondi-tions (given in ensures clauses), and invariants. Theseare all expressed as boolean expressions in JML’s exten-sion to Java’s expression syntax.

In addition to normal postconditions, the languagealso supports exceptional postconditions, specified us-ing signals clauses. These can be used to specify whatmust be true when a method throws an exception. Forexample, the signals clause in Figure 1’s debit methodspecifies that debit may throw a PurseException andthat the balance will not change in that case (as specifiedby the use of the \old keyword).

The assignable clause for the method debit spec-ifies a frame condition, namely that debit will assignonly to the balance field. Although not a traditionalpart of Design by Contract languages like Eiffel, suchframe conditions are essential for verification of codewhen using some of the tools described later.

There are many additional features of JML that arenot used in the example in Figure 1. We briefly discussthe most important of these below.

– Model variables, which play the role of abstract val-ues for abstract data types [23], allow specificationsthat hide implementation details. For example, if in-stead of a class Purse, we were specifying an inter-face PurseInterface, we could introduce the bal-ance as such a model variable. A class implementingthis interface could then specify how this model fieldis related to the class’s particular representation ofbalance.

– JML comes with an extensive library that providesJava types that can be used for describing behaviormathematically. This library includes such conceptsas sets, sequences, and relations. It is similar to li-

braries of mathematical concepts found in VDM, Z,LSL, or OCL, but allows such concepts to be used di-rectly in assertions, since they are embodied as Javaobjects.

– The semantics of JML forbids side-effects in asser-tions. This both allows assertion checks to be usedsafely during debugging and supports mathemati-cal reasoning about assertions. This semantics worksconservatively, by allowing a method to be used inassertions only if it is declared as pure, meaning themethod does not have any side-effects and does notperform any input or output [58]. For example, ifthere is a method getBalance() that is declared aspure,

/*@ pure @*/ int getBalance() { ... }

then this method can be used in the specificationinstead of the field balance.

– Finally, JML supports the Java modifiers (private,protected, and public) that control visibility ofspecifications. For example, an invariant can be de-clared to be protected if it is not observable byclients but is intended for use by programmers ofsubclasses. (Technically the invariants and methodspecifications in the Purse example of Figure 1 havedefault or package visibility, and thus would only bevisible to code in the same package.)

3 Tools for JML

For a specification language, just as for a programminglanguage, a range of tools is necessary to address thevarious needs of the specification language’s users suchas reading, writing, and checking JML annotations.

The most basic tool support for JML is parsing andtypechecking. This already provides an advantage overinformal comments, as parsing and typechecking willcatch any typos, type incompatibilities, references tonames that no longer exist, etc. The JML checker (jml)developed at Iowa State University performs parsing andtypechecking of Java programs and their JML annota-tions, and most of the other tools mentioned below in-corporate this functionality.

The rest of this paper describes the various toolsthat are currently available for JML. The following cat-egorization serves also as an organization for the imme-diately following sections of this paper. We distinguishtools for checking of assertions at runtime, tools for stat-ically checking of assertions (at or before compile-time),tools for generating specifications, and tools for docu-mentation.

3.1 Runtime assertion checking and testing

One way of checking the correctness of JML specifica-tions is by runtime assertion checking, i.e., simply run-

Burdy et al.: An overview of JML tools and applications 3

public class Purse {

final int MAX_BALANCE;

int balance;

//@ invariant 0 <= balance && balance <= MAX_BALANCE;

byte[] pin;

/*@ invariant pin != null && pin.length == 4

@ && (\forall int i; 0 <= i && i < 4;

@ 0 <= pin[i] && pin[i] <= 9);

@*/

/*@ requires amount >= 0;

@ assignable balance;

@ ensures balance == \old(balance) - amount

@ && \result == balance;

@ signals (PurseException) balance == \old(balance);

@*/

int debit(int amount) throws PurseException {

if (amount <= balance) { balance -= amount; return balance; }

else { throw new PurseException("overdrawn by " + amount); }

}

/*@ requires p != null && p.length >= 4;

@ assignable \nothing;

@ ensures \result <==> (\forall int i; 0 <= i && i < 4;

@ pin[i] == p[i]);

@*/

boolean checkPin(byte[] p) {

boolean res = true;

for (int i=0; i < 4; i++) { res = res && pin[i] == p[i]; }

return res;

}

/*@ requires 0 < mb && 0 <= b && b <= mb

@ && p != null && p.length == 4

@ && (\forall int i; 0 <= i && i < 4;

@ 0 <= p[i] && p[i] <= 9);

@ assignable MAX_BALANCE, balance, pin;

@ ensures MAX_BALANCE == mb && balance == b

@ && (\forall int i; 0 <= i && i < 4; p[i] == pin[i]);

@*/

Purse(int mb, int b, byte[] p) {

MAX_BALANCE = mb; balance = b; pin = (byte[]) p.clone();

}

}

Fig. 1. Example JML specification

ning the Java code and testing for violations of JML as-sertions. Such runtime assertion checks are accomplishedby using the JML compiler jmlc (Section 4.1).

Given that one often wants to do runtime assertionchecking in the testing phase, there is also a jmlunit tool(Section 4.2), which combines runtime assertion checkingwith unit testing.

3.2 Static checking and verification

More ambitious than testing if the code satisfies thespecifications at runtime is verifying that the code sat-isfies its specification statically. This can give more as-surance in the correctness of code as it establishes thecorrectness for all possible execution paths, whereas run-time assertion checking is limited by the execution pathsexercised by the test suite being used. Of course, correct-ness of a program with respect to a given specification isnot decidable in general. Any verification tool must trade


off the level of automation it offers (i.e., the ability todispense with user interaction) and the complexity of theproperties and code that it can handle. There are severaltools for statically checking or verifying JML assertions,providing different levels of automation and supportingdifferent levels of expressivity in specifications:

– The program checker ESC/Java (Section 5.1) canautomatically detect certain common errors in Javacode and check relatively simple assertions.

– ESC/Java2 (Section 5.2) extends ESC/Java to sup-port more of the JML syntax and to add other func-tionality.

– The LOOP tool (Section 5.3) translates code anno-tated with JML specifications to proof obligationsthat one can then try to prove using the theoremprover PVS. The LOOP tool can handle more com-plex specifications and code than automatic checkerslike ESC/Java can, but at the price of more userinteraction.

– The program checker JACK (Section 5.4) offers simi-lar functionality to ESC/Java, but is more ambitiousin attempting real program verification.

3.3 Generating specifications

In addition to these tools for checking specifications,there are also tools that help a developer write JMLspecifications, with the aim of reducing the cost and ef-fort of producing JML specifications:

– The Daikon tool (Section 6.1) infers likely invariantsby observing the runtime behavior of a program.

– The Houdini tool (Section 6.2) postulates annota-tions for code, then uses ESC/Java to check them.

– The jmlspec tool can produce a skeleton of a spec-ification file from Java source and can compare theinterfaces of two different files for consistency.

3.4 Documentation

Finally, in spite of all the tools mentioned above, ulti-mately human beings must read and understand JMLspecifications. Since JML specifications are also meantto be read and written by ordinary Java programmers, itis important to support the conventional ways that theseprogrammers create and use documentation. The jml-doc tool (Section 7.1) produces browsable HTML pagescontaining both the API and the specifications for Javacode, in the style of pages generated by javadoc [38].

4 Runtime Assertion Checking and Testing

The most obvious way to use JML annotations is to testthem at runtime and report any detected violations. Inthis section we discuss two tools, jmlc and jmlunit , thatwork this way.

4.1 Runtime Assertion Checking

4.1.1 Overview and Goals

The goal of the JML compiler, jmlc, also known as theruntime assertion checker, is to find inconsistencies be-tween specifications and code by executing assertions atruntime. The overall approach is to find such inconsis-tencies dynamically, by executing JML’s assertions whilethe program runs and notifying the user of any assertionviolations. As with other runtime assertion checkers, onenormally hopes to find that the code is incorrect with re-spect to the specification. However, it may also be thatthe specification itself is incorrect (with respect to whatthe user has in mind), but the code is correct. Find-ing problems in specifications is important for keepingthe specifications accurate and up-to-date; this solves acommon problem with informal documentation, whichcannot be mechanically checked against the program.

An important requirement for the runtime assertionchecker is that it be good at isolating problems, in thesense that users of the tool should be able to quicklypinpoint what in either the code or specifications mustbe changed to correct an inconsistency. For this purpose,jmlc must provide information that is helpful for users.This includes both static information, such as what partsof the specification were violated and where in the pro-gram the violation was detected, as well as dynamic in-formation about the values of variables and what methodcalls led to the violation (a stack backtrace).

It is also helpful, for isolating problems, if the runtimeassertion checker can execute as large a subset of theJML language as possible.

The runtime assertion checker must also be trustwor-thy, in the sense that it must not generate false reportsof assertion violations. That is, every assertion violationmust be a report of an assertion that is false, accordingto the JML semantics. In meeting this goal, the runtimeassertion checker can fail to report assertions that mightbe false. For example, JML includes a way to write in-formal descriptions in assertions; these informal descrip-tions are merely pieces of English text, and so only a hu-man reader can decide whether they are true or false. Ifthe runtime assertion checker were to assume some par-ticular truth value for these it might report an assertionviolation when none actually existed. In such cases it isbetter for the runtime assertion checker to not report aviolation. Similarly, it is also acceptable for the runtimeassertion checker to not execute some parts of assertions,especially in postconditions. However, not being able toexecute some precondition could cause a method to failunexpectedly; thus jmlc should give a warning for non-executable preconditions. In summary, it is better if theruntime assertion checker can execute all assertions andfind all assertion violations, but this is a goal that canbe incrementally approached during the development ofthe tool.


An important goal of the runtime assertion checker isthat its work should be transparent when no assertionsare violated. That is, except for time and space mea-surements, a correct program compiled with jmlc shouldbehave just as if compiled with a normal Java compiler.The transparency of runtime assertion checking is aidedby JML’s design, as assertions are not allowed to haveany side-effects [59].

Although jmlc does not have to be used with anyparticular methodology, there are some general ideas forusing such tools that are helpful for beginners [73]. Abasic technique for using the runtime assertion checkeris to first specify preconditions for the normal behaviorof methods. This is easily done and helps ensure thatall methods are called in expected states. For debuggingpurposes, it is also important to add toString methodsto all types involved, so that jmlc can display objectvalues in violation messages. Following this, one coulddefine invariants that describe the legal states of objectsof each class (see Section 6.1 for more on this topic). Tohelp debug implementations, one can then advance todescribing normal postconditions for methods. If one isdescribing a library for untrusted clients, it may also beuseful to document when various exceptions are thrownby writing exceptional postconditions.

4.1.2 Design of the Tool

The JML compiler was developed at Iowa State Univer-sity as an extension to the MultiJava compiler [24]. Itcompiles Java programs annotated with JML specifica-tions into Java bytecode [19,21]. The compiled bytecodeincludes instructions that check JML specifications suchas preconditions, normal and exceptional postconditions,invariants, and history constraints.

Because the JML language provides such a rich setof specification facilities, it presents new challenges inruntime assertion checking. One of these challenges thatthe current tool meets is supporting abstract specifica-tions written in terms of specification-only declarationssuch as model fields, ghost fields, and model methods.This aspect of the JML compiler represents a significantadvance over the state of the art in runtime assertionchecking as represented by Design by Contract tools suchas Eiffel [73] or by Java tools such as iContract [55] orJass [9]. Other advances over such tools include (stateful)interface specifications, multiple inheritance of specifica-tions from interfaces, various forms of quantifiers and setcomprehension notation, support for strong and weakbehavioral subtyping [68,28], and a contextual interpre-tation of undefinedness [21].

4.1.3 Example

The specifications and code in Figure 1 were debuggedusing the runtime assertion checker in combination withthe unit testing tool described in Section 4.2.3. Using

jmlc on the example is straightforward; the user simplytells the tool to compile the Purse.java file and thenruns a test driver using jmlrac as the virtual machine.The jmlrac command is a version of the java commandthat knows about the necessary runtime libraries for run-time assertion checking. Assertion violations are printedas messages on the console. We discuss details of thiskind of testing in Section 4.2.3.

4.1.4 Experience

The runtime assertion checker is one of the most widelyused JML tools. It has been used on several case stud-ies. One of the most demanding of these case studiesis the checking of the built-in model types for JML it-self, which have very rich and complete specifications. Ithas been used in several undergraduate classes, but inthose cases it has also been used for simple, Design byContract style, specifications. It has also been used inseveral of the other case studies mentioned in the restof this paper. It seems to be helpful to use the runtimeassertion checker before doing serious program verifica-tion, to make sure that the easily found bugs are removedbefore spending the effort to do verification.

In sum, the JML compiler brings programming ben-efits to formal interface specifications by allowing Javaprogrammers to use JML specifications as practical andeffective tools for debugging, testing, and Design by Con-tract.

4.1.5 Future Work

One of the main issues in the future work on jmlc is im-proving both the speed of compilation and the speed ofexecuting runtime assertion checks. For the latter, thereseem to be several simple things that can be done to im-prove execution speed. For example, caching the valuesof model fields instead of recomputing them in severalplaces within an assertion would be helpful.

Another direction for future work is being pursuedat Virginia Tech by Stephen Edwards and his studentRoy Tan. They are building a version of the JML com-piler that produces separate bytecode files for the nor-mal code and for a runtime assertion checking wrapper.Separating the runtime assertion checking code into thiswrapper has several advantages. In particular, decisionsabout what classes should be checked can be made whilethe program executes. It will also enable the addition ofruntime checks to code for which the source code is notavailable.

4.1.6 Availability

The runtime assertion checker is part of the main JMLtoolset available via www.jmlspecs.org, which is devel-oped as an open source project hosted at SourceForge.net.


4.2 Unit Testing


A formal specification can be viewed as a test oracle [84,3], and JML’s runtime assertion checker can be used asthe decision procedure for the test oracle [22]. This ideahas been implemented as a unit testing tool for Java,jmlunit , by combining JML with the popular unit testingtool JUnit [10].

The main goal of the jmlunit tool is to significantlyautomate unit testing of Java code. More specifically,the goal is to free the programmer from writing the codethat decides whether unit tests pass or fail.


The jmlunit tool, developed at Iowa State University,generates JUnit test classes that rely on the JML run-time assertion checker. The test classes send messagesto objects of the Java classes under test. The testingcode catches assertion violation errors from such methodcalls to decide if the test data violate the preconditionof the method under test; such assertion violation er-rors do not constitute test failures. When the methodunder test satisfies its precondition, but otherwise hasan assertion violation, then the implementation failed tomeet its specification, and hence the test data detectsa failure [22]. In other words, the generated test codeserves as a test oracle whose behavior is derived fromthe specified behavior of the class being tested.

The user is still responsible for generating test data;however, the generated test classes make it easy for theuser to supply this data. The tool comes with a frame-work that includes sample test data for the built-in Javavalue types. This framework allows one to combine, fil-ter, and compose test data in several different ways tocreate a variety of tests. In addition, the user can supplyhandwritten JUnit test methods if desired. Such hand-written tests are useful for exploring combinations ofmethod calls that the automatic testing ignores.

4.2.3 Example

In this subsection we discuss runtime assertion check-ing and unit testing with jmlunit , based on Figure 1.To do unit testing with jmlunit , one first runs the jm-lunit tool on the Purse.java file (technically, one hasto use an option to tell the tool to test methods andconstructors with package visibility). This produces afile, Purse JML TestData.java, into which test data isplaced, and another file Purse JML Test.java, whichcontains a driver to run the tests. In the first file wesupplied data of the various types used as arguments tothe methods being tested; this consists of integers (0,1, -1, -22, etc.), Purse objects (such as null and newPurse(1,1,p), where p is a 4-element array of bytes),

and fresh byte arrays (such as null, new byte[] {}, newbyte[] {0,0,0}, and new byte[] {0, 0, 0, 0}). Torun the tests, one first compiles the classes being testedwith jmlc (using a special option to flag unhandled andunspecified exceptions as errors); then the classes pro-duced by jmlunit are compiled with a normal Java com-piler; finally one executes the automatically-generateddriver class, Purse JML Test, using jmlrac.

If all the annotations are removed from Figure 1, thenthe unit testing process described in the previous para-graph does not detect any errors. This is because theunit testing tool is only testing for violations of asser-tions and, if there are no assertions, then no violationsare detected. This illustrates the important observationthat the quality of the testing that jmlunit provides isonly as good as the specifications.

Consider a version of Figure 1 that only includes thepreconditions of the methods and the constructor, butomits the invariants, the frame axioms, and all the nor-mal and exceptional postconditions. Testing of Purseproduces 11 failures, all of which are similar to thatshown in Figure 2. (Printing of Purse objects is handledby adding the obvious toString method to the code inFigure 1.)

This error is the result of not specifying (i.e., delet-ing) the exceptional postconditions of the debit method.It shows that the condition in an exceptional postcondi-tion can be alternatively considered as the negation ofa precondition for normal behavior; which makes sense,because throwing an exception is not normal behavior.If the precondition of the debit method is changed tothe following:

amount >= 0 && amount <= balance

then all of these failures go away. This also happens ifthe debit method has the exceptional postcondition re-stored from Figure 1, which tells the runtime assertionchecker that such exceptions are expected.

This kind of testing is also effective at finding variousomissions in preconditions. For example, if the precon-dition in the checkPin method or the constructor thatspecifies that the array must be of an appropriate lengthis omitted, then the tests will encounter failures.

Checking preconditions will not show places wherethe code is wrong, unless one method in the code callsanother incorrectly. For the most part, errors in code arerevealed by adding either invariants or postconditions.If we add the invariants back into the version of Purse,but still leave out the postconditions, then testing candetect omitted initialization of the MAX BALANCE field inthe constructor (although Java itself detects missing ini-tializations of final fields, so for JML to detect this error,one also has to omit the final attribute from that field).Similarly, with the invariants, the constructor’s precon-dition must have the first line shown in Figure 1, or manyviolations of the first invariant in the figure occur.


1) debit(Purse_JML_Test$TestDebit)junit.framework.AssertionFailedError:

Method ’debit’ applied to

Receiver: Purse(max=1, bal=0, pin={0123})

Argument amount: 1

Caused by: org.jmlspecs.jmlrac.runtime.JMLExceptionalPostconditionError:

by method Purse.debit regarding specifications at

File "Purse.java", line 9, character 17, when

’jml$e’ is PurseException: overdrawn by 1

at Purse.checkXPost$debit$Purse(Purse.java:256)

at Purse.debit(Purse.java:347)

Fig. 2. Example output from testing with jmlunit .

Adding postconditions from Figure 1 allows manyother errors in coding to be detected. For example, withall the postconditions restored, omissions of initializa-tions of the balance and pin fields are detected. Thepostconditions can also detect incorrect coding in theloop of the checkPin method, but doing so requires testdata for byte arrays that differ in only the positions notchecked by the code; we had to add such data to ourinitial set of test data, since the original test data didnot detect these errors. Figuring out the right test datato add in this case was subtle and could easily have beenmissed.

4.2.4 Experience

Our experience shows that the tool allows one to per-form unit testing with minimal coding effort and detectsmany kinds of errors. Ironically, about half of our testfailures were caused by specification errors, which showsthat the approach is also useful for debugging specifica-tions. In addition, the tool can report assertion coverageinformation, identifying assertions that are always trueor always false, and thus indicating deficiencies in theset of test cases. However, the approach requires specifi-cations to be fairly complete descriptions of the desiredbehavior, as the quality of the generated test oracles de-pends on the quality of the specifications. Thus, the ap-proach trades the effort one might spend in writing testcases for effort spent in writing formal specifications.

4.2.5 Future Work

JML/JUnit testing is limited in that it only detects prob-lems that are the result of single method or construc-tor calls. Thus test data has to be carefully crafted sothat the method is applied to objects in states that willfully exercise it. This process would be easier if the testdrivers would apply several methods in sequence to var-ious pieces of data. One alternative for doing this wouldbe to generate such sequences of method calls automat-ically. (An experimental version of Daikon can do this.)Another alternative is to augment JML with facilitiesto write specifications for blocks of example code to beused in testing.

4.2.6 Availability

jmlunit is part of the main JML toolset. This toolset isavailable via www.jmlspecs.org. It has been developedas an open source project hosted at SourceForge.net.

5 Static Checking and Verification

In this section, we describe several tools for staticallychecking—or verifying—JML annotations, providing dif-ferent degrees of rigor and automation.

5.1 Extended Static Checking with ESC/Java


The ESC/Java tool [36], originally developed at CompaqResearch, performs what is called extended static check-ing [27,60], compile-time checking that goes well beyondtype checking. It can check relatively simple assertionsand can check for certain kinds of common errors in Javacode, such as dereferencing null, indexing an array out-side its bounds, or casting a reference to an impermissi-ble type. ESC/Java supports a subset of JML. For thissubset it checks the consistency between the code andthe given JML annotations. The user’s interaction withESC/Java is quite similar to the interaction with a com-piler’s type checker: the user includes JML annotationsin the code and runs the tool, and the tool responds witha list of possible errors in the program.


JML annotations affect ESC/Java in two ways. First,the given JML annotations help ESC/Java suppress spu-rious warning messages. For example, in Figure 1, theconstructor’s precondition p != null lets ESC/Java de-termine that the dereference of p in the constructor’sbody is valid, and thus no null-dereference warning isproduced. Second, annotations make ESC/Java do addi-tional checks. For example, when checking a caller of thePurse constructor, the precondition p != null causesESC/Java to emit a warning if the actual parameter


for p may be passed in as null. In these two ways, theuse of JML annotations enables ESC/Java to producewarnings not at the source locations where errors man-ifest themselves at runtime, but at the source locationswhere the errors are committed.

An interesting property of ESC/Java is that it is nei-ther sound nor complete; that is, it neither warns aboutall errors, nor does it warn only about actual errors.This is a deliberate design choice: the aim is to increasethe cost-effectiveness of the tool. In some situations, con-vincing a mechanical checker of the absence of some par-ticular error may require a large number of JML anno-tations (consider, for example, a hypothetical programthat dereferences null if four of the program’s large-valued integer variables satisfy the equation in Fermat’sLast Theorem). To make the tool more cost-effective,it may therefore be prudent to ignore the possibility ofcertain errors, which is what ESC/Java has been de-signed to do. The ESC/Java User’s Manual [64] containsa list of all cases of unsoundness and incompleteness inESC/Java.

Under the hood, ESC/Java is powered by detailedprogram semantics and an automatic (non-interactive)theorem prover, Simplify [26]. ESC/Java translates agiven JML-annotated program into verification condi-tions [65,37,61]. Verification conditions are logical for-mulas that are valid if and only if the program is freeof the kinds of errors being analyzed. Any verification-condition counterexamples found by the theorem proverare turned into programmer-sensible warning messages,including the kind and source location of each potentialerror [62]. The User’s Manual for ESC/Java [64] alsoprovides a detailed description of the semantics of JMLannotations, as they pertain to ESC/Java.

5.1.3 Example

We refrain from giving details of an ESC/Java examplehere. Instead, we describe an example in the context ofESC/Java’s successor, ESC/Java2 , in Section 5.2.3.

5.1.4 Experience

The first major experience with ESC/Java was to ap-ply the tool to the sources of its own front end, over 40KLOC of Java. This source was “fully annotated”, mean-ing that enough specifications were given for ESC/Javato check the front end for run-time errors (like nulldereferences and array-index bounds errors) and spec-ification violations (like precondition violations) with-out producing any warnings. This and some other earlyexperiences are described in the ESC/Java overview pa-per [36].

Applications to Java Card are discussed in Section 8.The experience applying ESC/Java to Java Card wasone of the motivations for the work on ESC/Java2 , asmaintaining different versions of the API specification,

one using ESC/Java’s dialect of JML and one using thefull JML language, was becoming a lot of work.

5.1.5 Availability

The final binary release (version 1.2.4) of ESC/Java isavailable from Compaq/HP’s web site: www.research.compaq.com/downloads.html. The source code (includ-ing that of related tools, e.g. Houdini, Calvin, and Sim-plify) is available as well. This source code release is ob-scurely named the “Java Programming Toolkit SourceRelease.” ESC/Java only runs on x86 machines withLinux and Microsoft Windows, Sun’s SPARC with So-laris, and Alpha processors with Hewlett-Packard’s Tru64Unix.

5.2 ESC/Java2


Development of version 1 of ESC/Java had ceased bythe time the Compaq Systems Research Center becamepart of HP Labs, where it was later dissolved. Conse-quently, Cok and Kiniry have in progress a version 2 ofESC/Java, built on the source code release provided byCompaq and HP. This version has the following goals:

– to migrate the code base of ESC/Java and the codeaccepted by ESC/Java to Java 1.4;

– to update ESC/Java to accept annotations consis-tent with the current version of JML;

– to increase the amount of JML that is checked, whileremaining true to the original engineering goals ofESC/Java.


ESC/Java2 follows the design of ESC/Java. In addition,ESC/Java2 , like ESC/Java, recognizes that the state-of-the-art of static checking is such that not all mismatchesbetween code and specifications are reported by staticchecking tools; that is, there are aspects which are un-sound, typically because some of the Java semantics arenot yet fully modeled. Similarly, some generated warn-ings are not actually errors in the program; that is, thereare aspects which are incomplete, typically because cur-rent theorem provers are insufficiently powerful. It is agoal of all such tools, including ESC/Java2 , to be assound and complete as is possible within reasonable en-gineering limits, but since no existing tools fully modelor fully prove full multi-threaded Java (indeed, portionsof the semantics of the language are still being debated),the authors of both ESC/Java and ESC/Java2 believethat it is in the interests of users to be explicit about theknown sources of unsoundness and incompleteness.

ESC/Java2 does include improvements to ESC/Javain the following areas, while retaining backwards com-patibility in all but a few features:


– It parses Java 1.4 (ESC/Java only parsed Java 1.3).In particular ESC/Java2 handles the Java assertstatement. A tool option allows the user to choosewhether Java assert statements are treated as state-ments that may throw exceptions (per the Java se-mantics) or whether they are treated like assertstatements in JML, which are checked by the staticchecker.

– It handles the current binary format for Java classes.– It parses all of current JML. This is a somewhat mov-

ing target, since JML is the subject of ongoing dis-cussion and research. Nevertheless the core part ofJML is stable and that is the portion that ESC/Java2attempts to statically check. Some of the more eso-teric features of JML (e.g. model programs) are onlyparsed and are ignored for purposes of static check-ing.

– It allows specifications to be placed in (multiple) filesseparate from the implementation, using JML’s re-finement features. ESC/Java2 makes checks by com-bining all available specifications and implementa-tions. It also checks these specifications for consis-tency.

– It follows the JML semantics for specification inher-itance. The constructs specific to ESC/Java version1 (also_requires, etc.) were dropped.

– It enlarges the set of JML features that are staticallychecked, most importantly:– Pure methods, which may be included in annota-

tions;– Most aspects of assignable clauses;– Model fields, with the associated represents, in

and maps annotations.

5.2.3 Example

As an example, if the second invariant in Figure 1 isomitted and the current ESC/Java2 tool is applied tothe source code, the warnings shown in Figure 3 are pro-duced. The warning messages indicate the likely problemand the source code location that violates the implicitor explicit specification, namely, in this case, the implicitspecification that the left-hand operand of the derefer-ence operation must not be a null reference and that theindex of an array reference must be less than the arraylength.

If ESC/Java2 is applied to Purse.java as it stands(using a current version of JML’s specifications for Javasystem classes), a warning will be produced reflectingthe fact that the specifications of the behavior of cloneare not yet completed.

A source of unsoundness in ESC/Java(2) that is rel-evant in the Purse example is its handling of loops:by default, it will not attempt verification of the loopin checkPin, but simply unroll it once. This makes iteasy for the programmer, who doesn’t have to supply aloop invariant, but it may also miss errors. In contrast,

LOOP and JACK (and ESC/Java2 with the -loopSafeswitch) handle loops soundly, but then require users tosupply loop invariants. For this case, the loop invariantas illustrated in Figure 4 would have to be given.

5.2.4 Experience

The first major partial verification using ESC/Java2 wasdone in early 2004 when the Dutch Parliament decidedin 2003 to construct an Internet-based remote votingsystem for use by Dutch expatriates. The SoS group atthe University of Nijmegen was part of an expert reviewpanel for the system and also performed a black-boxnetwork and system security evaluation of this systemin late 2003. They also were responsible for designing,implementing, and verifying the vote tally subsystemof this system in early 2004. This implementation usedJML and ESC/Java2 extensively.

ESC/Java2 made a very positive impression on theSoS developers. Its increased capabilities as comparedto Compaq ESC/Java, particularly with regards to han-dling the full JML language, the ability to reason withmodels and specifications with pure methods, are veryimpressive. And, while the tool is still classified as an“alpha” release, we found it to be quite robust (per-haps unsurprising given its history, the use of JML andESC/Java2 in and on its own source code, and the factthat it is passed through seven alpha releases thus far).But there are still a number of issues with ESC/Java2and JML that were highlighted by this verification effortand are discussed in another paper [54].

5.2.5 Future Work

There are a number of major areas of development ofESC/Java2 that will improve overall usability of the tool,besides performance improvements.

– The use of model variables and method calls in anno-tation expressions. Model variables are an importantabstraction mechanism in writing specifications andmodel methods allow much more readable and com-pact specifications [23]. This is a current topic of re-search and experimentation; most of what is neededto support these features is a part of the current al-pha release of ESC/Java2 [25].

– Checking of the frame conditions specified by JML’sassignable clause (also known as modifies). It is anacknowledged unsoundness of ESC/Java that theseare not checked and faulty assignable clauses canbe a subtle source of errors. ESC/Java2 checks mostaspects of assignable clauses. However, the defaultassignable clause in JML specifications is that ev-erything is potentially modified; this interpretationis not currently implemented.

– Arithmetic. JML needs to have available for specifi-cations both mathematical integers and reals as wellas the finite-precision approximations that are used


Purse.java:31: Warning: Possible null dereference (Null)


^

---------------------------------------------------------------------

Purse.java:31: Warning: Array index possibly too large (IndexTooBig)


^

Fig. 3. Example ESC/Java2 warnings

in computer programs. There is some initial work [18]incorporating these into JML but as yet no axioma-tization that enables reasoning with ESC/Java2 .

The most significant aspect of future work, however,is experimentation with specification and static checkingof larger, more varied, and real-world bodies of sourcecode. Such experimentation is needed to verify that JMLhas the facilities that are needed for realistic specifica-tions and that static checking tools such as ESC/Java2are capable of providing a benefit to working program-mers.

5.2.6 Availability

An alpha version of ESC/Java2 is available from http://www.cs.kun.nl/sos/research/escjava. The tool isa Java program that is fairly platform-independent, butit uses the Simplify prover, which is only available onLinux, Windows, Solaris, and MacOSX platforms.

5.3 Program Verification with LOOP


The LOOP project at the University of Nijmegen startedout as an exploration of the semantics of object-orientedlanguages in general, and Java in particular. Only laterdid it evolve to investigate verification of JML-annotatedJava. For a detailed overview of the LOOP project werefer to [50].


The project began with the formalization of a denota-tional semantics of sequential Java [51] in the languageof the theorem prover PVS [82]. An associated compiler,called the LOOP tool [11], was developed, which trans-lates any given sequential Java class into PVS theoriesdescribing its semantics. In order to conveniently use thisas a basis for the specification and verification of Javacode, the LOOP tool was then extended to also providea formal semantics of JML, so that the tool now trans-lates JML-annotated Java code into proof obligations forPVS, which one can try to prove interactively, in PVS.These proof obligations are expressed as a special kind ofHoare statements about methods, and they are proved

using an associated Hoare logic [49] and weakest-pre-condition calculus [47] for Java and JML, both of whichhave been formalized in PVS.

A difference between LOOP and both ESC/Java(2)and JACK (see Section 5.4 for the JACK tool) is thatit provides a so-called shallow embedding of Java andJML in PVS, defining a formal denotational semanticsof both Java and JML in PVS. This has its advantages.The Hoare logic and wp-calculi that are used have beencompletely formalized and proven sound with respectto these semantics in PVS, whereas both ESC/Java(2)and JACK directly rely on an axiomatic semantics. Also,our semantics of Java in PVS is still (symbolically) ex-ecutable to a degree, as it lets PVS evaluate the de-notation of a program. This has been very useful in theextensive testing and debugging of our formal semantics,where we compared the results of the normal executionof a Java program, i.e. the result of executing its byte-code on a Java VM, and the symbolic execution of itssemantics in PVS.

5.3.3 Example

Using the LOOP tool to verify the example in Figure 1fails for the constructor, as it did for ESC/Java, becausethe specifications of the behavior of clone are incom-plete. The verification of the methods is fully automaticusing LOOP , using its weakest precondition calculus,except that the verification of checkPin needs manualinteraction in PVS to supply the loop invariant, as thetool doesn’t handle JML’s loop_invariant yet.

5.3.4 Experience

Case studies with the LOOP tool are discussed in [12,46,48]. Verification of JML-annotated code with the LOOPtool (especially the required interactive theorem provingwith PVS) can be very labor-intensive, but allows verifi-cation of more complicated properties than can be han-dled by fully automated extended static checking usingESC/Java. Because of this labor-intensive nature, onewill typically first want to use other, less labor-intensive,approaches, such as runtime assertion checking or ex-tended static checking, to remove some of the errors inthe code or specifications before turning to the LOOPtool. Experiences with such a combined approach aredescribed in [13]. The possibility to do this is an impor-


tant —if not crucial— advantage of using a specificationlanguage that is supported by a range of tools.

The LOOP tool generates a single proof obligationfor each method and constructor, expressed as a Hoarestatement. It does not, as commonly done in verificationcondition generators, split this up into smaller verifica-tion conditions. Instead, this splitting up is done insidethe theorem prover PVS, using dedicated proof strate-gies. A disadvantage of this is that the size of proof obli-gations that can be comfortably handled in PVS hasbecome a bottleneck.

5.3.5 Future Work

Ongoing work on the LOOP tool includes support forthe different forms of arithmetic as proposed in [18] andinvestigations into proving information flow properties.The longer term plans for the LOOP tool are currentlynot clear.

5.3.6 Availability

The LOOP tool is not publicly available, simply becauseit is not easy to use without intensive user support anddocumentation that we cannot provide. Actually, LOOPitself is easy enough to use — it is simply a compiler thatoutputs PVS — but dealing with the large and numer-ous PVS theories it outputs requires considerable (PVS)expertise.

5.4 Static Verification with JACK


The JACK [15] tool was initially developed at the re-search lab of Gemplus, a manufacturer of smartcards andsmartcard software. Further development is now happen-ing at INRIA. JACK aims to provide an environmentfor Java and Java Card program verification using JMLannotations. It implements a fully automated weakestprecondition calculus in order to generate proof obli-gations from JML-annotated Java sources. Those proofobligations can then be discharged using different theo-rem provers.

The main design goals are an easily accessible userinterface, a high degree of automation, a high correctnessassurance, and prover independence.


The main goal of JACK is that it should be usable bynormal Java developers, allowing them to validate theirown code, following, in this way, the JML philosophy.Thus, care has been taken to hide the mathematical for-mulation of the underlying concepts. To allow develop-ers to work in a familiar environment, JACK is inte-

grated as a plug-in to the Eclipse2 IDE. This plug-inallows users to generate proof obligations, to run theautomatic provers, and to inspect the generated lem-mas. To facilitate this last task, JACK provides a ded-icated proof obligation viewer. This viewer presents theproof obligations as execution paths within the program,highlighting the source code relevant to the proof obli-gations. Moreover, goals and hypotheses are displayedin a Java/JML-like notation. The user can then workwithin its current development tool, add the JML anno-tations and check partially the correctness of the codein a familiar environment.

JACK ’s core is an implementation, in Java, of aweakest precondition calculus. This ensures proof obliga-tion generation without user interaction. Following thisstep, automatic provers are used to prove the generatedlemmas. Users then have to check whether any remain-ing lemmas are valid or not. To reduce the remainingcostly manual task – creating the JML annotation as-sertions – we have developed and integrated in JACK aprototype that annotates source code with assertions bypropagation of pre and post conditions. This is a way toreduce the cost of using JML, since, at the moment, themain issue when using JACK is the time spent annotat-ing classes.

JACK is not based on a formalization of Java asLOOP is; thus one cannot easily prove the formal cor-rectness of the tool, and the implementation of the weak-est precondition calculus can contain bugs. Nevertheless,the aim of the tool is to be complete and sound (i.e. togenerate all proof obligations that are valid if and only ifthe application respects its formalization). So, users canchoose to check partially the correctness of their applica-tion by just reviewing the unproved proof obligations, orthey can also prove all the proof obligations using an in-teractive theorem prover, thereby obtaining a completeassurance on the development correctness.

JACK provides an interface to automatic theoremprovers. Currently, the prover of the Atelier B toolkit,Simplify (the prover used in ESC/Java), and PVS areintegrated. These provers are integrated as plug-ins inJACK . Since JACK is based on an intermediate lemmaformulation language, it is quite easy to integrate newprovers by implementing a translator from this interme-diate language to the prover input. Interfacing severalprovers increases the automatic proof ratio. This also al-lows people to prove any remaining lemmas interactivelyin their preferred prover.

The actually interfaced automatic provers can usu-ally automatically prove up to 90% of the proof obliga-tions. The remaining ones have to be proved outside ofJACK , using the classical B proof tool, PVS, or the Coqproof assistant. However, JACK is meant to be used byJava developers, who cannot be expected to use a proofassistant. Therefore, in addition to the proved and un-

2 http://www.eclipse.org


proved states, JACK adds a checked state, which allowsdevelopers to indicate that they have manually checkedthe proof obligation. In order to better handle thosecases, other different approaches could be investigated,such as integration with test tools such as jmlunit , in-tegration of other proof assistants, or perhaps supportfrom a proof-expert team.

5.4.3 Example

The code of the class given in the Figure 1 was provedusing JACK . To generate proof obligations automati-cally, loop invariants have to be given explicitly in thecode. Here, the JML annotation of Figure 4 is addedin the body of the method checkPin before the forstatement. When this annotation is added, one can runJACK , which then calculates proof obligations automat-ically and proves them using the automated provers.Here, only three proof obligations remain unproved dueto, yet again, the lack of complete specification of theclone() method in the constructor.

5.4.4 Experience

Like ESC/Java, JACK tries to hide the complications ofthe underlying theorem prover from the user, by provid-ing a push-button tool that normal Java developers, andnot just formal methods experts, can and would like touse. We believe that this may be a way to let non-expertsventure into the world of formal verification.

ESC/Java, LOOP , and JACK all use (or, in thecase of LOOP , have the option of using) a weakest pre-condition calculus to generate verification conditions.ESC/Java and LOOP generate one verification condi-tion per method implementation, whereas JACK gener-ates roughly one verification condition per syntactic codepath through the code. So each of JACK ’s verificationconditions is smaller than those generated by ESC/Javaand LOOP . On the other hand, JACK may generatea very large number of verification conditions. Thoughit generates just one verification condition per method,ESC/Java factors its verification conditions differentlythan the other two tools (see [37,61]) and therefore isable to keep the one verification condition reasonablysmall. More important than size, verification conditionsgenerated by ESC/Java often let the theorem proveravoid redundant work. JACK ’s approach has the ad-vantage that it is easy to pass the different verificationconditions to different theorem provers.

5.4.5 Future Work

To increase the automation of this validation phase, weare currently thinking of interfacing JACK with a coun-terexample detector or runtime test generator. We arealso still investigating the annotation generation andpropagation techniques since we consider that it can bea way to reduce the cost of using the tool.

5.4.6 Availability

JACK is currently not publicly available.

6 Generating Specifications

Apart from checking that implementations meet specifi-cations, a considerable barrier to entry in the use of anyformal specification language is writing specifications inthe first place. The JML tools discussed so far assumethe existence of a JML specification, and leave the taskof writing it to the programmer. This task can be time-consuming, tedious, and error-prone, so tools that canhelp in this task can be of great benefit.

6.1 Invariant Detection with Daikon


The Daikon invariant detector [31,32] is a tool that pro-vides assistance in creating a specification. Daikon out-puts observed program properties in JML syntax (as wellas other output formats) and automatically inserts theminto a target program.


The Daikon tool dynamically detects likely program in-variants. In other words, given program executions, itreports properties that were true over those executions.The set of reported properties is also known as an op-erational abstraction. Dynamic invariant detection op-erates by observing values that a program computesat runtime, generalizing over those values, and report-ing the resulting properties. The properties reported byDaikon encompass numbers (x <= y, y == ax+ b), col-lections (mytree.contains(x ), mylist .isSorted()), point-ers (n == n.next .prev), and implications (p != null==> p.value > x ); a complete list appears in the Daikonuser manual.

Like any dynamic analysis, the accuracy of the in-ferred invariants depends in part on the quality and com-pleteness of the test cases, and other executions may fal-sify some of the reported properties. (Furthermore, theactual behavior of the program is not necessarily thesame as its intended behavior.) However, Daikon usesstatic analysis, statistical tests, and other mechanismsto reduce the number of false positives [33]. Even if aproperty is not true in general, Daikon’s output providesvaluable information about the test suite over which theprogram was run. Combining invariant detection witha static verifier such as ESC/Java helps to overcomethe problems of both techniques: the unsoundness of thedynamic analysis and the static analysis’s need for an-notations.


//@ loop_invariant 0 <= i <= 5;

//@ loop_invariant res == (\forall int j; 0 <= j && j < i; pin[j] == p[j]);

Fig. 4. Loop invariant

6.1.3 Example

In order to apply Daikon to a program, a user runs aninstrumented version of the program to create a datatrace file, then runs Daikon over the data trace file toproduce likely invariants. The instrumented version ofthe program contains, at program points such as proce-dure entries and exits, code that writes the values of allvariables in scope to a trace file. In some cases (as forDaikon’s C front end), the instrumentation is performedautomatically on a compiled executable by a special run-time system. In other cases (as for Daikon’s Java frontend), the user runs a source-to-source translator thatinstruments the program, then runs the instrumentedprogram in place of the original.

Given a simple test suite that creates 1000 randomPurse objects and invokes debit on each one, the Daikontool automatically generates the annotations of Figure 1,except that the current version of Daikon does not gen-erate JML’s signals clauses. Daikon’s output is correctJML that is parseable by the JML toolset.

6.1.4 Experience

Even with modest test suites, Daikon’s output is re-markably accurate. In one set of experiments [80], over90% of the properties that it reported were verifiable byESC/Java (the other properties were true, but were be-yond the capabilities of ESC/Java), and it reported over90% of the properties that ESC/Java needed in order tocomplete its verification. For example, if Daikon gener-ated 100 properties, users had only to delete less than10 properties and to add another 10 properties in orderto have a verifiable set of properties. In another experi-ment [81], users who were provided with Daikon output(even from unrealistically bad test suites) performed sta-tistically significantly better on a program verificationtask than did users who did not have such assistance.

In addition to aiding the task of static checking asdescribed above, operational abstractions generated bythe Daikon invariant detector have been used to generateand improve test suites [44,93,40], automate theorem-proving [78,79], identify refactoring opportunities [53],aid program analysis [29,30], choose modalities [67], pre-dict incompatibilities in component upgrades [71,72], de-tect anomalies and bugs [89,43,87,14,70], and isolate er-rors [92,39,66], among other uses.

6.1.5 Future Work

As noted above, Daikon does not generate JML signalsclauses for exceptional method exits. Doing so requires

enhancements to the language-specific front ends, butno significant changes to Daikon proper. Another in-strumentation enhancement that we are pursuing is re-placing the current Java instrumenter (which performsa source-to-source translation) by one that is embed-ded in the Java Virtual machine and works on compiledJava programs. This change will simplify using Daikonby reducing the work required of a user. Finally, mak-ing Daikon work online — taking data from a runningprogram rather than from a trace file — will reduce thenumber of steps to 1, which is the same as currently re-quired to run any Java program (via the java command).

Our main research thrust is not to improve Daikonitself, but to find more uses for the operational abstrac-tions that it produces. Linking it to verification toolsfrom the JML toolset is just one application; some oth-ers were noted above in Section 6.1.4.

6.1.6 Availability

Daikon is publicly available, in both source and com-piled form, from http://pag.csail.mit.edu/daikon/.Daikon includes front ends for Java, C, Perl, and otherlanguages and input formats.

Several other implementations of dynamic invariantdetection exist [43,87,45]. However, they do not produceoutput in JML format, they are not publicly available,and they check and report only a small fraction of theproperties that Daikon does [83].

6.2 Inferring annotations with Houdini


An obstacle to using program verification tools such asESC/Java on legacy code is the lack of annotations insuch a program. The warnings more likely point outmissing annotations than errors in the code. The Hou-dini tool [35,34] attempts to alleviate this problem bysupplying many of the missing annotations.


Houdini works by making up candidate annotations forthe given program. Such candidate annotations comparefields and array lengths to -1, 0, 1, constants used inarray constructors, null, true, and false (dependingon the type of the field), and indicate that arrays andsub-arrays contain no null elements. To find which ofthe candidate annotations hold for the program, Hou-dini repeatedly invokes ESC/Java, removing those can-


didate annotations that ESC/Java finds to be inconsis-tent with the code. When all remaining candidate an-notations are consistent with the code, Houdini invokesESC/Java a final time to produce warnings that are thenpresented to the user. Houdini thus retains the precisionof ESC/Java, trading quick turnaround for a reducedannotation effort.

Note that any user-supplied JML annotations in theprogram still get used by Houdini , since they becomepart of each invocation of ESC/Java. Thus, the benefitsof using JML annotations are the same for Houdini as forESC/Java, but Houdini can find program errors from asmaller set of user-supplied JML annotations.

6.2.3 Example

If the class in Figure 1 is given to Houdini withoutany annotations, then Houdini will produce a numberof candidate annotations, including the invariants 0 <=balance and 1 <= balance and the Purse-constructorpreconditions 0 <= b and 1 <= b. If the given programcontains a call to the Purse constructor that passes in 0for b, then the candidate precondition 1 <= b is refutedand removed. Since the constructor assigns b to balance,the candidate invariant 1 <= balance will then eventu-ally also become refuted.

Houdini will also include balance <= MAX BALANCEamong many other candidate annotations, but will notinclude, for example, the universal quantifications shownin Figure 1.

6.2.4 Experience

Houdini has been applied to a number of real applica-tion programs, the initial account of which is reportedin [35]. For each of the applications, Houdini (in concertwith ESC/Java) was able to find errors. The number ofwarnings produced was generally larger than the num-ber of warnings inspected by a user. For example, in the36-KLOC program “Cobalt” [35], only 200 of the 540warnings were inspected by a user, though this inspec-tion revealed 8 errors. In the largest program to whichHoudini was applied, a systems administration tool com-prising 500 KLOC of Java, the number of warnings pro-duced was too large to be particularly useful, though aninspection of 10 of the warnings still revealed 2 programerrors. The experience with Houdini , albeit limited, sug-gests that it is possible for a user to inspect a programfor errors at a rate of upwards of 1000 LOC per hour.

6.2.5 Future Work

Though Houdini has found real errors, some problemsmake the tool less effective than one would like. We men-tion three such problems here.

First, Houdini ’s simple strategy for producing candi-date annotations limits the number of ESC/Java warn-

ings it can suppress. Future work might consider apply-ing more static analysis or dynamic profiling to improvethe initial set of candidate annotations.

Second, to reduce the number of warnings produced,it is important for Houdini to infer good class invari-ants. Even in the cases where Houdini ’s candidate setincludes the necessary invariants, Houdini may fail toinfer them because the first point at which they hold isunknown. For example, ESC/Java generally checks thatan object’s invariant has been established before the ob-ject’s constructor invokes any method on the object. Butthe purpose of such a method invocation is sometimesto help establish the object’s invariant in the first place.In an attempt to improve this situation, Houdini uses aspecial mode of ESC/Java, where ESC/Java inlines anymethod call from a constructor. This mode allows Hou-dini to infer better invariants, but sometimes producesenormous verification conditions in ESC/Java. Futurework might find a better solution to this problem.

Third, to avoid forcing users to write loop invariants,ESC/Java by default analyzes only a fixed number ofunrollings of each loop. If the loop is known always togo through more iterations than are unrolled (for exam-ple, if a for loop iterates exactly 10 times, where 10is a constant mentioned in the loop head), then the ef-fect is that ESC/Java’s analysis doesn’t ever reach theother side of the loop. This may be acceptable in a man-ual application of ESC/Java, since ESC/Java performsmodular checking method by method, and therefore thechecking of other methods is unaffected. However, forHoudini , whose inference is more like that of a whole-program analysis, this situation can have a paralyzingeffect on the entire program analysis. Houdini side stepsthis situation by using a special mode of ESC/Java,where ESC/Java in effect introduces a jump from itslast unrolling of the loop until after the loop. This ismuch better for Houdini , but it also introduces execu-tion paths that don’t exist in the given program, whichleads to other problems. Perhaps there are better solu-tions.

6.2.6 Availability

Work on the Houdini tool petered out in 2001 with thetransformation of the Compaq Systems Research Center.The sources of the final version of Houdini are availablein the ESC/Java source distribution, named the “JavaProgramming Toolkit Source Release”, at http://www.research.compaq.com/downloads.html.

7 Documentation

Generating human-readable web pages from JML spec-ifications is accomplished by the jmldoc tool.


7.1 jmldoc


The goal of the jmldoc tool is to produce HTML pageslike those produced by the javadoc tool, but includinginformation from JML annotations as well. JML allowsspecifications to be spread across a number of refinementfiles. This is essential, for example, in the case that theJava source code may not be modified to include specifi-cations directly in the source code. Even within one file,the specifications relevant to the class may be spreadthroughout the file, making easy spotting of a relevantinvariant difficult. Also, JML enforces behavioral inher-itance, in which an overriding method must satisfy thespecification of the methods it overrides. Accordingly,jmldoc includes in the HTML representation of the spec-ifications of a method the specifications of the methodsit overrides. By combining and grouping these specifica-tions appropriately, jmldoc makes them more accessibleto the programmer. Particularly for those accustomedto browsing the javadoc documentation of an API, theinclusion of the additional specifications in a formal no-tation as provided by jmldoc is expected to be a conve-nience.


The jmldoc tool is designed to leverage as much of boththe JML tools and the javadoc tool as possible. It usesthe classes of the JML checker to parse, typecheck, andprovide an AST that includes specifications of each classand method being documented. The javadoc tool pro-vides a doclet API that allows some reuse of the javadocframework. Many of the contributed doclets use the pro-vided classes to parse valid Java source code with javadoccomments and then to do checks or alternative process-ing on those files, such as producing PDF rather thanHTML or checking that all methods do indeed havejavadoc comments. The jmldoc tool instead alters (byderivation) the mechanism that generates the HTMLpages in order that the output will contain in additioninformation about the JML annotations in the sourcefiles, as provided by the JML-generated AST. In thisway, the jmldoc tool remains consistent with the otherJML tools in their handling of the JML language, butit also produces HTML pages consistent with other cur-rent javadoc documentation and with that produced bythe javadoc tool itself. Aside from accepting additionalcommand-line options appropriate to JML, jmldoc is in-tended to be a drop-in replacement for javadoc.

7.1.3 Example

An example of jmldoc’s output is shown in Figure 5;it shows the current output produced for the methodHashMap.size as currently specified by JML specifica-tions for Java system classes.

7.1.4 Experience

The main experience we have with jmldoc is in documen-tation of packages that ship with JML, such as JML’sbuilt-in types for modeling and its samples, and withdocumentation of parts of the Java standard libraries.While these are used by JML users on a daily basis,there have been no formal case studies of the usefulnessof jmldoc. Informal reports, however, have been positive.

7.1.5 Future Work

The tool is being maintained as part of the JML toolset,but not being extended further other than to keep pacewith changes in the definition of JML itself. Extensivemaintenance is also needed to keep pace with changesin the doclet API with each new version of Java. Asit happens, the portions of the doclet API that are ex-tended by jmldoc have been changing significantly evenbetween minor releases of javadoc. If this rate of changecontinues, the JML project may need to seek an alter-native design that is not tied as closely to the currentappearance of javadoc documentation in order to lessenthe maintenance burden.

7.1.6 Availability

The jmldoc tool was authored by David Cok along thelines of the goals espoused by Raghavan [88]. It is part ofthe main JML toolset available via www.jmlspecs.org,which is developed as an open source project hosted atSourceForge.net.

8 Applications of JML to Java Card

Although JML is able to specify arbitrary sequentialJava programs, most of the serious applications of JMLand JML tools up to now have targeted Java Card.Java CardTM is a dialect of Java specifically designedfor the programming of the latest generation of smart-cards. Java Card is adapted to the hardware limitationsof smartcards; for instance, it does not support floatingpoint numbers, strings, object cloning3, or threads.

Java Card is a well-suited target for the applicationof formal methods. It is a relatively simple languagewith a restricted API. Moreover, Java Card programs,called applets, are small, typically on the order of severalKBytes of bytecode. Additionally, correctness of JavaCard programs is of crucial importance, since they areused in sensitive applications, e.g. as bank cards, iden-tity cards, and in mobile phones. Furthermore, once such

3 The fact that Java Card does not have cloning means that aversion of the Purse example in Figure 1 rewritten to Java Cardrather than Java does verify using ESC/Java, LOOP , or JACK .Indeed, the absence of clone in Java Card is a reason why dealingwith clone has not been a priority in these tools.


Fig. 5. Example jmldoc output

smartcards are issued, it is difficult, if not impossible, tofix any software errors.

JML, and several tools for JML, have been used forJava Card, especially in the context of the EU-supportedproject VerifiCard (www.verificard.org).

JML has been used to write a formal specificationof almost the entire Java Card API [86]. This experi-ence has shown that JML is expressive enough to specifynon-trivial existing API classes. The runtime assertionchecker has been used to specify and verify a componentof a smartcard operating system [85].

ESC/Java has been used with great success to verifya realistic example of an electronic purse implementationin Java Card [16]. This case study was instrumental inconvincing industrial users of the usefulness of JML andfeasibility of automated program checking by ESC/Javafor Java Card applets. In fact, this case study providedthe motivation for the development of the JACK tooldiscussed earlier, which is specifically designed for JavaCard programs. One of the classes of the electronic pursehas also been verified using the LOOP tool [12]. Anoverview of the work on this electronic purse, and theway in which ESC/Java and LOOP can be used to com-plement each other, is given in [13].

As witnessed by the development of the JACK toolby Gemplus, Java Card smartcard programs may beone of the niche markets where formal methods havea promising future. Here, the cost that companies arewilling to pay to ensure the absence of certain kinds ofbugs is quite high. It seems that, given the current stateof the art, using static checking techniques to ensurerelatively simple properties (e.g., that no runtime ex-ception ever reaches the top-level without being caught)seems to provide an acceptable return-on-investment. Itshould be noted that the very simplicity of Java Card isnot without its drawbacks. In particular, the details ofits very primitive communication with smartcards (viaa byte array buffer) is not easily abstracted away from.It will be interesting to investigate if J2ME (Java 2 Mi-cro Edition), which targets a wider range of electronic

consumer products, such as mobile phones and PDAs, isalso an interesting application domain for JML.

9 Related Work

9.1 Java

Many runtime assertion checkers for Java exist, for ex-ample Jass, iContract, and Parasoft’s jContract, to namejust a few. Each of these tools has its own specificationlanguage, thus specifications written for one tool do notwork in any other tool. And while some of these toolssupport higher-level constructs such as quantifiers, allare quite primitive when compared to JML. For exam-ple, none include support for purity specification andchecking, model methods, refinements, or unit test inte-gration. The developers of Jass have expressed interestin moving to JML as their specification language.

The ChAsE tool [17] is a static checker for JML’sassignable clauses. It performs a syntactic check onsuch clauses, which, in the spirit of ESC/Java, is nei-ther sound nor complete, but which spots many mis-takes made in the user’s assignable clauses. ChAsE wasdeveloped to complement the functionality missing inother tools: not checking assignable clauses was one ofthe sources of unsoundness of ESC/Java. Also, assign-able clauses are not checked by the runtime assertionchecker, making errors in assignable clauses hard to de-tect. The functionality to check assignable clauses is nowincorporated in ESC/Java2 . Also, the JML runtime as-sertion checker has started to incorporate some of thisfunctionality.

In addition to ESC/Java(2), LOOP , and JACK , sev-eral other tools exist for the verification of Java code,for instance Krakatoa [69], Jive [74], and KeY [1]. TheKrakatoa tool also uses JML as specification language; itproduces proof obligations for the theorem prover Coq.It is planned that Jive will also start supporting JML.


The KeY tool uses OCL instead as its specification lan-guage, and is integrated with a commercial CASE tool.

9.2 Other languages

SPARK (www.sparkada.com, [4]) is an initiative similarto JML in many respects, but much more mature, andtargeting Ada rather than Java. SPARK (which standsfor Spade Ada Kernel) is a language designed for pro-gramming high-integrity systems. It is a subset of Ada95(with no object references and subclasses, for example)enriched with annotations to enable tool support. Thisincludes tools for data- and information-flow analysis,and for code verification, in particular to ensure the ab-sence of runtime exceptions [2]. Spark has been success-fully used to construct high-integrity systems that havebeen certified using the Common Criteria, the ISO stan-dard for the certification of information technology secu-rity. SPARK and the associated tools are marketed byPraxis Critical Systems Ltd., demonstrating that thistechnology is commercially viable.

A more recent initiative that is very similar to JMLis Spec# [6]. The Spec# language extends C# withcontract specifications, analogously to the way JML ex-tends Java. The Spec# compiler then introduces runtimechecks for the declared specifications (akin to jmlc), andthe Boogie program verifier tries to prove these speci-fications statically using an automatic theorem prover(akin the tools described in Section 5). One differencebetween Spec# and JML is that Spec# builds in a newmethodology for object invariants [5,63,7], trading re-strictions on the kinds of programs that can be writtenfor a sound modular reasoning technique.

9.3 OCL: UML’s constraint language

Despite the similarity in the acronyms, JML is very dif-ferent in its aims from UML [90]. The most basic differ-ence is that the UML aims to cover all phases of anal-ysis and design with many notations, and it tries to beindependent of programming language, while JML onlydeals with detailed designs (for APIs) and is tied to Java.The model in JML refers to abstract, specification-onlyfields that can be used to describe the behavior of vari-ous types. By contrast, the model of UML refers to thegeneral modeling process (analysis and design) and isnot limited to abstractions of individual types.

JML does have some things in common with the Ob-ject Constraint Language (OCL) [91], which is part ofthe UML standard. Like JML, OCL can be used to spec-ify invariants and pre- and postconditions. An importantdifference is that JML explicitly targets Java, whereasOCL is not specific to any one programming language.One could say that JML is related to Java in the sameway that OCL is related to UML.

JML clearly has the disadvantage that it can not beused for, say, C++ programs, whereas OCL can. But it

also has obvious advantages when it comes to syntax,semantics, and expressivity. Because JML sticks to theJava syntax and typing rules, a typical Java programmerwill prefer JML notation over OCL notation, and, forinstance, prefer to write (in JML):

invariant pin != null && pin.length == 5;

rather than the OCL:

inv: pin <> null and pin->size() = 5

JML supports all the Java modifiers such as static,private, public, etc., and these can be used to recorddetailed design decisions for different readers. Further-more, there are legal Java expressions that can be usedin JML specifications but that cannot be expressed inOCL.

More significant than these limitations, or differencesin syntax, are differences in semantics. JML builds onthe (well-defined) semantics of Java. So, for instance,equals has the same meaning in JML and Java, as does==, and the same rules for overriding, overloading, andhiding apply. One cannot expect this for OCL, althoughefforts to define a semantics for OCL are underway.

In all, we believe that a language like JML, whichis tailored to Java, is better suited for recording the de-tailed design of Java programs than a generic languagelike OCL. Even if one uses UML in the development ofa Java application, it may be better to use JML ratherthan OCL for the specification of object constraints, es-pecially in the later stages of the development. Therehas been work on automatically translating OCL to JML[42].

10 Conclusions

We believe that JML presents a promising opportunityto gently introduce formal specification into industrialpractice. It has the following strong points:

1. JML is easy to learn for any Java programmer, sinceits syntax and semantics are very close to Java. Webelieve this a crucial advantage, as a big hurdle tointroducing formal methods in industry is often thatpeople are not willing, or do not have the time, tolearn yet another language.

2. There is no need to invest in the construction of aformal model before one can use JML. Or rather: thesource code is the formal model. This brings furtheradvantages:– It is easy to introduce the use of JML gradually,

simply by adding the odd assertion to some Javacode.

– JML can be used for existing (legacy) code andAPIs. Indeed, most applications of JML and itstools to date have involved existing APIs andcode.


– There is no discrepancy between the actual codeand the formal model. In traditional applicationsof formal methods there is often a gap betweenthe formal model and the actual implementation,which means that some bugs in the implementa-tion cannot be found, because they are not partof the formal model, and, conversely, some prob-lems discovered in the formal model may not berelevant for the implementation.

3. There is a growing availability of a wide range of toolsupport for JML.

Unlike B, JML does not impose a particular designmethodology on its users. Unlike UML, VDM, and Z,JML is tailored to specifying both the syntactic interfaceof Java code and its behavior. Therefore, JML is bettersuited than these alternative languages for documentingthe detailed design of existing Java programs.

As a common notation shared by many tools, JMLoffers users multiple tools supporting the same notation.This frees users from having to learn a whole new lan-guage before they can start using a new tool. The sharednotation also helps the economics both for users and toolbuilders. Any industrial use of formal methods will haveto be economically justified, by comparing the costs (theextra time and effort spent) against the benefits (im-provements in quality, number of bugs found). Havinga range of tools, offering different levels of assuranceat different costs, makes it much easier to start usingJML. One can begin with a technique that requires theleast time and effort (perhaps runtime assertion check-ing) and then move to more labor-intensive techniques ifand when that seems worthwhile, until one has reached acombination of tools and techniques that is cost-effectivefor a particular situation.

Using any of the tools for static checking or verifi-cation requires formal specifications of the APIs of anysystem libraries used, and the cost of developing suchspecifications is very high. Indeed, the largest case studyto date in using JML for specification is the ongoing workin developing specifications for substantial parts of theJava system libraries. Being able to reuse these samespecifications for different tools is an important advan-tage.

Future Work

There are still many opportunities for further develop-ment of both the JML language and its tools. For in-stance, we would also like to see support for JML inintegrated development environments (such as Eclipse)and integration with other kinds of static checkers.

A major recent extension to JML concerns the sup-port for different forms of arithmetic, providing normalmathematical integers in addition to Java’s n-bit 2’s-complements integers [18].

One important aspect of future work is experiment-ing with the use JML for specification of real-world codeand APIs, and using the associated tools. There hasbeen a lot of work on producing JML specifications ofthe Java system libraries (these can be downloaded fromwww.jmlspecs.org), but more work is needed.

Using JML to specify real-world code raises manyinteresting issues. For instance, JML allows pure meth-ods to be used in annotations, where pure methods aredefined as those which have no side-effects. But this isa very strict definition, which can be impractical whenwriting specifications, as many methods (including somein core Java libraries) that programmers intuitively as-sume to be pure are not pure, due to unobservable andbenevolent side-effects [59]. Work continues on a betterand more useful definition of purity, e.g. [8].

With more tools supporting JML, and the specifica-tion language JML growing in complexity due to the dif-ferent features that are useful for the different tools, oneimportant challenge is maintaining agreement on the se-mantics of the language between the different tools. Onething that has become very clear in the course of devel-oping JML is that precisely defining the semantics of aspecification language such as JML is very tricky.

More generally, there are several fundamental issuesin the specification of object-oriented systems that arestill active topics of investigation. The notion of objectinvariant is tricky in the presence of callbacks [5,7,63,77]. Another largely open issue is how concurrency prop-erties should be specified.

As always in imperative programming, aliasing is amajor source of complications, and an important sourceof bugs. For example, in the example in Figure 1 it isprobably important that in the constructor the field pinis not simply aliased to the argument p, but that a newarray is created. However, the current specification doesnot demand this. JML should offer practical ways toconstrain potential aliasing. A first proposal is given in[76].

The subtleties involved in such open problems are ev-idenced by the slightly different ways in which differenttools approach these problems. This reflects the research(as opposed to industrial development) focus of most ofthose involved in JML and its tools. Nevertheless, JMLseems to be successful in providing a common notationand a semantics that is, at least for a growing core sub-set, shared by many tools, and as a common notation,JML is already proving to be useful to both tool devel-opers and users.

Acknowledgements. Despite our long list of co-authors, stillmore people have been involved in developing the tools dis-cussed in this paper, including Joachim van den Berg, Ab-hay Bhorkar, Kristina Boysen, Cees-Bart Breunesse, NestorCatano, Patrice Chalin, Curtis Clifton, Kui Dai, Werner Di-etl, Marko van Dooren, Cormac Flanagan, Mark Lillibridge,Marieke Huisman, Bart Jacobs, Jean-Louis Lanet, Todd Mill-


stein, Peter Mueller, Greg Nelson, Jeremy Nimmer, CarlosPacheco, Arun Raghavan, Antoine Requet, Frederic Rioux,Clyde Ruby, Jim Saxe, Raymie Stata, Roy Tan, and MartijnWarnier. Thanks to Raymie Stata for his initiative in gettingthe JML and ESC/Java projects to agree on a common syn-tax, and to Michael Moller for the logo. Work on the JMLtools at Iowa State builds on the MultiJava compiler writtenby Curtis Clifton as an adaptation of the Kopi Java compiler.

References

1. W. Ahrendt, Th. Baar, B. Beckert, R. Bubel, M. Giese,R. Hahnle, W. Menzel, W. Mostowski, A. Roth,S. Schlager, and P. H. Schmitt. The KeY tool. Soft-ware and System Modeling, 2004. To appear.

2. Peter Amey and Roderick Chapman. Industrial strengthexception freedom. In ACM SigAda 2002, pages 1–9.ACM, 2002.

3. Sergio Antoy and Dick Hamlet. Automatically checkingan implementation against its formal specification. IEEETransactions on Software Engineering, 26(1):55–69, Jan-uary 2000.

4. John Barnes. High Integrity Software: The SPARK Ap-proach to Safety and Security. Addison Wesley, 2003.

5. Mike Barnett, Robert DeLine, Manuel Fahndrich,K. Rustan M. Leino, and Wolfram Schulte. Verificationof object-oriented programs with invariants. Journal ofObject Technology, 3(6):27–56, 2004.

6. Mike Barnett, K. Rustan M. Leino, and Wolfram Schulte.The Spec# programming system: An overview. In Con-struction and Analysis of Safe, Secure and InteroperableSmart devices (CASSIS), LNCS, 2004. To appear.

7. Mike Barnett and David Naumann. Friends need a bitmore: Maintaining invariants over shared state. In Dex-ter Kozen, editor, Mathematics of Program Construction,volume 3125 of LNCS, pages 54–84. Springer, July 2004.

8. Mike Barnett, David A. Naumann, Wolfram Schulte, andQi Sun. 99.44% pure: Useful abstractions in specifi-cations. In Formal Techniques for Java-like Programs.Proceedings of the ECOOP’2004 Workshop, pages 11–18, 2004. Technical Report NIII-R0426, University ofNijmegen.

9. D. Bartetzko, C. Fischer, M. Moller, and H. Wehrheim.Jass — Java with assertions. In Workshop on RuntimeVerification at CAV’01, 2001. Published in ENTCS, K.Havelund and G. Rosu (eds.), 55(2), 2001.

10. Kent Beck and Erich Gamma. Test infected: Program-mers love writing tests. Java Report, 3(7):37–50, 1998.

11. Joachim van den Berg and Bart Jacobs. The LOOPcompiler for Java and JML. In T. Margaria and W. Yi,editors, TACAS’01, number 2031 in LNCS, pages 299–312. Springer–Verlag, 2001.

12. Cees-Bart Breunesse, Joachim van den Berg, and BartJacobs. Specifying and verifying a decimal representationin Java for smart cards. In H. Kirchner and C. Ringeis-sen, editors, AMAST’02, number 2422 in LNCS, pages304–318. Springer–Verlag, 2002.

13. Cess-Bart Breunesse, Nestor Catano, Marieke Huisman,and Bart Jacobs. Formal methods for smart cards: anexperience report. Technical report, University of Ni-jmegen, 2003. NIII Technical Report NIII-R0316. Toappear in Science of Computer Programming, Elsevier.

14. Yuriy Brun and Michael D. Ernst. Finding latent codeerrors via machine learning over program executions.In ICSE’04, Proceedings of the 26th International Con-ference on Software Engineering, Edinburgh, Scotland,May 26–28, 2004.

15. Lilian Burdy, Antoine Requet, and Jean-Louis Lanet.Java applet correctness: A developer-oriented approach.In D. Mandrioli K. Araki, S. Gnesi, editor, FME 2003,volume 2805 of LNCS, pages 422–439. Springer–Verlag,2003.

16. Nestor Catano and Marieke Huisman. Formal specifica-tion of Gemplus’s electronic purse case study. In L. H.Eriksson and P. A. Lindsay, editors, FME 2002, volumeLNCS 2391, pages 272–289. Springer–Verlag, 2002.

17. Nestor Catano and Marieke Huisman. CHASE: A staticchecker for JML’s assignable clause. In Lenore D.Zuck, Paul C. Attie, Agostino Cortesi, and SupratikMukhopadhyay, editors, VMCAI: Verification, ModelChecking, and Abstract Interpretation, volume 2575 ofLNCS, pages 26–40. Springer–Verlag, 2003.

18. Patrice Chalin. JML support for primitive arbitrary pre-cision numeric types: Definition and semantics. Journalof Object Technology, 3(6):57–79, 2004.

19. Yoonsik Cheon. A runtime assertion checker for the JavaModeling Language. Technical Report 03-09, Depart-ment of Computer Science, Iowa State University, Ames,IA, April 2003. The author’s Ph.D. dissertation. Avail-able from archives.cs.iastate.edu.

20. Yoonsik Cheon and Gary T. Leavens. TheLarch/Smalltalk interface specification language. ACMTransactions on Software Engineering and Methodology,3(3):221–253, July 1994.

21. Yoonsik Cheon and Gary T. Leavens. A runtime as-sertion checker for the Java Modeling Language (JML).In Hamid R. Arabnia and Youngsong Mun, editors, theInternational Conference on Software Engineering Re-search and Practice (SERP ’02), pages 322–328. CSREAPress, June 2002.

22. Yoonsik Cheon and Gary T. Leavens. A simple and prac-tical approach to unit testing: The JML and JUnit way.In Boris Magnusson, editor, ECOOP 2002, volume 2374of LNCS, pages 231–255. Springer–Verlag, June 2002.

23. Yoonsik Cheon, Gary T. Leavens, Murali Sitaraman, andStephen Edwards. Model variables: Cleanly supportingabstraction in design by contract. Technical Report 03-10, Department of Computer Science, Iowa State Uni-versity, April 2003.

24. Curtis Clifton. MultiJava: Design, implementation, andevaluation of a Java-compatible language supportingmodular open classes and symmetric multiple dispatch.Technical Report 01-10, Department of Computer Sci-ence, Iowa State University, Ames, Iowa, 50011, Novem-ber 2001. Available from www.multijava.org.

25. David R. Cok. Reasoning with specifications containingmethod calls in jml. In Formal Techniques for Java-likePrograms. Proceedings of the ECOOP’2004 Workshop,pages 41–48, 2004. Technical Report NIII-R0426, Uni-versity of Nijmegen.

26. David Detlefs, Greg Nelson, and James B. Saxe. Sim-plify: A theorem prover for program checking. TechnicalReport HPL-2003-148, HP Labs, July 2003.

27. David L. Detlefs, K. Rustan M. Leino, Greg Nelson, andJames B. Saxe. Extended static checking. Research Re-


port 159, Compaq Systems Research Center, December1998.

28. Krishna Kishore Dhara and Gary T. Leavens. Forcingbehavioral subtyping through specification inheritance.In 18th International Conference on Software Engineer-ing, pages 258–267. IEEE Computer Society Press, 1996.

29. Nii Dodoo, Alan Donovan, Lee Lin, and Michael D.Ernst. Selecting predicates for implications in programanalysis, March 16, 2002. Draft. http://pag.lcs.mit.edu/~mernst/pubs/invariants-implications.ps.

30. Nii Dodoo, Lee Lin, and Michael D. Ernst. Selecting,refining, and evaluating predicates for program analysis.Technical Report MIT-LCS-TR-914, Massachusetts In-stitute of Technology, Laboratory for Computer Science,Cambridge, MA, July 21, 2003.

31. Michael D. Ernst. Dynamically Discovering Likely Pro-gram Invariants. PhD thesis, University of WashingtonDepartment of Computer Science and Engineering, Seat-tle, Washington, August 2000.

32. Michael D. Ernst, Jake Cockrell, William G. Griswold,and David Notkin. Dynamically discovering likely pro-gram invariants to support program evolution. IEEETransactions on Software Engineering, 27(2):1–25, 2001.

33. Michael D. Ernst, Adam Czeisler, William G. Griswold,and David Notkin. Quickly detecting relevant programinvariants. In ICSE 2000, Proceedings of the 22nd In-ternational Conference on Software Engineering, pages449–458, 2000.

34. Cormac Flanagan, Rajeev Joshi, and K. Rustan M.Leino. Annotation inference for modular checkers. In-formation Processing Letters, 77(2–4):97–108, February2001.

35. Cormac Flanagan and K. Rustan M. Leino. Houdini, anannotation assistant for ESC/Java. In J. N. Oliveira andP. Zave, editors, FME 2001, volume LNCS 2021, pages500–517. Springer–Verlag, 2001.

36. Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge,Greg Nelson, James B. Saxe, and Raymie Stata. Ex-tended static checking for Java. In ACM SIGPLAN 2002Conference on Programming Language Design and Im-plementation (PLDI’2002), pages 234–245, 2002.

37. Cormac Flanagan and James B. Saxe. Avoiding expo-nential explosion: Generating compact verification con-ditions. In Conference Record of the 28th Annual ACMSymposium on Principles of Programming Languages,pages 193–205. ACM, January 2001.

38. Lisa Friendly. The design of distributed hyperlinkedprogramming documentation. In S. Fraisse, F. Gar-zotto, T. Isakowitz, J. Nanard, and M. Nanard, editors,IWHD’95, pages 151–173. Springer–Verlag, 1995.

39. Alex Groce and Willem Visser. What went wrong: Ex-plaining counterexamples. In 10th International SPINWorkshop on Model Checking of Software, pages 121–135, Portland, Oregon, May 9–10, 2003.

40. Neelam Gupta and Zachary V. Heidepriem. A new struc-tural coverage criterion for dynamic detection of programinvariants. In Proceedings of the 13th Annual Interna-tional Conference on Automated Software Engineering(ASE 2003), Montreal, Canada, October 8–10, 2003.

41. John V. Guttag, James J. Horning, et al. Larch: Lan-guages and Tools for Formal Specification. Springer–Verlag, New York, NY, 1993.

42. Ali Hamie. Translating the Object Constraint Languageinto the Java Modeling Language. In Symposium onApplied Computing. Proceedings of the 2004 ACM sym-posium on applied computing (SAC’2004), pages 1531–1535. ACM, 2004.

43. Sudheendra Hangal and Monica S. Lam. Tracking downsoftware bugs using automatic anomaly detection. InICSE’02, Proceedings of the 24th International Confer-ence on Software Engineering, pages 291–301, Orlando,Florida, May 22–24, 2002.

44. Michael Harder, Jeff Mellen, and Michael D. Ernst.Improving test suites via operational abstraction. InICSE’03, Proceedings of the 25th International Confer-ence on Software Engineering, pages 60–71, Portland,Oregon, May 6–8, 2003.

45. Johannes Henkel and Amer Diwan. Discovering alge-braic specifications from Java classes. In ECOOP 2003— Object-Oriented Programming, 15th European Con-ference, Darmstadt, Germany, July 23–25, 2003.

46. B. Jacobs, J. Kiniry, and M. Warnier. Java programverification challenges. In FMCO 2002, volume 2852 ofLNCS, pages 202–219. Springer–Verlag, 2003.

47. Bart Jacobs. Weakest precondition reasoning for Javaprograms with JML annotations. Journal of Logic andAlgebraic Programming, 58(1-2):61–88, 2004.

48. Bart Jacobs, Martijn Oostdijk, and Martijn Warnier.Source Code Verification of a Secure Payment Applet.Journ. of Logic and Algebraic Programming, 58(1-2):107–120, 2004.

49. Bart Jacobs and Erik Poll. A logic for the Java Model-ing Language JML. In H. Hussmann, editor, Fundamen-tal Approaches to Software Engineering (FASE), volume2029 of LNCS, pages 284–299. Springer–Verlag, 2001.

50. Bart Jacobs and Erik Poll. Java program verificationat Nijmegen: Developments and perspective. In Inter-national Symposium on Software Security (ISSS’2003),number 3233 in LNCS, pages 134–153. Springer–Verlag,2004.

51. Bart Jacobs, Joachim van den Berg, Marieke Huisman,Martijn van Berkum, Ulrich Hensel, and Hendrik Tews.Reasoning about Java classes (preliminary report). InOOPSLA’98, volume 33(10) of ACM SIGPLAN Notices,pages 329–340. ACM, October 1998.

52. Cliff B. Jones. Systematic Software Development UsingVDM. International Series in Computer Science. PrenticeHall, Englewood Cliffs, N.J., second edition, 1990.

53. Yoshio Kataoka, Michael D. Ernst, William G. Griswold,and David Notkin. Automated support for programrefactoring using invariants. In ICSM 2001, Proceed-ings of the International Conference on Software Main-tenance, pages 736–743, Florence, Italy, November 6–10,2001.

54. Joseph R. Kiniry and David R. Cok. ESC/Java2: UnitingESC/Java and JML: Progress and issues in building andusing ESC/Java2 and a report on a case study involvingthe use of ESC/Java2 to verify portions of an internetvoting tally system. In Construction and Analysis ofSafe, Secure and Interoperable Smart devices (CASSIS),LNCS. Springer–Verlag, 2004. To appear.

55. Reto Kramer. iContract – the Java design by con-tract tool. TOOLS 26: Technology of Object-OrientedLanguages and Systems, Los Alamitos, California, pages295–307, 1998.


56. Gary T. Leavens. An overview of Larch/C++: Behav-ioral specifications for C++ modules. In Haim Kilovand William Harvey, editors, Specification of Behav-ioral Semantics in Object-Oriented Information Model-ing, chapter 8, pages 121–142. Kluwer Academic Publish-ers, Boston, 1996. An extended version is TR #96-01d,Department of Computer Science, Iowa State University,Ames, Iowa, 50011.

57. Gary T. Leavens, Albert L. Baker, and Clyde Ruby.JML: A notation for detailed design. In Haim Kilov,Bernhard Rumpe, and Ian Simmonds, editors, Behav-ioral Specifications of Businesses and Systems, pages175–188. Kluwer Academic Publishers, Boston, 1999.

58. Gary T. Leavens, Albert L. Baker, and Clyde Ruby. Pre-liminary design of JML: A behavioral interface specifica-tion language for Java. Technical Report 98-06u, IowaState University, Department of Computer Science, April2003.

59. Gary T. Leavens, Yoonsik Cheon, Curtis Clifton, ClydeRuby, and David R. Cok. How the design of JML accom-modates both runtime assertion checking and formal ver-ification. In FMCO 2002, volume 2852 of LNCS, pages262–284. Springer–Verlag, 2003. Also appears as tech-nical report TR03-04, Dept. of Computer Science, IowaState University.

60. K. Rustan M. Leino. Extended static checking: Aten-year perspective. In Reinhard Wilhelm, editor,Informatics—10 Years Back, 10 Years Ahead, volume2000 of LNCS. Springer–Verlag, 2000.

61. K. Rustan M. Leino. Efficient weakest preconditions.Technical Report MSR-TR-2004-34, Microsoft Research,April 2004.

62. K. Rustan M. Leino, Todd Millstein, and James B. Saxe.Generating error traces from verification-condition coun-terexamples. Science of Computer Programming, 2004.To appear.

63. K. Rustan M. Leino and Peter Mller. Object invariants indynamic contexts. In ECOOP 2004 — Object-OrientedProgramming, 18th European Conference, pages 491–516,Olso, Norway, June 16–18, 2004.

64. K. Rustan M. Leino, Greg Nelson, and James B. Saxe.ESC/Java user’s manual. Technical Note 2000-002, Com-paq SRC, October 2000.

65. K. Rustan M. Leino, James B. Saxe, and Raymie Stata.Checking Java programs via guarded commands. Tech-nical Note 1999-002, Compaq SRC, May 1999.

66. Ben Liblit, Alex Aiken, Alice X. Zheng, and Michael I.Jordan. Bug isolation via remote program sampling.In Proceedings of the ACM SIGPLAN 2003 Conferenceon Programming Language Design and Implementation,pages 141–154, San Diego, CA, June 9–11, 2003.

67. Lee Lin and Michael D. Ernst. Improving adaptabilityvia program steering. In ISSTA 2004, Proceedings of the2004 International Symposium on Software Testing andAnalysis, Boston, MA, USA, July 12–14, 2004.

68. Barbara Liskov and Jeannette Wing. A behavioral notionof subtyping. ACM Transactions on Programming Lan-guages and Systems, 16(6):1811–1841, November 1994.

69. C. Marche, C. Paulin-Mohring, and X. Urbain. TheKrakatoa tool for certification of Java/JavaCard pro-grams annotated in JML. Journal of Logic and AlgebraicProgramming, 58(1-2):89–106, 2004.

70. Leonardo Mariani and Mauro Pezze. A technique for ver-ifying component-based software. In International Work-shop on Test and Analysis of Component Based Systems,Barcelona, Spain, March 27–28, 2004.

71. Stephen McCamant and Michael D. Ernst. Predictingproblems caused by component upgrades. In Proceedingsof the 10th European Software Engineering Conferenceand the 11th ACM SIGSOFT Symposium on the Founda-tions of Software Engineering, pages 287–296, Helsinki,Finland, September 3–5, 2003.

72. Stephen McCamant and Michael D. Ernst. Early identifi-cation of incompatibilities in multi-component upgrades.In ECOOP 2004 — Object-Oriented Programming, 18thEuropean Conference, Olso, Norway, June 16–18, 2004.

73. Bertrand Meyer. Object-oriented Software Construction.Prentice Hall, New York, NY, second edition, 1997.

74. J. Meyer and A. Poetzsch-Heffter. An architecturefor interactive program provers. In S. Graf andM. Schwartzbach, editors, TACAS’00, number 1785 inLNCS, pages 63–77. Springer, Berlin, 2000.

75. Carroll Morgan. Programming from Specifications: Sec-ond Edition. Prentice Hall International, Hempstead,UK, 1994.

76. P. Muller, A. Poetzsch-Heffter, and G.T. Leavens. Modu-lar specification of frame properties in JML. Concurrencyand Computation: Practice and Experience, 15(2):117–154, 2003.

77. Peter Muller, Arnd Poetzsch-Heffter, and Gary T. Leav-ens. Modular invariants for object structures. TechnicalReport 424, ETH Zurich, October 2003.

78. Toh Ne Win and Michael D. Ernst. Verifying distributedalgorithms via dynamic analysis and theorem proving.Technical Report 841, Massachusetts Institute of Tech-nology, Laboratory for Computer Science, Cambridge,MA, May 25, 2002.

79. Toh Ne Win, Michael D. Ernst, Stephen J. Garland, Dil-sun Kırlı, and Nancy Lynch. Using simulated executionin verifying distributed algorithms. Software Tools forTechnology Transfer, 6(1):67–76, 2004.

80. Jeremy W. Nimmer and Michael D. Ernst. Automaticgeneration of program specifications. In ISSTA 2002,International Symposium on Software Testing and Anal-ysis, pages 232–242, Rome, Italy, 2002.

81. Jeremy W. Nimmer and Michael D. Ernst. Invariantinference for static checking: An empirical evaluation. InACM SIGSOFT 10th International Symposium on theFoundations of Software Engineering (FSE 2002), pages11–20, 2002.

82. S. Owre, S. Rajan, J.M. Rushby, N. Shankar, and M. Sri-vas. PVS: Combining specification, proof checking, andmodel checking. In R. Alur and T.A. Henzinger, editors,Computer Aided Verification, number 1102 in LNCS,pages 411–414. Springer–Verlag, 1996.

83. Jeff H. Perkins and Michael D. Ernst. Efficient incremen-tal algorithms for dynamic detection of likely invariants.In ACM SIGSOFT 12th International Symposium on theFoundations of Software Engineering (FSE 2004), New-port Beach, CA, USA, November 2004.

84. Dennis K. Peters and David Lorge Parnas. Using testoracles generated from program documentation. IEEETransactions on Software Engineering, 24(3):161–173,1998.


85. Erik Poll, Pieter Hartel, and Eduard de Jong. A Javareference model of transacted memory for smart cards.In Smart Card Research and Advanced Application Con-ference (CARDIS’2002), pages 75–86. USENIX, 2002.

86. Erik Poll, Joachim van den Berg, and Bart Jacobs. For-mal specification of the Java Card API in JML: theAPDU class. Computer Networks, 36(4):407–421, 2001.

87. Brock Pytlik, Manos Renieris, Shriram Krishnamurthi,and Steven P. Reiss. Automated fault localization usingpotential invariants. In AADEBUG’2003, Fifth Interna-tional Workshop on Automated and Algorithmic Debug-ging, Ghent, Belgium, September8–10, 2003.

88. Arun D. Raghavan. Design of a JML documentationgenerator. Technical Report 00-12, Iowa State Univer-sity, Department of Computer Science, July 2000.

89. Orna Raz, Philip Koopman, and Mary Shaw. Se-mantic anomaly detection in online data sources. InICSE’02, Proceedings of the 24th International Confer-ence on Software Engineering, pages 302–312, Orlando,Florida, May 22–24, 2002.

90. Jim Rumbaugh, Ivar Jacobson, and Grady Booch. TheUnified Modeling Language Reference Manual. Addison-Wesley Publishing Company, 1998.

91. Jos Warmer and Anneke Kleppe. The Object ConstraintLanguage: Precise Modeling with UML. Addison-WesleyPublishing Company, 1999.

92. Tao Xie and David Notkin. Checking inside the blackbox: Regression fault exposure and localization based onvalue spectra differences. Technical Report UW-CSE-02-12-04, University of Washington Department of Com-puter Science and Engineering, Seattle, WA, USA, De-cember 2002.

93. Tao Xie and David Notkin. Tool-assisted unit test selec-tion based on operational violations. In Proceedings ofthe 13th Annual International Conference on AutomatedSoftware Engineering (ASE 2003), Montreal, Canada,October 8–10, 2003.

an overview of jml tools and applicationsleavens/jml/oldreleases/sttt04.pdf · this article appears...

Documents