spec# k. rustan m. leino senior researcher programming languages and methods microsoft research,...
TRANSCRIPT
Spec#Spec#
K. Rustan M. LeinoSenior ResearcherProgramming Languages and MethodsMicrosoft Research, Redmond, WA, USA
K. Rustan M. LeinoSenior ResearcherProgramming Languages and MethodsMicrosoft Research, Redmond, WA, USA
Microsoft Research faculty summit, Redmond, WA, USA, 17 July 2006
joint work withMike Barnett, Robert DeLine, Manuel Fähndrich, Wolfram Schulte, Herman Venter,
and internsBor-Yuh Evan Chang, Ádám Darvas, Bart Jacobs, Daan Leijen, Angela Wallenburg,
and visiting researchersFrancesco Logozzo, Peter Müller, David A. Naumann, Arnd Poetzsch-Heffter
Software engineering problemSoftware engineering problem
Building and maintaining large systems that are correctBuilding and maintaining large systems that are correct
ApproachApproach
Specifications record design decisionsbridge intent and code
Tools amplify human effortmanage details
find inconsistencies
ensure quality
Specifications record design decisionsbridge intent and code
Tools amplify human effortmanage details
find inconsistencies
ensure quality
Research goalsResearch goals
Build the best such system we can build today
Experiment with the system to get a feel for what it is like to use
Advance the state of the art
Build the best such system we can build today
Experiment with the system to get a feel for what it is like to use
Advance the state of the art
Spec#Spec#Experimental mix of contracts and tool support
Aimed at experienced developers who know the high cost of testing and maintenance
Superset of C#non-null types
pre- and postconditions
object invariants
Tool supportmore type checking
compiler-emitted run-time checks
static program verification
Experimental mix of contracts and tool support
Aimed at experienced developers who know the high cost of testing and maintenance
Superset of C#non-null types
pre- and postconditions
object invariants
Tool supportmore type checking
compiler-emitted run-time checks
static program verification
C#contracts
everywhere
type checking
static verification
into the future
run-time checks
degree of checking,effort
familiar
Spec# demo: ChunkerSpec# demo: Chunker
Some design issuesSome design issues
0. Non-null types
1. C# compatibility
2. Preconditions
3. Object invariants
4. Program verifier architecture
5. Verification-condition generation
0. Non-null types
1. C# compatibility
2. Preconditions
3. Object invariants
4. Program verifier architecture
5. Verification-condition generation
T x; The value of x is null ora reference to an object whose type is a subtype of T.
T ! y; The value of y isa reference to an object whose type is a subtype of T,not null.
T x; The value of x is null ora reference to an object whose type is a subtype of T.
T ! y; The value of y isa reference to an object whose type is a subtype of T,not null.
0. Non-null types0. Non-null types
Non-null escape hatch: castNon-null escape hatch: cast
object o;string s;
…
string! a = (string!)o;
string! b = (!)s;
object o;string s;
…
string! a = (string!)o;
string! b = (!)s;
Comparing against nullComparing against null
public void M( T x ) {
if (x == null) {…
} else {int y = ((!)x).f;…
}}
public void M( T x ) {
if (x == null) {…
} else {int y = ((!)x).f;…
}}
Comparing against nullComparing against null
public void M( T x ) {
if (x == null) {…
} else {int y = x.f;…
}}
public void M( T x ) {
if (x == null) {…
} else {int y = x.f;…
}} Spec# performs a data-flow analysis to
allow this (similar to definite assignment)
Non-null instance fieldsNon-null instance fields
class C : B {T ! x;public C(T ! y): base(){
this.x = y;}public override int M() { return x.f; }
}
class C : B {T ! x;public C(T ! y): base(){
this.x = y;}public override int M() { return x.f; }
}
Is this code type safe?No!
abstract class B {public B() { this.M(); }
public abstract int M();
}
null dereferencenull dereference
Non-null instance fieldsNon-null instance fields
class C : B {T ! x;public C(T ! y){
this.x = y;base();
}public override int M() { return x.f; }
}
class C : B {T ! x;public C(T ! y){
this.x = y;base();
}public override int M() { return x.f; }
}
Spec# allows x to beassigned before baseconstructor is called.
Other non-null issuesOther non-null issues
Comparing a field against nullif (this.f != null) {
// …this.f.M(…);
}
Static fieldsstatic T g = new T();
ArraysT![ ] a = new T![100];
GenericsList<T!> myList = new List<T!>();
Comparing a field against nullif (this.f != null) {
// …this.f.M(…);
}
Static fieldsstatic T g = new T();
ArraysT![ ] a = new T![100];
GenericsList<T!> myList = new List<T!>();
Spec# is superset of C#
From C# to Spec#:accept every C# program
compile it to have the same behavior
Consequences“Possible null dereference” is just a warning
“Must initialize non-null fields before calling base constructor” is an error
Support for out-of-band contracts
Spec# is superset of C#
From C# to Spec#:accept every C# program
compile it to have the same behavior
Consequences“Possible null dereference” is just a warning
“Must initialize non-null fields before calling base constructor” is an error
Support for out-of-band contracts
1. C# compatibility1. C# compatibility
From Spec# to C#or: Leveraging wiz-bang features of Visual Studio 2005From Spec# to C#or: Leveraging wiz-bang features of Visual Studio 2005
class B : A {string! src;public B(string! source, int x)
requires 0 <= x;{
this.src = source;base(x);
}
class B : A {string! src;public B(string! source, int x)
requires 0 <= x;{
this.src = source;base(x);
}
From Spec# to C#or: Leveraging wiz-bang features of Visual Studio 2005From Spec# to C#or: Leveraging wiz-bang features of Visual Studio 2005
class B : A {string! src;public B(string! source, int x)
//^ requires 0 <= x;{
this.src = source;base(x);
}
class B : A {string! src;public B(string! source, int x)
//^ requires 0 <= x;{
this.src = source;base(x);
}
From Spec# to C#or: Leveraging wiz-bang features of Visual Studio 2005From Spec# to C#or: Leveraging wiz-bang features of Visual Studio 2005
class B : A {string/*!*/ src;public B(string/*!*/ source, int x)
//^ requires 0 <= x;{
this.src = source;base(x);
}
class B : A {string/*!*/ src;public B(string/*!*/ source, int x)
//^ requires 0 <= x;{
this.src = source;base(x);
}
From Spec# to C#or: Leveraging wiz-bang features of Visual Studio 2005From Spec# to C#or: Leveraging wiz-bang features of Visual Studio 2005
class B : A {string/*!*/ src;public B(string/*!*/ source, int x)
//^ requires 0 <= x; : base(x){
this.src = source;//^ base;
}
class B : A {string/*!*/ src;public B(string/*!*/ source, int x)
//^ requires 0 <= x; : base(x){
this.src = source;//^ base;
}
2. Preconditions2. Preconditions
StringBuilder.Append Method (Char[ ], Int32, Int32)Appends the string representation of a specified subarray of Unicode characters to the end of this instance.
public StringBuilder Append(char[] value, int startIndex, int charCount);
Parameters
valueA character array.
startIndexThe starting position in value.
charCountThe number of characters append.
Return Value
A reference to this instance after the append operation has occurred.
Exceptions
Contracts todayContracts today
Exception Type Condition
ArgumentNullException value is a null reference, and startIndex and charCount are not zero.
ArgumentOutOfRangeException charCount is less than zero.
-or-
startIndex is less than zero.
-or-
startIndex + charCount is less than the length of value.
Contract in Spec#Contract in Spec#
public StringBuilder Append(char[ ] value, int startIndex,int charCount );
requires value == null ==> startIndex == 0 && charCount == 0;
requires 0 <= startIndex;
requires 0 <= charCount;
requires value == null || startIndex + charCount <= value.Length;
public StringBuilder Append(char[ ] value, int startIndex,int charCount );
requires value == null ==> startIndex == 0 && charCount == 0;
requires 0 <= startIndex;
requires 0 <= charCount;
requires value == null || startIndex + charCount <= value.Length;
Exception Type Condition
ArgumentNullException value is a null reference, and startIndex and charCount are not zero.
ArgumentOutOfRangeException charCount is less than zero.
-or-
startIndex is less than zero.
-or-
startIndex + charCount is less than the length of value.
Otherwise clausesOtherwise clauses
public StringBuilder Append(char[ ] value, int startIndex,int charCount );
requires value == null ==> startIndex == 0 && charCount == 0otherwise ArgumentNullException;
requires 0 <= startIndexotherwise ArgumentOutOfRangeException;
…
public StringBuilder Append(char[ ] value, int startIndex,int charCount );
requires value == null ==> startIndex == 0 && charCount == 0otherwise ArgumentNullException;
requires 0 <= startIndexotherwise ArgumentOutOfRangeException;
…
Exception Type Condition
ArgumentNullException value is a null reference, and startIndex and charCount are not zero.
ArgumentOutOfRangeException charCount is less than zero.
-or-
startIndex is less than zero.
-or-
startIndex + charCount is less than the length of value.
Inheriting contractsInheriting contracts
interface J {void M(int x); requires P;
}
class A {public abstract void M(int x); requires Q;
}
class B : A, J {public override void M(int x){ … }
}
interface J {void M(int x); requires P;
}
class A {public abstract void M(int x); requires Q;
}
class B : A, J {public override void M(int x){ … }
}
3. Object invariants3. Object invariants
When do object invariants hold?When do object invariants hold?
class C {
private int x;private int y;
invariant x < y;
public C() { x = 0; y = 1; }
public void M(){
int t = 100 / (y – x);x = x + 1;P(t);y = y + 1;
}
…
}
class C {
private int x;private int y;
invariant x < y;
public C() { x = 0; y = 1; }
public void M(){
int t = 100 / (y – x);x = x + 1;P(t);y = y + 1;
}
…
}
invariant assumed to holdon entry to method
invariant checked to holdon exit from method
invariant checked to holdat end of constructor
invariant may betemporarily broken here
invariant is restored here
what if P calls back into M?
Object statesObject states
MutableObject invariant may not hold
Field updates allowed
ValidObject invariant holds
Field updates not allowed
MutableObject invariant may not hold
Field updates allowed
ValidObject invariant holds
Field updates not allowed
Valid vs. mutable objectsValid vs. mutable objectsclass C {
private int x;private int y;invariant x < y;
public void M()requires this.inv == Valid;
{expose (this) {
int t = 100 / (y – x);x = x + 1;P(t);y = y + 1;
}}…
}
class C {private int x;private int y;invariant x < y;
public void M()requires this.inv == Valid;
{expose (this) {
int t = 100 / (y – x);x = x + 1;P(t);y = y + 1;
}}…
}
represent explicitlythat invariant holds(without revealing
what the invariant is)
change this.invfrom Valid to Mutable
check invariant;then, change this.invfrom Mutable to Valid
field updates allowedonly on Mutable objects
Summary of object invariantsSummary of object invariants
invariant …
inv : { Mutable, Valid }
expose
updates of o.f require o.inv = Mutable
(o ･ o.inv = Mutable Inv (o))
invariant …
inv : { Mutable, Valid }
expose
updates of o.f require o.inv = Mutable
(o ･ o.inv = Mutable Inv (o))
4. Spec# verifier architecture4. Spec# verifier architecture
V.C. generator
automatictheorem prover
verification condition
Spec#
“correct” or list of errors
Spec# compiler
MSIL (“bytecode”)
bytecode translator
Boogie PL
inference engine
Spec# program verifier (aka Boogie)
BoogiePLBoogiePL
Intermediate languagetheory part
imperative part
Semantics of Spec# is encoded in BoogiePL
Can be used for other program-verification tasks, like verifying other source languages
Intermediate languagetheory part
imperative part
Semantics of Spec# is encoded in BoogiePL
Can be used for other program-verification tasks, like verifying other source languages
Example BoogiePL (0)Example BoogiePL (0)var $Heap: [ref,name]any where IsHeap($Heap);function IsHeap(h: [ref,name]any) returns (bool);
const Chunker: name;axiom Chunker <: System.Object;
const Chunker.n: name;function DeclType(field: name) returns (class: name);axiom DeclType(Chunker.n) = Chunker;
const $allocated: name;axiom ( h: [ref,name]any, o: ref, f: name •
IsHeap(h) h[o, $allocated] h[h[o, f],$allocated]);
const $inv: name;axiom ( $oi: ref, $h: [ref,name]any •
IsHeap($h) $h[$oi, $inv] <: Chunker0 < $h[$oi, Chunker.ChunkSize] 0 ≤ $h[$oi, Chunker.n] $h[$oi, Chunker.n] ≤ $Length($h[$oi, Chunker.src]));
var $Heap: [ref,name]any where IsHeap($Heap);function IsHeap(h: [ref,name]any) returns (bool);
const Chunker: name;axiom Chunker <: System.Object;
const Chunker.n: name;function DeclType(field: name) returns (class: name);axiom DeclType(Chunker.n) = Chunker;
const $allocated: name;axiom ( h: [ref,name]any, o: ref, f: name •
IsHeap(h) h[o, $allocated] h[h[o, f],$allocated]);
const $inv: name;axiom ( $oi: ref, $h: [ref,name]any •
IsHeap($h) $h[$oi, $inv] <: Chunker0 < $h[$oi, Chunker.ChunkSize] 0 ≤ $h[$oi, Chunker.n] $h[$oi, Chunker.n] ≤ $Length($h[$oi, Chunker.src]));
Example BoogiePL (1)Example BoogiePL (1)procedure Chunker.NextChunk(this: ref) returns ($result: ref); requires $Heap[this, $inv] = Chunker; requires $Heap[this, $ownerFrame] = $PeerGroupPlaceholder
¬($Heap[$Heap[this, $ownerRef], $inv] <: $Heap[this, $ownerFrame]); free requires $Heap[this, $allocated] = true $IsNotNull(this, Chunker); modifies $Heap; ensures $Length($result) ≤ $Heap[this, Chunker.ChunkSize]; ensures ( $pc: ref • $pc ≠ null $Heap[$pc, $allocated] = true
$Heap[$pc, $ownerRef] = $Heap[$result, $ownerRef] $Heap[$pc, $ownerFrame] = $Heap[$result, $ownerFrame]($Heap[$pc, $ownerFrame] = $PeerGroupPlaceholder
¬($Heap[$Heap[$pc, $ownerRef], $inv] <: $Heap[$pc, $ownerFrame])) $Heap[$pc, $inv] = $typeof($pc));
free ensures $Heap[$result, $allocated] = true $IsNotNull($result, System.String); free ensures ( $o: ref • $o ≠ null ¬old($Heap)[$o, $allocated] $Heap[$o, $allocated]
$Heap[$o, $inv] = $typeof($o)); ensures ( $o: ref • $o ≠ null old($Heap)[$o, $allocated] = true
old($Heap)[$Heap[$o, $ownerRef], $allocated] = trueold($Heap)[$o, $ownerRef] = $Heap[$o, $ownerRef] old($Heap)[$o, $ownerFrame] = $Heap[$o, $ownerFrame]);
free ensures ( $o: ref, $f: name • $f ≠ $inv $o ≠ null old($Heap)[$o, $allocated] = true (old($Heap)[$o, $ownerFrame] = $PeerGroupPlaceholder
¬(old($Heap)[old($Heap)[$o, $ownerRef], $inv] <: old($Heap)[$o, $ownerFrame])) (¬IsStaticField($f) ¬IsDirectlyModifiableField($f)) old($Heap)[$o, $f] = $Heap[$o, $f]);
free ensures ( $o: ref • old($Heap)[$o, $inv] = $Heap[$o, $inv] old($Heap)[$o, $allocated] ≠ true);
free ensures ( $o: ref • old($Heap)[$o, $allocated] $Heap[$o, $allocated]);
procedure Chunker.NextChunk(this: ref) returns ($result: ref); requires $Heap[this, $inv] = Chunker; requires $Heap[this, $ownerFrame] = $PeerGroupPlaceholder
¬($Heap[$Heap[this, $ownerRef], $inv] <: $Heap[this, $ownerFrame]); free requires $Heap[this, $allocated] = true $IsNotNull(this, Chunker); modifies $Heap; ensures $Length($result) ≤ $Heap[this, Chunker.ChunkSize]; ensures ( $pc: ref • $pc ≠ null $Heap[$pc, $allocated] = true
$Heap[$pc, $ownerRef] = $Heap[$result, $ownerRef] $Heap[$pc, $ownerFrame] = $Heap[$result, $ownerFrame]($Heap[$pc, $ownerFrame] = $PeerGroupPlaceholder
¬($Heap[$Heap[$pc, $ownerRef], $inv] <: $Heap[$pc, $ownerFrame])) $Heap[$pc, $inv] = $typeof($pc));
free ensures $Heap[$result, $allocated] = true $IsNotNull($result, System.String); free ensures ( $o: ref • $o ≠ null ¬old($Heap)[$o, $allocated] $Heap[$o, $allocated]
$Heap[$o, $inv] = $typeof($o)); ensures ( $o: ref • $o ≠ null old($Heap)[$o, $allocated] = true
old($Heap)[$Heap[$o, $ownerRef], $allocated] = trueold($Heap)[$o, $ownerRef] = $Heap[$o, $ownerRef] old($Heap)[$o, $ownerFrame] = $Heap[$o, $ownerFrame]);
free ensures ( $o: ref, $f: name • $f ≠ $inv $o ≠ null old($Heap)[$o, $allocated] = true (old($Heap)[$o, $ownerFrame] = $PeerGroupPlaceholder
¬(old($Heap)[old($Heap)[$o, $ownerRef], $inv] <: old($Heap)[$o, $ownerFrame])) (¬IsStaticField($f) ¬IsDirectlyModifiableField($f)) old($Heap)[$o, $f] = $Heap[$o, $f]);
free ensures ( $o: ref • old($Heap)[$o, $inv] = $Heap[$o, $inv] old($Heap)[$o, $allocated] ≠ true);
free ensures ( $o: ref • old($Heap)[$o, $allocated] $Heap[$o, $allocated]);
Example BoogiePL (2)Example BoogiePL (2)implementation Chunker.NextChunk(this: ref) returns ($result: ref){ var temp0: ref, local4: ref, stack0i: int, stack1i: int, stack1o: ref,
stack0b: bool, stack0o: ref, stack2i: int, s: ref, return.value: ref,SS$Display.Return.Local: ref;
entry: // … // ----- load field ----- Chunker.ssc(14,7) assert this ≠ null; stack1o := $Heap[this, Chunker.src]; // … // ----- binary operator ----- Chunker.ssc(14,7) stack0b := stack0i > stack1i; // ----- branch ----- Chunker.ssc(14,7) goto true5814to5848, false5814to5831;
true5814to5848: assume stack0b = true; …
implementation Chunker.NextChunk(this: ref) returns ($result: ref){ var temp0: ref, local4: ref, stack0i: int, stack1i: int, stack1o: ref,
stack0b: bool, stack0o: ref, stack2i: int, s: ref, return.value: ref,SS$Display.Return.Local: ref;
entry: // … // ----- load field ----- Chunker.ssc(14,7) assert this ≠ null; stack1o := $Heap[this, Chunker.src]; // … // ----- binary operator ----- Chunker.ssc(14,7) stack0b := stack0i > stack1i; // ----- branch ----- Chunker.ssc(14,7) goto true5814to5848, false5814to5831;
true5814to5848: assume stack0b = true; …
Example BoogiePL (3)Example BoogiePL (3)
// ----- call ----- Chunker.ssc(17,9) assert stack0o ≠ null; call s := System.String.Substring$System.Int32(stack0o, stack1i); // … // ----- store field ----- Chunker.ssc(19,7) assert this ≠ null; assert ¬($Heap[this, $inv] <: Chunker); $Heap[this, Chunker.n] := stack0i; // … // ----- return $result := stack0o; return;}
// ----- call ----- Chunker.ssc(17,9) assert stack0o ≠ null; call s := System.String.Substring$System.Int32(stack0o, stack1i); // … // ----- store field ----- Chunker.ssc(19,7) assert this ≠ null; assert ¬($Heap[this, $inv] <: Chunker); $Heap[this, Chunker.n] := stack0i; // … // ----- return $result := stack0o; return;}
5. Verification conditions5. Verification conditions
Automatic theorem provercan be hidden from programmer
generates counterexamples
Interactive theorem proverrequires gurus
not limited by built-in decision procedures
Automatic theorem provercan be hidden from programmer
generates counterexamples
Interactive theorem proverrequires gurus
not limited by built-in decision procedures
Performance considerationsPerformance considerations
Generate verification conditions that the theorem prover can handle quickly
“Efficient” encodings of axioms
“Efficient” weakest preconditions
Generate verification conditions that the theorem prover can handle quickly
“Efficient” encodings of axioms
“Efficient” weakest preconditions
Initial verifier experienceInitial verifier experience
Pilot production project“it changes how you think”
Several smaller (300-1500 lines) case studies
Parts of Spec# program verifier
External academic use
Pilot production project“it changes how you think”
Several smaller (300-1500 lines) case studies
Parts of Spec# program verifier
External academic use
download Spec#from here
ConclusionsConclusionsBecause of tool support, we’re ready for programming at the next level of rigorCurrent work
Specification/programming/verification methodologyPerformanceTechnology transferEngineering effort
Technology sharingTeachingCase studiesBoogiePL as common intermediate logic
Because of tool support, we’re ready for programming at the next level of rigorCurrent work
Specification/programming/verification methodologyPerformanceTechnology transferEngineering effort
Technology sharingTeachingCase studiesBoogiePL as common intermediate logic
http://research.microsoft.com/~leino
http://research.microsoft.com/specsharp