http://research.microsoft.com/contracts. public int abs(int x) { if (x < 0) return -x; else...
Post on 01-Apr-2015
244 Views
Preview:
TRANSCRIPT
Practical verification with abstract interpretationFrancesco Logozzo
Joint work with Manuel Fahndrich
http://research.microsoft.com/contracts
Exercise: Specify Abs(int x)
Precondition?Postcondition?
public int Abs(int x) { if (x < 0) return -x; else return x; }
Little reminder:
-(-231)== -231
Specifications via Contracts
PreconditionWhat I expect from the caller?e.g. A non-null parameter
PostconditionWhat I ensure to the caller?e.g. The returned value is non-negative
Object InvariantWhat holds in the stable states of an object?e.g. This field is non-null
Contracts
Not a new idea…Eiffel, JML, Spec# …
General consensus on their usefulnessEven in dynamic languages communities!
However, not mainstream (yet). Why???Two main problems
Require changes to the build environmentNew compiler/language/ …
Static checking either absent or to painful to use
Over-specification
CodeContracts
Idea: Use code to specify code
public int Abs(int x) { Contract.Requires(x != Int32.MinValue); Contract.Ensures(Contract.Result<int>() >= 0);
if (x < 0) return -x; else return x; }
CodeContracts
Pragmatic solution to the two problemsThe Contract Language is a .NET Library
No changes in/of the compilerTransparently use C#, VB, F#, Delphi …
Leverage IDE supportIntellisense, type checking …
The static checker Abs. Interpretation based
Infer loop invariantsFine tuning of the algorithmsFocuses on the properties of interestPredictable!!!!
Let’s demo!
CCCheck (aka Clousot)
Goal: Prove contracts at static timeAbstract interpretation-based
Loop invariants inferenceTunable
Focuses on properties of interestDifferent from usual WP-based provers
Optimistic hypotheses for aliasingConservative otherwise
Clousot main loop
For each assembly, class, method 1. Collect the proof obligations• What should I prove?
2. Run the analyses• Discover facts about the program
3. Discharge the proof obligations• Using the inferred facts
4. On failure, use a more refined analysis5. Otherwise, report warning
Proof obligationsImplicit
NonNull checkingBounds checkingDivisions by zero, overflows, float comparisons…
Explicit AssertionsWhen calling a method, its preconditionWhen returning from a method
its postconditionits object invariant
Analysis steps
1. Read the bytecode, extract contracts2. Program transformations:
1. De-Stack2. CFG Construction3. De-heap4. Expression recovery
3. Value Analyses• Non-null, numerical, containers, buffers
…4. Checking5. Inference propagation
Why Analyzing Bytecode?
Stable, standard formatLanguages change, bytecode does notC# 2.0 -> C# 3.0 -> C# 4.0
Analyze one instead of manyC#, VB, Managed C++, F#, Delphi, Oxygen …
Leverage the compiler workType inference, generics …
Main Drawback: Structure lostShould Reconstruct it!
Base Analysis Stack
Expression Reconstruction
Heap Analysis
Destack
Heap abstraction public class Alias { int x;
public void Foo(bool b) { Contract.Ensures(tmp.x >= -20);
Alias tmp = new Alias(); tmp.x = -11;
Alias alias = tmp;
if(b) { alias.x = 10; }
} }
public class Alias { public void Foo(bool b) { int svX = -11; assume (b) { svX = 10; } assert (svX >= -20); } }
Output: program in scalar formOptimistic assumptions on external aliasing
Value Analyses
Built on the top of the stackNon-Null
Is this object valid?Numerical
Ranges and linear restraints over variables
ArraysCompound structure values
EnumsWhich enum values are legal?
Intervals
public class Alias { public void Foo(int f, int max) { int x = 0;
while (x < max) { x++; }
Contract.Assert(x >= -20); } }
Infer x ∈[0, +oo]
Check: Ok!
No overflow!
DisIntervals
Disjunctions of intervalspublic enum ItalianBikeBrand { DeRosa=0, Colnago=2, Pinarello=4, Daccordi=6 }
public string CityFor(ItalianBikeBrand bike){ switch(bike) { case ItalianBikeBrand.DeRosa: return "Milan"; case ItalianBikeBrand.Daccordi: return "Pisa"; case ItalianBikeBrand.Pinarello: return "Treviso"; case ItalianBikeBrand.Colnago: return "Milan"; default: Contract.Assert(false); // Should prove unreachable return null; }}
DisIntervals infer[-∞,-1] [1,1] [3,3] [5,5] [7,
+∞]
Check: ⊥
Admissible values[0,0] [2,2] [4,4] [6,6]
Basic Numerical domain
Reduced product ofDisIntervals
x ∈ [a0, b0] ∨ [a1, b1] ∨… ∨ [an, bn]
LTx < { y0, y1, y2 … }
Leqx ≤ { y0, y1, y2 … }
Linear Equalitiesa0 ⋅ x0 + … + an ⋅ xn = b
Main advantage: It’s fast!Most operations are linear
Example of reduction
public static void F(){ int x = 5, y = 100;
while (x >= 0) { x = x - 1; y = y + 10; } Contract.Assert(y == 160);}
Linear equalities10 * x + y == 150
Check: ok!
Intervals inferx ∈[-1, -1]
y ∈[100, +∞]
SubPolyhedra
Often we need the full linear inequalities
Polyhedra do not scale up (we tried them)public void Count(int[] values){ int neg = 0, pos = 0, j= 0; foreach (var x in values) { if (x < 0) { neg++; j++; } else if (x > 0) { pos++; j++; } } Contract.Assert(neg + pos == j); Contract.Assert(neg + pos <= values.Length);}
Proven by Linear equalities
Proven by SubPolyhedra
SubPolyhedra (with V. Laviron)
As expressive as PolyhedraFull linear inequalitiesNo templates!
Give up some of the inference powerIdea: ∑aixi ≤ b ⇔ ∑aixi = β ∧ β ∈ [-∞, b]
Combination of Linear Equalities and Intervals
Challenge: JoinThe point wise join is too imprecise
Scales up to hundreds of variables
Join algorithm : SubPolyhedra1. Uniform slack variables2. Reduce the states3. Do the pair-wise join4. Recover precision using hints• Deleted equalities • Templates• 2D Convex Hull• Annotations • …
Example : Join Step 1
Entry State:s0 : ⟨x - y == β, β ∈ [-∞, 0] ⟩s1 : ⟨T, x ∈ [0,0] ⋀ y ∈ [1,1]⟩
Step 1 (uniform slack variables) s’0 : ⟨x - y == β, β ∈ [-∞, 0] ⟩s’1 : ⟨x - y == β, x ∈ [0,0] ⋀ y ∈ [1,1]⟩
Example: Join steps 2-3
Step 2 (Reduction)s’’0 : ⟨x - y == β, β ∈ [-∞, 0]⟩
s’’1 : ⟨x - y == β, x ∈ [0,0] ⋀ y ∈ [1,1] ⋀ β ∈ [-1,-1]⟩
Step 3 (Pair-wise join)s2 : ⟨x - y == β, β ∈ [-∞, 0]⟩
Example: Join Step 4
Recover lost relations
assume x == y x = 0; y = 1
assert x<= y
⟨x - y == 0, T⟩⟨T, x ∈ [0,0] ⋀ y ∈
[1,1]⟩
⟨T, T⟩⟨x - y == β, β ∈ [-1, 0]⟩
Critical operation: Reduction
Infer tightest boundsInstance of a Linear programming problem
Solution in polynomial timeBut may still be too expensive
We have implementedSimplex
Theoretically completeRounding problems
Basis explorationIncompleteNo rounding problems
Precis
ion/ Cost
SubPolyhedra: a family of domains
Hints for Join/Widening
Reduct
ion a
lgori
thm
,
Basis exploration
Simplex with floats
Exact Simplex
….
No Hint
Die-Hard
Semantic hints
2D Convex hull
….
Incremental analysis in Clousot
First analyze with “cheap” domainsIf check is definitive (True, False, Bottom)
Done!Otherwise
Try a more precise domainOn average great performance gains
Abstract Domain
AD1
Abstract Domain
AD2
Abstract Domain
AD3
Array Content analysis
public void Init(int N){ Contract.Requires(N > 0);
int[] a = new int[N]; int i = 0;
while (i < N) { a[i] = 222; i = i + 1; }
Contract.Assert(∀ k ∈ [0, N). a[k] == 222);}
If i == 0 then a not initializedelse if i > 0 a[0] == … a[i] == 222else impossible
Challenge 1:Effective handling of disjunction
Challenge 2:Infer all the elements initialized
Needed to prove quantified factsExtensions to enumerators
Not the firsts …
Many approaches using:Human helpUnder- and over-approximationsTemplatesTheorem proversModel checking…
We tried some of them in Clousot but not practical
Many hidden hypothesesScalability is an issue
Our idea (with P&R Cousot)
Precise and very very fast!More in the POPL’11 Talk & Paper
[222, 222]0i, k
[0, 0] N
Segment bounds Uniform content abstraction
?
0 ≤ i, 0 ≤ k i == k i < N, k < N
Disjunction
Inter-method Inference
By default, infer getter/setter ensuresReduce the initial annotation burdenDo not propagate over assemblies
Suggest immediate preconditions
Talk @ VMCAI on Tuesday
public static int[] Factory(int len) { return new int[len]; }
Caching (with J.-H. Jourdan)
Idea: Hash the annotated programPersist output of the analyzer
ChallengesCaching of Metadata
Inheritance, enums, templates …
Inferred expressionsBe conservative
Calls to Enum.IsDefined(…)Semantics given via reflection
Warning scoring
For each warning, compute a Semantic score
Use info from the abstract domains≠ the syntactic algorithm of FindBugs
Have thresholds for masking warningsLow, Medium, HiFound tenth of bugs with Low
In production, well-tested code
Use scoring to sort warnings
Further …
Goal direct backward propagationTimeouts
The analysis of a method can take too much
Message suppressionWeakness of the checker? Of the analysis?
Selective VerificationStart by focusing on most core code
CodeContracts Impact
API in .NET 4.0Externally available ~14 months
>30,000 downloads, very active forum3 book chapters on CodeContractsMany dozens of blog articles
Publications, talks, lecturesPOPL, ECOOP, OOPSLA, VMCAI, APLAS, SAS, SAC, FoVeOOS …
Internal usage Integrated into CLR buildA few groups
Conclusions & Next
CCCheck externally availableBing for “CodeContracts MSDN”
Tenths of Thousands of downloads
Or try it at http://pexforfun.com/absverified
Abstract interpretation-basedAutomatic
Inference: loop invariants, pre/post/invariants
Tunable, predicatableDogfood: Run on itself at each build
Thanks!!!!
To the Chairs & VMCAI PC for inviting meTo our colleagues
M. Barnett, H. Venter & RiSETo the visitors and interns
P. & R. CousotP. Ferrara, V. Laviron, M. Peron, M. Monereau, J.-H. Jourdan
To the hundreds of users in the forumTo push us to make CCCheck better!
Next…
Re-architecture to integrate with Z3To leverage the decision procedures in Z3To share code
E-graph, etc.
To improve reasoning on implicationsNote: ≠ from WP-based provers
No blind axiomatizationClousot is still in control, uses Z3 as oracle
top related