automated program analysis with software model checking · search of the state space of the design...
TRANSCRIPT
![Page 1: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/1.jpg)
© 2016 Carnegie Mellon University
Automated Program Analysis with Software Model Checking
Arie GurfinkelSoftware Engineering InstituteCarnegie Mellon University
February, 2016
![Page 2: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/2.jpg)
2Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Static Program Analysis
Reasoning statically about behavior of a program without executing it• compile-time analysis• exhaustive, considers all possible executions under all possible environments
and inputs
The algorithmic discovery of properties of program by inspection of the source text
Manna and Pnueli, “Algorithmic Verification”
Also known as static analysis, program verification, formal methods, etc.
Automated
Analysis
Correct
Incorrect
Program
Specification
![Page 3: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/3.jpg)
3Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Turing, 1936: “undecidable”
![Page 4: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/4.jpg)
4Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Undecidability
The halting problem• does a program P terminates on input I• proved undecidable by Alan Turing in 1936• https://en.wikipedia.org/wiki/Halting_problem
Rice’s Theorem• for any non-trivial property of partial functions, no general and effective
method can decide whether an algorithm computes a partial function with that property
• in practice, this means that there is no machine that can always decide whether the language of a given Turing machine has a particular nontrivial property
• https://en.wikipedia.org/wiki/Rice%27s_theorem
![Page 5: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/5.jpg)
5Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Living with Undecidability
“Algorithms” that occasionally diverge
Limit programs that can be analyzed• finite-state, loop-free
Partial (unsound) verification• analyze only some executions up-to a fixed number of steps
Incomplete verification / Abstraction• analyze a superset of program executions
Programmer Assistance• annotations, pre-, post-conditions, inductive invariants
![Page 6: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/6.jpg)
6Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
(Temporal Logic) Model Checking
Automatic verification technique for finite state concurrent systems.
• Developed independently by Clarke and Emerson and by Queille and Sifakis in early 1980’s.
• ACM Turing Award 2007
Specifications are written in propositional temporal logic. (Pnueli 77)
• Computation Tree Logic (CTL), Linear Temporal Logic (LTL), …
Verification procedure is an intelligent exhaustive search of the state space of the design
• Statespace explosion
![Page 7: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/7.jpg)
7Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Model Checking since 1981
1981 Clarke / Emerson: CTL Model CheckingSifakis / Quielle
1982 EMC: Explicit Model CheckerClarke, Emerson, Sistla
1990 Symbolic Model CheckingBurch, Clarke, Dill, McMillan
1992 SMV: Symbolic Model VerifierMcMillan
1998 Bounded Model Checking using SATBiere, Clarke, Zhu
2000 Counterexample-guided Abstraction RefinementClarke, Grumberg, Jha, Lu, Veith
105
10100
101000
1990s: Formal Hardware Verification in Industry:Intel, IBM, Motorola, etc.
![Page 8: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/8.jpg)
8Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
1981 Clarke / Emerson: CTL Model CheckingSifakis / Quielle
1982 EMC: Explicit Model CheckerClarke, Emerson, Sistla
1990 Symbolic Model CheckingBurch, Clarke, Dill, McMillan
1992 SMV: Symbolic Model VerifierMcMillan
1998 Bounded Model Checking using SATBiere, Clarke, Zhu
2000 Counterexample-guided Abstraction RefinementClarke, Grumberg, Jha, Lu, Veith
CBMC
SLAM,MAGIC,BLAST, …
Model Checking since 1981
![Page 9: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/9.jpg)
© 2016 Carnegie Mellon University
Temporal Logic Model Checking
![Page 10: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/10.jpg)
10Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
10
Temporal Logic Model Checking
Yes/No +
Counter-example
SW/HW
Artifact
Correctness
properties
Temporal
logicFinite
Model
Model
ExtractionTranslation
Model
Checker
Correct?
Abstraction
![Page 11: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/11.jpg)
11Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Models: Kripke Structures
Conventional state machines• K = (V, S, s0, I , R)• V is a (finite) set of atomic propositions• S is a (finite) set of states• s0 ∈ S is a start state• I: S → 2V is a labelling function that maps
each state to the set of propositional variables that hold in it – That is, I(S) is a set of interpretations
specifying which propositions are true in each state
• R ⊆ S × S is a transition relation
req req,busy
busy
s0
s2
s1
s3
![Page 12: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/12.jpg)
12Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Propositional Variables
Fixed set of atomic propositions, e.g, {p, q, r}
Atomic descriptions of a system“Printer is busy”
“There are currently no requested jobs for the printer”
“Conveyer belt is stopped”
Do not involve time!
![Page 13: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/13.jpg)
13Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Modal Logic
Extends propositional logic with modalities to qualify propositions• “it is raining” – rain• “it will rain tomorrow” –☐rain– it is raining in all possible futures
• “it might rain tomorrow” – ⃟rain– it is raining in some possible futures
Modal logic formulas are interpreted over a collection of possible worldsconnected by an accessibility relation
Temporal logic is a modal logic that adds temporal modalities: next, always, eventually, and until
![Page 14: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/14.jpg)
14Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Computation Tree Logic (CTL)
CTL: Branching-time propositional temporal logicModel - a tree of computation paths
S1 S2
S3
S2
S1 S3
S1 S3S2
S2
S1
S1 S3 S1 S3
Tree of computationKripke Structure
![Page 15: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/15.jpg)
15Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
CTL: Computation Tree Logic
Propositional temporal logic with explicit quantification over possible futuresSyntax:
True and False are CTL formulas;propositional variables are CTL formulas;
If ϕ and ψ are CTL formulae, then so are: ¬ ϕ , ϕ ∧ ψ , ϕ ∨ ψ
EX ϕ : ϕ holds in some next state
EF ϕ : along some path, ϕ holds in a future state
E[ϕ U ψ] : along some path, ϕ holds until ψ holds
EG ϕ : along some path, ϕ holds in every state
• Universal quantification: AX ϕ , AF ϕ , A[ϕ U ψ], AG ϕ
![Page 16: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/16.jpg)
16Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Examples: EX and AX
ϕ
EX ϕ (exists next)
ϕ
AX ϕ (all next)
ϕ
![Page 17: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/17.jpg)
17Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Examples: EG and AG
ϕ
ϕ
ϕ
ϕ
EG ϕ (exists global)
ϕ
ϕ
ϕ ϕϕ
ϕϕ ϕ ϕ ϕ
AG ϕ (all global)
ϕ
![Page 18: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/18.jpg)
18Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Examples: EF and AF
ϕ
EF ϕ (exists future)
ϕ
ϕ ϕ
AF ϕ (all future)
ϕ
![Page 19: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/19.jpg)
19Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Examples: EU and AU
ϕ
ϕ
ψ
E[ϕ U ψ] (exists until)
ϕ
ϕ
ϕ ψ
ψ
A[ϕ U ψ] (all until)
ψ
![Page 20: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/20.jpg)
20Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
CTL Examples
Properties that hold:• (AX busy)(s0)• (EG busy)(s3)• A (req U busy) (s0) • E (¬req U busy) (s1) • AG (req ⇒ AF busy) (s0)
Properties that fail:• (AX (req ∨ busy))(s3)
req req,busy
busy
s0
s2
s1
s3
![Page 21: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/21.jpg)
21Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Some Statements To Express
An elevator can remain idle on the third floor with its doors closed • EF (state=idle ∧ floor=3 ∧ doors=closed)
When a request occurs, it will eventually be acknowledged• AG (request ⇒ AF acknowledge)
A process is enabled infinitely often on every computation path• AG AF enabled
A process will eventually be permanently deadlocked• AF AG deadlock
Action s precedes p after q• A[¬q U (q ∧ A[¬p U s])]
• Note: hard to do correctly. Use property patterns
![Page 22: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/22.jpg)
22Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Semantics of CTL
K,s ⊨ ϕ – means that formula ϕ is true in state s. K is often omitted since we always talk about the same Kripke structure• E.g., s ⊨ p ∧¬qπ = π0 π1 … is a pathπ0 is the current state (root)πi+1 is a successor state of πi. Then,AX ϕ = ∀π ⋅ π1 ⊨ ϕ EX ϕ = ∃π ⋅ π1 ⊨ ϕAG ϕ = ∀π ⋅ ∀i ⋅ πi ⊨ ϕ EG ϕ = ∃π ⋅ ∀i ⋅ πi ⊨ ϕAF ϕ = ∀π ⋅ ∃i ⋅ πi ⊨ ϕ EF ϕ = ∃π ⋅ ∃i ⋅ πi ⊨ ϕA[ϕ U ψ] = ∀π ⋅ ∃i ⋅ πi ⊨ ψ ∧ ∀ j ⋅ 0 ≤ j < i ⇒ πj ⊨ ϕE[ϕ U ψ] = ∃π ⋅ ∃i ⋅ πi ⊨ ψ ∧ ∀ j ⋅ 0 ≤ j < i ⇒ πj ⊨ ϕ
![Page 23: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/23.jpg)
23Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Linear Temporal Logic (LTL)
For reasoning about complete traces through the system
Allows to make statements about a trace
S1 S2
S3
S2 S1S1 S2 S1
S2 S1S1 S2 S3
S2 S3S1 S3 S3
S2 S3S1 S1 S2
S2 S3S1 S3 S1
![Page 24: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/24.jpg)
24Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
LTL Syntax
If ϕ is an atomic propositional formula, it is a formula in LTLIf ϕ and ψ are LTL formulas, so are ϕ∧ ψ, ϕ∨ ψ, ¬ ϕ, ϕ U ψ (until), X ϕ (next), Fϕ (eventually), G ϕ (always)Interpretation: over computations π: ω ⇒ 2V which assigns truth values to the elements of V at each time instantπ ⊨ X ϕ iff π 1 ⊨ ϕπ ⊨ G ϕ iff ∀i ⋅ π i ⊨ ϕπ ⊨ Fϕ iff ∃i ⋅ π i ⊨ ϕπ ⊨ ϕ U ψ iff ∃i ⋅ π i ⊨ ψ ∧ ∀ j ⋅ 0 ≤ j < i ⇒ π j ⊨ ϕHere, π i is the i ’th state on a path
![Page 25: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/25.jpg)
25Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Expressing Properties in LTL
Good for safety (G ¬) and liveness (F) propertiesExpress:• When a request occurs, it will eventually be acknowledged– G (request ⇒ F acknowledge)
• Each path contains infinitely many q’s– G F q
• At most a finite number of states in each path satisfy ¬q (or property qeventually stabilizes)– F G q
• Action s precedes p after q– [¬q U (q ∧ [¬p U s])]– Note: hard to do correctly.
![Page 26: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/26.jpg)
26Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Safety and Liveness
Safety: Something “bad” will never happen• AG ¬bad• e.g., mutual exclusion: no two processes are in their critical section at once• Safety = if false then there is a finite counterexample• Safety = reachability
Liveness: Something “good” will always happen• AG AF good• e.g., every request is eventually serviced• Liveness = if false then there is an infinite counterexample• Liveness = termination
Every universal temporal logic formula can be decomposed into a conjunction of safety and liveness
![Page 27: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/27.jpg)
27Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
State Explosion
How fast do Kripke structures grow? • Composing linear number of structures yields exponential growth!
How to deal with this problem?• Symbolic model checking with efficient data structures (BDDs, SAT). – Do not need to represent and manipulate the entire model
• Abstraction – Abstract away variables in the model which are not relevant to the formula
being checked – Partial order reduction (for asynchronous systems)– Several interleavings of component traces may be equivalent as far as
satisfaction of the formula to be checked is concerned• Composition– Break the verification problem down into several simpler verification
problems
27
![Page 28: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/28.jpg)
28Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Representing Models Symbolically
A system state represents an interpretation (truth assignment) for a set of propositional variables V• Formulas represent sets of states that satisfy it– False = ∅, True = S– req – set of states in which req is– true – {s0, s1}– busy – set of states in which busy is– true – {s1, s3}– req∨ busy = {s0, s1 , s3}
• State transitions are described by relations over two sets of variables: V (source state) and V’ (destination state)– Transition (s2, s3) is ¬req∧ ¬ busy ∧ ¬req’∧ busy’– Relation R is described by disjunction of formulas for individual transitions
28
req req,busy
busy
s0
s2
s1
s3
![Page 29: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/29.jpg)
29Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Pros and Cons of Model-Checking
Often cannot express full requirements• Instead check several smaller simpler properties
Few systems can be checked directly• Must generally abstract parts of the system and model the environment
Works better for certain types of problems• Very useful for control-centered concurrent systems– Avionics software– Hardware– Communication protocols
• Not very good at data-centered systems– User interfaces, databases
![Page 30: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/30.jpg)
30Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Pros and Cons of Model Checking (Cont’d)
Largely automatic and fast
Better suited for debugging • … rather than assurance
Testing vs model-checking• Usually, find more problems by
exploring all behaviours of a downscaled system than by
testing some behaviours of the full system
![Page 31: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/31.jpg)
© 2016 Carnegie Mellon University
SAT and SMT
![Page 32: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/32.jpg)
32Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Boolean Satisfiability
Let V be a set of variablesA literal is either a variable v in V or its negation ~vA clause is a disjunction of literals• e.g., (v1 || ~v2 || v3)
A Boolean formula in Conjunctive Normal Form (CNF) is a conjunction of clauses• e.g., (v1 || ~v2) && (v3 || v2)
An assignment s of Boolean values to variables satisfies a clause c if it evaluates at least one literal in c to trueAn assignment s satisfies a formula C in CNF if it satisfies every clause in CBoolean Satisfiability Problem (SAT): • determine whether a given CNF C is satisfiable
![Page 33: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/33.jpg)
33Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
CNF Examples
CNF 1• ~b• ~a || ~b || ~c• a• sat: s(a) = True; s(b) = False; s(c) = False
CNF 2• ~b• ~a || b || ~c• a• ~a || c• unsat
![Page 34: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/34.jpg)
34Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Algorithms for SAT
SAT is NP-complete
DPLL (Davis-Putnam-Logemman-Loveland, ‘60)• smart enumeration of all possible SAT assignments• worst-case EXPTIME• alternate between deciding and propagating variable assignments
CDCL (GRASP ‘96, Chaff ‘01)• conflict-driven clause learning• extends DPLL with– smart data structures, backjumping, clause learning, heuristics, restarts…
• scales to millions of variables• N. Een and N. Sörensson, “An Extensible SAT-solver”, in SAT 2013.
![Page 35: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/35.jpg)
35Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
DPLL by Example
From http://homepage.cs.uiowa.edu/~tinelli/classes/196/Fall09/notes/dpll.pdf
DPLL Example by Prof. Cesare Tinelli
![Page 36: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/36.jpg)
36Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
S. A. Seshia 1
Some Experience with SAT Solving Sanjit A. Seshia
Speed-up of 2012 solver over other solvers
1
10
100
1,000
Solver
Spee
d-up
(log
sca
le)
Figure 4: SAT Solvers Performance%labelfigure
20
from M. Vardi, https://www.cs.rice.edu/~vardi/papers/highlights15.pdf
![Page 37: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/37.jpg)
37Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
SMT: Satisfiability Modulo Theory
Satisfiability of Boolean formulas over atoms in a theory• e.g., (x < 0) && (x >= 0)
Extends syntax of Boolean formulas with functions and predicates• +, -, div, select, store, bvadd, etc.
Existing solvers support many theories useful for program analysis• Equality and Uninterpreted Functions: f(x)• Real/Integer Linear Arithmetic: x + 2*y <= 3• Unbounded Arrays: a[i], a[i := v]• Bitvectors (a.k.a. machine integers): x >> 3, x/3• Floating point: 3.0 * x• …
![Page 38: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/38.jpg)
38Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
SMT-LIB: http://smt-lib.org
International initiative for facilitating research and development in SMTProvides rigorous definition of syntax and semantics for theoriesSMT-LIB syntax• based on s-expressions (LISP-like)• common syntax for interpreted functions of different theories– e.g. (and (= x y) (<= (* 2 x) z))
• commands to interact with the solver– (declare-fun …) declares a constant/function symbol– (assert p) conjoins formula p to the curent context– (check-sat) checks satisfiability of the current context– (get-model) prints current model (if the context is satisfiable)
• see examples at http://rise4fun.com/z3
![Page 39: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/39.jpg)
39Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
SMT Example
http://rise4fun.com/z3
![Page 40: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/40.jpg)
40Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
SAT/SMT Revolution
Solve any computational problem by effective reduction to SAT/SMT• iterate as necessary
Problem
encode
decode
SAT/SMTSolver
![Page 41: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/41.jpg)
© 2016 Carnegie Mellon University
Software Model Checking
![Page 42: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/42.jpg)
42Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
42
Software Model Checking
Yes/No
Answer
Program
(e.g., C)
Correctness
propertyModel of
the program
Model Extraction
Model Checker
1: int x = 2;int y = 2;
2: while (y <= 2)3: y = y – 1;4: if (x == 2)5: error();6:
EF (pc = 5)
![Page 43: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/43.jpg)
43Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
43
In Our Programming Language…
All variables are globalFunctions are in-linedint is integer • i.e., no overflow
Special statements:
skip do nothingassume(e) if e then skip else abortx,y=e1,e2 x, y are assigned e1,e2 in parallelx=nondet() x gets an arbitrary valuegoto L1,L2 non-deterministically go to L1 or L2
![Page 44: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/44.jpg)
44Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
44
From Programs to Kripke Structures
1: int x = 2;int y = 2;
2: while (y <= 2)3: y = y – 1;4: if (x == 2)5: error();6:
Program
pc x y …
3 1 3 …
State
pc x y …
2 1 2 …
Step
Property: EF (pc = 5)
![Page 45: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/45.jpg)
45Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
45
Programs as Control Flow Graphs
1: int x = 2;int y = 2;
2: while (y <= 2)3: y = y – 1;4: if (x == 2)5: error();6:
Program Labeled CFG
Semantics S
1:
2:
3:4:
5:
6:
x,y=2,2
y<=2y>2
x==2
x!=2
y=y-1
![Page 46: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/46.jpg)
46Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Modeling in Software Model Checking
Software Model Checker works directly on the source code of a program• but it is a whole-program-analysis technique• requires the user to provide the model of the environment with which the
program interacts– e.g., physical sensors, operating system, external libraries, specifications,
etc.
Programing languages already provide convenient primitives to describe behavior• programming languages are extended to modeling and specification
languages by adding three new features– non-determinism: like random values, but without a probability distribution– assumptions: constraints on “random” values– assertions: an indication of a failure
![Page 47: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/47.jpg)
47Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
From Programming to Modeling
Extend C programming language with 3 modeling features
Assertions• assert(e) – aborts an execution when e is false, no-op otherwise
Non-determinism• nondet_int() – returns a non-deterministic integer value
Assumptions• assume(e) – “ignores” execution when e is false, no-op otherwise
void assert (bool b) { if (!b) error(); }
int nondet_int () { int x; return x; }
void assume (bool e) { while (!e) ; }
![Page 48: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/48.jpg)
48Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Non-determinism vs. Randomness
A deterministic function always returns the same result on the same input• e.g., F(5) = 10
A non-deterministic function may return different values on the same input• e.g., G(5) in [0, 10] “G(5) returns a non-deterministic value between 0 and 10”
A random function may choose a different value with a probability distribution• e.g., H(5) = (3 with prob. 0.3, 4 with prob. 0.2, and 5 with prob. 0.5)
Non-deterministic choice cannot be implemented! • used to model the worst possible adversary/enviroment
![Page 49: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/49.jpg)
49Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Modeling with Non-determinism
int x, y;
void main (void){x = nondet_int ();
assume (x > 10);assume (x <= 100);y = x + 1;
assert (y > x);assert (y < 200);
}
![Page 50: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/50.jpg)
50Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Using nondet for modeling
Library spec:• “foo is given via grab_foo(), and is busy until returned via return_foo()”
Model Checking stub:
int nondet_int ();
int is_foo_taken = 0;
int grab_foo () {
if (!is_foo_taken)
is_foo_taken = nondet_int ();
return is_foo_taken; }
void return_foo ()
{ is_foo_taken = 0; }
![Page 51: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/51.jpg)
51Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Dangers of unrestricted assumptions
Assumptions can lead to vacuous correctness claims!!!
if (x > 0) {
assume (x < 0);
assert (0); }Is this program correct?
Assume must either be checked with assert or used as an idiom:
x = nondet_int ();
y = nondet_int ();
assume (x < y);
![Page 52: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/52.jpg)
52Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Software Model Checking Workflow
1. Identify module to be analyzed – e.g., function, component, device driver, library, etc.
2. Instrument with property assertions– e.g., buffer overflow, proper API usage, proper state change, etc.– might require significant changes in the program to insert necessary
monitors3. Model environment of the module under analysis– provide stubs for functions that are called but are not analyzed
4. Write verification harness that exercises module under analysis– similar to unit-test, but can use symbolic values– tests many executions at a time
5. Run Model Checker
6. Repeat as needed
![Page 53: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/53.jpg)
53Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
http://seahorn.github.io
![Page 54: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/54.jpg)
54Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
SeaHorn Verification Framework
Automated C program verifier for• buffer- and integer-overflow, API usage rules, and user-specified assertions
Integrates with industrial-strength LLVM compiler frameworkBased on our research in software model checking and abstract interpretation Developed jointly by the SEI, CMU CyLab, and NASA Ames
![Page 55: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/55.jpg)
55Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
SeaHorn Usage
> sea pf FILE.cOutputs sat for unsafe (has counterexample); unsat for safe Additional options• --cex=trace.xml outputs a counter-example in SV-COMP’15 format• --show-invars displays computed invariants• --track={reg,ptr,mem} track registers, pointers, memory content• --step={large,small} verification condition step-semantics– small == basic block, large == loop-free control flow block
• --inline inline all functions in the front-end passesAdditional commands• sea smt – generates CHC in extension of SMT-LIB2 format• sea clp -- generates CHC in CLP format (under development)• sea lfe-smt – generates CHC in SMT-LIB2 format using legacy front-end
![Page 56: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/56.jpg)
56Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Verification Pipeline
clang | pp | ms |opt | horn
front-end
compile pre-process
mixed semantics
optimize
VC gen & solve
![Page 57: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/57.jpg)
57Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Current Application
Verification of resource usage rules in Linux device drivers• e.g., locks are acquired and released, buffers are initialized, etc.• specifications and verification environment provided by the Open-Source
Linux Device Verification (LDV) project
NASA's Lunar Atmosphere and Dust Environment Explorer (LADEE)• conformance of auto-generated code with Simulink models• absence of buffer overflows
![Page 58: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/58.jpg)
58Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Types of Software Model Checking
Bounded Model Checking (BMC)• look for bugs (bad executions) up to a fixed bound• usually bound depth of loops and depth of recursive calls• reduce the problem to SAT/SMT
Predicate Abstraction with CounterExample Guided Abstraction Refinement (CEGAR)• Construct finite-state abstraction of a program• Analyze using finite-state Model Checking techniques• Automatically improve / refine abstraction until the analysis is conclusive
Interpolation-based Model Checking (IMC)• Iteratively apply BMC with increasing bound• Generalize from bounded-safety proofs • reduce the problem to many SAT/SMT queries and generalize from SAT/SMT
reasoning
![Page 59: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/59.jpg)
© 2016 Carnegie Mellon University
Bounded Model Checking
![Page 60: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/60.jpg)
60Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Bug Catching with SAT-Solvers
Main Idea: Given a program and a claim use a SAT-solver to find whether there exists an execution that violates the claim.
Program
ClaimAnalysisEngine
SATSolver
UNSAT
(no counterexample found)
SAT
(counterexample exists)
CNF
![Page 61: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/61.jpg)
61Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Programs and Properties
Arbitrary ANSI-C programs• With bitvector arithmetic, dynamic memory, pointers, …
Simple Safety Properties• Array bound checks (i.e., buffer overflow)• Division by zero• Pointer checks (i.e., NULL pointer dereference)• Arithmetic overflow• User supplied assertions (i.e., assert (i > j) )
![Page 62: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/62.jpg)
62Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Why use a SAT Solver?
SAT Solvers are very efficient
Analysis is completely automated
Analysis as good as the underlying SAT solver
Allows support for many features of a programming language• bitwise operations, pointer arithmetic, dynamic memory, type casts
![Page 63: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/63.jpg)
63Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
A (very) simple example (1)
int x;
int y=8,z=0,w=0;
if (x)
z = y – 1;
else
w = y + 1;
assert (z == 7 ||
w == 9)
y = 8,
z = x ? y – 1 : 0,
w = x ? 0 :y + 1,
z != 7,
w != 9
Program Constraints
UNSAT
no counterexample
assertion always holds!
![Page 64: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/64.jpg)
64Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
A (very) simple example (2)
int x;
int y=8,z=0,w=0;
if (x)
z = y – 1;
else
w = y + 1;
assert (z == 5 ||
w == 9)
y = 8,
z = x ? y – 1 : 0,
w = x ? 0 :y + 1,
z != 5,
w != 9
Program Constraints
SAT
counterexample found!
y = 8, x = 1, w = 0, z = 7
![Page 65: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/65.jpg)
65Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
What about loops?!
SAT Solver can only explore finite length executions!Loops must be bounded (i.e., the analysis is unsound)
Program
ClaimAnalysisEngine
SATSolver
UNSAT(no counterexample of
bound n is found)
SAT(counterexample exists)
CNF
Bound (n)
![Page 66: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/66.jpg)
66Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
CBMC: C Bounded Model Checker
Started at CMU by Daniel Kroening and Ed Clarke
Available at: http://www.cprover.org/cbmc• On Ubuntu: apt-get install cbmc
Supported platforms: Windows, Linux, OSX
Has a command line, Eclipse CDT, and Visual Studio interfaces
Scales to programs with over 30K LOC
Found previously unknown bugs in MS Windows device drivers
![Page 67: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/67.jpg)
67Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
How does it work
Transform a programs into a set of equations1. Simplify control flow 2. Unwind all of the loops3. Convert into Single Static Assignment (SSA)4. Convert into equations5. Bit-blast6. Solve with a SAT Solver7. Convert SAT assignment into a counterexample
![Page 68: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/68.jpg)
68Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
CBMC: Bounded Model Checker for CA tool by D. Kroening/Oxford and Ed Clarke/CMU
Parser Static Analysis
CNF-genSAT solver
CEX-gen CBMC
C Program
SAFE
UNSAFE + CEX
SAT
UNSAT CNF
goto-program
equations
![Page 69: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/69.jpg)
69Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Control Flow Simplifications
l All side effect are removed• e.g., j=i++ becomes j=i;i=i+1
• Control Flow is made explicit• continue, break replaced by goto
• All loops are simplified into one form• for, do while replaced by while
![Page 70: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/70.jpg)
70Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Loop Unwinding
• All loops are unwound• can use different unwinding bounds for different loops
• to check whether unwinding is sufficient special “unwinding assertion” claims are added
• If a program satisfies all of its claims and all unwinding assertions then it is correct!
• Same for backward goto jumps and recursive functions
![Page 71: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/71.jpg)
71Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Loop Unwinding
while() loops are unwound iteratively
Break / continue replaced by goto
void f(...) {...while(cond) {
Body;}Remainder;
}
![Page 72: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/72.jpg)
72Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Loop Unwinding
while() loops are unwound iteratively
Break / continue replaced by goto
void f(...) {...if(cond) {
Body;while(cond) {
Body;}
}Remainder;
}
![Page 73: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/73.jpg)
73Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Loop Unwinding
while() loops are unwound iteratively
Break / continue replaced by goto
void f(...) {...if(cond) {
Body;if(cond) {
Body;while(cond) {
Body;}
}}Remainder;
}
![Page 74: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/74.jpg)
74Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Unwinding assertion
while() loops are unwound iteratively
Break / continue replaced by goto
Assertion inserted after last iteration: violated if program runs longer than bound permits
void f(...) {...if(cond) {
Body;if(cond) {
Body;if(cond) {
Body;while(cond) {
Body;}
}}
}Remainder;
}
![Page 75: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/75.jpg)
75Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Unwinding assertion
while() loops are unwound iteratively
Break / continue replaced by goto
Assertion inserted after last iteration: violated if program runs longer than bound permits
Sound results!
void f(...) {...if(cond) {
Body;if(cond) {
Body;if(cond) {
Body;assert(!cond);
}}
}}Remainder;
}
Unwindingassertion
![Page 76: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/76.jpg)
76Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Example: Sufficient Loop Unwinding
void f(...) {j = 1if(j <= 2) {
j = j + 1;if(j <= 2) {
j = j + 1;if(j <= 2) {
j = j + 1;assert(!(j <= 2));}
}}
}Remainder;
}
void f(...) {j = 1while (j <= 2)
j = j + 1;Remainder;
}
unwind = 3
![Page 77: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/77.jpg)
77Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Example: Insufficient Loop Unwinding
void f(...) {j = 1if(j <= 10) {
j = j + 1;if(j <= 10) {
j = j + 1;if(j <= 10) {
j = j + 1;assert(!(j <= 10));}
}}
}Remainder;
}
void f(...) {j = 1while (j <= 10)
j = j + 1;Remainder;
}
unwind = 3
![Page 78: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/78.jpg)
78Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Transforming Loop-Free Programs Into Equations (1)
Easy to transform when every variable is only assigned once!
x = a;
y = x + 1;
z = y – 1;
Program Constraints
x = a &&
y = x + 1 &&
z = y – 1 &&
![Page 79: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/79.jpg)
79Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Transforming Loop-Free Programs Into Equations (2)
When a variable is assigned multiple times, use a new variable for the RHS of each assignment
Program SSA Program
![Page 80: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/80.jpg)
80Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
What about conditionals?
Program SSA Program
if (v)
x = y;
else
x = z;
w = x;
if (v0)
x0 = y0;
else
x1 = z0;
w1 = x??;
What should ‘x’ be?
![Page 81: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/81.jpg)
81Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
What about conditionals?
For each join point, add new variables with selectors
Program SSA Program
if (v)
x = y;
else
x = z;
w = x;
if (v0)
x0 = y0;
else
x1 = z0;
x2 = v0 ? x0 : x1;
w1 = x2
![Page 82: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/82.jpg)
82Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Adding Unbounded Arrays
Arrays are updated “whole array” at a time
A[1] = 5;
A[2] = 10;
A[k] = 20;
A1=λ i : i == 1 ? 5 : A0[i]
A2=λ i : i == 2 ? 10 : A1[i]
A3=λ i : i == k ? 20 : A2[i]
Examples: A2[2] == 10 A2[1]==5 A2[3] == A0[3]
A3[2] == (k==2 ? 20 : 10)
Uses only as much space as there are uses of the array!
![Page 83: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/83.jpg)
83Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Example
![Page 84: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/84.jpg)
84Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Pointers
While unwinding, record right hand side of assignments to pointersThis results in very precise points-to information• Separate for each pointer• Separate for each instance of each program location
Dereferencing operations are expanded intocase-split on pointer object (not: offset)• Generate assertions on offset and on type
Pointer data type assumed to be part of bit-vector logic• Consists of pair <object, offset>
![Page 85: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/85.jpg)
85Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
BMC: Summary
An effective way to look for bugs• reduce analysis to SAT/SMT• creating effective and precise encoding is very hard
Mature tools available from several academic groups• CBMC: http://www.cprover.org/cbmc/• LLBMC: http://llbmc.org/
Starting point for many other approaches• deductive verification: user provides inductive invariants for loops• Interpolation-based Model Checking (later in the lecture)• (dynamic) symbolic execution
![Page 86: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/86.jpg)
© 2016 Carnegie Mellon University
Predicate Abstraction and CounterExample Guided Abstraction-Refinement
![Page 87: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/87.jpg)
87Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Model Checking Software by Abstraction
Programs are not finite state• integer variables• recursion • unbounded data structures• dynamic memory allocation • dynamic thread creation • pointers • …
87
Program
Model Checker
Ü Build a finite abstraction Ä … small enough to analyzeÄ … rich enough to give conclusive
results
Abstraction
![Page 88: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/88.jpg)
88Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
88
Software Model Checking and Abstraction
ProgramP
Boolean Program
BP
KripkeStructure
K
AbstractKripke
K’
Semantics
Abstraction AbstractSemantics
Soundness of Abstraction:
BP abstracts P implies that K’ approximates K
![Page 89: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/89.jpg)
89Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
CounterExample Guided Abstraction Refinement (CEGAR)
Software Model Checking, SLAM
Project, Microsoft, Ball & Rajamani
Counterexample-guided Abstraction
Refinement for Symbolic Model
Checking, Clarke et al., CMU
Localization Reduction, Kurshan,
Bell LabsPredicate
Abstraction Model Checking
Predicate Refinement
Counterexample Valid?
Abstract Model
Candidate Counter-example
Better Predicates
Program
Initial Predicates
No
No
Yes
Yes
System OK
Problem Found
SMT Solver
![Page 90: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/90.jpg)
90Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
90
The Running Example
1: int x = 2;int y = 2;
2: while (y <= 2)3: y = y – 1;4: if (x == 2)5: error();6:
EF (pc = 5)
Program PropertyExpected
Answer
False
![Page 91: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/91.jpg)
91Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
91
An Example Abstraction
1: int x = 2;int y = 2;
2: while (y <= 2)3: y = y – 1;4: if (x == 2)5: error();6:
bool b is (y <= 2)1: b = T;
2: while (b)3: b = ch(b,f);4: if (*)5: error();6:
Program Abstraction
(with y<=2)
![Page 92: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/92.jpg)
92Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
92
Boolean (Predicate) Programs (BP)
Variables correspond to predicatesUsual control flow statementswhile, if-then-else, goto
Expressionsusual Boolean expressions, plus
*
ch(a,b)
Parallel Assignmentp1 = ch(a1,b1), p2 = ch(a2,b2), ...
b1 = ch(b1,¬b1), b2 = ch(b1⋁b2, f), b3=ch(f,f)
unknown
if a then true
else if b then
false else *
![Page 93: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/93.jpg)
93Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
93
Detour: Pre- and Post-Conditions
A Hoare triple {P} C {Q} is a logical statement that holds when
For any state s that satisfies P, if executing statement C on sterminates with a state s’, then s’ satisfies Q.
{P} C {Q}Statement
Pre-condition
(boolean formula)
Post-condition
(boolean formula)
![Page 94: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/94.jpg)
94Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Detour: Weakest Liberal Pre-Condition
The weakest liberal precondition of a statement C with respect to a post-condition Q (written WLP(C,Q)) is a formula P such that
1. {P} C {Q}
2. for all other P’ such that {P’} C {Q}, P’ ⇒ P (P is weaker then P’).
![Page 95: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/95.jpg)
95Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Detour: Weakest Liberal Preconditions
95
{P} C {Q}Statement
Pre-condition
(boolean formula)
Post-condition
(boolean formula)
{3>y} x = 3 {x>y}
{x>0} x = 2+y {y>0}
{*x>3 ⋁ x = &y} y=5 {*x>3}
{false} y=5 {y<0}
✘
✔
✔
✔
![Page 96: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/96.jpg)
96Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
96
Calculating Weakest Preconditions
Assignment (easy)• WLP (x=e, Q) = Q[x/e]– Intuition: after an assignment, x gets the value of e, thus Q[x/e] is
required to hold before x=e is executed
Examples:WLP (x:=0, x=y) = (x=y)[x/0] = (0==y)WLP (x:=0, x=y+1) = (x=y+1)[x/0] = (0 == y+1)WLP (y:=y-1,y<=2) = (y<=2)[y/y-1] = (y-1 <= 2)WLP(y:=y-1,x=2) = (x=2)[y/y-1] = (x == 2)
![Page 97: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/97.jpg)
97Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
97
Boolean Program Abstraction
Update p = ch(a, b) is an approximation of a concrete statement Siff {a}S{p} and {b}S{¬p} are valid• i.e., y = y – 1 is approximated by– (x == 2) = ch(x ==2, x!=2), and– (y <= 2) = ch(y<=2,false)
Parallel assignment approximates a concrete statement S iff all of its updates approximate S• i.e., y = y – 1 is approximated by
(x == 2) = ch(x ==2, x!=2),
(y <= 2) = ch(y<=2,false)
A Boolean program approximates a concrete program iff all of its statements approximate corresponding concrete statements
![Page 98: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/98.jpg)
98Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
98
Computing An Abstract Update// S a statement under abstraction
// P a list of predicates used for abstraction
// t a target predicate for the update
absUpdate (Statement S, List<Predicates> P, Predicate q) {
resT, resF = false, false;
// foreach monomial (full conjunction of literals) in P
foreach m : monomials(P) {
if (SMT_IS_VALID(“m ⇒ WLP(S,q)”) resT = resT ⋁ m;
if (SMT_IS_VALID(“m ⇒ WLP(S,¬q)”) resF = resF ⋁ m;
}
return “q = ch(resT, resF)”
}
![Page 99: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/99.jpg)
99Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
99
absUpdate (y=y-1, P={y<=2}, q=(y<=2))
y = y - 1;
(y<=2) = ch (y<=2,f)
P is {y <= 2}q is (y <= 2)
SMT Queries:(y<=2) ⇒ (y–1) <= 2¬(y<=2)⇒ (y–1) <= 2(y<=2) ⇒ (y–1) > 2¬(y<=2)⇒ (y–1) > 2
absUpdate
✔
✘
✘
✘
WLP(y=y-1,y<=2) is (y-1) <= 2
WLP(y=y-1,¬(y<=2)) is (y–1) > 2
![Page 100: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/100.jpg)
100Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
100
The result of abstraction
1: int x = 2;int y = 2;
2: while (y <= 2)3: y = y – 1;4: if (x == 2)5: error();6:
bool b is (y <= 2)1: b = T;
2: while (b)3: b = ch(b,f);4: if (*)5: error();6:
Program Abstraction
(with y<=2)
But what is the semantics of Boolean programs?
![Page 101: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/101.jpg)
101Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
101
BP Semantics: Overview
Over-Approximation• treat “unknown” as non-deterministic• good for establishing correctness of universal properties
Under-Approximation• treat “unknown” as abort• good for establishing failure of universal properties
Exact Approximation• Treat “unknown” as a special unknown value• good for verification and refutation• good for universal, existential, and mixed properties
![Page 102: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/102.jpg)
102Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
102
BP Semantics: Over-Approximation
1: ;2: if (nondet) {3: if (*)4: error();5: if (nondet)6: error();7: }
Abstraction
1:
2:
3:
5:
4:
7:
6:
Over-
Approximation
Unknown is treated as non-deterministic
![Page 103: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/103.jpg)
103Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
103
BP Semantics: Under-Approximation
1: ;2: if (nondet) {3: if (*)4: ERROR;5: if (nondet)6: ERROR;7: }
Abstraction
1:
2:
3:
5:
4:
7:
6:
Under-
Approximation
Unknown is treated as abort
![Page 104: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/104.jpg)
104Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
104
BP Semantics: Exact Approximation
1: ;2: if (nondet) {3: if (*)4: ERROR;5: if (nondet)6: ERROR;7: }
Abstraction
1:
2:
3:
5:
4:
7:
6:
Exact
Belnap KS
t
f
⊥ ⊤
“unknown”
“non-deterministic”
Unknown is treated as unknown
![Page 105: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/105.jpg)
105Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
105
Summary: The Three Semantics
b1 = ch(b1,f);b2 = ch(b2,¬b2)
b1
b2
b2
t
f
⊥ ⊤
Abstract
Over-Approx Belnap (Exact) Under-Approx
y = y - 1;
Concreteb1 is (y <= 2)b2 is (x == 2)
b1
b2
b2
b1?
b2
b1
b2?
b1
b2
b2
b1?
b2
b1
b2?
![Page 106: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/106.jpg)
106Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Summary: Program Abstraction
Abstract a program P by a Boolean program BPPick an abstract semantics for this BP:• Over-approximating • Under-approximating• Belnap (Exact)
Yield relationship between K and K’:• Over-approximation • Under-approximation • Belnap abstraction
106
ProgramP
Boolean Program
BP
KripkeStructure
K
AbstractKripke
K’Semantics
Abstraction Abstract
Semantics
![Page 107: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/107.jpg)
107Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
CounterExample Guided Abstraction Refinement (CEGAR)
Software Model Checking, SLAM
Project, Microsoft, Ball & Rajamani
Counterexample-guided Abstraction
Refinement for Symbolic Model
Checking, Clarke et al., CMU
Localization Reduction, Kurshan,
Bell LabsPredicate
Abstraction Model Checking
Predicate Refinement
Counterexample Valid?
Abstract Model
Candidate Counter-example
Better Predicates
Program
Initial Predicates
No
No
Yes
Yes
System OK
Problem Found
SMT Solver
![Page 108: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/108.jpg)
108Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Example: Is ERROR Unreachable?
1: int x = 2;int y = 2;
2: while (y <= 2)3: y = y – 1;4: if (x == 2)5: error();6:
1: ;
2: while (*)3: ;4: if (*)5: error();6:
1:
2:
3:4:
5:
6:
Need This!
Program Abstraction Over-Approximation
Abstract Translate Check Validate
CEGAR steps
Repeat
![Page 109: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/109.jpg)
109Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Example: Is ERROR Unreachable?
1: int x = 2;int y = 2;
2: while (y <= 2)3: y = y – 1;4: if (x == 2)5: error();6:
bool b is (y <= 2)1: b = T;
2: while (b)3: b = ch(b,f);4: if (*)5: error();6:
Program Abstraction(with y<=2)
Over-Approximation
1:
2:b=T
3:b=T4:b=F
5:b=F
6:b=F
2:b=F
UNREACHABLE
Abstract Translate Check NO ERROR
CEGAR steps
![Page 110: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/110.jpg)
110Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
110
Using Cex for Refinement
s0
s1
s2s3
ERROR
s5 s4
MCIs ERROR Reachable?EF (ERROR)
UNKNOWN
Counterexample
s0
s1
s2
ERROR
s4t
f
⊥ ⊤
![Page 111: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/111.jpg)
111Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
111
can stop here
causeUsing Proofs for Refinement
EF (ERROR) (s0) = ⊥∃n EFn (ERROR)(s0) = ⊥
EF4 (ERROR)(s0) = ⊥s0→s1 EF3(ERROR)(s1) = ⊥
s1→s2 EF2(ERROR)(s2) = ts0
s1
s2s3
ERROR
s5 s4
MCIs ERROR Reachable?EF (ERROR)
UNKNOWN
Why?
RefineHERE
t
f
⊥ ⊤
![Page 112: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/112.jpg)
112Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
112
Finding Refinement Predicates
Recall• each abstract state is a conjunction of predicates– i.e., y<=2⋀x==2 y>2 ⋀ x!=2 etc.
• each abstract transition corresponds to a program statement
Result from a partial proof
Unknown transitions1→s2
MC needs to knowvalidity of
{s1} C {s2}
C is the statementcorresponding to
the transition
![Page 113: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/113.jpg)
113Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
113
Refinement via Weakest Liberal Precondition
If s1→s2 corresponds to a conditional statement• refine by adding the condition as a new predicate
If s1→s2 corresponds to a statement C• Find a predicate p in s2 with uncertain value
– i.e., {s1}C{p} is not valid• refine by adding WLP(C,p)
![Page 114: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/114.jpg)
114Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
114
An Example
{y>2⋀x==2} y = y-1 {y>2⋀x==2}
s1 → s2 is unknown
WLP(y = y–1, y>2) = y>3
{y>2⋀x==2} y = y-1 {y>2} ✘
new predicate
{y>2⋀x==2} y = y-1 {x==2} ✔
pc=2
y>2
x==2
pc=3
y>2
x==2
![Page 115: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/115.jpg)
115Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
115
Summary: Predicate Abstraction and CEGAR
Predicate abstraction with CEGAR is an effective technique for analyzing behavioral properties of software systems
Combines static analysis and traditional model-checking
Abstraction is essential for scalability• Boolean programs are used as an intermediate step• Different abstract semantics lead to different abs.– over-, under-, Belnap
Automatic abstraction refinement finds the “right” abstraction incrementally
![Page 116: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/116.jpg)
© 2016 Carnegie Mellon University
Interpolation-based Model Checking
![Page 117: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/117.jpg)
117Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Programs, Safety, Cexs, Invariants
A transition system P = (V, Init, Tr, Bad)
P is UNSAFE if and only if there exists a number N s.t.
P is SAFE if and only if there exists a safe inductive invariant Inv s.t.
Inductive
Safe
Init(X0) ^
N�1̂
i=0
Tr(Xi, Xi+1)
!^ Bad(XN ) 6) ?
Init ) Inv
Inv(X) ^ Tr(X,X 0) ) Inv(X 0)
Inv ) ¬Bad
![Page 118: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/118.jpg)
118Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Verification by Successive Under-Approximation
bounded proof
Lemma2
Lemma1Lemma3
Inductive?
bounded proof
Lemma2
Lemma1Lemma3
Inductive?
bounded proof
Lemma2
Lemma1Lemma3
Inductive?No No No
BMC BMC BMC
bound 1 bound 2 bound 3
![Page 119: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/119.jpg)
119Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
INIT
Reachability Analysis
119
Bad
Is Bad reachable?
R1R2
…Rn
![Page 120: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/120.jpg)
120Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Interpolating Model Checking
Key Idea• turn SAT/SMT proofs of bounded safety to inductive traces• repeat forever until a counterexample or inductive invariant are found
Introduced by McMillan in 2003• Kenneth L. McMillan: Interpolation and SAT-Based Model Checking. CAV2003: 1-
13• based on pairwise Craig interpolation
Extended to sequences and DAGs• Yakir Vizel, Orna Grumberg: Interpolation-sequence based model checking.
FMCAD 2009: 1-8– uses interpolation sequence
• Kenneth L. McMillan: Lazy Abstraction with Interpolants. CAV 2006: 123-136– IMPACT: interpolation sequence on each program path
• Aws Albarghouthi, Arie Gurfinkel, Marsha Chechik: From Under-Approximations to Over-Approximations and Back. TACAS 2012: 157-172– UFO: interpolation sequence on the DAG of program paths
![Page 121: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/121.jpg)
121Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
IMC: Interpolating Model Checking
N=1
BMCN
SeqItp
trace F = [F0, …, FN]
Is F closed
N:=N+1
CEX
SAFE
SAT
UNSAT
YesNo
![Page 122: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/122.jpg)
122Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Bounded Model Checking
INIT
R1 R2
……
INIT(V0)
Rk
∧Tr(V0,V1)∧…∧Tr(Vk-1,Vk)∧Bad(Vk)
![Page 123: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/123.jpg)
123Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Inductive Trace
An inductive trace of a transition system P = (V, Init, Tr, Bad) is a sequence of formulas [F0, …, FN] such that• Init → F0
• 8 0 · i < N , Fi(v) Æ Tr (v, u) → Fi+1 (u), or, in Hoare Logic {Fi} Tr {Fi+1}
A trace is safe iff 8 0 · i · N , Fi → ¬Bad
A trace is monotone iff 8 0 · i < N , Fi → Fi+1
A trace is closed iff 9 1 · i · N, Fi → (F0 Ç … Ç Fi-1)
A transition system P is SAFE iff it admits a safe closed trace
![Page 124: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/124.jpg)
124Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
INIT
Inductive Trace in Pictures
124
Bad
F1F2
…FN
![Page 125: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/125.jpg)
125Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Craig Interpolation Theorem
Theorem (Craig 1957)Let A and B be two First Order (FO) formulae such that A ) ¬B, then there exists a FO formula I, denoted ITP(A, B), such that
A ) I I ) ¬B atoms(I) 2 atoms(A) Å atoms(B)
A Craig interpolant ITP(A, B) can be effectively constructed from a resolution proof of unsatisfiability of A Æ B
In Model Cheching, Craig Interpolation Theorem is used to safely over-approximate the set of (finitely) reachable states
![Page 126: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/126.jpg)
126Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
A
Craig Interpolant
126
B
I
![Page 127: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/127.jpg)
127Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Craig Interpolant Examples
Boolean logic• A is {!b, (!a || b || c), a} B is !a || !c• Itp is a && c
EUF (equality with uninterpreted functions)• A is {f(a) = b, p(f(a))} B is {b=c, !p(c)}• Itp is p(b)
Linear Arithmetic• A is {z+x+y > 10, z < 5} B is {x < -5, y < -3}• Itp is x+y>5
![Page 128: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/128.jpg)
128Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Craig Interpolant as a Circuit
Let F = A(x, z) Æ B(z, y) be UNSAT, where x and y are distinct• Note that for any assignment v to z either– A(x, v) is UNSAT, or– B(v, y) is UNSAT
An interpolant is a circuit I(z) such that for every assignment v to z• I(v) = A only if A(x, v) is UNSAT• I(v) = B only if B(v, y) is UNSAT
A proof system S has a feasible interpolation if for every refutation ¼ of F in S, F has an interpolant polynomial in the size of ¼• propositional resolution has feasible interpolation• extended resolution does not have feasible interpolation
![Page 129: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/129.jpg)
129Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Craig Interpolation for Linear Arithmetic
Useful properties of existing interpolation algorithms [CGS10] [HB12]• I 2 ITP (A, B) then ¬I 2 ITP (B, A)• if A is syntactically convex (a monomial), then I is convex• if B is syntactically convex, then I is co-convex (a clause)• if A and B are syntactically convex, then I is a half-space
A
I = interpolant
![Page 130: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/130.jpg)
130Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
) ) ) )))
Interpolation Sequence
Given a sequence of formulas A = {Ai}i=0n, an interpolation
sequence ItpSeq(A) = {I1, …, In-1} is a sequence of formulas such that• Ik is an ITP (A0 Æ … Æ Ak-1, Ak Æ … Æ An), and• 8 k<n . Ik Æ Ak+1) Ik+1
A0 A1 A2 A3 A4 A5 A6
I0 I1 I2 I3 I4 I5
Can compute by pairwise interpolation applied to different cuts of a fixed resolution proof (very robust property of interpolation)
![Page 131: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/131.jpg)
131Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
From Interpolants to Traces
A Sequence Interpolant of a BMC instance is an inductive trace
( Init(v0) )0 Æ ( Tr (v0,v1) )1 Æ … Æ ( Tr (vN-1, vN) )N Æ Bad(vN)
F0(v0) F1(v1) FN(vN)
A trace computed by a sequence interpolant is • safe• NOT necessarily monotone• NOT necessarily closed
BMCN
trace
![Page 132: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/132.jpg)
132Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
INIT
Inductive Trace in Pictures
132
Bad
F1F2
…FN
![Page 133: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/133.jpg)
133Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
ImcMkSafe
IMC: Interpolating Model Checking
N=1
BMCN
SeqItp
trace F = [F0, …, FN]
Is F closed
N:=N+1
CEX
SAFE
SAT
UNSAT
YesNo
![Page 134: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/134.jpg)
134Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
IMC: Strength and Weaknesses
Strength• elegant• global bounded safety proof• many different interpolation algorithms available• easy to extend to SMT theories
Weaknesses• the naïve version does not converge easily– interpolants are weaker towards the end of the sequence
• not incremental– no information is reused between BMC queries
• size of interpolants• hard to guide
![Page 135: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/135.jpg)
© 2016 Carnegie Mellon University
Trust in Formal Methods
![Page 136: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/136.jpg)
136Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Idealized Development w/ Formal Methods
No expensive testing!•Verification is exhaustive
Simpler certification!• Just check formal arguments
Design Develop Verify (with FM) Certify Deploy
Can we trust formal methods tools? What can go wrong?
![Page 137: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/137.jpg)
137Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Trusting Automated Verification Tools
How should automatic verifiers be qualified for certification?
What is the basis for automatic program analysis (or other automatic formal methods) to replace testing?
Verify the verifier• (too) expensive• verifiers are often very complex tools• difficult to continuously adapt tools to project-specific needs
Proof-producing (or certifying) verifier• Only the proof is important – not the tool that produced it• Only the proof-checker needs to be verified/qualified• Single proof-checker can be re-used in many projects
![Page 138: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/138.jpg)
138Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Active research area• proof carrying code, certifying model checking, model carrying code etc.• Few tools available. Some preliminary commercial application in the telecom domain.• Static context. Good for ensuring absence of problems.• Low automation. Applies to source or binary. High confidence.
Evidence Producing Analysis
X witnesses that P satisfies Q. X can be objectively and independently verified. Therefore, EPA is outside the Trusted Computing Base (TCB).
Program P
Property Q
Proof XEPA
do not trust “easy” to verify
Not that simple in practice !!!
![Page 139: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/139.jpg)
139Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
An In-Depth Look…
Low level property
Program = (Text, Semantics)
Verifier
Proof Checker
Front-EndEnvironment model
VC
No + Counterexample
Yes + Proof
Good Bad
Compiler
Executable
Real Env HardwareGood
Bad?=?
Hard to verify
Hard to get right
Diff semused by diff tools
Hard to get right
![Page 140: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/140.jpg)
140Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Five Hazards (Gaps) of Automated Verification
Soundness Gap• Intentional and unintentional unsoundness in the verification engine • e.g., rational instead of bitvector arithmetic, simplified memory model, etc.
Semantic Gap• Compiler and verifier use different interpretation of the programming
languageSpecification Gap• Expressing high-level specifications by low-level verifiable properties
Property Gap• Formalizing low-level properties in temporal logic and/or assertions
Environment Gap• Too coarse / unsound / unfaithful model of the environment
![Page 141: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/141.jpg)
141Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Mitigating The Soundness Gap
Proof-producing verifier makes the soundness gap explicit• the soundness of the proof can be established by a “simple” checker• all assumptions are stated explicitly
Open questions:• how to generate proofs for explicit Model Checking – e.g., SPIN, Java PathFinder
• how to generate partial proofs for non-exhaustive methods – e.g., KLEE, Sage
• how to deal with “intentional” unsoundness – e.g., rational arithmetic instead of bitvectors, memory models, …
![Page 142: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/142.jpg)
142Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Vacuity: Mitigating Property Gap
Model Checking Perspective: Never trust a True answer from a Model Checker
When a property is violated, a counterexample is a certificate that can be examined by the user for validity
When a property is satisfied, there is no feedback!
It is very easy to formally state something very trivial in a very complex way
![Page 143: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/143.jpg)
143Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
MODULE main
VARsend : {s0,s1,s2};recv : {r0,r1,r2};
ack : boolean;req : boolean;
ASSIGNinit(ack):=FALSE;init(req):=FALSE;
init(send):= s0;init(recv):= r0;
next (send) := case
send=s0:{s0,s1};send=s1:s2;send=s2&ack:s0;TRUE:send;
esac;
next (recv) := case
recv=r0&req:r1;recv=r1:r2;recv=r2:r0;TRUE: recv;
esac;
next (ack) :=case
recv=r2:TRUE;TRUE: ack;
esac;
next (req) := case
send=s1:FALSE;TRUE: req;
esac;
SPEC AG (req -> AF ack)
![Page 144: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/144.jpg)
144Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Five Hazards (Gaps) of Automated Verification
Soundness Gap• Intentional and unintentional unsoundness in the verification engine • e.g., rational instead of bitvector arithmetic, simplified memory model, etc.
Semantic Gap• Compiler and verifier use different interpretation of the programming
languageSpecification Gap• Expressing high-level specifications by low-level verifiable properties
Property Gap• Formalizing low-level properties in temporal logic and/or assertions
Environment Gap• Too coarse / unsound / unfaithful model of the environment
![Page 145: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/145.jpg)
145Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Verification Competitions
Multitude of events where solvers and analysis engines competeSAT-RACE• competitive event for SAT solvers• http://baldur.iti.kit.edu/sat-race-2015/
SMT-COMP• competitive event for SMT solvers• http://www.smtcomp.org
SV-COMP• Software Verification Competition– open to all, but most tools are based on Model Checking
• http://sv-comp.sosy-lab.org/2016/CASC• competitive event for Automated Theorem Proving• http://www.cs.miami.edu/~tptp/CASC/
![Page 146: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/146.jpg)
146Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
References
Software Model Checking and Program Analysis• Vijay D'Silva, Daniel Kroening, Georg Weissenbacher: A Survey of
Automated Techniques for Formal Software Verification. IEEE Trans. on CAD of Integrated Circuits and Systems 27(7): 1165-1178 (2008)
• Ranjit Jhala, Rupak Majumdar: Software model checking. ACM Comput. Surv. 41(4) (2009)
Symbolic Execution• Cristian Cadar, Patrice Godefroid, Sarfraz Khurshid, Corina S. Pasareanu,
Koushik Sen, Nikolai Tillmann, Willem Visser: Symbolic execution for software testing in practice: preliminary assessment. ICSE 2011: 1066-1071
SMT and Decision Procedures• Daniel Kroening, Ofer Strichman: Decision Procedures - An Algorithmic Point
of View. Texts in Theoretical Computer Science. An EATCS Series, Springer 2008, ISBN 978-3-540-74104-6, pp. 1-304
• The SMT-LIB v2 Language and Tools: A Tutorial, by David R. Cokk
![Page 147: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/147.jpg)
© 2016 Carnegie Mellon University
Extra Slides
![Page 148: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/148.jpg)
148Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Hoare Triples
A Hoare triple {Pre} P {Post} is valid iff every terminating execution of Pthat starts in a state that satisfies Pre ends in a state that satisfies PostInductive Loop Invariant
Function Application
Recursion
Pre ) Inv {InvÆC} Body {Inv} InvƬC ) Post
{Pre} while C do Body {Post}
(PreÆp=a) ) P {P} BodyF {Q} (QÆp,r=a,b) )Post
{Pre} b = F(a) {Post}
{Pre} b = F(a) {Post} ` {Pre} BodyF {Post}
{Pre} b = F(a) {Post}
![Page 149: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/149.jpg)
149Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Weakest Liberal Pre-Condition
Validity of Hoare triples is reduced to FOL validity by applying a predicate transformer
Dijkstra’s weakest liberal pre-condition calculus [Dijkstra’75]
wlp (P, Post)
weakest pre-condition ensuring that executing P ends in Post
{Pre} P {Post} is valid , Pre ) wlp (P, Post)
![Page 150: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/150.jpg)
150Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
A Simple Programming Language
Prog ::= def Main(x) { bodyM }, …, def P (x) { bodyP }
body ::= stmt (; stmt)*
stmt ::= x = E | assert (E) | assume (E) | while E do S | y = P(E) |L:stmt | goto L (optional)
E := expression over program variables
![Page 151: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/151.jpg)
151Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Horn Clauses by Weakest Liberal Precondition
Prog ::= def Main(x) { bodyM }, …, def P (x) { bodyP }
wlp (x=E, Q) = let x=E in Qwlp (assert(E) , Q) = E Æ Qwlp (assume(E), Q) = E → Qwlp (while E do S, Q) = I(w) Æ
8w . ((I(w) Æ E) → wlp (S, I(w))) Æ ((I(w) Æ ¬E) → Q))wlp (y = P(E), Q) = ppre(E) Æ (8 r. p(E, r) → Q[r/y])
ToHorn (def P(x) {S}) = wlp (x0=x;assume(ppre(x)); S, p(x0, ret))ToHorn (Prog) = wlp (Main(), true) Æ 8{P 2 Prog} . ToHorn (P)
![Page 152: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/152.jpg)
152Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Example of a WLP Horn Encoding
{y ¸ 0} P {x = xold+yold} is true iff the query C3 is satisfiable
{Pre: y¸ 0}xo = x;yo = y; while y > 0 dox = x+1;y = y−1;
{Post: x=xo+yo}
C1: I(x,y,x,y) à y>=0.C2: I(x+1,y-1,xo,yo) à I(x,y,xo,yo), y>0.C3: false à I(x,y,xo,yo), y·0, x≠xo+yo
ToHorn
![Page 153: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/153.jpg)
153Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Single Static Assignment
SSA == every value has a unique assignment (a definition)A procedure is in SSA form if every variable has exactly one definition
SSA form is used by many compilers• explicit def-use chains• simplifies optimizations and improves analyses
PHI-function are necessary to maintain unique definitions in branching control flow
x = PHI ( v0:bb0, …, vn:bbn) ) (phi-assignment)
“x gets vi if previously executed block was bbi”
![Page 154: Automated Program Analysis with Software Model Checking · search of the state space of the design • Statespace explosion. 7 Software Model Checking Gurfinkel, Feb. 2016 ... Modal](https://reader034.vdocuments.us/reader034/viewer/2022042211/5eb1048a239e961bcd3f9eb9/html5/thumbnails/154.jpg)
154Software Model CheckingGurfinkel, Feb. 2016© 2016 Carnegie Mellon University
Single Static Assignment: An Example0: goto 11: x_0 = PHI(0:0, x_3:5);
y_0 = PHI(y:0, y_1:5);if (x_0 < N) goto 2 else goto 6
2: if (y_0 > 0) goto 3 else goto 4
3: x_1 = x_0 + y_0; goto 5
4: x_2 = x_0 – y_0; goto 5
5: x_3 = PHI(x_1:3, x_2:4);y_1 = -1 * y_0;goto 1
6:
int x, y, n;
x = 0;while (x < N) {if (y > 0)
x = x + y;else
x = x – y;y = -1 * y;
}
val:bb