Agenda of this course• Introduction to Software Design• Modeling notations• Design Patterns• Model Validation: Model simulation and model-based

testing• Performance validation - Timing analysis and prediction• Performance validation - Scheduling methods• Software validation - Trace analysis and Debugging

methods• Software validation - Static property checking of software• Validation of communication behavior

Introduction to Embedded Systems Verification

Embedded Systems

Entertainment

Polar Lander logic-error

Rover (2004) file-system error

Everywhere

Alice

Hall of Shame Therac-25 Radiation error

Control systems, hardware and software, with many sensors, signal & data processing algorithms, communications over networks

Airbus

Rigorous Verification and Validation indispensable

Motivation for Embedded System Verification

• Unlike “one-shot” function computations, computations of embedded systems are infinite streams– Example: Rudder positions

computed by an autopilot program: L, R, R, R, C, L, …

• Testing and simulations (at least their naïve applications) can check for only finitely many behaviors

• Not sufficient for covering all behaviors of Embedded Systems

• Is this a real problem?• Yes!“June 4, 1996 -- Ariane 5 Flight 501. Working code for the Ariane 4rocket is reused in the Ariane 5, but the Ariane 5's faster enginestrigger a bug in an arithmetic routine inside the rocket's flightcomputer. The error is in the code that converts a 64-bit floating-point number to a 16-bit signed integer. The faster engines causethe 64-bit numbers to be larger in the Ariane 5 than in the Ariane 4,triggering an overflow condition that results in the flight computercrashing.” --- History's Worst Software Bugs, SimsonGarfinkel, Wired 2005

Generic structure of an Embedded System today

Analog Digital Analog

Memory

Coprocessors

Controllers

Converters

Processor

Interface

Software(Application Programs)

ASIC

Embedded Systems Design Flow

ConceptSpecification

HW/SWPartitioning

Hardware Components

Software Components

Estimation -Exploration

Hardware

Software

Validation and Evaluation (area, power, performance, …)

System Type Formal

specsCore

Problems

DiscreteContinuous

HybridBoolean Logic

Temporal Logic

FSM

Equations

Hybrid AutomataSatisfiability

Coverage

Synthesizability

Formal Verification

Dimensions of the challenge

Non-Linear

• Task: Verify if a system design meet its specification• Standard Testing methods losing steam (system dynamics,

stochastic, non-linear, mixed, thousands of states….)

• Formal Verification showing much promise in recent times

Architecturevalidation

Microcodevalidation

Timingvalidation

Powervalidation

Protocolvalidation

System

Unit

Cluster

Reliabilityvalidation

The promise of Formal Verification

• An algorithm which takes as input– (a) a model of a system A and – (b) a property P

and terminates with output – (c) a proof that all the behaviors of A satisfy P OR– (d) a particular behavior of A that violates P

• Examples: 1. A: model of autonomous vehicle P: always stays on the road2. A: model of a traffic control system P: vehicles do not collide

• Is completely automatic

P

Verification Algorithm

A A satisfies P

Trace of A violating P

The overall picture

system model

M

property

ψ

Formal Engine

(Does M satisfy ψ ?)

Yes!

No! +“counterexample”

Where do we get the system model?

hardware

e.g., Verilog or VHDL, source code

System model

abstraction & other (semi-)automated transformations

software

e.g., C, C++ , or Java, source

code

Extended design models

Where do we get the properties?

requirements documentation

+(insight)

formal properties

(typically based on temporal logic or

automata)

canned standard properties & templates

(e.g., “deadlock-freedom”)

Formal Verification

System A mathematical model M

Desired behavior A formal specification ψ

The system has the required

behaviorM satisfies ψ

Model checking

Formal Verification

• Developed independently by Clarke and Emerson and by Queille and Sifakis in early 1980’s.

• Properties are written in propositional temporal logic.

• Systems are modeled by finite state machines.

• Verification procedure is an exhaustive search of the state space of the design.

• Model checking complements testing/simulation.

13

14

Mutual Exclusion Example

N1 → T1T1 ∧ S0 → C1 ∧ S1 C1 → N1 ∧ S0

N2 → T2T2 ∧ S0 → C2 ∧ S1C2 → N2 ∧ S0||

• Two process mutual exclusion with shared semaphore• Each process has three states

• Non-critical (N)• Trying (T)• Critical (C)

• Semaphore can be available (S0) or taken (S1) • Initially both processes are in the Non-critical state andthe semaphore is available --- N1 N2 S0

15

Mutual Exclusion Example

N1N2S0

C1N2S1T1T2S0

N1T2S0T1N2S0

N1C2S1

T1C2S1C1T2S1

K ╞ AG EF (N1 ∧N2 ∧S0)

No matter where you are there is always a way to get to the initial state

16

Mutual Exclusion Example

N1N2S0

C1N2S1T1T2S0

N1T2S0T1N2S0

N1C2S1

T1C2S1C1T2S1

K ╞ AG EF (N1 ∧N2 ∧S0)

17

Mutual Exclusion Example

N1N2S0

C1N2S1T1T2S0

N1T2S0T1N2S0

N1C2S1

T1C2S1C1T2S1

K ╞ AG EF (N1 ∧N2 ∧S0)

18

Mutual Exclusion Example

N1N2S0

C1N2S1T1T2S0

N1T2S0T1N2S0

N1C2S1

T1C2S1C1T2S1

K ╞ AG EF (N1 ∧N2 ∧S0)

19

Mutual Exclusion Example

N1N2S0

C1N2S1T1T2S0

N1T2S0T1N2S0

N1C2S1

T1C2S1C1T2S1

K ╞ AG EF (N1 ∧N2 ∧S0)

20

Mutual Exclusion Example

N1N2S0

C1N2S1T1T2S0

N1T2S0T1N2S0

N1C2S1

T1C2S1C1T2S1

K ╞ AG EF (N1 ∧N2 ∧S0)

Advantage of Model Checking

Simulation Checks Only the Values We SelectEven Small Systems Have Trillions (of Trillions) of Possible Tests!

Advantage of Model CheckingModel Checker Tries Every Possible Input and State!

Transition System

System Description(e.g. VERILOG, VHDL, etc.)

Informal Specification

Temporal Logic Formula(CTL, LTL, etc.)

Formal Verification

Transition System

Informal Specification

Temporal Logic Formula(CTL, LTL, etc.)

Safety Property:bad state unreachable:

satisfied

Initial State

Counterexamples

Program or circuit

Transition System

Program or circuitInformal

Specification

Temporal Logic Formula(CTL, LTL, etc.)

Initial State

Safety Property:bad state unreachable

Counterexample

Counterexamples

Transition System

Program or circuitInformal

Specification

Temporal Logic Formula(CTL, LTL, etc.)

Initial State

Safety Property:bad state unreachable

Counterexamples

Counterexample

Advantages of Formal Verification

• No proofs!!!

• Fast (compared to other rigorous methods)

• Diagnostic counterexamples

• Logics can easily express many concurrency properties

27

Formal Verification since 1981

1981 Clarke / Emerson: CTL Model CheckingSifakis / Quielle

1982 EMC: Explicit Model CheckerClarke, Emerson, Sistla

1990 Symbolic Model CheckingBurch, Clarke, Dill, McMillan

1992 SMV: Symbolic Model VerifierMcMillan

1998 Bounded Model Checking using SATBiere, Clarke, Zhu

2000 Counterexample-guided Abstraction RefinementClarke, Grumberg, Jha, Lu, Veith

105

10100

101000

1990s: Formal Hardware Verification in Industry:Intel, IBM, Motorola, etc.

28

1

2

3

a

b

c

|| n states,m processes

1,a

2,a 1,b

2,b3,a 1,c

3,b 2,c

3,c

nm states

Main Disadvantage (Cont.)

The State Explosion Problem

System Description

State Transition Graph

Combinatorial explosion of system states renders explicit

model construction infeasible.

Exponential Growth of …… global state space in number of concurrent components.… memory states in memory size.

Feasibility of model checking inherently tied to handling state explosion. 30

Guarded Optimism• Hardware verification (model checking) is now part of engineering practice in the

industry

• The Intel Pentium bug, was the “disaster” that got model checking on the map in the hardware industry

• Synchronous languages like Esterel and Lustre and their analysis suites are well-adopted in avionics and process control industries

• Commercial and non-profit verification enterprises– Big EDA: Synopsys, Mentor Graphics, Cadence– Jasper, Coverity, Galois, SRI, etc.

• Explosive growth in academic research– International conferences: HSCC, EMSoft, ICCPS, CAV, TACAS…

• Plenty of room for research

Formal Verification of Software: A much bigger challenge

What makes Software Model Checking different ?

32

What Makes Software Model Checking Different ?

• Large/unbounded base types: int, float, string• User-defined types/classes• Pointers/aliasing + unbounded #’s of heap-

allocated cells• Procedure calls/recursion/calls through

pointers/dynamic method lookup/overloading• Concurrency + unbounded #’s of threads

33

Grand Challenge: Model Check Software !

Early attempts in the 1980s failed to scale.

2000s: renewed interest / demand:Java Pathfinder: NASA AmesSLAM: MicrosoftBandera: Kansas StateBLAST: BerkeleySPIN…SLAM shipped to Windows device driver developers.

34

What Makes Software Model Checking Different ?

• Large/unbounded base types: int, float, string• User-defined types/classes• Pointers/aliasing + unbounded #’s of heap-

allocated cells• Procedure calls/recursion/calls through

pointers/dynamic method lookup/overloading• Concurrency + unbounded #’s of threads

35

Model Checking

• Algorithm for answering queries about behaviors of state machines– Given a state machine M and a query φ does M satisfy φ ?

• Standard formulation:– M is a Kripke structure– φ is a temporal logic formula

• Computational Tree Logic (CTL)• Linear Temporal Logic (LTL)

• Discovered independently by Clarke & Emerson and Queille & Sifakis in the early 1980’s

Models of C Code

if (x) {y = x;

} else {y = x + 1;

}assert (y);

Program: Syntax Control Flow Graph

y = x + 1

if (x)

y = x

no yes

assert (y)

Model: Semantics

x=1 y=0

x=1 y=1

x=0 y=0

x=0 y=0

x=0 y=1

x=1 y=0

…

…

Infinite State

Existential Abstraction

M

Mα

Given an abstraction function α : S → Sα, the concrete states are grouped and mapped into abstract states:

α α α

Preservation Theorem Theorem (Clarke, Grumberg, Long) If property

holds on abstract model, it holds on concrete model

Technical conditions Property is universal i.e., no existential quantifiers Atomic formulas respect abstraction mapping

Converse implication is not true !

Spurious Behavior

AGAF red“Every path necessarily leads

back to red.”

Spurious Counterexample:<go><go><go><go> ...

“red”

“go”

Artifact of the abstraction !

Automatic Abstraction

MOriginal Model

Refinement

Refinement

Mα Initial AbstractionSpurious

Spuriouscounterexample

Validation orCounterexample Correct !

Example of Existential Abstraction

2

1

3

6

5

4

9

7

10

8

Concrete State

Concrete Transition

Abstract State

Abstract Transition

Abstractly and Concretely Unreachable

Abstractly Reachable but

Concretely Unreachable

Concrete Initial State

Abstract Initial State

Example of Existential Abstraction

p

Concrete State

Concrete Transition

Abstract State

Abstract Transition

Abstractly and Concretely Unreachable

Abstractly Reachable but

Concretely Unreachable

G(~p)

Predicate Abstraction

y = x + 1

if (x)

y = x

no yes

assert (y)

Partition the statespace based on values of a finite set of predicateson program variables

Predicate Abstraction

y = x + 1

if (x)

y = x

no yes

assert (y)

P ≡ ( y == 0 )

:P

States where y ≠ 0

P

States where y = 0

ERROR

φ = G(: ERROR)

CEGAR CounterExample-Guided Abstraction Refinement

Circuit orProgram

InitialAbstraction

Simulator

No erroror bug found

Propertyholds

Simulationsucessful

Bug found

Abstraction refinement Refinement

ModelChecker

Verification

Spurious counterexample

Counterexample

Abstract Model

Software Example: Device Driver Code

Also according to Wired News:

“Microsoft has developed a tool called Static Device Verifier or SDV, that uses ‘Model Checking’ to analyze the source code for Windows drivers and see if the code that the programmer wrote matches a mathematical model of what a Windows device driver should do. If the driver doesn’t match the model, the SDV warns that the driver might contain a bug.”

System Type Formal

specsCore

Problems

DiscreteContinuous

HybridBoolean Logic

Temporal Logic

FSM

Equations

Hybrid AutomataSatisfiability

Coverage

Synthesizability

Formal Verification

Dimensions of this course

Non-Linear

Architecturevalidation

Microcodevalidation

Timingvalidation

Powervalidation

Protocolvalidation

System

Unit

Cluster

Reliabilityvalidation