A brief history ofmodel checking

Ken McMillanCadence Berkeley Labsmcmillan@cadence.com

Outline

• Part I -- Introduction to model checking– Automatic formal verification of finite-state

systems– Applications

• Commercial hardware design• Avionics, chemical plant control, automotive, etc.

• Part II -- A brief history of model checking– Influence of many abstract ideas from logic on

the development of model checking

The Verification Problem

• Debugging chips by simulation...– consumes greater than half of design time,– is unreliable

• “Escapes” can cost up to $500M,

– is increasing in cost as chip densities scale up

2

Model Checking

• input:– temporal logic spec– finite-state model

• output– yes– no +

counterexample

(look ma, no test vectors!)

MC

G(p F q)yes

nop

q

p

q

5

Temporal logic (LTL)• A logical notation that allows to:

– specify relations in time– conveniently express finite control properties

• Temporal operators– G p “henceforth p”– F p “eventually p”– X p “p at the next time”– p W q“p unless q”

6

Types of temporal properties• Safety (nothing bad happens)

G ~(ack1 & ack2) “mutual exclusion”G (req (req W ack)) “req must hold

until ack”

• Liveness (something good happens)

G (req F ack) “if req, eventually ack”

• Fairness

GF req GF ack “if infinitely often req, infinitely often ack”

7

Computation tree logic (CTL)

• Branching time model• Path quantifiers

– A = “for all future paths”– E = “for some future path”

• Example: AF p = “inevitably p”

AFp

p

p

p

9

CTL model checking algorithm• Example: AF p = “inevitably p”

• Complexity– linear in size of model (FSM)– linear in size of specification formula p

Note: LTL is exponential in formula size

AFpAFp

AFpAFp

10

Example: traffic light controller

• Guarantee no collisions• Guarantee eventual service

E

S

N

14

Specifications

• Safety (no collisions)AG (E_Go (N_Go | S_Go));

• LivenessAG ( N_Go N_Sense AF N_Go);

AG ( S_Go S_Sense AF S_Go); AG ( E_Go E_Sense AF E_Go);

• Fairness constraintsinfinitely often (N_Go N_Sense);

infinitely often (S_Go S_Sense); infinitely often (E_Go E_Sense);

(assume each sensor off infinitely often)

15

Counterexample

• East and North lights on at same time...

E_Go

E_Sense

NS_Lock

N_Go

N_Req

N_Sense

S_Go

S_Req

S_Sense

E_ReqN light goes on atsame time S light goesoff.

S takes priority andresets NS_Lock

N light goes on atsame time S light goesoff.

S takes priority andresets NS_Lock

20

State explosion problem

• What if the state space is too large?– too much parallelism– data in model

• Approaches– Abstraction/reduction– “Symbolic” methods– Exploiting symmetry – “Partial order” methods

21

Binary Decision Diagrams

• Ordered decision tree for f = ab + cd

0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 1

d d d d d d d d

c c c c

0 1

0 1 0 1

0 1 0 1 0 1 0 1

b b

a

22

OBDD reduction

• Reduced (OBDD) form:

0 1

d

c

01

0 1

0 1

b

a

0

1

Key idea: combine equivalent subcases

24

Symbolic model checking

• Basic idea:– Use BDD’s to represent sets and relations– Avoid explicitly representing states

• Transition relations

a,b a’,b’R(a,b,a’,b’)

25

Image computation

• EX p = states that can reach p in one step

EXp p

EX p = v’. (R(v,v’) p(v’))

Note: a. f = f |a=0 + f |a=1

26

Fixed point iteration

• EF p = states that can reach p

S0 = pS1...Sw

Si+1 = Si \/ EX Si

...Model checking without building state graph

33

Example: “Gigamax” cache protocol

• First commercial application• Method scales well with system size• Finds very subtle “escapes”

M P P . . .

cluster bus

M P P . . .

. . .

global bus

UIC

UIC

UIC

. . .

Genealogy of model checking

Logics ofPrograms

Temporal/Modal Logics

CTL ModelChecking

SymbolicModel Checking

-automataS1S

LTLMC

ATV

Tarski

-calc

QBF BDD

Many ideas from logic influence development of model checking...

Logics of programs

• Floyd/Hoare/Dijkstra– Give precise definitions of programming

languages– Allows reasoning about programs

(proofs/derivations)– Pre-post conditions/ weakest precondition

• example: assignment axioms

{true} x :=y {x = y}

{P} x := y {P} (no x in P)

• Pnueli– Concurrent vs. sequential programming

– need to characterize execution sequences– proposes use of temporal logic

Concurrent programs

sequential

A B

concurrent

A B

call

ret

Temporal and modal logics

• Roots in philosophical logic– Tense logic -- formalizing linguistic time

“If a, then b before c”

– Modal logic -- reasoning about possibility“If I had run I would have caught my plane”

• New use in computer science:– characterize the interactions of parallel

processesG req F ack

Genealogy

Logics ofPrograms

Temporal/Modal Logics

Pnueli, late 70’s

Floyd/Hoarelate ‘60’s

Aristotle 300’sBCEKripke ‘59

CTL Model checking

• Reasoning about properties of non-deterministic programs– branching time properties of programs– fixed point characterizations (Tarski)

• every monotonic function has least/greatest fixed point

– key idea: apply to finite graphs, not infinite trees• can directly calculate Tarski fixed points

• Applications– finite state machines in hardware– protocols– proved incorrectness of some published designs

Genealogy, cont

Logics ofPrograms

Temporal/Modal Logics

CTL ModelChecking

Tarski

Clarke/EmersonEarly 80’s

50’s

Some published circuits are proved incorrect

Decidable logics and automata

• Büchi– S1S -- reason about sets of natural numbers– Automata on infinite words

• characterize set of models of formula• example: sets that contain the odd numbers

– Deep connection between logics and automata

0,1

0,1

1

0

LTL model checking

• Vardi and Wolper– Apply Büchi’s technique to LTL– Automaton construction yields optimal decision

algorithm

• Kurshan– Specify properties directly as automata

• example: infinitely often p (GFp)

p

true

p

Genealogy

Logics ofPrograms

Temporal/Modal Logics

CTL ModelChecking

-automataS1S

LTLMC

ATV

TarskiBüchi, 60

Kurshan Vardi/Wolper

mid 80’s

Symbolic Model Checking

• State explosion problem– graph model guarantees worst-case complexity

• Characterize sets and relations by Boolean formulas– compute Tarski fixed points directly on formulas

– Use BDD’s to represent formulas• efficient canonical form

EXp = v. (R p(QBF)

Mu-calculus

• Park’s Mu-Calculus– Logic of relations with fixed point operator– Can express transitive closure– Nicely characterizes what SMC can compute

• SMC algorithm for Mu-calculus

– Use to express symbolic algorithms for• CTL, LTL model checking

• Automaton containment, etc...

– Note: bad specification logic, but good for describing algorithms

AFp = Q. p AX Q

Genealogy, cont.

– Note first commercial application in 1990• Encore Gigamax cache protocols

Logics ofPrograms

Temporal/Modal Logics

CTL ModelChecking

SymbolicModel Checking

-automataS1S

LTLMC

ATV

Tarski

-calc

QBF BDD

Park60’s

Bryantmid 80’s

late 80’s

Applications

• Hardware Design– Encore Gigamax– Intel instruction decoder– SGI cache protocol chip

• Other areas– Avionics (TCAS)– Chemical plant control– Nuclear storage facilities (!)

• Commercial tools– Cadence, IBM, Synopsys

A convergence of research areas in logic

• Many areas of logic have shaped the discourse in model checking– Logics of programs – Temporal/Modal logics– Tarski fixed point theory– Decidable logics -- S1S/automata– Park’s mu-calculus

• Much of this work is quite abstract, but has strongly influenced practical work in model checking