Carnegie Mellon University

Formal VerificationFormal Verificationof Infinite-State Systemsof Infinite-State SystemsUsing Boolean MethodsUsing Boolean Methods

Formal VerificationFormal Verificationof Infinite-State Systemsof Infinite-State SystemsUsing Boolean MethodsUsing Boolean Methods

http://www.cs.cmu.edu/~bryant

Randal E. Bryant

Contributions by former graduate students:Sanjit Seshia, Shuvendu Lahiri

– 2 –

OutlineOutline

TaskTask Formally verify abstract models of hardware and software

systems Build on success in verifying finite models

Infinite-State ModelsInfinite-State Models Need logic that is suitably expressive, yet remains

reasonably tractable

Verification TechniquesVerification Techniques Solve problems by mapping into propositional logic

Proof engines can use powerful Boolean methods

Different levels of automation and capacity

– 3 –

Theoretically Infinite-State SystemsTheoretically Infinite-State Systems

Systems with unbounded buffersEven though can’t really build one

• • • ••• •••

tailtail headhead

In Use

– 4 –

Arbitrarily Large Finite-State Systems Arbitrarily Large Finite-State Systems

Synchronization protocol that should work for arbitrary number of processes

Verify for arbitrary N

Circular buffer with fixed, but arbitrary capacity

Verify for arbitrary value of Max

••••••

tailtailheadhead

In Use

•••

0 0 Max-1Max-1

P 2 • • • P NP 1

– 5 –

Existing Automatic Verification MethodsExisting Automatic Verification Methods

Simulators, model checkers, …

All Operate at Bit LevelAll Operate at Bit Level State model

State encoded as words and arrays of wordsComprised of bits

Must track how each bit of state gets updated

Only Verify Single Instance of DesignOnly Verify Single Instance of Design Fixed values for parameters

Word sizeBuffer sizesNumber of processes

– 6 –

What About Theorem Provers?What About Theorem Provers?

Traditional Tool for Formal VerificationTraditional Tool for Formal Verification Allow many forms of abstraction

Hard to UseHard to Use Lots of manual effort & expertise required

Question:Question: Can we incorporate some of these abstraction abilities into

an automated tool?

– 7 –

Data Abstraction #1: Bits → IntegersData Abstraction #1: Bits → Integers

View Data as Symbolic WordsView Data as Symbolic Words Arbitrary integers

No assumptions about size or encodingClassic model for reasoning about software

Can store in memories & registers

x0

x1

x2

xn-1

x

– 8 –

Data PathData Path

Com.Log.

1

Com.Log.

2

Abstracting Data BitsAbstracting Data Bits

Control LogicControl Logic

Data PathData Path

Com.Log.

1

Com.Log.

1? ?

What do we do about logic functions?

– 9 –

Abstraction #2:Uninterpreted Functions

Abstraction #2:Uninterpreted Functions

For any Block that Transforms or Evaluates Data:For any Block that Transforms or Evaluates Data: Replace with generic, unspecified function Only assumed property is functional consistency:

a = x b = y f (a, b) = f (x, y)

ALUf

– 10 –

Abstracting FunctionsAbstracting Functions

For Any Block that Transforms Data:For Any Block that Transforms Data: Replace by uninterpreted function Ignore detailed functionality Conservative approximation of actual system

Data PathData Path

Control LogicControl Logic

Com.Log.

1

Com.Log.

1F1 F2

– 11 –

Modeling Data-Dependent ControlModeling Data-Dependent Control

Model by Uninterpreted PredicateModel by Uninterpreted Predicate Yields arbitrary Boolean value for each control + data

combination Produces same result when arguments match

Cond

Adata

Bdata

Branch?

Bra

nch

Lo

gic

p

– 12 –

Abstraction #3: Modeling Memories as Mutable FunctionsAbstraction #3: Modeling Memories as Mutable Functions

Memory M Modeled as FunctionMemory M Modeled as Function

M(a): Value at location a

InitiallyInitially

Arbitrary state Modeled by uninterpreted function m0

Ma

M

a m0

– 13 –

Effect of Memory Write OperationEffect of Memory Write Operation

Writing Transforms MemoryWriting Transforms Memory M = Write(M, wa, wd)

Reading from updated memory M(a):Address wa will get wdOtherwise get what’s already in M

M

Ma 1

0

wd

=wa

– 14 –

Systems with BuffersSystems with Buffers

Modeling MethodModeling Method Mutable function to describe buffer contents Integers to represent head & tail pointers

• • • ••• •••

tailtail headhead

In Use

••••••

tailtailheadheadheadhead

In Use

•••

0 0 0 0 MaxMax--11MaxMax--11

Unbounded Buffer Circular Queue

– 15 –

UCLIDUCLID

Seshia, Lahiri, Bryant, CAV ‘02

Term-Level Verification SystemTerm-Level Verification System Language for describing systems

Inspired by CMU SMV

Symbolic simulatorGenerates integer expressions describing system state after

sequence of steps

Decision procedureDetermines validity of formulas

Support for multiple verification techniques

Available by DownloadAvailable by Downloadhttp://www.cs.cmu.edu/~uclid

– 16 –

System ModelSystem Model

State Variable TypesState Variable Types Boolean

Control signals

Integer Data, addresses

Function Memories, buffers

System OperationSystem Operation Synchronous

All state variables updated on each step of operation

Interleaving One (set of) state variable(s) updated at a time Simulate in synchronous model with uninterpreted scheduling function

PresentState

NextState

Inputs(Arbitrary)

Reset

– 17 –

Pipeline

Modeling ExampleModeling ExampleDLX PipelineDLX Pipeline

Single-issue, 5-stage pipeline

pc

pPC

fd

Valid

PC

Type

Instr

de

Valid

PC

Type

Instr

Arg2

Arg1

mw

Valid

Dest

Data

em

Branch

Valid

Type

Instr

Arg2

Value

Target

RF Mem

Fetch Decode Execute MemoryWriteBack

Integer state

Boolean state

Function state

– 18 –

Writing & Reading Register FileWriting & Reading Register File

fd

Instr

de

Arg2

Arg1

mw

Valid

Dest

Data

RF

DecodeWriteBack

src1

src2

– 19 –

Writing Register FileWriting Register File

mw

Valid

Dest

Data

RF

WriteBack

init[RF] := rf0; (* Uninterpreted Function *) next[RF] := Lambda(a) . case mw_Valid & (a = mw_Dest) : mw_Data; default : RF(a); esac;

– 20 –

Reading Register FileReading Register File

fd

Instr

de

Arg2

Arg1

RF

Decode

src1

src2

init[de_Arg1] := dea10; (* Initially arbitary *) next[de_Arg1] := next[RF](src1(fd_Instr)); init[de_Arg2] := dea20; (* Initially arbitary *) next[de_Arg2] := next[RF](src2(fd_Instr));

Write-before-read semantics

– 21 –

Underlying LogicUnderlying LogicScalar Data TypesScalar Data Types

Formulas (F ) Boolean ExpressionsControl signals

Terms (T ) Integer ExpressionsData values

Functional Data TypesFunctional Data Types Functions (Fun) Integer Integer

Immutable: Functional unitsMutable: Memories

Predicates (P) Integer Boolean Immutable: Data-dependent controlMutable: Bit-level memories

– 22 –

CLU LogicCLU Logic Counter Arithmetic, Lambda Expressions and Uinterpreted

Functions

Terms (Terms (T T )) Integer ExpressionsInteger ExpressionsITE(F, T1, T2) If-then-else

Fun (T1, …, Tk) Function application

succ (T) Increment

pred (T) Decrement

Formulas (Formulas (F F )) Boolean ExpressionsBoolean ExpressionsF, F1 F2, F1 F2 Boolean connectives

T1 = T2 Equation

T1 < T2 Inequality

P(T1, …, Tk) Predicate applicationTo support pointer operations

– 23 –

CLU Logic (Cont.)CLU Logic (Cont.)Functions (Functions (FunFun)) Integer Integer Integer Integer

f Uninterpreted function symbol

x1, …, xk . T Function definition

Predicates (Predicates (PP)) Integer Integer Boolean Booleanp Uninterpreted predicate

symbol

x1, …, xk . F Predicate definition

– 24 –

Decision ProblemDecision ProblemCircuit Representation of FormulaCircuit Representation of Formula

Truth ValuesDashed LinesModel ControlLogical connectivesEquations

Integer ValuesSolid linesModel DataUninterpreted functions If-Then-Else operation

TaskTask Determine whether formula F is universally valid

True for all interpretations of variables and function symbolsOften expressed as (un)satisfiability problem

» Prove that formula F is not satisfiable

=

f

T

F

T

F

fT

F

=

e1

e0x0

d0

T

F

T

F

T

F

e1

e0x0

d0

=

f

f

=

– 25 –

=

f

T

F

T

F

fT

F

=

e1

e0x0

d0

T

F

T

F

T

F

e1

e0x0

d0

=

f

f

=

Finite Model PropertyFinite Model Property

ObservationObservation Any formula has limited number of distinct expressions Only property that matters is whether or not different terms

are equal

x0 d0 f (x0) f (d0)

– 26 –

Boolean Encoding of Integer ValuesBoolean Encoding of Integer Values

For Each ExpressionFor Each Expression Either equal to or distinct from each preceding expression

Boolean EncodingBoolean Encoding Use Boolean values to encode integers over small range CLU formula can be translated into propositional logic

Logic circuit with multiplexors, comparators, logic gates Tautology iff original formula valid

ExpressionExpression Possible Possible ValuesValues

Bit Bit EncodingEncoding

x0 {0}{0} 00 00

d0 {0,1}{0,1} 00 bb1010

f (x0) {0,1,2}{0,1,2} bb2121 bb2020

f (d0) {0,1,2,3}{0,1,2,3} bb3131 bb3030

– 27 –

Recent Progress in SAT SolvingRecent Progress in SAT Solving

766

147 118 81 46

3600

0

1,000

2,000

3,000

Gra

sp (2

000)

zChaf

f (200

1)

BerkM

in (2

002)

zChaf

f (200

3-04

)

Siege

(200

4)

SatElit

eGTI (

2005

)

Ru

n-t

ime

(sec

.)

– 28 –

ReachableStates

Verifying Safety PropertiesVerifying Safety Properties

Prove: System will never reach bad stateProve: System will never reach bad state

ResetStates

BadStates

PresentState

NextState

Inputs(Arbitrary)

Reset

– 29 –

Reachable

• • •

Rn

R2

Bounded Model CheckingBounded Model Checking

Repeatedly Perform Image Repeatedly Perform Image ComputationsComputations Set of all states reachable

by one more state transition

Easy to ImplementEasy to Implement

Underapproximation of Underapproximation of Reachable State SetReachable State Set But, typically catch most

bugs with 8–10 steps

BadStates

R1

ResetStates

– 30 –

Implementing BMCImplementing BMC

Construct verification condition formula for step n by symbolically simulating system for n cycles

Check with decision procedure Do as many cycles as tractable

S

X1 X2 Xn

Bad

ResetSatisfiable?

– 31 –

• • •

Rn

R2

True Model CheckingTrue Model Checking

Reach Fixed-PointReach Fixed-Point Rn = Rn+1 = Reachable

Impractical for Term-Level Impractical for Term-Level ModelsModels

Many systems never reach fixed point

Can keep adding elements to buffer

Convergence test undecidable

BadStates

R1

ResetStates

– 32 –

I

Inductive Invariant CheckingInductive Invariant Checking

Key Properties of System that Make it Operate CorrectlyKey Properties of System that Make it Operate Correctly Formulate as formula I

Prove InductiveProve Inductive Holds initially I(s0)

Preserved by all state changes I(s) I((i, s))

ReachableStates

ResetStates

BadStates

– 33 –

An Out-of-order Processor (OOO)An Out-of-order Processor (OOO)

Data Dependencies Resolved by Register RenamingData Dependencies Resolved by Register Renaming Map register ID to instruction in reorder buffer that will generate

register value

Inorder Retirement Managed by Retirement BufferInorder Retirement Managed by Retirement Buffer FIFO buffer keeping pending instructions in program order

Reorder BufferFields

PC

Programmemory

Reorder Buffer

validvaluesrc1validsrc1valsrc1tagsrc2validsrc2valsrc2tagdestop

result bus

DECODE Register

Rename Unit

valid tag val

ALU

head tail

incrdispatch

retire

execute

result

1st

Operand

2nd

Operand

– 34 –

Verifying OOOVerifying OOO Lahiri, Seshia, & Bryant,

FMCAD 2002

GoalGoal Show that OOO implements

Instruction Set Architecture (ISA) model

For all possible execution sequences

ChallengeChallenge OOO holds partially executed

instructions in reorder bufferStates of two systems match

only when reorder buffer flushed

ISA

Reg.File

PC

OOO

Reg.File

PCReorder Buffer

– 35 –

Adding Shadow StateAdding Shadow State

McMillan, ‘98 Arons & Pnueli, ‘99

Provides Link Between ISA Provides Link Between ISA & OOO Models& OOO Models Additional info. in ROB

Do not affect OOO behavior

Generated when instruction dispatched

Predict values of operands and result

From ISA model

ISA

Reg.File

PC

OOO

Reg.File

PCReorder Buffer

– 36 –

Invariant CheckingInvariant Checking

Formulas Formulas II11, …, , …, IInn

Ij(s0) holds for any initial state s0, for 1 j n

I1(s) I2(s) … In(s) Ij(s ) for any current state s and successor state s for 1 j n

Invariants for OOO (13)Invariants for OOO (13) Refinement maps (2)

Show relation between ISA and OOO models

Shadow state (3)Shadow values correctly predict OOO values

State consistency (8)Properties of OOO state that ensure proper operation

Overall CorrectnessOverall Correctness Follows by induction on time

– 37 –

State Consistency Invariant Examples State Consistency Invariant Examples

Register Renaming invariants (2)Register Renaming invariants (2) Any mapped register should be in the ROB, and the

destination register should matchr.reg.valid(r) (rob.head reg.tag(r) < rob.tail

rob.dest(reg.tag(r)) = r )

For any ROB entry, the destination should have reg.valid as false and tag should be to this or later instructionrobt.(reg.valid(rob.dest(t))

t reg.tag(rob.dest(t)) < rob.tail)

– 38 –

Extending the OOO ProcessorExtending the OOO Processor

baseExecutes ALU instructions only

excHandles arithmetic exceptionsMust flush reorder buffer

exc/brHandles branchesPredicts branch & speculatively executes along path

exc/br/mem-simpAdds load & store instructionsStore commits as instruction retires

exc/br/memStores held in bufferCan commit laterLoads must scan buffer for matching addresses

– 39 –

Comparative Verification EffortComparative Verification Effort

base exc exc / br exc / br / mem-simp

exc / br / mem

Total Invariants

13 34 39 67 71

UCLID time

54 s 236 s 403 s 1594 s 2200 s

Person time

2 days 7 days 9 days 24 days 34 days

(Person time shown cumulatively)

– 40 –

“I Just Want a Loaf of Bread”“I Just Want a Loaf of Bread”Ingredients

Recipe Result

– 41 –

Cooking with InvariantsCooking with InvariantsIngredients: Predicates

Recipe: Invariants

Result: Correctness

reg.valid(r)

r,t.reg.valid(r) reg.tag(r) = t (rob.head reg.tag(r) < rob.tail rob.dest(t) = r )

rob.head reg.tag(r)

reg.tag(r) = t

rob.dest(t) = r

– 42 –

Automatic Recipe GenerationAutomatic Recipe Generation

Want Something MoreWant Something More Given any set of ingredients Generate best recipe possible

Ingredients

Recipe Creator Result

– 43 –

Automatic Predicate AbstractionAutomatic Predicate Abstraction

Graf & Saïdi, CAV ‘97

IdeaIdea Given set of predicates P1(s), …, Pk(s)

Boolean formulas describing properties of system state

View as abstraction mapping: States {0,1}k

Defines abstract FSM over state set {0,1}k

Form of abstract interpretationDo reachability analysis similar to symbolic model checking

ImplementationImplementation Early ones had weak inference capabilities

Call theorem prover or decision procedure to test each potential transition

Recent ones make better use of symbolic encodings

– 44 –

Abstract State SpaceAbstract State Space

ConcreteStates

AbstractStates

P1(s), …, Pk(s)

s

AbstractionFunction

t

Abstraction

ConcreteStates

AbstractStates

s t

ConcretizationFunction

Concretization

– 45 –

Abstract State MachineAbstract State Machine

Transitions in abstract system mirror those in concrete

Abstract

ConcreteSystem

AbstractSystem

s

Concretize

t t

s

Concrete Transition

Abstract Transition

– 46 –

Generating Concrete InvariantGenerating Concrete Invariant

Reach Fixed-Point on Reach Fixed-Point on Abstract SystemAbstract System Termination guaranteed,

since finite state

Equivalent to Computing Equivalent to Computing Invariant for Concrete Invariant for Concrete SystemSystem Strongest possible

invariant that can be expressed by formula over these predicates

• • •Rn

R2

R1

ResetStates

A

AbstractSystem

Concretize

ConcreteSystem

I

ResetStates

C

– 47 –

Quantified Invariant GenerationQuantified Invariant Generation(Lahiri & Bryant, VMCAI 2004) User supplies predicates containing free variables Generate globally quantified invariant

ExampleExample Predicates

p1: reg.valid(r)

p2: rob.dest(t) = r

p3: reg.tag(r) = t

Abstract state satisfying (p1 p2 p3) corresponds to concrete state satisfying r,t[reg.valid(r) reg.tag(r) = t

rob.dest(t) = r] rather than r[reg.valid(r)] r,t[reg.tag(r) = t]

r,t[rob.dest(t) = r]

– 48 –

Systems Verified with Predicate AbstractionSystems Verified with Predicate Abstraction

Safety properties only

Model Predicates Iterations CPU Time

Out-Of-Order Execution Unit 25 9 1,207s

German’s Cache Protocol 13 9 14s

German’s Protocol, unbounded channels

24 17 427s

Lamport’s Bakery Algorithm 33 18 471s

– 49 –

Future ProspectsFuture Prospects

EvaluationEvaluation Demonstrated ability to verify complex, parameterized

systems

Predicate Abstraction Shows PromisePredicate Abstraction Shows Promise Provides key automation advantage of model checking

Successful Application to Program ApplicationSuccessful Application to Program Application Qadeer & Lahiri, POPL ’06 Generate loop invariants for list manipulation programs

– 50 –

– 51 –

Automatic Predicate DiscoveryAutomatic Predicate Discovery

Strength of Predicate AbstractionStrength of Predicate Abstraction If give it right set of predicates, PA will put them together

into invariant

WeaknessWeakness Gets nowhere without right set of predicates Typical failure mode: Generate “true” as invariant

ChallengesChallenges Too many predicates will overwhelm PA engine Our use of quantified invariants precludes counterexample-

generated refinement techniques

– 52 –

Implementation of Predicate DiscoveryImplementation of Predicate Discovery

Lahiri & Bryant, CAV ’04 Initially: Extract predicates from verification condition Iterate: Add new predicates by composing next-state

formulasWith some heuristics thrown in

ExperienceExperience Can automatically generate invariants for real examples ~10X slower than for hand-selected predicates