formal methods jeannette m. wing computer science department carnegie mellon university

Formal Methods

Jeannette M. Wing

Computer Science Department

Carnegie Mellon University

Formal Methods Overview Jeannette M. Wing2

What are Formal Methods?

Formal methods are mathematically based languages, techniques, and tools for specifying and verifying hardware and software systems.

system

property

yes / no / don’t know

verifier

specifications

The Enterprise of Formal Methods


Power of Formal Methods

• Mathematical properties• Precise, concise, consistent, unambiguous.

• Expressive power• Not necessarily executable, quantifiers, infinite domains, abstraction.

• Predictive power• Reason in terms of model of system.

• Amenable to machine analysis• Induction, exhaustive case analysis.


Formal Methods: Foci of Research Energy

• Specification languages, for describing– Program modules, e.g., pre/post-conditions– Software design, e.g., Z– Protocols, e.g., state machines– Properties, e.g., temporal logics, automata

• Verification techniques– Model checking– Theorem proving

• Application domains– Hardware: circuit-level verification– Software: C programs, safety-critical systems, security properties,

system architecture– Embedded systems


Model Checking: Overview

System (Finite Model)

Property

Counterexample

Model Checker

No

Yes• States and Transitions

• Temporal Logic

• Abstract Automaton

• Explicit State

• Symbolic

• Trace


Glimpse at Six Projects at Carnegie Mellon

• Bounded model checking (Ed Clarke, CSD)– Application area: embedded systems, C code

• Verifying system invariants via predicate abstraction (Bryant, CSD)– Application area: hardware, cache coherence, synchronization protocols

• Hybrid systems model checking (Bruce Krogh, ECE)– Application area: embedded systems

• Probabilistic model checking (Reid Simmons, CSD/Robo)– Application area: hybrid dynamic systems

• Model checking to generate attack graphs (Jeannette Wing, CSD)

– Application area: security

• Model checking for software architecture (David Garlan, ISRI)– Application area: self-healing systems


CBMC: Embedded Systems Verification

• Method:Bounded Model Checking

• Implemented GUI to make it look like debugger

• Applications:– Part of train controller from GE– Cryptographic algorithms (DES,

AES, SHS)– C Models of ASICs provided by

nVidia


MAGIC

RequirementsRequirements SpecificationValidation

SpecificationValidationSpecificationSpecification

CodeCode CodeValidation

CodeValidation

ConformanceCheck

ConformanceCheck

• Verify C programs against finite state machine specs

– Automated abstraction refinement• Concurrency

– Compositionality• Protocols, controllers, OS

• OpenSSL• Micro-C OS

– 6000 LOC– Bug found

• Industrial IPC module– Over 30 billion states– Bug found despite years of

testing • Metal casting controller

– 30 KLOC

www.cs.cmu.edu/~chaki/magic

Predicate Abstraction


Abstract System

Concrete System

Verifying System Invariants Via Predicate Abstraction

process state array •••••• •••i j

i,j ( state[i] = critical state[j] = critical i = j )

MutualExclusion:

PijPi Pj

AbstractMapping


Implementing Predicate Abstraction

• Algorithms Inspired by Symbolic Model Checking– Determine set of reachable states in abstract model

• Expand breadth-first from initial state set until converge

– Determine whether invariant holds for all reachable states• Implementation

– Encode abstraction & transition as first-order predicate logic formula

– Heuristically instantiate quantifiers– Translate into Boolean formula– Extract next states with Boolean satisfiability solver


Systems Verified with Predicate Abstraction

– Very general models• Unbounded processes, buffers, cache lines, …

– Safety properties only

Model Predicates Iterations CPU Time

Out-Of-Order Execution Unit 25 9 2,613s

German’s Cache Protocol 21 9 122s

German’s Protocol, unbounded channels 30 19 15,000s

Bounded Retransmission Buffer 22 9 11s

Lamport’s Bakery Algorithm 24 24 5,211s

Model Checking Hybrid Systems


Verification of Hybrid Systems

• Hybrid Systems: mixed discrete & continuous variables

– embedded control systems (continuous dynamic environment)

– mixed-signal circuits (analog + digital)

• Objective: Develop effective methods to extend model checking techniques from finite-state systems to hybrid systems

• Solution approach:

– Construct finite-state abstractions for the (infinite state) hybrid system

– Apply model checking to the conservative abstraction

– Refine the abstraction if necessary


CheckMate: Hybrid System Verification Tool

MATLAB/Simulink model1. Constructs finite-state

abstraction with transition relation based on polyhedral representations of continuous flows

Specifications over discrete states• Reachability• ACTL

Polyhedral sets of initial continuous states & parameters

('1,p',q')

'1'2

('2,p',q')

(,p,q)

p p'

q q'

('1,p',q')

'1'2

('2,p',q')

(,p,q)

p p'

q q'

2. Applies model checking to resulting transition system.

3. R

efin

es a

bstr

actio

n if

nece

ssar

y.

www.ece.cmu.edu/~webk/checkmate/


Recent Application of CheckMate: Delta-Sigma ADC (mixed-signal circuit)

Hyperplanes defining various regions for the

quantizer input

“zero_threshold” : x > 0

“overload” : -2 < x < 2

Noise-Shaping & LPF Filters

Hyperplane defining the desired region of the LPF

“LPF_okay” : -0.1 < x < 0.1

QuantizerFSM

Low Pass FilterFSM

quantizerthreshold

quantizer overload(first violations)

Circuit Simulation Model

Reachability ResultsCheckMate Model

Probabilistic Model Checking


StatisticalProbabilistic Model Checking

• Verification of stochastic systems– “Is the probability greater than 0.1 that the system will fail in

the next 60 minutes?”• Why statistical approach?

– Insensitive to size of system– Easy to trade accuracy for speed– Easy to parallelize


Statistical Solution Method

• Use sequential acceptance sampling to verify probabilistic properties

RejectReject

AcceptAccept

Continue samplingContinue sampling

Number of samples

Nu

mb

er

of

posi

tive s

am

ple

s Acceptance line

Rejection line

Start here

Generate samplesusing simulation

Continue until aline is crossed


Numerical vs. Statistical Probabilistic Model Checking

T=40 (numerical)T=20 ( " )T=10 ( " )T=40 (statistical)T=20 ( " )T=10 ( " )

Veri

fica

tion

tim

e (

seco

nds)

Size of state space

10−2

10−1

100

101

102

103

104

105

106

102 104 106 108 1010 1012 1014

==10−2

=0.5·10−2

=10−6

serv1 Pr≥0.5(true U≤T poll1)

Model Checking Applied to Security


Example of Attack Graph Developedby Professional Red Team

• Sandia Red Team “White Board” attack tree from DARPA CC20008 Information battle space preparation experiment

Sandia Red Team “White Board” attack graph from DARPA CC20008 Information battle space preparation experiment

Drawn By Hand


Automatic Generation of Attack Graphs

Model Checker

Model of Target System and Attacker

• finite state machine

Statement of Threat

• negation of secure-state property,


Performance (Explicit-State)

Linear Regression R2 = 0.9967

0

5

10

15

20

25

30

35

40

45

0 100000 200000 300000 400000 500000 600000 700000 800000 900000

Reachable Transitions (Edges)

Gen

era

tio

n T

ime (

sec)

Linear coefficient 1.12 x 10-4

Model Checking Applied to Self-Healing Systems


Understanding Self-Healing Systems

Increasingly, systems– are composed of parts built by many organizations– must run continuously– operate in environments where resources change frequently– are used by mobile users

For such systems, traditional methods break down– exhaustive verification and testing is not possible– manual reconfiguration does not scale– off-line repair and enhancement is not an option

New requirement: systems must automatically adapt to handle – changes in user needs,variable resources, faults, mobility

But how?


Approach

Maintain formal system models at run time as a basis for – monitoring– problem detection– repair

ConstraintEvaluator

RepairHandler

Interpreter

Model Layer

Formal Model

GenericAPI

MonitoringMechanisms

Executing System

Implementation Layer

Translator

RuntimeManager


Recent (1996-2003) Grads

• Robert Allen (Garlan), IBM/Vermont– A Formal Approach to Software Architecture

• Sergey Berezin (Clarke) Stanford– Model Checking and Theorem Proving: A Unified Framework

• Sergio Campos (Clarke), Federal University of Minas Gerais, Brazil– A Quantitative Approach to the Formal Verification of Real-Time Systems

• Yirng-An Chen (Bryant), Synopsis– Arithmetic Circuit Verification Based on Word-Level Decision Diagrams

• Craig Damon (Wing, Jackson), University of Vermont– Selective Enumeration

• Somesh Jha (Clarke), University of Wisconsin– Symmetry and Induction in Model Checking

• Darrell Kindred (Wing), Network Associates Laboratories– Theory Generation for Security Protocols

• Charles Krueger (Garlan, Habermann), BigLever Software– Modeling and Simulating a Software Architecture Design Space


Recent (1996-2003) Students

• Will Marrero (Clarke) DePaul University– Brutus: A Model Checker for Security Protocols

• Marius Minea (Clarke) Politehnica University of Timisoara– Partial Order Reduction for Verification of Timed Systems

• Bob Monroe (Garlan), FreeMarkets– Capturing Software Architecture Design Expertise with Armani

• Rob O’Callahan (Wing), IBM/Hawthorne– Generalized Aliasing as a Basis for Program Analysis Tools

• John Ockerbloom (Garlan), University of Pennsylvania– Mediating Among Diverse Data Formats

• Bridget Spitznagel (Garlan)– Compositional Transformation of Software Connectors

• Hao-Chi Wong (Wing), Federal University of Minas Gerais, Brazil– Protecting Individuals’ Interests in Electronic Commerce Protocols

• Xudong Zhao (Clarke), Intel– Verification of Arithmetic Circuits


Faculty and Their Current Students (CSD)

• Randy Bryant (CSD)– Miroslav Velev, Sanjit Seshia, Amit Goel, Shuvendu Lahiri, Nitin Sharma

• Ed Clarke (CSD)– Sagar Chaki, Pankajkumar Chauhan, Flavio Lerda, Anubhav Gupta, Alex

Groce, Stephen Magill, Nishant Sinha, Muralidhar Talupur

• David Garlan (ISRI)– Owen Chang, George Fairbanks, Jung Soo Kim, Vahe Poladian, Joao

Pedro Sousa, Hong Yan, Wei Zhang

• Bruce Krogh (ECE)– Smriti Gupta, Zhi Han, James Kapinksi, Rajesh Kumar, Haotian Zhang

• Reid Simmons (Robotics/CSD)– Allison Bruce, Rachel Gockley, Marek Michalowski, Maayan Roth, Brennan

Sellner, Trey Smith, Christopher Urmson, Vandi Verma, Hakan Younes

• Jeannette Wing (CSD)– Arvind Kannan, Pratyusa Manadhata, Oleg Sheyner (defending April 14!!!),

Meera Sridhar


For More Information

• Specification and Verification Center http://www.cs.cmu.edu/~svc

http://www.cs.cmu.edu/~svc

formal methods jeannette m. wing computer science department carnegie mellon university

Documents