serdar tasiran systems research center, hp labs (formerly compaq)
DESCRIPTION
Using Formal Specifications to Monitor and Guide Simulation: Verifying the Cache Coherence Engine of the Alpha 21364 Microprocessor. Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/1.jpg)
Using Formal Specifications to Monitor and Guide Simulation:
Verifying the Cache Coherence Engine of the Alpha 21364 Microprocessor
Serdar TasiranSystems Research Center, HP Labs (formerly Compaq)
Yuan Yu (Microsoft Research, formerly Compaq)Brannon Batson, Scott Kreider, (Intel, formerly Compaq)
![Page 2: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/2.jpg)
The Problem
Given A complex protocol specified formally A hardware implementation
Verify that All executions of implementation are consistent
with protocol spec An “implementation verification” problem
Properties of protocol verified separately
![Page 3: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/3.jpg)
Alpha 21364 (EV7) System Block Diagram
1M
IO0
M
IO2
M
IO3
M
IO
5M
IO4
M
IO6
M
IO7
M
IO
9M
IO8
M
IO10
M
IO11
M
IO
Distributed shared memory Up to 256 processors, 32 GB per processor
![Page 4: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/4.jpg)
EV7 Cache Coherence:
Implementation in hardware (~20K lines of HDL code)
C
Z1
mem
systemdata
buffers
R
core&
cache
Z0
mem
EV7
SVDB,FB0,FB1
Cache coherence protocol spec (~3K lines of TLA, written by architects and us)
Executable Not list of properties
Does the hardware implement the protocol spec correctly?
![Page 5: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/5.jpg)
Why is the problem difficult? Thousands of state variables per processor
Parallelism, deep pipelining, speculative execution, redundancy, ...
Need 4+ processors for an interesting system
Out of reach of automatic formal methods
Limited to several hundred state variables
Decomposition methods difficult for non-specialists, large design teams
Complete verification of hardware against protocol not practical
Simulation only viable approach
Even simulation is expensive
![Page 6: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/6.jpg)
Validation Guided by Formal Spec Coverage
Simulation
Correctness checking using
formal spec
Automated input
generation
Coverage analysis on formal spec
![Page 7: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/7.jpg)
Validation Guided by Formal Spec Coverage
Simulation
Correctness checking using
formal spec
“Automated” input
generation
Coverage analysis on formal spec
1
![Page 8: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/8.jpg)
Contributions:
Spec written in formal language Properties of spec can be verified formally Model checker checks properties satisfied during simulation:
More reliable than hand-written code More flexible than automatically generated assertions
Must relate Implementation state Spec state Devised two-phase mapping approach Applicable to complex designs by non-specialists
1: Formal Spec + Model Checker as Monitor
![Page 9: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/9.jpg)
Validation Guided by Formal Spec Coverage
Simulation
Correctness checking using
formal spec
“Automated” input
generation
Coverage analysis on formal spec
2
![Page 10: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/10.jpg)
Contributions
2: Coverage analysis and input generation using formal protocol spec
Formal spec encapsulates design intent Full coverage = All scenarios exercised Spec at same level of abstraction as existing coverage data
Model checker used to Measure coverage, detect gaps Generate simulation input traces to reach coverage holes Determine if unexercised scenario actually possible
![Page 11: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/11.jpg)
Outline
The cache-coherence protocol The EV7 cache-coherence engine Spec-guided simulation Conclusion
![Page 12: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/12.jpg)
The EV7 Coherence Protocol
Distributed shared memory Each address belongs to a
“home node” but may be in other caches
Directory-based protocol Cache states:
Modified (Dirty), Exclusive (Clean), Shared, Invalid Directory states: Local, Shared, Exclusive, Incoherent
Directory distributed with memory at each node CPU requests that miss in local caches are sent to
home node Home node may forward request to other nodes
Directory In Flight Table (DIFT) keeps track of pending requests
1M
IO0
M
IO2
M
IO3
M
IO
5M
IO4
M
IO6
M
IO7
M
IO
9M
IO8
M
IO10
M
IO11
M
IO
![Page 13: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/13.jpg)
Example: write, remote sharers
Conditions: home is remote, directory state is shared
Actions: read-exclusive request to home home sends invalidation requests to sharers, sends data back to
requester with invalidation count (early exclusive reply) sharing nodes reply to requester with invalidation acknowledgements requester proceeds when data arrives, but must “stall” incoming
requests and potential writeback of line until all InvalAcks are received
ReadMod SharedInv
HR S
BlkExclusive
InvalAck
SS
![Page 14: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/14.jpg)
TLA Description for Protocol
Temporal Logic of Actions [Leslie Lamport] Formal language for writing high-level specs of
concurrent, reactive systems Very expressive. Incorporates
First-order logic, set theory, temporal operators Sets, queues, records, tuples, …
EV7 protocol description is a TLA formula
![Page 15: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/15.jpg)
One Protocol Action
ReadMod SharedInv
HR SS
SBlkExclusiveCnt
Preconditions
Messages sentand state variablesupdated
![Page 16: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/16.jpg)
Outline
The cache-coherence protocol The EV7 cache-coherence engine Spec-guided simulation Conclusion
![Page 17: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/17.jpg)
Alpha 21364 Chip Block Diagram
L2DataArray
L2 TagArray
Core
L2 CacheController
L1 Cache
Router
MemoryController 0
MemoryController 1
DataBuffers
DataAddress & Control
IPx4
RDRAM
RDRAM
I/O
![Page 18: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/18.jpg)
Directory in Flight Table (DIFT)
Cache State Lookup (from cache controller)Cache State Lookup (from cache controller)
Directory State from MemoryDirectory State from Memory
CoherenceCoherenceEngineEngine
New directoryNew directory to Memoryto Memory
Forwards andForwards andresponses to responses to other CPUsother CPUs
DIFTDIFT
RequestRequest
Front end of memory controller Tracks up to 32 in-flight transactions
![Page 19: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/19.jpg)
DIFT block diagram
From CBox
To RBox
zc_dft_acc
zc_dft_ros Address File (zx_dft_af*)
addr maf pid vdba vdbv fb directory
Event File
cmd src ack vic lpr
From CBoxFrom Back EndNew Request
Decode
Proto File
rd fwd rsp inv wr vcp akp
Protocol Logic(zx_dft_plc)
Next DIFT State
New DirectoryLogic (zx_dft_plt)
New DirectoryOutput (aod)
To Back End
Issue Logic(zx_dft_isp*)
Grant logic to everywhere
AddressOutputLogic(ao*)
Messagesto the Ring
Rd/Wr Requests tothe Zbox middle
DIFT Free List
DIFT Conflict Array
![Page 20: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/20.jpg)
Outline
The cache-coherence protocol The EV7 cache-coherence engine Spec-guided simulation Conclusion
![Page 21: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/21.jpg)
Validation Guided by Formal Spec Coverage
Simulation
Correctness checking using
formal spec
“Automated” input
generation
Coverage analysis on formal spec
1
![Page 22: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/22.jpg)
Formal Spec as Simulation Monitor
SpecState-Space
fabs
Implementation State-Space
fabs : Abstraction
mappingfabs
Model checker (TLC) checks if transition
is legal
![Page 23: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/23.jpg)
Refinement Mapping Issues Protocol transactions look instantaneous at spec level but in
the implementation they happen over many clock cycles, interleaved with other actions
Want designers, developers to write the mapping
Implementation has parallelism, pipelining, speculative execution, redundancy AND many processors
Burch, Dill style “flushing” not practical
Actioni1 (pj1, ak1)
Actioni2 (pj2, ak2)
Actioni3 (pj3, ak3)
time
![Page 24: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/24.jpg)
The Refinement Mapping
Preconditions
Messages sentand state variablesupdated
Actioni1 (pj1, ak1)
Actioni2 (pj2, ak2)
Actioni3 (pj3, ak3)
time
![Page 25: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/25.jpg)
A two-phase recipe for refinement mappings Collect “tokens”, e.g., cache state looked-up, invalidate message
sent, directory state written1. Implementation state Intermediate state Determine
interfaces: write, read ports state machinesthat relate to protocol state
Watch Messages crossing interfaces Updates to state machines
Record in intermediate state
C
Z1
mem
systemdata
buffers
R
core&
cache
Z0
mem
IO
EV7
Actioni3 (pj3, ak3) time
Record in intermediate state for processor pj3 address ak3
![Page 26: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/26.jpg)
A two-phase recipe for refinement mappings All tokens collected. “Fire” action.
2. Intermediate state Protocol spec state For each protocol transaction, check when
All preconditions hold All state updates happen,
all required messages sent Update abstract state
Actioni3 (pj3, ak3)
time
Preconditions and implementation state updates related to Actioni3 (pj3, ak3)
Abstract state getsupdated here
![Page 27: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/27.jpg)
A two-phase recipe for refinement mappings
1. Implementation state Intermediate state Hardware signal transitions Protocol events (tokens)
2. Intermediate state Protocol spec state Protocol event sequences Protocol transactions (actions)
Component implementers can write step 1 System architects can put together step 2
Distinguishes protocol errors from component implementation errors
Well-defined, clean interface Easier to keep implementation and spec consistent
throughout design process Modular description makes reasoning easier
![Page 28: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/28.jpg)
Validation Guided by Formal Spec Coverage
Simulation
Correctness checking using
formal spec
“Automated” input
generation
Coverage analysis on formal spec
2
![Page 29: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/29.jpg)
Formal Spec as Coverage Model
SpecState-Space
fabs
Implementation State-Space
fabs : Abstraction
mappingfabs
Model checker (TLC) records visited states
![Page 30: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/30.jpg)
Model Checker Tracks and Improves Coverage
SpecState-Space
Non-coveredstate
Path generatedby model checker
Identify parts of spec not exercised by simulation Path in spec state space = unexamined scenario Problems:
Spec has too many states Not feasible to track coverage, generate paths for each
Want to explore “qualitatively distinct” scenarios
![Page 31: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/31.jpg)
Coverage Metric Defined on Spec States
CoverageState-Space
SpecState-Space
c0c1
c2
c3
c4
c5
c6
c7
c8
![Page 32: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/32.jpg)
Coverage Metric Examples All possible directory state transitions:
Invalid Exclusive Shared SharedMask All legal combinations of
Request type Source of request (Local or remote) Cache state Directory state
All possible transitions of some protocol state field in the DIFT
WaitingForAck WaitingForVictim, …
![Page 33: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/33.jpg)
Conclusions
Novel approach uses formal spec and model checker to monitor simulation to identify coverage gaps to guide input generation
Found valuable by architects and verification engineers EV7 verification engineers want to use model checker
to analyze their coverage data EV8 design started with formal specification first! First attempt at verifying industrial implementation
of this scale and complexity against formal spec
![Page 34: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/34.jpg)
Actioni1 (pj1, ak1)
Actioni2 (pj2, ak2)
Actioni3 (pj3, ak3)
time
Actioni3 (pj3, ak3)
time
Preconditions and implementation state updates related to Actioni3 (pj3, ak3)
Abstract state getsupdated here
![Page 35: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/35.jpg)
ReadMod SharedInv
HR SS
SBlkExclusiveCnt
![Page 36: Serdar Tasiran Systems Research Center, HP Labs (formerly Compaq)](https://reader035.vdocuments.us/reader035/viewer/2022062423/56814491550346895db13016/html5/thumbnails/36.jpg)
1M
IO0
M
IO2
M
IO3
M
IO
5M
IO4
M
IO6
M
IO7
M
IO
9M
IO8
M
IO10
M
IO11
M
IO