lecture 23: thread level parallelism --introduction, smp ... · 2 topics for thread level...
Post on 19-Jul-2020
5 Views
Preview:
TRANSCRIPT
Lecture 23: Thread Level Parallelism-- Introduction, SMP and Snooping Cache
Coherence Protocol
CSCE 513 Computer Architecture
Department of Computer Science and Engineering
Yonghong Yanyanyh@cse.sc.edu
https://passlab.github.io/CSCE513
2
Topics for Thread Level Parallelism (TLP)§ Parallelism (centered around … )
– Instruction Level Parallelism– Data Level Parallelism– Thread Level Parallelism
§ TLP Introduction– 5.1
§ SMP and Snooping Cache Coherence Protocol– 5.2, 5.3
§ Distributed Shared-Memory and Directory-Based Coherence– 5.4
§ Synchronization Basics and Memory Consistency Model– 5.5, 5.6
§ Others
3
Moore’s Law• Long-term trend on the density of transistor per integrated
circuit• Number of transistors/in2 double every ~18-24 month
4
What do we do with that many transistors?
§ Optimizing the execution of a single instruction stream through– Pipelining
» Overlap the execution of multiple instructions» Example: all RISC architectures; Intel x86 underneath the
hood– Out-of-order execution:
» Allow instructions to overtake each other in accordance with code dependencies (RAW, WAW, WAR)
» Example: all commercial processors (Intel, AMD, IBM, Oracle)
– Branch prediction and speculative execution: » Reduce the number of stall cycles due to unresolved
branches» Example: (nearly) all commercial processors
5
What do we do with that many transistors? (II)
– Multi-issue processors: » Allow multiple instructions to start execution per clock
cycle» Superscalar (Intel x86, AMD, …) vs. VLIW architectures
– VLIW/EPIC architectures: » Allow compilers to indicate independent instructions per
issue packet» Example: Intel Itanium
– SIMD units:» Allow for the efficient expression and execution of vector
operations» Example: Vector, SSE - SSE4, AVX instructions
Everything we have learned so far
6
Limitations of optimizing a single instruction stream
§ Problem: within a single instruction stream we do not find enough independent instructions to execute simultaneously due to– data dependencies– limitations of speculative execution across multiple branches– difficulties to detect memory dependencies among
instruction (alias analysis)§ Consequence: significant number of functional units are
idling at any given time § Question: Can we maybe execute instructions from
another instructions stream – Another thread?– Another process?
7
Thread-level parallelism§ Problems for executing instructions from multiple
threads at the same time– The instructions in each thread might use the same register
names– Each thread has its own program counter
§ Virtual memory management allows for the execution of multiple threads and sharing of the main memory
§ When to switch between different threads:– Fine grain multithreading: switches between every instruction– Course grain multithreading: switches only on costly stalls
(e.g. level 2 cache misses)
8
Power, Frequency and ILP§ Moore’s Law to processor speed (frequency)
Note: Even Moore’s Law is ending around 2021:
http://spectrum.ieee.org/semiconductors/devices/transistors-could-stop-shrinking-in-2021
https://www.technologyreview.com/s/601441/moores-law-is-dead-now-what/
http://www.forbes.com/sites/timworstall/2016/07/26/economics-is-important-the-end-of-moores-law
CPU frequency increase was flattened around 2000-2005
Two main reasons:1. Limited ILP and 2. Power consumption and
heat dissipation
9
History – Past (2000) and Today
10
Flynn’s Taxonomy
https://en.wikipedia.org/wiki/Flynn%27s_taxonomy
✔✔
✔
11
Examples of MIMD Machines§ Symmetric Shared-Memory
Multiprocessor (SMP)– Multiple processors in box with
shared memory communication– Current Multicore chips like this– Every processor runs copy of OS
§ Distributed/Non-uniform Shared-Memory Multiprocessor – Multiple processors
» Each with local memory» general scalable network
– Extremely light “OS” on node provides simple services
» Scheduling/synchronization– Network-accessible host for I/O
§ Cluster– Many independent machine
connected with general network – Communication through messages
P P P P
Bus
Memory
P/M P/M P/M P/M
P/M P/M P/M P/M
P/M P/M P/M P/M
P/M P/M P/M P/M
Host
Network
12
Symmetric (Shared-Memory) Multiprocessors (SMP)
§ Small numbers of cores– Typically eight or fewer, and
no more than 32 in most cases§ Share a single centralized
memory that all processors have equal access to, – Hence the term symmetric.
§ All existing multicores are SMPs.
§ Also called uniform memory access (UMA) multiprocessors– all processors have a uniform
latency
13
Bus-Based Symmetric Shared Memory§ Still an important architecture – even on chip (until very recently)
– Building blocks for larger systems; arriving to desktop§ Attractive as throughput servers and for parallel programs
– Fine-grain resource sharing– Uniform access via loads/stores– Automatic data movement and coherent replication in caches– Cheap and powerful extension
§ Normal uniprocessor mechanisms to access data– Key is extension of memory hierarchy to support multiple processors
I/O devicesMem
P1
$ $
Pn
Bus
14
Centralized shared memory system (I)§ Multi-core processors
– Typically connected over a cache, – Previous SMP systems were typically connected over the
main memory
§ Intel X7350 quad-core (Tigerton)– Private L1 cache: 32 KB instruction, 32 KB data – Shared L2 cache: 4 MB unified cache
Core
L1
Core
L1shared L2
Core
L1
Core
L1shared L2
1066 MHz FSB
15
Centralized shared memory systems (II)§ Intel X7350 quad-core (Tigerton) multi-processor
configuration
C0
C1
L2
C8
C9
L2
C2
C3
L2
C10
C11
L2
C4
C5
L2
C12
C13
L2
C6
C7
L2
C14
C15
L2
Socket 0 Socket 1 Socket 2 Socket 3
Memory ControllerHub (MCH)
Memory Memory Memory Memory
8 GB/s8 GB/s8 GB/s8 GB/s
16
Distributed Shared-Memory Multiprocessor
§ Large processor count– 64 to 1000s
§ Distributed memory– Remote vs local memory– Long vs short latency– High vs low latency
§ Interconnection network– Bandwidth, topology, etc
§ Nonuniform memory access (NUMA)
§ Each processor may has local I/O
17
Distributed Shared-Memory Multiprocessor (NUMA)
§ Reduces the memory bottleneck compared to SMPs§ More difficult to program efficiently
– E.g. first touch policy: data item will be located in the memory of the processor which uses a data item first
§ To reduce effects of non-uniform memory access, caches are often used– ccNUMA: cache-coherent non-uniform memory access
architectures§ Largest example as of today: SGI Origin with 512
processors
18
Shared-Memory Multiprocessor§ SMP and DSM are all shared memory multiprocessors
– UMA or NUMA§ Multicore are SMP shared memory§ Most multi-CPU machines are DSM
– NUMA
§ Shared Address Space (Virtual Address Space)– Not always shared memory
19
Performance Metrics§ Speedup: how much faster does a problem run on p
processors compared to 1 processor?
– Optimal: S(p) = p (linear speedup)
§ Parallel Efficiency: Speedup normalized by the number of processors
– Optimal: E(p) = 1.0
)()1()(pT
TpStotal
total=
ppSpE )()( =
20
Amdahl’s Law§ Most applications have a (small) sequential fraction,
which limits the speedup
f: fraction of the code which can only be executed sequentially
§ Assumes the problem size is constant– In most applications, the sequential part is independent of the
problem size– The part which can be executed in parallel depends.
pffT
pff
TpStotal
total
-+
=-
+= 1
1
)1()1(
)1()(
TotalTotalparallelsequentialtotal TffTTTT )1( -+=+=
21
Challenges of Parallel Processing§ 1. Limited parallelism available in programs
– Amdahl’s Law
§ 0.25% can be sequential
22
Cache in Shared Memory System (UMA or NUMA)
P1
Switch
Main memory
Pn
(Interleaved)
(Interleaved)
First-level $
P1
$
Interconnection network
$
Pn
Mem Mem
P1
$
Interconnection network
$
Pn
Mem MemShared Cache
UMA
Scale
NUMA
23
Caches and Cache Coherence§ Caches play key role in all cases
– Reduce average data access time– Reduce bandwidth demands placed on shared interconnect
§ Private processor caches create a problem– Copies of a variable can be present in multiple caches – A write by one processor may not become visible to others
» They’ll keep accessing stale value in their caches
Þ Cache coherence problem
§ What do we do about it?– Organize the mem hierarchy to make it go away – Detect and take actions to eliminate the problem
24
Example Cache Coherence Problem
Things to note:Processors see different values for u after event 3With write back caches, value written back to memory depends on happenstance of which cache flushes or writes back value and when
Processes accessing main memory may see very stale valueUnacceptable to programs, and frequent!
I/O devices
Memory
P1
$ $ $
P2 P3
5
u = ?4
u = ?
u:51
u :5
2
u :5
3
u= 7
int count = 5;int * u= &count;…. a1 = *u;
a3 = *u;*u = 7;
b1 = *ua2 = *u
T1 (P1) T2 (P2) T3 (P3)
25
Cache Coherence§ Typical solution:
– Caches keep track on whether a data item is shared between multiple processes
– Upon modification of a shared data item, ‘notification’ of other caches has to occur
– Other caches will have to reload the shared data item on the next access into their cache
§ Cache coherence is only an issue in case multiple tasks access the same item and one is to write– Multiple threads– Multiple processes have a joint shared memory segment– Process is being migrated from one CPU to another
26
Cache Coherence Protocols§ Snooping Protocols
– Send all requests for data to all processors, the address– Processors snoop a bus to see if they have a copy and
respond accordingly – Requires broadcast, since caching information is at
processors– Works well with bus (natural broadcast medium)– Dominates for centralized shared memory machines
§ Directory-Based Protocols – Keep track of what is being shared in centralized location– Distributed memory => distributed directory for scalability
(avoids bottlenecks)– Send point-to-point requests to processors via network– Scales better than Snooping– Commonly used for distributed shared memory machines
27
Snoopy Cache-Coherence Protocols
§ Works because bus is a broadcast medium & Caches know what they have
§ Cache Controller “snoops” all transactions on the shared bus– relevant transaction if for a block it contains– take action to ensure coherence
» invalidate, update, or supply value– depends on state of the block and the protocol
State
Address
Data
I/O devicesMem
P1
$
Bus snoop
$
Pn
Cache-memorytransaction
Broadcast msg:
28
Basic Snoopy Protocols§ Write Invalidate Protocol:
– Multiple readers, single writer– Write to shared data: an invalidate is sent to all caches which
snoop and invalidate any copies– Read Miss:
» Write-through: memory is always up-to-date» Write-back: snoop in caches to find most recent copy
§ Write Update Protocol (typically write through):– Write to shared data: broadcast on bus, processors snoop,
and update any copies– Read miss: memory is always up-to-date
§ Write serialization: bus serializes requests!– Bus is single point of arbitration
29
Write Invalidate Protocol§ Basic Bus-Based Protocol
– Each processor has cache, state– All transactions over bus snooped
§ Writes invalidate all other caches– can have multiple simultaneous readers
of block, but write invalidates them§ Two states per block in each cache
– as in uniprocessor– state of a block is a p-vector of states– Hardware state bits associated with
blocks that are in the cache – other blocks can be seen as being in
invalid (not-present) state in that cache I
VBusWr / -
PrRd/ --
PrWr / BusWr
PrWr / BusWr
PrRd / BusRd
State Tag Data
I/O devicesMem
P1
$ $
Pn
Bus
State Tag Data
30
Example: Write Invalidate
I/O devices
Memory
P1
$ $ $
P2 P3
5
u = ?4
u = ?
u:51
u :5
2
u :5
3
u= 7
u= 7
u :7
31
Write-Update (Broadcast) § Update all the cached copies of a data item when that
item is written. – Even a processor may not need the updated copy in the
future§ Consumes considerably more bandwidth§ Recent multiprocessors have opted to implement a
write invalidate protocol
I/O devices
Memory
P1
$ $ $
P2 P3
5
u = ?4
u = ?
u:51
u :5
2
u :5
3
u= 7
u= 7
u=7
32
Implementation of Cache Coherence Protocol -- 1
§ When data are coherent, the cache block is shared– “Memory” could be the last level shared cache, e.g. shared L3
1. When there is a write by CPU 0, Invalidate the shared copies in the cache of other processors/cores
– Copy in CPU 0’s cache is exclusive/unshared, – CPU 0 is the owner of the block– For write-through cache, data is also written to the memory
» Memory has the latest– For write-back cache: data in memory is obsoleted– For snooping protocol, invalidate signals are broadcasted by CPU 0
» CPU 0 broadcasts; and CPU 1 snoops, compares and invalidates
Memory
CPU 0
Cache
CPU 1
Cache Written by CPU 0
Invalidated by CPU 0
33
Implementation of Cache Coherence Protocol -- 2
§ CPU 0 owned the block (exclusive or unshared)2. When there is a read/write by CPU 1 or others à Miss since already invalidated
– For write-through cache: read from memory– For write-back cache: supply from CPU 0 and abort memory access– For snooping: CPU 1 broadcasts mem request because of a miss;
CPU 0 snoops, compares and provides cache block (aborts the memory request)
Memory
CPU 0
Cache
CPU 1
Cache Owned by CPU 0
Read/write miss
34
An Example Snoopy Protocol§ Invalidation protocol, write-back cache§ Each block of memory is in one state:
– Clean in all caches and up-to-date in memory (Shared)– OR Dirty in exactly one cache (Exclusive)– OR Not in any caches
§ Each cache block is in one state (track these):– Shared : block can be read– OR Exclusive : cache has only copy, its writeable, and dirty– OR Invalid : block contains no data
§ Read misses: cause all caches to snoop bus§ Writes to clean line are treated as misses
35
Shared Memory Multiprocessor
Use snoopy mechanism to keep all processors’ view of memory coherent
CPU1
CPU2
CPU3
SnoopyCache
DMA
PhysicalMemory
MemoryBus
SnoopyCache
SnoopyCache DISKS
36
Cache Line for Snooping § Cache tags for implementing snooping
– Compares the addresses on the bus with the tags of the cache line
§ Valid bit for being invalidated§ State bit for shared/exclusive
§ We will use write-back cache– Lower bandwidth requirement than write-through cache– Dirty bit for write-back– Write-buffer complicates things
37
Snoopy-Cache State Machine-I § State machine
for CPU requestsfor each cache block Invalid
Shared(read/only)
Exclusive(read/write)
CPU Read
CPU Write
Place read miss on bus
Place Write Miss on bus
CPU Write missPlace Write Miss on Bus
CPU Read missPlace read miss on bus
CPU Write MissWrite back cache blockPlace write miss on bus
CPU read hitCPU write hit
CPU read hit
38
Snoopy-Cache State Machine-II§ State machine
for bus requestsfor each cache block Invalid Shared
(read/only)
Exclusive(read/write)
Write BackBlock; (abortmemory access)
Write missfor this block Read miss
for this block
Write missfor this block
Write BackBlock; (abortmemory access)
Invalidate for this block
Read miss
39
Snoopy-Cache State Machine-III § State machine
for CPU requestsfor each cache block andfor bus requestsfor each cache block
Place read misson bus
InvalidShared
(read/only)
Exclusive(read/write)
CPU Read
CPU Write
CPU Read hit
Place Write Miss on busCPU read missWrite back block,Place read misson bus CPU Write
Place Write Miss on Bus
CPU Read missPlace read miss on bus
CPU Write MissWrite back cache blockPlace write miss on bus
CPU read hitCPU write hit
Write missfor this block
Write BackBlock; (abortmemory access)
Write missfor this block
Read miss for this block
Write BackBlock; (abortmemory access)
40
State Table of Snoopy Protocol
41
Example
P1 P2 Bus Memorystep State Addr Value State Addr Value Action Proc. Addr Value Addr Value
P1: Write 10 to A1P1: Read A1P2: Read A1
P2: Write 20 to A1P2: Write 40 to A2
Assumes initial cache state
is invalid and A1 and A2 map to same cache block,
but A1 != A2
Processor 1 Processor 2 Bus Memory
Remote
WriteWrite Back
Remote Write
Invalid Shared
Exclusive
CPU Read hit
Readmiss on bus
Writemiss on bus CPU Write
Place Write Miss on Bus
CPU read hit
CPU write hit
Remote ReadWrite Back
CPU Write Miss
Write Back
CPU Read Miss
42
Example: Step 1
P1 P2 Bus Memorystep State Addr Value State Addr Value Action Proc. Addr Value Addr Value
P1: Write 10 to A1 Excl. A1 10 WrMs P1 A1P1: Read A1P2: Read A1
P2: Write 20 to A1P2: Write 40 to A2
Assumes initial cache state
is invalid and A1 and A2 map to same cache block,
but A1 != A2.
Active arrow =
Remote
WriteWrite Back
Remote Write
Invalid Shared
Exclusive
CPU Read hit
Readmiss on bus
Writemiss on bus CPU Write
Place Write Miss on Bus
CPU read hit
CPU write hit
Remote ReadWrite Back
CPU Write Miss
Write Back
CPU Read Miss
43
P1 P2 Bus Memorystep State Addr Value State Addr Value Action Proc. Addr Value Addr Value
P1: Write 10 to A1 Excl. A1 10 WrMs P1 A1P1: Read A1 Excl. A1 10P2: Read A1
P2: Write 20 to A1P2: Write 40 to A2
Example: Step 2
Assumes initial cache state
is invalid and A1 and A2 map to same cache block,
but A1 != A2Remote
WriteWrite Back
Remote Write
Invalid Shared
Exclusive
CPU Read hit
Readmiss on bus
Writemiss on bus CPU Write
Place Write Miss on Bus
CPU read hit
CPU write hit
Remote ReadWrite Back
CPU Write Miss
Write Back
CPU Read Miss
44
Example: Step 3
P1 P2 Bus Memorystep State Addr Value State Addr Value Action Proc. Addr Value Addr Value
P1: Write 10 to A1 Excl. A1 10 WrMs P1 A1P1: Read A1 Excl. A1 10P2: Read A1 Shar. A1 RdMs P2 A1
Shar. A1 10 WrBk P1 A1 10 10Shar. A1 10 RdDa P2 A1 10 10
P2: Write 20 to A1 10P2: Write 40 to A2 10
10
Assumes initial cache state
is invalid and A1 and A2 map to same cache block,
but A1 != A2.Remote
WriteWrite Back
Remote Write
Invalid Shared
Exclusive
CPU Read hit
Readmiss on bus
Writemiss on bus CPU Write
Place Write Miss on Bus
CPU read hit
CPU write hit
Remote ReadWrite Back
A1A1
CPU Write Miss
Write Back
CPU Read Miss
45
Example: Step 4P1 P2 Bus Memory
step State Addr Value State Addr Value Action Proc. Addr Value Addr ValueP1: Write 10 to A1 Excl. A1 10 WrMs P1 A1
P1: Read A1 Excl. A1 10P2: Read A1 Shar. A1 RdMs P2 A1
Shar. A1 10 WrBk P1 A1 10 10Shar. A1 10 RdDa P2 A1 10 10
P2: Write 20 to A1 Inv. Excl. A1 20 WrMs P2 A1 10P2: Write 40 to A2 10
10
Assumes initial cache state
is invalid and A1 and A2 map to same cache block,
but A1 != A2Remote
WriteWrite Back
Remote Write
Invalid Shared
Exclusive
CPU Read hit
Readmiss on bus
Writemiss on bus CPU Write
Place Write Miss on Bus
CPU read hit
CPU write hit
Remote ReadWrite Back
A1A1A1
CPU Write Miss
Write Back
CPU Read Miss
46
Remote
WriteWrite Back
Remote Write
Invalid Shared
Exclusive
CPU Read hit
Readmiss on bus
Writemiss on bus CPU Write
Place Write Miss on Bus
CPU read hit
CPU write hit
Remote ReadWrite Back
Example: Step 5P1 P2 Bus Memory
step State Addr Value State Addr Value Action Proc. Addr Value Addr ValueP1: Write 10 to A1 Excl. A1 10 WrMs P1 A1
P1: Read A1 Excl. A1 10P2: Read A1 Shar. A1 RdMs P2 A1
Shar. A1 10 WrBk P1 A1 10 10Shar. A1 10 RdDa P2 A1 10 10
P2: Write 20 to A1 Inv. Excl. A1 20 WrMs P2 A1 10P2: Write 40 to A2 WrMs P2 A2 10
Excl. A2 40 WrBk P2 A1 20 20
A1
A1
Assumes initial cache state
is invalid and A1 and A2 map to same cache block,
but A1 != A2
A1A1A1
CPU Write Miss
Write Back
CPU Read Miss
47
Categories of cache misses§ Up to now:
– Compulsory Misses: first access to a block cannot be in the cache (cold start misses)
– Capacity Misses: cache cannot contain all blocks required for the execution
– Conflict Misses: cache block has to be discarded because of block replacement strategy
§ In multi-processor systems:– Coherence Misses: cache block has to be discarded because
another processor modified the content» true sharing miss: another processor modified the
content of the request element» false sharing miss: another processor invalidated the
block, although the actual item of interest is unchanged.
48
False Sharing
§ A cache line contains more than one word
§ Cache-coherence is done at the line-level and not word-level
§ Suppose M1 writes wordi and M2 writes wordk and– both words have the same line address.
§ What can happen?
state line addr data0 data1 ... dataN
49
Avoid False Sharing in Programming§ False sharing
– When at least one thread write to a cache line while others access it
» Thread 0: = A[1] (read)» Thread 1: A[0] = … (write)
§ Solution: use array padding
int a[16]; // can fill in a full 64-byte cache line
#pragma omp parallel for num_threads (16) schedule(static,1)
for(int i=0; i<16; i++) a[i] +=i;
int a[16][16];
#pragma omp parallel for num_threads (16) schedule(static,1)
for(int i=0; i<16; i++) a[i][0] +=i;
Getting OpenMP Up To Speed
RvdP/V1 Tutorial IWOMP 2010 – CCS Un. of Tsukuba, June 14, 2010
False Sharing
CPUs Caches Memory
A store into a shared cache line invalidates the other copies of that line:
The system is not able to distinguish between changes
within one individual line
A
T0
T1
50
Class Lectures End Here!
51
Performance§ Coherence influences cache miss rate
–Coherence misses» True sharing misses
• Write to shared block (transmission of invalidation)• Read an invalidated block
» False sharing misses• Read an unmodified word in an invalidated block
52
Example: True v. False Sharing v. Hit?
Time P1 P2 True,False,Hit?Why?1 Writex1
2 Readx2
3 Writex1
4 Writex2
5 Readx2
• Assumex1andx2insamecacheline.P1andP2bothreadx1andx2before.
Truemiss;invalidatex1inP2Falsemiss;x1irrelevanttoP2Falsemiss;x2irrelevanttoP1Falsemiss;invalidatex2inP1
Truemiss;invalidatex2inP1
53
Performance Study: Commercial Workload
54
Performance Study: Commercial Workload
55
Performance Study: Commercial Workload
56
Performance Study: Commercial Workload
57
Snooping Cache Variations
§ Owner can update via bus invalidate operation§ Owner must write back when replaced in cache
– If read sourced from memory, then Private Clean– if read sourced from other cache, then Shared– Can write in cache if held private clean or dirty
Berkeley Protocol
Owned Exclusive
Owned Shared
Shared
Invalid
Basic Protocol
Exclusive
Shared
Invalid
Illinois Protocol
Private Dirty
Private Clean
Shared
Invalid
MESI Protocol
Modfied (private,!=Memory)Exclusive (private,=Memory)
Shared (shared,=Memory)Invalid
58
Implementing Snooping Caches§ Multiple processors must be on bus, access to both
addresses and data§ Add a few new commands to perform coherency,
in addition to read and write§ Processors continuously snoop on address bus
– If address matches tag, either invalidate or update§ Since every bus transaction checks cache tags,
could interfere with CPU cache access: – solution 1: duplicate set of tags for L1 caches just to allow
checks in parallel with CPU– solution 2: L2 cache already duplicate,
provided L2 obeys inclusion with L1 cache» block size, associativity of L2 affects L1
59
Implementing Snooping Caches§ Bus serializes writes, getting bus ensures no one else
can perform memory operation§ On a miss in a write back cache, may have the
desired copy of the cache block and its dirty, so must reply– Add extra state bit to cache to determine shared or not– Add 4th state (MESI)
60
Complications§ The simple cache protocol is correct, it omits a
number of complications that make the implementation much trickier.
§ Operations are atomic– For example, the protocol described assumes that write
misses can be detected, acquire the bus, and receive a response as a single atomic action. In reality this is not true.
– In fact, even a read miss might not be atomic; after detecting a miss in the L2 of a multi-core, the core must arbitrate for access to the bus connecting to the shared L3.
§ Nonatomic actions introduce the possibility that the protocol can deadlock
§ With multicore processors, the coherence among the processor cores is all implemented on chip, using either a snooping or simple central directory proto-col.
61
Extensions to Snoopy: MESI§ MESI: adds the state Exclusive to the basic MSI
– Exclusive: indicate when a cache block is resident only in a single cache but is clean.
– If a block is in the E state, it can be written without generating any invalidates
» optimizes the case where a block is read by a single cache before being written by that same cache.
– When a read miss to a block in the E state occurs, the block must be changed to the S state to maintain coherence.
– The advantage: a subsequent write to a block in the exclusive state by the same core need not acquire bus access or generate an invalidate, since the block is known to be exclusively in this local cache; the processor merely changes the state to modified.
– This state is easily added by using the bit that encodes the coherent state as an exclusive state and using the dirty bit to indicate that a bock is modified.
– The popular MESI protocol (Modified, Exclusive, Shared, and Invalid), is used in Intel i7 as a variant called MESIF, which adds a state (Forward) to designate which sharing processor should respond to a request. It is designed to enhance performance in distributed memory organizations.
62
MESI (4-state) Invalidation Protocol§ Four States:
– “M”: “Modified”– “E”: “Exclusive”– “S”: “Shared”– “I”: “Invalid”
§ Add exclusive state– distinguish exclusive (writable) and owned (written)– Main memory is up to date, so cache not necessarily owner– can be written locally
§ States– invalid– exclusive or exclusive-clean (only this cache has copy, but not modified)– shared (two or more caches may have copies)– modified (dirty)
§ I -> E on PrRd if no cache has copy=> How can you tell?
63
Hardware Support for MESI§ All cache controllers snoop on BusRd§ Assert ‘shared’ if present (S? E? M?)§ Issuer chooses between S and E
– how does it know when all have voted?
I/O devices
Memory
u:5
P0 P1 P4
shared signal
- wired-OR
64
MESI State Transition Diagram§ BusRd(S) means shared
line asserted on BusRdtransaction
§ Flush’: if cache-to-cache xfers– only one cache flushes data
§ Replacement:– S®I can happen without
telling other caches– E®I, M®I
§ MOESI protocol: Owned state: exclusive but memory not valid
PrWr/—
BusRd/Flush
PrRd/
BusRdX/Flush
PrWr/BusRdX
PrWr/—
PrRd/—
PrRd/—BusRd/Flush’¢
E
M
I
S
PrRd
BusRd(S)
BusRdX/Flush’¢
BusRdX/Flush
BusRd/Flush
PrWr/BusRdX
PrRd/BusRd (S )
65
MESI: An Enhanced MSI protocolincreased performance for private data
M E
S I
M: Modified ExclusiveE: Exclusive but unmodifiedS: SharedI: Invalid
Each cache line has a tag
Address tagstatebits
Write miss
Other processorintent to write
Read miss,shared
Other processorintent to write
P1 write
Read by anyprocessor
Other processor readsP1 writes back
P1 readP1 writeor read
Cache state in processor P1
P1 intent to write
Read miss, not sharedOther
processorreads
Other processor intent to write, P1 writes back
66
Dragon Write-back Update Protocol§ 4 states
– Exclusive-clean or exclusive (E): I and memory have it– Shared clean (Sc): I, others, and maybe memory, but I’m not owner– Shared modified (Sm): I and others but not memory, and I’m the owner
» Sm and Sc can coexist in different caches, with only one Sm– Modified or dirty (D): I and, noone else
§ No invalid state– If in cache, cannot be invalid– If not present in cache, view as being in not-present or invalid state
§ New processor events: PrRdMiss, PrWrMiss– Introduced to specify actions when block not present in cache
§ New bus transaction: BusUpd– Broadcasts single word written on bus; updates other relevant caches
67
Dragon State Transition Diagram
E Sc
Sm M
PrWr/—PrRd/—
PrRd/—
PrRd/—
PrRdMiss/BusRd(S)PrRdMiss/BusRd(S)
PrWr/—
PrWrMiss/(BusRd(S); BusUpd) PrWrMiss/BusRd(S)
PrWr/BusUpd(S)
PrWr/BusUpd(S)
BusRd/—
BusRd/Flush
PrRd/— BusUpd/Update
BusUpd/Update
BusRd/Flush
PrWr/BusUpd(S)
PrWr/BusUpd(S)
68
Extensions to Snoopy: MOESI§ Add the state Owned to the MESI protocol
– indicate that the associated block is owned by that cache and out-of-date in memory.
§ In MSI and MESI protocols, when there is an attempt to share a block in the Modified state, the state is changed to Shared (in both the original and newly sharing cache), and the block must be written back to memory. In a MOESI protocol, the block can be changed from the Modified to Owned state in the original cache without writing it to memory. Other caches, which are newly sharing the block, keep the block in the Shared state; the O state, which only the original cache holds, indicates that the main memory copy is out of date and that the designated cache is the owner. The owner of the block must supply it on a miss, since memory is not up to date and must write the block back to memory if it is replaced. The AMD Opteron uses the MOESI protocol.
69
Optimized Snoop with Level-2 Caches
Snooper Snooper Snooper Snooper
• Processorsoftenhavetwo-levelcaches• smallL1,largeL2(onchip)
• Inclusionproperty:entriesinL1mustbeinL2invalidationinL2Þ invalidationinL1
• SnoopingonL2doesnotaffectCPU-L1bandwidth
Whatproblemcouldoccur?
CPU
L1$
L2$
CPU
L1$
L2$
CPU
L1$
L2$
CPU
L1$
L2$
70
Intervention
Whenaread-missforA occursincache-2,areadrequestforA isplacedonthebus
• Cache-1needstosupply&changeitsstatetoshared• Thememorymayrespond totherequestalso!
Doesmemoryknowithasstaledata?Cache-1needstointervenethroughmemorycontrollertosupplycorrectdatatocache-2
cache-1A 200
CPU-Memorybus
CPU-1 CPU-2
cache-2
memory(staledata)A 100
71
Coherence Protocols: Extensions§ Shared memory bus and
snooping bandwidth is bottleneck for scaling symmetric multiprocessors– Duplicating tags– Place directory in
outermost cache– Use crossbars or point-to-
point networks with banked memory
72
Coherence Protocols§ AMD Opteron:
– Memory directly connected to each multicore chip in NUMA-like organization
– Implement coherence protocol using point-to-point links– Use explicit acknowledgements to order operations
top related