cs 252 graduate computer architecture recap: pipeline performance lecture...
TRANSCRIPT
CS 252 Graduate Computer Architecture
Lecture 5: Instruction-Level Parallelism
(Part 2)
Krste AsanovicElectrical Engineering and Computer Sciences
University of California, Berkeley
http://www.eecs.berkeley.edu/~krste
http://inst.cs.berkeley.edu/~cs252
9/13/2007 2
Recap: Pipeline Performance
• Exploit implicit instruction-level parallelism with morecomplex pipelines and dynamic scheduling
• Execute instructions out-of-order while preserving:– True dependences (RAW)
– Precise exceptions
• Register renaming removes WAR and WAW hazards
• Reorder buffer holds completed results beforecommitting to support precise exceptions
• Branches are frequent and limit achievableperformance due to control hazards
9/13/2007 3
Recap: Overall Pipeline Structure
• Instructions fetched and decoded into instruction reorder buffer in-order• Execution is out-of-order ( ! out-of-order completion)
• Commit (write-back to architectural state, i.e., regfile & memory) is in-order
Temporary storage needed to hold results before commit(shadow registers and store buffers)
Fetch Decode
Execute
CommitReorder Buffer
In-order In-orderOut-of-order
KillKill Kill
Exception?Inject handler PC
9/13/2007 4
I-cache
FetchBuffer
IssueBuffer
Func.Units
Arch.State
Execute
Decode
ResultBuffer Commit
PC
Fetch
Branchexecuted
Next fetchstarted
Modern processors may have> 10 pipeline stages betweennext PC calculation and branchresolution !
Control Flow Penalty
How much work is lost ifpipeline doesn’t followcorrect instruction flow?
9/13/2007 5
Instruction Taken known? Target known?
J
JR
BEQZ/BNEZ
MIPS Branches and Jumps
Each instruction fetch depends on one or two piecesof information from the preceding instruction:
1) Is the preceding instruction a taken branch?
2) If so, what is the target address?
9/13/2007 6
Branch Penalties in Modern Pipelines
A PC Generation/Mux
P Instruction Fetch Stage 1
F Instruction Fetch Stage 2
B Branch Address Calc/Begin Decode
I Complete Decode
J Steer Instructions to Functional units
R Register File Read
E Integer Execute
Remainder of execute pipeline (+ another 6 stages)
UltraSPARC-III instruction fetch pipeline stages(in-order issue, 4-way superscalar, 750MHz, 2000)
BranchTargetAddressKnown
BranchDirection &JumpRegisterTargetKnown
9/13/2007 7
Reducing Control Flow Penalty
Software solutions• Eliminate branches - loop unrolling
Increases the run length• Reduce resolution time - instruction scheduling
Compute the branch condition as earlyas possible (of limited value)
Hardware solutions• Find something else to do - delay slots
Replaces pipeline bubbles with useful work(requires software cooperation)
• Speculate - branch predictionSpeculative execution of instructions beyondthe branch
9/13/2007 8
Branch Prediction
Motivation:Branch penalties limit performance of deeply pipelinedprocessors
Modern branch predictors have high accuracy(>95%) and can reduce branch penalties significantly
Required hardware support:Prediction structures:
• Branch history tables, branch target buffers, etc.
Mispredict recovery mechanisms:• Keep result computation separate from commit• Kill instructions following branch in pipeline• Restore state to state following branch
9/13/2007 9
Static Branch Prediction
Overall probability a branch is taken is ~60-70% but:
ISA can attach preferred direction semantics to branches,e.g., Motorola MC88110
bne0 (preferred taken) beq0 (not taken)
ISA can allow arbitrary choice of statically predicteddirection, e.g., HP PA-RISC, Intel IA-64 typically reported as ~80% accurate
JZ
JZbackward
90%forward
50%
9/13/2007 10
Dynamic Branch Predictionlearning based on past behavior
Temporal correlationThe way a branch resolves may be a goodpredictor of the way it will resolve at the nextexecution
Spatial correlationSeveral branches may resolve in a highlycorrelated manner (a preferred path ofexecution)
9/13/2007 11
• Assume 2 BP bits per instruction• Change the prediction after two consecutive mistakes!
¬takewrong
taken¬ taken
taken
taken
taken¬takeright
takeright
takewrong
¬ taken
¬ taken¬ taken
BP state:(predict take/¬take) x (last prediction right/wrong)
Branch Prediction Bits
9/13/2007 12
Branch History Table
4K-entry BHT, 2 bits/entry, ~80-90% correct predictions
0 0Fetch PC
Branch? Target PC
+
I-Cache
Opcode offset
Instruction
k
BHT Index
2k-entryBHT,2 bits/entry
Taken/¬Taken?
9/13/2007 13
Exploiting Spatial CorrelationYeh and Patt, 1992
History register, H, records the direction of the lastN branches executed by the processor
if (x[i] < 7) then
y += 1;
if (x[i] < 5) then
c -= 4;
If first condition false, second condition also false
9/13/2007 14
Two-Level Branch PredictorPentium Pro uses the result from the last two branchesto select one of the four sets of BHT bits (~95% correct)
0 0
kFetch PC
Shift inTaken/¬Takenresults of eachbranch
2-bit global branchhistory shift register
Taken/¬Taken?
9/13/2007 15
Limitations of BHTs
Only predicts branch direction. Therefore, cannot redirectfetch stream until after branch target is determined.
UltraSPARC-III fetch pipeline
Correctly
predicted
taken branchpenalty
Jump Registerpenalty
A PC Generation/Mux
P Instruction Fetch Stage 1
F Instruction Fetch Stage 2
B Branch Address Calc/Begin Decode
I Complete Decode
J Steer Instructions to Functional units
R Register File Read
E Integer Execute
Remainder of execute pipeline(+ another 6 stages)
9/13/2007 16
Branch Target Buffer
BP bits are stored with the predicted target address.
IF stage: If (BP=taken) then nPC=target else nPC=PC+4later: check prediction, if wrong then kill the instruction and update BTB & BPb else update BPb
IMEM
PC
Branch Target Buffer (2k entries)
k
BPbpredicted
target BP
target
9/13/2007 17
Address Collisions
What will be fetched after the instruction at 1028?BTB prediction = Correct target =
!
Assume a 128-entry BTB
BPbtarget
take236
1028 Add .....
132 Jump 100
InstructionMemory
9/13/2007 18
BTB is only for Control Instructions
BTB contains useful information for branch andjump instructions only
! Do not update it for other instructions
For all other instructions the next PC is PC+4 !
How to achieve this effect without decoding theinstruction?
9/13/2007 19
Branch Target Buffer (BTB)
• Keep both the branch PC and target PC in the BTB• PC+4 is fetched if match fails• Only taken branches and jumps held in BTB• Next PC determined before branch fetched and decoded
2k-entry direct-mapped BTB(can also be associative)
I-Cache PC
k
Valid
valid
Entry PC
=
match
predicted
target
target PC
9/13/2007 20
Consulting BTB Before Decoding
1028 Add .....
132 Jump 100
BPbtarget
take236
entry PC
132
• The match for PC=1028 fails and 1028+4 is fetched ! eliminates false predictions after ALU instructions
• BTB contains entries only for control transfer instructions! more room to store branch targets
9/13/2007 21
Combining BTB and BHT• BTB entries are considerably more expensive than BHT, but can
redirect fetches at earlier stage in pipeline and can accelerateindirect branches (JR)
• BHT can hold many more entries and is more accurate
A PC Generation/Mux
P Instruction Fetch Stage 1
F Instruction Fetch Stage 2
B Branch Address Calc/Begin Decode
I Complete Decode
J Steer Instructions to Functional units
R Register File Read
E Integer Execute
BTB
BHTBHT in laterpipeline stagecorrects whenBTB misses apredictedtaken branch
BTB/BHT only updated after branch resolves in E stage
9/13/2007 22
Uses of Jump Register (JR)
• Switch statements (jump to address of matching case)
• Dynamic function call (jump to run-time function address)
• Subroutine returns (jump to return address)
How well does BTB work for each of these cases?
9/13/2007 23
Subroutine Return Stack
Small structure to accelerate JR for subroutine returns,typically much more accurate than BTBs.
&fb()
&fc()
Push call address whenfunction call executed
Pop return addresswhen subroutinereturn decoded
fa() { fb(); }
fb() { fc(); }
fc() { fd(); }
&fd()k entries(typically k=8-16)
9/13/2007 24
Mispredict Recovery
In-order execution machines:
– Assume no instruction issued after branch can write-back beforebranch resolves
– Kill all instructions in pipeline behind mispredicted branch
–Multiple instructions following branch in programorder can complete before branch resolves
Out-of-order execution?
9/13/2007 25
In-Order Commit for Precise Exceptions
• Instructions fetched and decoded into instruction reorder buffer in-order• Execution is out-of-order ( ! out-of-order completion)
• Commit (write-back to architectural state, i.e., regfile & memory, is in-order
Temporary storage needed in ROB to hold results before commit
Fetch Decode
Execute
CommitReorder Buffer
In-order In-orderOut-of-order
KillKill Kill
Exception?Inject handler PC
9/13/2007 26
Branch Misprediction in Pipeline
Fetch Decode
Execute
CommitReorder Buffer
Kill
Kill Kill
BranchResolution
Inject correct PC
• Can have multiple unresolved branches in ROB• Can resolve branches out-of-order by killing all the instructions in ROB that follow a mispredicted branch
BranchPrediction
PC
Complete
9/13/2007 27
t vt vt v
Recovering ROB/Renaming Table
RegisterFile
Reorder buffer Load
UnitFU FU FU
Store Unit
< t, result >
t1
t2
.
.tn
Ins# use exec op p1 src1 p2 src2 pd dest data
Commit
Rename Table r1
t v
r2
Take snapshot of register rename table at each predictedbranch, recover earlier snapshot if branch mispredicted
Rename Snapshots
Ptr2
next to commit
Ptr1
next available
rollback
next available
9/13/2007 28
Speculating Both Directions
An alternative to branch prediction is to executeboth directions of a branch speculatively
9/13/2007 29
CS252 Administrivia
• Prereq quiz
• Next reading assignment “Limits of ILP” by DavidWall. Read pages 1-35 (back contains longappendices). Summarize in one page, and includedescriptions of any flaws you found in study. Discussin class on Tuesday Sep 18.
9/13/2007 30
“Data in ROB” Design(HP PA8000, Pentium Pro, Core2Duo)
• On dispatch into ROB, ready sources can be in regfile or in ROBdest (copied into src1/src2 if ready before dispatch)• On completion, write to dest field and broadcast to src fields.• On issue, read from ROB src fields
Register Fileholds onlycommitted state
Reorderbuffer
Load Unit
FU FU FUStore Unit
< t, result >
t1
t2
.
.tn
Ins# use exec op p1 src1 p2 src2 pd dest data
Commit
9/13/2007 31
Unified Physical Register File(MIPS R10K, Alpha 21264, Pentium 4)
• One regfile for both committed and speculative values (no data in ROB)
• During decode, instruction result allocated new physical register, source regs translated to physical regs through rename table• Instruction reads data from regfile at start of execute (not in decode)• Write-back updates reg. busy bits on instructions in ROB (assoc. search)• Snapshots of rename table taken at every branch to recover mispredicts• On exception, renaming undone in reverse order of issue (MIPS R10000)
Rename Table
r1 tir2 tj
FU FUStore Unit
< t, result >
FULoad Unit
FU
t1t2.tn
RegFile
Snapshots formispredict recovery
(ROB not shown)
9/13/2007 32
Pipeline Design with Physical Regfile
FetchDecode &Rename
Reorder BufferPC
BranchPrediction
Update predictors
Commit
BranchResolution
BranchUnit
ALU MEMStoreBuffer
D$
Execute
In-Order
In-OrderOut-of-Order
Physical Reg. File
kill
kill
kill
kill
9/13/2007 33
Lifetime of Physical Registers
ld r1, (r3)
add r3, r1, #4
sub r6, r7, r9
add r3, r3, r6
ld r6, (r1)
add r6, r6, r3
st r6, (r1)
ld r6, (r11)
ld P1, (Px)
add P2, P1, #4
sub P3, Py, Pz
add P4, P2, P3
ld P5, (P1)
add P6, P5, P4
st P6, (P1)
ld P7, (Pw)
Rename
When can we reuse a physical register?
• Physical regfile holds committed and speculative values
• Physical registers decoupled from ROB entries (no data in ROB)
9/13/2007 34
Physical Register Management
op p1 PR1 p2 PR2exuse Rd PRdLPRd
<R6>P5<R7>P6<R3>P7
P0
Pn
P1P2P3P4
R5P5R6P6R7
R0P8R1
R2P7R3
R4
ROB
RenameTable
Physical Regs Free List
ld r1, 0(r3)
add r3, r1, #4
sub r6, r7, r6
add r3, r3, r6
ld r6, 0(r1)
ppp
P0P1P3P2P4
(LPRd requiresthird read port
on RenameTable for eachinstruction)
<R1>P8 p
9/13/2007 35
Physical Register Management
op p1 PR1 p2 PR2exuse Rd PRdLPRd
ROB
ld r1, 0(r3)
add r3, r1, #4
sub r6, r7, r6
add r3, r3, r6
ld r6, 0(r1)
Free ListP0P1P3P2P4
<R6>P5<R7>P6<R3>P7
P0
Pn
P1P2P3P4
Physical Regs
ppp
<R1>P8 p
x ld p P7 r1 P0
R5P5R6P6R7
R0P8R1
R2P7R3
R4
RenameTable
P0
P8
9/13/2007 36
Physical Register Management
op p1 PR1 p2 PR2exuse Rd PRdLPRd
ROB
ld r1, 0(r3)
add r3, r1, #4
sub r6, r7, r6
add r3, r3, r6
ld r6, 0(r1)
Free ListP0P1P3P2P4
<R6>P5<R7>P6<R3>P7
P0
Pn
P1P2P3P4
Physical Regs
ppp
<R1>P8 p
x ld p P7 r1 P0
R5P5R6P6R7
R0P8R1
R2P7R3
R4
RenameTable
P0
P8P7
P1
x add P0 r3 P1
9/13/2007 37
Physical Register Management
op p1 PR1 p2 PR2exuse Rd PRdLPRd
ROB
ld r1, 0(r3)
add r3, r1, #4
sub r6, r7, r6
add r3, r3, r6
ld r6, 0(r1)
Free ListP0P1P3P2P4
<R6>P5<R7>P6<R3>P7
P0
Pn
P1P2P3P4
Physical Regs
ppp
<R1>P8 p
x ld p P7 r1 P0
R5P5R6P6R7
R0P8R1
R2P7R3
R4
RenameTable
P0
P8P7
P1
x add P0 r3 P1P5
P3
x sub p P6 p P5 r6 P3
9/13/2007 38
Physical Register Management
op p1 PR1 p2 PR2exuse Rd PRdLPRd
ROB
ld r1, 0(r3)
add r3, r1, #4
sub r6, r7, r6
add r3, r3, r6
ld r6, 0(r1)
Free ListP0P1P3P2P4
<R6>P5<R7>P6<R3>P7
P0
Pn
P1P2P3P4
Physical Regs
ppp
<R1>P8 p
x ld p P7 r1 P0
R5P5R6P6R7
R0P8R1
R2P7R3
R4
RenameTable
P0
P8P7
P1
x add P0 r3 P1P5
P3
x sub p P6 p P5 r6 P3P1
P2
x add P1 P3 r3 P2
9/13/2007 39
Physical Register Management
op p1 PR1 p2 PR2exuse Rd PRdLPRd
ROB
ld r1, 0(r3)
add r3, r1, #4
sub r6, r7, r6
add r3, r3, r6
ld r6, 0(r1)
Free ListP0P1P3P2P4
<R6>P5<R7>P6<R3>P7
P0
Pn
P1P2P3P4
Physical Regs
ppp
<R1>P8 p
x ld p P7 r1 P0
R5P5R6P6R7
R0P8R1
R2P7R3
R4
RenameTable
P0
P8P7
P1
x add P0 r3 P1P5
P3
x sub p P6 p P5 r6 P3P1
P2
x add P1 P3 r3 P2x ld P0 r6 P4P3
P4
9/13/2007 40
op p1 PR1 p2 PR2exuse Rd PRdLPRd
ROB
x ld p P7 r1 P0x add P0 r3 P1x sub p P6 p P5 r6 P3
x ld p P7 r1 P0
Physical Register Management
ld r1, 0(r3)
add r3, r1, #4
sub r6, r7, r6
add r3, r3, r6
ld r6, 0(r1)
Free ListP0P1P3P2P4
<R6>P5<R7>P6<R3>P7
P0
Pn
P1P2P3P4
Physical Regs
ppp
<R1>P8 p
R5P5R6P6R7
R0P8R1
R2P7R3
R4
RenameTable
P0
P8P7
P1
P5
P3
P1
P2
x add P1 P3 r3 P2x ld P0 r6 P4P3
P4
Execute &Commit
p
p
p<R1>
P8
x
9/13/2007 41
op p1 PR1 p2 PR2exuse Rd PRdLPRd
ROB
x sub p P6 p P5 r6 P3x add P0 r3 P1x add P0 r3 P1
Physical Register Management
ld r1, 0(r3)
add r3, r1, #4
sub r6, r7, r6
add r3, r3, r6
ld r6, 0(r1)
Free ListP0P1P3P2P4
<R6>P5<R7>P6<R3>P7
P0
Pn
P1P2P3P4
Physical Regs
ppp
P8
x x ld p P7 r1 P0
R5P5R6P6R7
R0P8R1
R2P7R3
R4
RenameTable
P0
P8P7
P1
P5
P3
P1
P2
x add P1 P3 r3 P2x ld P0 r6 P4P3
P4
Execute &Commitp
p
p<R1>
P8
x
p
p<R3>
P7
9/13/2007 42
Reorder Buffer HoldsActive Instruction Window
…
ld r1, (r3)
add r3, r1, r2
sub r6, r7, r9
add r3, r3, r6
ld r6, (r1)
add r6, r6, r3
st r6, (r1)
ld r6, (r1)
…
(Older instructions)
(Newer instructions)
Cycle t
…
ld r1, (r3)
add r3, r1, r2
sub r6, r7, r9
add r3, r3, r6
ld r6, (r1)
add r6, r6, r3
st r6, (r1)
ld r6, (r1)
…
Commit
Fetch
Cycle t + 1
Execute
9/13/2007 43
Superscalar Register Renaming• During decode, instructions allocated new physical destination register
• Source operands renamed to physical register with newest value• Execution unit only sees physical register numbers
Rename Table
Op Src1 Src2Dest Op Src1 Src2Dest
RegisterFree List
Op PSrc1 PSrc2PDestOp PSrc1 PSrc2PDest
UpdateMapping
Does this work?
Inst 1 Inst 2
Read Addresses
Read Data
Wri
tePort
s
9/13/2007 44
Superscalar Register Renaming
Rename Table
Op Src1 Src2Dest Op Src1 Src2Dest
RegisterFree List
Op PSrc1 PSrc2PDestOp PSrc1 PSrc2PDest
UpdateMapping
Inst 1 Inst 2
Read Addresses
Read Data
Wri
tePort
s
=?=?
Must check forRAW hazardsbetweeninstructionsissuing in samecycle. Can bedone in parallelwith renamelookup.
MIPS R10K renames 4 serially-RAW-dependent insts/cycle
9/13/2007 45
Memory Dependencies
st r1, (r2)
ld r3, (r4)
When can we execute the load?
9/13/2007 46
In-Order Memory Queue
• Execute all loads and stores in program order
=> Load and store cannot leave ROB for execution untilall previous loads and stores have completedexecution
• Can still execute loads and stores speculatively, andout-of-order with respect to other instructions
9/13/2007 47
Conservative O-o-O Load Execution
st r1, (r2)
ld r3, (r4)
• Split execution of store instruction into two phases: addresscalculation and data write
• Can execute load before store, if addresses known and r4 != r2
• Each load address compared with addresses of all previousuncommitted stores (can use partial conservative check i.e.,bottom 12 bits of address)
• Don’t execute load if any previous store address not known
(MIPS R10K, 16 entry address queue)
9/13/2007 48
Address Speculation
• Guess that r4 != r2
• Execute load before store address known
• Need to hold all completed but uncommitted load/store addresses inprogram order
• If subsequently find r4==r2, squash load and all followinginstructions
=> Large penalty for inaccurate address speculation
st r1, (r2)
ld r3, (r4)
9/13/2007 49
Memory Dependence Prediction(Alpha 21264)
st r1, (r2)
ld r3, (r4)
• Guess that r4 != r2 and execute load before store
• If later find r4==r2, squash load and all followinginstructions, but mark load instruction as store-wait
• Subsequent executions of the same load instruction willwait for all previous stores to complete
• Periodically clear store-wait bits
9/13/2007 50
Speculative Loads / Stores
Just like register updates, stores should not modifythe memory until after the instruction is committed
- A speculative store buffer is a structure introduced to holdspeculative store data.
9/13/2007 51
Speculative Store Buffer
• On store execute:– mark entry valid and speculative, and save data and tag of instruction.
• On store commit:– clear speculative bit and eventually move data to cache
• On store abort:– clear valid bit
Data
Load Address
Tags
Store Commit Path
SpeculativeStoreBuffer
L1 DataCache
Load Data
Tag DataSVTag DataSVTag DataSVTag DataSVTag DataSVTag DataSV
9/13/2007 52
Speculative Store Buffer
• If data in both store buffer and cache, which should we use:
• If same address in store buffer twice, which should we use:
Data
Load Address
Tags
Store Commit Path
SpeculativeStoreBuffer
L1 DataCache
Load Data
Tag DataSVTag DataSVTag DataSVTag DataSVTag DataSVTag DataSV
9/13/2007 53
FetchDecode &Rename
Reorder BufferPC
BranchPrediction
Update predictors
Commit
Datapath: Branch Predictionand Speculative Execution
BranchResolution
BranchUnit
ALU
Reg. File
MEMStoreBuffer
D$
Execute
kill
kill
killkill
9/13/2007 54
Paper Discussion: CISC vs RISC
Recommended optional further reading:• D. Bhandarkar and D. W. Clark. “Performance from
architecture: Comparing a RISC and a CISC with similarhardware organization”, In Intl. Conf. on ArchitecturalSupport for Prog. Lang. and Operating Sys., ASPLOS-IV, Santa Clara, CA, Apr. 1991, pages 310--319 -conclusion is RISC is 2.7x better than CISC!