1
Lecture 8: Instruction Fetch, ILP Limits
• Today: advanced branch prediction, limits of ILP (Sections 3.4-3.5, 3.8-3.14)
2
1-Bit Prediction
• For each branch, keep track of what happened last time and use that outcome as the prediction
• What are prediction accuracies for branches 1 and 2 below:
while (1) { for (i=0;i<10;i++) { branch-1 … } for (j=0;j<20;j++) { branch-2 … } }
3
2-Bit Prediction
• For each branch, maintain a 2-bit saturating counter: if the branch is taken: counter = min(3,counter+1) if the branch is not taken: counter = max(0,counter-1)
• If (counter >= 2), predict taken, else predict not taken
• Advantage: a few atypical branches will not influence the prediction (a better measure of “the common case”)
• Especially useful when multiple branches share the same counter (some bits of the branch PC are used to index into the branch predictor)
• Can be easily extended to N-bits (in most processors, N=2)
4
Correlating Predictors
• Basic branch prediction: maintain a 2-bit saturating counter for each entry (or use 10 branch PC bits to index into one of 1024 counters) – captures the recent “common case” for each branch
• Can we take advantage of additional information? If a branch recently went 01111, expect 0; if it recently went 11101, expect 1; can we have a separate counter for each case? If the previous branches went 01, expect 0; if the previous branches went 11, expect 1; can we have a separate counter for each case?
Hence, build correlating predictors
5
Local/Global Predictors
• Instead of maintaining a counter for each branch to capture the common case,
Maintain a counter for each branch and surrounding pattern If the surrounding pattern belongs to the branch being predicted, the predictor is referred to as a local predictor If the surrounding pattern includes neighboring branches, the predictor is referred to as a global predictor
6
Global Predictor
A single register that keeps trackof recent history for all branches
00110101
Branch PC
8 bits6 bits
Table of16K entries
of 2-bitsaturatingcounters
Also referred to as a two-level predictor
7
Local Predictor
Branch PC
Table of16K entries
of 2-bitsaturatingcounters
Table of 64 entries of 14-bithistories for a single branch
10110111011001
Use 6 bits of branch PC toindex into local history table
14-bit historyindexes into
next level
Also a two-level predictor that onlyuses local histories at the first level
8
Tournament Predictors
• A local predictor might work well for some branches or programs, while a global predictor might work well for others
• Provide one of each and maintain another predictor to identify which predictor is best for each branch
TournamentPredictor
Branch PC
Table of 2-bitsaturating counters
LocalPredictor
GlobalPredictor
MUX
Alpha 21264:1K entries in level-11K entries in level-2
4K entries12-bit global history
4K entries
Total capacity: ?
9
Predictor Comparison
• Note that predictors of equal capacity must be compared• Sizes of each level have to be selected to optimize prediction accuracy• Influencing factors: degree of interference between branches, program likely to benefit from local/global history
10
Branch Target Prediction
• In addition to predicting the branch direction, we must also predict the branch target address
• Branch PC indexes into a predictor table; indirect branches might be problematic
• Most common indirect branch: return from a procedure – can be easily handled with a stack of return addresses
11
Multiple Instruction Issue
• The out-of-order processor implementation can be easily extended to have multiple instructions in each pipeline stage
• Increased complexity (lower clock speed!): more reads and writes per cycle to register map table more read and write ports in issue queue more tags being broadcast to issue queue every cycle higher complexity for bypassing/forwarding among FUs more register read and write ports more ports in the LSQ more ports in the data cache more ports in the ROB
12
ILP Limits
• The perfect processor: Infinite registers (no WAW or WAR hazards) Perfect branch direction and target prediction Perfect memory disambiguation Perfect instruction and data caches Single-cycle latencies for all ALUs Infinite ROB size (window of in-flight instructions) No limit on number of instructions in each pipeline stage
• The last instruction may be scheduled in the first cycle
• The only constraint is a true dependence (register or memory RAW hazards) (with value prediction, how would the perfect processor behave?)
13
Infinite Window Size and Issue Rate
14
Effect of Window Size
• Window size is effected by register file/ROB size, branch mispredict rate, fetch bandwidth, etc.• We will use a window size of 2K instrs and a max issue rate of 64 for subsequent experiments
15
Imperfect Branch Prediction
• Note: no branch mispredict penalty; branch mispredict restricts window size• Assume a large tournament predictor for subsequent experiments
16
Effect of Name Dependences
• More registers fewer WAR and WAW constraints (usually register file size goes hand in hand with in-flight window size)• 256 int and fp registers for subsequent experiments
17
Memory Dependences
18
Limits of ILP – Summary
• Int programs are more limited by branches, memory disambiguation, etc., while FP programs are limited most by window size
• We have not yet examined the effect of branch mispredict penalty and imperfect caching
• All of the studied factors have relatively comparable influence on CPI: window/register size, branch prediction, memory disambiguation
• Can we do better? Yes: better compilers, value prediction, memory dependence prediction, multi-path execution
19
Pentium III (P6 Microarchitecture) Case Study
• 14-stage pipeline: 8 for fetch/decode/dispatch, 3+ for o-o-o, 3 for commit branch mispredict penalty of 10-15 cycles
• Out-of-order execution with a 40-entry ROB (40 temporary or virtual registers) and 20 reservation stations
• Each x86 instruction gets converted into RISC-like micro-ops – on average, one CISC instr 1.37 micro-ops
• Three instructions in each pipeline stage 3 instructions can simultaneously leave the pipeline ideal CPI = 0.33 ideal CPI = 0.45
20
Branch Prediction
• 512-entry global two-level branch predictor and 512-entry BTB 20% combined mispredict rate
• For every instruction committed, 0.2 instructions on the mispredicted path are also executed (wasted power!)
• Mispredict penalty is 10-15 cycles
21
Where is Time Lost?
• Branch mispredict stalls
• Cache miss stalls (dominated by L1D misses)
• Instruction fetch stalls (happens often because subsequent stages are stalled, and occasionally because of an I-cache miss
22
CPI Performance
• Owing to stalls, the processor can fall behind (no instructions are committed for 55% of all cycles), but then recover with multi-instruction commits (31% of all cycles) average CPI = 1.15 (Int) and 2.0 (FP)• Overlap of different stalls CPI is not the sum of individual stalls• IPC is also an attractive metric
23
Title
• Bullet