computer architecture and design – elen 350 part 8 [some slides adapted from m. irwin, d....
TRANSCRIPT
Computer Architecture and Design – ELEN 350
Part 8
[Some slides adapted from M. Irwin, D. Paterson. D. Garcia and others]
Review: Single Cycle vs. Multiple Cycle Timing
Clk Cycle 1
Multiple Cycle Implementation:
IFetch Dec Exec Mem WB
Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9Cycle 10
IFetch Dec Exec Mem
lw sw
IFetch
R-type
Clk
Single Cycle Implementation:
lw sw Waste
Cycle 1 Cycle 2
multicycle clock slower than 1/5th of single cycle clock due to stage register overhead
How Can We Make It Even Faster?
Split the multiple instruction cycle into smaller and smaller steps
There is a point of diminishing returns where as much time is spent loading the state registers as doing the work
Start fetching and executing the next instruction before the current one has completed
Pipelining – (all?) modern processors are pipelined for performance
Fetch (and execute) more than one instruction at a time (out-of-order superscalar and VLIW )
Fetch (and execute) instructions from more than one instruction stream (multithreading)
Pipelining
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Time76 PM 8 9 10 11 12 1 2 AM
A
B
C
D
Taskorder
Taskorder
Assume 30 min. each task – wash, dry, fold, store – and that separate tasks use separate hardware and so can be overlapped
Pipelined
Not pipelined
A Pipelined MIPS Processor Start the next instruction before the current one has
completed improves throughput - total amount of work done in a given
time instruction latency (execution time, delay time, response
time - time from the start of an instruction to its completion) is not reduced
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5
IFetch Dec Exec Mem WBlw
Cycle 7Cycle 6 Cycle 8
sw IFetch Dec Exec Mem WB
R-type IFetch Dec Exec Mem WB
- clock cycle (pipeline stage time) is limited by the slowest stage
- for some instructions, some stages are wasted cycles
Single Cycle, Multiple Cycle, vs. Pipeline
Multiple Cycle Implementation:
Clk
Cycle 1
IFetch Dec Exec Mem WB
Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9Cycle 10
IFetch Dec Exec Mem
lw sw
IFetch
R-type
lw IFetch Dec Exec Mem WB
Pipeline Implementation:
IFetch Dec Exec Mem WBsw
IFetch Dec Exec Mem WBR-type
Clk
Single Cycle Implementation:
lw sw Waste
Cycle 1 Cycle 2
MIPS Pipeline Datapath Modifications What do we need to add/modify in our MIPS datapath?
State registers between each pipeline stage to isolate them
ReadAddress
InstructionMemory
Add
PC
4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register
File
Read Data 1
Read Data 2
16 32
ALU
Shiftleft 2
Add
DataMemory
Address
Write Data
ReadDataIF
etc
h/D
ec
De
c/E
xe
c
Ex
ec
/Me
m
Me
m/W
B
IF:IFetch ID:Dec EX:Execute MEM:MemAccess
WB:WriteBack
System Clock
SignExtend
?
MIPS Pipeline Control Path Modifications All control signals can be determined during Instruction
Decode Step and held in the state registers between pipeline stages
ReadAddress
InstructionMemory
Add
PC
4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register
File
Read Data 1
Read Data 2
16 32
ALU
Shiftleft 2
Add
DataMemory
Address
Write Data
ReadData
IF/ID
SignExtend
ID/EXEX/MEM
MEM/WB
Control
Pipelining the MIPS ISAWhat makes it easy
all instructions have the same length (32 bits)- can fetch in the 1st stage and decode in the 2nd stage
few instruction formats (three) with symmetry across formats
- can begin reading register file in 2nd stage memory operations can occur only in loads and stores
- can use the execute stage to calculate memory addresses each MIPS instruction writes at most one result (i.e.,
changes the machine state) and does so near the end of the pipeline (MEM and WB)
What makes it hard structural hazards: what if we had only one memory? control hazards: what about branches? data hazards: what if an instruction’s input operands
depend on the output of a previous instruction?
Pipelined Datapath
IF/ID
Pipeline registers
5 516
RD1
RD2
RN1 RN2
WD
Register File ALU
EXTND
16 32
RD
WD
DataMemory
ADDR
Instruction I32
MUX
<<2RD
InstructionMemory
ADDR
PC
4
ADD
ADD
MUX
32
ID/EX EX/MEM MEM/WB
Zero
64 bits
97 bits 64 bits
128 bits
wide enough to hold data coming in
Pipelined Example Consider the following instruction sequence:
lw $t0, 10($t1) sw $t3, 20($t4) add $t5, $t6, $t7 sub $t8, $t9, $t10
Single-Clock-Cycle Diagram:Clock Cycle 1LW
5
RD1
RD2
RN1
RN2
WN
WD
RegisterFile
ALU
EXTND
16 32
RD
WD
DataMemory
ADDR
32
MUX
<<2
RD
InstructionMemory
ADDR
PC
4
ADD
ADD
MUX
5
5
5
IF/ID ID/EX EX/MEM MEM/WB
Zero
5
RD1
RD2
RN1
RN2
WN
WD
RegisterFile
ALU
EXTND
16 32
RD
WD
DataMemory
ADDR
32
MUX
<<2
RD
InstructionMemory
ADDR
PC
4
ADD
ADD
MUX
5
5
5
IF/ID ID/EX EX/MEM MEM/WB
Zero
Single-Clock-Cycle Diagram: Clock Cycle 2LWSW
5
RD1
RD2
RN1
RN2
WN
WD
RegisterFile
ALU
EXTND
16 32
RD
WD
DataMemory
ADDR
32
MUX
<<2
RD
InstructionMemory
ADDR
PC
4
ADD
ADD
MUX
5
5
5
IF/ID ID/EX EX/MEM MEM/WB
Zero
Single-Clock-Cycle Diagram: Clock Cycle 3
LWSWADD
5
RD1
RD2
RN1
RN2
WN
WD
RegisterFile
ALU
EXTND
16 32
RD
WD
DataMemory
ADDR
32
MUX
<<2
RD
InstructionMemory
ADDR
PC
4
ADD
ADD
MUX
5
5
5
IF/ID ID/EX EX/MEM MEM/WB
Zero
Single-Clock-Cycle Diagram: Clock Cycle 4
LWSWADDSUB
Single-Clock-Cycle Diagram: Clock Cycle 5
5
RD1
RD2
RN1
RN2
WN
WD
RegisterFile
ALU
EXTND
16 32
RD
WD
DataMemory
ADDR
32
MUX
<<2
RD
InstructionMemory
ADDR
PC
4
ADD
ADD
MUX
5
5
5
IF/ID ID/EX EX/MEM MEM/WB
Zero
LWSWADDSUB
Single-Clock-Cycle Diagram: Clock Cycle 6
5
RD1
RD2
RN1
RN2
WN
WD
RegisterFile
ALU
EXTND
16 32
RD
WD
DataMemory
ADDR
32
MUX
<<2
RD
InstructionMemory
ADDR
PC
4
ADD
ADD
MUX
5
5
5
IF/ID ID/EX EX/MEM MEM/WB
Zero
SWADDSUB
Single-Clock-Cycle Diagram: Clock Cycle 7
5
RD1
RD2
RN1
RN2
WN
WD
RegisterFile
ALU
EXTND
16 32
RD
WD
DataMemory
ADDR
32
MUX
<<2
RD
InstructionMemory
ADDR
PC
4
ADD
ADD
MUX
5
5
5
IF/ID ID/EX EX/MEM MEM/WB
Zero
ADDSUB
Single-Clock-Cycle Diagram: Clock Cycle 8
5
RD1
RD2
RN1
RN2
WN
WD
RegisterFile
ALU
EXTND
16 32
RD
WD
DataMemory
ADDR
32
MUX
<<2
RD
InstructionMemory
ADDR
PC
4
ADD
ADD
MUX
5
5
5
IF/ID ID/EX EX/MEM MEM/WB
Zero
SUB
Pipelined Datapath with Control I
PC
Instructionmemory
Address
Inst
ruct
ion
Instruction[20– 16]
MemtoReg
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0Registers
Writeregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux
1Write
data
Read
data Mux
1
ALUcontrol
RegWrite
MemRead
Instruction[15– 11]
6
IF/ID ID/EX EX/MEM MEM/WB
MemWrite
Address
Datamemory
PCSrc
Zero
AddAdd
result
Shiftleft 2
ALUresult
ALU
Zero
Add
0
1
Mux
0
1
Mux
Same controlsignals as thesingle-cycledatapath
Pass control signals along just like the data – extend each pipeline register to hold needed control bits for succeeding stages
Note: The 6-bit funct field of the instruction required in the EX stage to generate ALU control can be retrieved as the 6 least significant bits of the immediate field which is sign-extended and passed from the IF/ID register to the ID/EX register
Pipeline Control Implementation
Control
EX
M
WB
M
WB
WB
IF/ID ID/EX EX/MEM MEM/WB
Instruction
Pipelined Datapath with Control II
PC
Instructionmemory
Inst
ruct
ion
Add
Instruction[20– 16]
Me
mto
Re
g
ALUOp
Branch
RegDst
ALUSrc
4
16 32Instruction[15– 0]
0
0
Mux
0
1
Add Addresult
RegistersWriteregister
Writedata
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Signextend
Mux
1
ALUresult
Zero
Writedata
Readdata
Mux
1
ALUcontrol
Shiftleft 2
Re
gWrit
e
MemRead
Control
ALU
Instruction[15– 11]
6
EX
M
WB
M
WB
WBIF/ID
PCSrc
ID/EX
EX/MEM
MEM/WB
Mux
0
1
Me
mW
rite
AddressData
memory
Address
Control signalsemanate from the controlportions of the pipeline registers
Pipelined Execution and Control
Instruction sequence:
Label “before<i>” meansi th instruction beforelw
Clock cycle 1
lw $10, 20($1)sub $11, $2, $3and $12, $4, $7or $13, $6, $7add $14, $8, $9
Clock cycle 2
lw $10, 20($1)sub $11, $2, $3and $12, $4, $7or $13, $6, $7add $14, $8, $9
Clock cycle 3
lw $10, 20($1)sub $11, $2, $3and $12, $4, $7or $13, $6, $7add $14, $8, $9
Clock cycle 4
lw $10, 20($1)sub $11, $2, $3and $12, $4, $7or $13, $6, $7add $14, $8, $9
Clock cycle 5
lw $10, 20($1)sub $11, $2, $3and $12, $4, $7or $13, $6, $7add $14, $8, $9
Clock cycle 6
lw $10, 20($1)sub $11, $2, $3and $12, $4, $7or $13, $6, $7add $14, $8, $9
Clock cycle 7
lw $10, 20($1)sub $11, $2, $3and $12, $4, $7or $13, $6, $7add $14, $8, $9
Clock cycle 8
lw $10, 20($1)sub $11, $2, $3and $12, $4, $7or $13, $6, $7add $14, $8, $9
Clock cycle 9
WB
EX
M
Instructionmemory
Address
Mem
toR
eg
ALUOp
Branch
RegDst
ALUSrc
4
0
0
1
Add Addresult
1
ALUresult
Zero
ALUcontrol
Shiftleft 2
Reg
Writ
e
M
WB
Inst
ruct
ion
IF/ID EX/MEMID/EX
ID: after<3> EX: after<2> MEM: after<1> WB: add $14, . . .
MEM/WB
IF: after<4>
000
00
0000
000
00
000
0
00
00
0
1
0
Mux
0
1
Add
PC
0Writedata
Mux
1
Control
Registers
Readdata 1
Readdata 2
Readregister 1
Readregister 2
Instruction[20– 16]
Instruction[15– 0] Sign
extend
Instruction[15– 11]
MemRead
Mem
Wri
te
Mux
Mux
ALUReaddata
WB
14
14
Writeregister
Writedata
Datamemory
Address
Clock 9
lw $10, 20($1)sub $11, $2, $3and $12, $4, $7or $13, $6, $7add $14, $8, $9
Can Pipelining Get Us Into Trouble?
Yes: Pipeline Hazards structural hazards: attempt to use the same resource by
two different instructions at the same time data hazards: attempt to use data before it is ready
- An instruction’s source operand(s) are produced by a prior instruction still in the pipeline
control hazards: attempt to make a decision about program control flow before the condition has been evaluated and the new PC target address calculated
- branch and jump instructions, exceptions
Can always resolve hazards by waiting pipeline control must detect the hazard and take action to resolve hazards
Graphically Representing MIPS Pipeline
Can help with answering questions like: How many cycles does it take to execute this code? What is the ALU doing during cycle 4? Is there a hazard, why does it occur, and how can it be
fixed?
AL
UIM Reg DM Reg
Why Pipeline? For Performance!
Instr.
Order
Time (clock cycles)
Inst 0
Inst 1
Inst 2
Inst 4
Inst 3
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM RegA
LUIM Reg DM Reg
AL
UIM Reg DM Reg
Once the pipeline is full,
one instruction is completed
every cycle so complete an
instruction every cycle
(CPI = 1)
Time to fill the pipeline
Instr.
Order
Time (clock cycles)
lw
Inst 1
Inst 2
Inst 4
Inst 3
AL
UMem Reg Mem Reg
AL
UMem Reg Mem Reg
AL
UMem Reg Mem RegA
LUMem Reg Mem Reg
AL
UMem Reg Mem Reg
A Single Memory Would Be a Structural Hazard
Reading data from memory
Reading instruction from memory
Can fix with separate instr and data memories
How About Register File Access?
Instr.
Order
Time (clock cycles)
add $1,
Inst 1
Inst 2
add $2,$1,
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
How About Register File Access?
Instr.
Order
Time (clock cycles)
Inst 1
Inst 2
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM RegA
LUIM Reg DM Reg
Fix register file access hazard by
doing reads in the second half of the
cycle and writes in the first half
add $1,
add $2,$1,
clock edge that controls register writing
clock edge that controls loading of pipeline state registers
Register Usage Can Cause Data Hazards
Instr.
Order
add $1,$2,$5
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9A
LUIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
Dependencies backward in time cause hazards
Read before write data hazard
Register Usage Can Cause Data Hazards
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
Dependencies backward in time cause hazards
add $1,$2,$5
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9
Read before write data hazard
Loads Can Cause Data Hazards
lw $1,4($2)
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9A
LUIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
Dependencies backward in time cause hazards
Loads Can Cause Data Hazards
lw $1,4($2)
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9A
LUIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
Dependencies backward in time cause hazards
Load-use data hazard
stall
stall
One Way to “Fix” a Data Hazard
add $1,
AL
UIM Reg DM Reg
sub $4,$1,$5
and $6,$1,$7
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
Can fix data hazard by
waiting – stall
Another Way to “Fix” a Data Hazard
add $1,
AL
UIM Reg DM Reg
sub $4,$1,$5
and $6,$1,$7A
LUIM Reg DM Reg
AL
UIM Reg DM Reg
Fix data hazards by forwarding results as soon as they are available to where they are needed
xor $4,$1,$5
or $8,$1,$9
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
Another Way to “Fix” a Data Hazard
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
Instr.
Order
add $1,
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9
Fix data hazards by forwarding results as soon as they are available to where they are needed
Forwarding Hardware with Control
PCInstruction
memory
Registers
Mux
Mux
Control
ALU
EX
M
WB
M
WB
WB
ID/EX
EX/MEM
MEM/WB
Datamemory
Mux
Forwardingunit
IF/ID
Inst
ruct
ion
Mux
RdEX/MEM.RegisterRd
MEM/WB.RegisterRd
Rt
Rt
Rs
IF/ID.RegisterRd
IF/ID.RegisterRt
IF/ID.RegisterRt
IF/ID.RegisterRs
Datapath with forwarding hardware and control wires – certain details,e.g., branching hardware, are omitted to simplify the drawingNote: so far we have only handled forwarding to R-type instructions…!
Forwarding with Load-use Data Hazards
Instr.
Order
lw $1,4($2)
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9A
LUIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
Forwarding with Load-use Data Hazards
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
Will still need one stall cycle even with forwarding
Instr.
Order
lw $1,4($2)
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9
Adding the Hazard Hardware
ReadAddress
InstructionMemory
Add
PC
4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register
File
Read Data 1
Read Data 2
16 32
ALU
Shiftleft 2
Add
DataMemory
Address
Write Data
ReadData
IF/ID
SignExtend
ID/EXEX/MEM
MEM/WB
Control
ALUcntrl
Branch
PCSrc
ForwardUnit
HazardUnit
0
1
Adding the Hazard Hardware
ReadAddress
InstructionMemory
Add
PC
4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register
File
Read Data 1
Read Data 2
16 32
ALU
Shiftleft 2
Add
DataMemory
Address
Write Data
ReadData
IF/ID
SignExtend
ID/EXEX/MEM
MEM/WB
Control
ALUcntrl
Branch
PCSrc
ForwardUnit
HazardUnit
0
1
ID/EX.RegisterRt
0
ID/EX.MemRead
Stall Hardware Along with the Hazard Unit, we have to implement the stall Prevent the instructions in the IF and ID stages from
progressing down the pipeline – done by preventing the PC register and the IF/ID pipeline register from changing
Hazard detection Unit controls the writing of the PC (PC.write) and IF/ID (IF/ID.write) registers
Insert a “bubble” between the lw instruction (in the EX stage) and the load-use instruction (in the ID stage) (i.e., insert a noop in the execution stream)
Set the control bits in the EX, MEM, and WB control fields of the ID/EX pipeline register to 0 (noop). The Hazard Unit controls the mux that chooses between the real control values and the 0’s.
Let the lw instruction and the instructions after it in the pipeline (before it in the code) proceed normally down the pipeline
Many Other Pipeline Structures Are Possible
What about the (slow) multiply operation? Make the clock twice as slow or … let it take two cycles (since it doesn’t use the DM stage)
AL
UIM Reg DM Reg
MUL
AL
UIM Reg DM1 RegDM2
What if the data memory access is twice as slow as the instruction memory?
make the clock twice as slow or … let data memory access take two cycles (and keep
the same clock rate)
Control Hazards When the flow of instruction addresses is not
sequential (i.e., PC = PC + 4) Conditional branches (beq, bne) Unconditional branches (j, jal, jr) Exceptions
Branches Cause Control Hazards
lw
Inst 4
Inst 3
beq
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
Dependencies backward in time cause hazards
Instr.
Order
Control Hazards Possible “solutions”
Stall (impacts performance) Move branch decision point as early in the pipeline as
possible, thereby reducing the number of stall cycles Delay decision (requires compiler support) Predict and hope for the best !
Control hazards occur less frequently than data hazards, but there is nothing as effective against control hazards as forwarding is for data hazards
flush
flush
flush
One Way to “Fix” a Branch Control Hazard
beq
AL
UIM Reg DM Reg
beq target
AL
UIM Reg DM Reg
AL
U
Inst 3IM Reg DM
Fix branch hazard by waiting –
flush – but affects CPI
AL
UIM Reg DM Reg
AL
UIM Reg DM RegA
LUIM Reg DM Reg
Instr.
Order
flush
Another Way to “Fix” a Branch Control Hazard
beq
beq target
AL
UIM Reg DM Reg
Inst 3
AL
UIM Reg DM
Fix branch hazard by waiting –
flush
AL
UIM Reg DM Reg
Move branch decision hardware back to as early in the pipeline as possible – i.e., during the decode cycle
AL
UIM Reg DM Reg
Instr.
Order
flush
Jumps Incur One Stall
j
j targetA
LUIM Reg DM Reg
AL
UIM Reg DM Reg
Fortunately, jumps are very infrequent – only 3% of the SPECint instruction mix
Jumps not decoded until ID, so one flush is needed
Fix jump hazard by waiting –
flush
AL
UIM Reg DM Reg
Instr.
Order
Delayed Decision If the branch hardware has been moved to the ID stage,
then we can eliminate all branch stalls with delayed branches which are defined as always executing the next sequential instruction after the branch instruction – the branch takes effect after that next instruction
MIPS compiler moves an instruction to immediately after the branch that is not affected by the branch (a safe instruction) thereby hiding the branch delay
With deeper pipelines, the branch delay grows requiring more than one delay slot
Delayed branches have lost popularity compared to more expensive but more flexible (dynamic) hardware branch prediction
Growth in available transistors has made hardware branch prediction relatively cheaper
Scheduling Branch Delay Slots
A is the best choice, fills delay slot and reduces IC In B and C, the sub instruction may need to be copied, increasing IC In B and C, must be okay to execute sub when branch fails
add $1,$2,$3if $2=0 then
delay slotadd $1,$2,$3if $1=0 then
delay slot
add $1,$2,$3if $1=0 then
delay slot
sub $4,$5,$6
sub $4,$5,$6
if $2=0 then
add $1,$2,$3add $1,$2,$3if $1=0 then
sub $4,$5,$6
add $1,$2,$3if $1=0 then
sub $4,$5,$6
flush
Yet Another Way to “Fix” a Control Hazard
4 beq $1,$2,2Instr.
Order
AL
UIM Reg DM Reg
16 and $6,$1,$7
20 or r8,$1,$9
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg
AL
UIM Reg DM Reg8 sub $4,$1,$5
“Predict branches are always not taken – and take corrective action when wrong (i.e., taken)
Static Branch Prediction Resolve branch hazards by assuming a given outcome
and proceeding without waiting to see the actual branch outcome
1. Predict not taken – always predict branches will not be taken, continue to fetch from the sequential instruction stream, only when branch is taken does the pipeline stall If taken, flush instructions after the branch (earlier in the pipeline)
- in IF, ID, and EX stages if branch logic in MEM – three stalls- In IF and ID stages if branch logic in EX – two stalls- in IF stage if branch logic in ID – one stall
ensure that those flushed instructions haven’t changed the machine state – automatic in the MIPS pipeline since machine state changing operations are at the tail end of the pipeline (MemWrite (in MEM) or RegWrite (in WB))
restart the pipeline at the branch destination
Branching Structures Predict not taken works well for “top of the loop”
branching structures Loop: beq $1,$2,Out 1nd loop instr . . . last loop instr j LoopOut: fall out instr
But such loops have jumps at the bottom of the loop to return to the top of the loop – and incur the jump stall overhead
Predict not taken doesn’t work well for “bottom of the loop” branching structuresLoop: 1st loop instr
2nd loop instr . . . last loop instr bne $1,$2,Loop fall out instr
Dynamic Branch Prediction A branch prediction buffer (aka branch history table
(BHT)) in the IF stage addressed by the lower bits of the PC, contains a bit passed to the ID stage through the IF/ID pipeline register that tells whether the branch was taken the last time it was executed
Prediction bit may predict incorrectly (may be a wrong prediction for this branch this iteration or may be from a different branch with the same low order PC bits) but it doesn’t affect correctness, just performance
- Branch decision occurs in the ID stage after determining that the fetched instruction is a branch and checking the prediction bit
If the prediction is wrong, flush the incorrect instruction(s) in pipeline, restart the pipeline with the right instruction, and invert the prediction bit
- A 4096 bit BHT varies from 1% misprediction (nasa7, tomcatv) to 18% (eqntott)
Branch Target Buffer The BHT predicts when a branch is taken, but does not
tell where its taken to! A branch target buffer (BTB) in the IF stage can cache the
branch target address, but we also need to fetch the next sequential instruction. The prediction bit in IF/ID selects which “next” instruction will be loaded into IF/ID at the next clock edge
- Would need a two read port instruction memory
If the prediction is correct, stalls can be avoided no matter which direction they go
Or the BTB can cache the branch taken instruction while the instruction memory is fetching the next sequential instruction
ReadAddress
InstructionMemory
PC 0
BTB
1-bit Prediction Accuracy A 1-bit predictor will be incorrect twice when not taken
For 10 times through the loop we have a 80% prediction accuracy for a branch that is taken 90% of the time
Assume predict_bit = 0 to start (indicating branch not taken) and loop control is at the bottom of the loop code
1. First time through the loop, the predictor mispredicts the branch since the branch is taken back to the top of the loop; invert prediction bit (predict_bit = 1)
2. As long as branch is taken (looping), prediction is correct
3. Exiting the loop, the predictor again mispredicts the branch since this time the branch is not taken falling out of the loop; invert prediction bit (predict_bit =
0)
Loop: 1st loop instr 2nd loop instr . . . last loop instr bne $1,$2,Loop fall out instr
2-bit Predictors A 2-bit scheme can give 90% accuracy since a prediction
must be wrong twice before the prediction bit is changed
PredictTaken
PredictNot Taken
PredictTaken
PredictNot Taken
TakenNot taken
Not taken
Not taken
Not taken
Taken
Taken
Taken
Loop: 1st loop instr 2nd loop instr . . . last loop instr bne $1,$2,Loop fall out instr
2-bit Predictors A 2-bit scheme can give 90% accuracy since a prediction
must be wrong twice before the prediction bit is changed
PredictTaken
PredictNot Taken
PredictTaken
PredictNot Taken
TakenNot taken
Not taken
Not taken
Not taken
Taken
Taken
Taken
Loop: 1st loop instr 2nd loop instr . . . last loop instr bne $1,$2,Loop fall out instr
wrong on loop fall out
0
1 1
right 9 times
right on 1st iteration
0 BHT also stores the initial FSM state
10
11
01
00
Pipelining Summary All modern day processors use pipelining Pipelining doesn’t help latency of single task, it
helps throughput of entire workload Potential speedup: a really fast clock cycle and
able to complete one instruction every clock cycle (CPI)
Pipeline rate limited by slowest pipeline stage Unbalanced pipe stages makes for inefficiencies The time to “fill” pipeline and time to “drain” it can impact
speedup for deep pipelines and short code runs
Must detect and resolve hazards Stalling negatively affects CPI (makes CPI less than the
ideal of 1)