pipelined implementation part ii

PipelinedPipelinedImplementationImplementation

Part IIPart II

PipelinedPipelinedImplementationImplementation

Part IIPart II

– 2 – Processor

OverviewOverview

Make the pipelined processor work!

Data HazardsData Hazards Instruction having register R as source follows shortly after

instruction having register R as destination Common condition, don’t want to slow down pipeline

Control HazardsControl Hazards Mispredict conditional branch

Our design predicts all branches as being takenNaïve pipeline executes two extra instructions

Getting return address for ret instructionPIPE- executes three extra instructions

Making Sure It Really WorksMaking Sure It Really Works What if multiple special cases happen simultaneously?

– 3 – Processor

Suggested ReadingSuggested Reading

- - Chap 4.5Chap 4.5

– 4 – Processor

Branch Misprediction ExampleBranch Misprediction Example

Should only execute first 7 instructions

0x000: xorl %eax,%eax 0x002: jne t # Not taken 0x007: irmovl $1, %eax # Fall through 0x00d: nop 0x00e: nop 0x00f: nop 0x010: halt 0x011: t: irmovl $3, %edx # Target (Should not execute) 0x017: irmovl $4, %ecx # Should not execute 0x01d: irmovl $5, %edx # Should not execute

demo-j.ys

– 5 – Processor

Branch Misprediction TraceBranch Misprediction Trace

0x000: xorl %eax,%eax

1 2 3 4 5 6 7 8 9

F D E M

W0x002: jne t # Not taken F D E M

W

0x011: t: irmovl $3, %edx # Target F D E M W0x017: irmovl $4, %ecx # Target+1 F D E M W0x007: irmovl $1, %eax # Fall Through F D E M W

# demo-j

F D E M W

Cycle 5

E

valE 3dstE = %edx

E

valE 3dstE = %edx

MM_Bch = 0M_valA = 0x007

D

valC = 4dstE = %ecx

D

valC = 4dstE = %ecx

F

valC 1rB %eax

F

valC 1rB %eax

Incorrectly execute two instructions at branch target

– 6 – Processor

0x000: irmovl Stack,%esp # Intialize stack pointer 0x006: nop # Avoid hazard on %esp 0x007: nop 0x008: nop 0x009: call p # Procedure call 0x00e: irmovl $5,%esi # Return point 0x014: halt 0x020: .pos 0x20 0x020: p: nop # procedure 0x021: nop 0x022: nop 0x023: ret 0x024: irmovl $1,%eax # Should not be executed 0x02a: irmovl $2,%ecx # Should not be executed 0x030: irmovl $3,%edx # Should not be executed 0x036: irmovl $4,%ebx # Should not be executed 0x100: .pos 0x100 0x100: Stack: # Stack: Stack pointer

Return ExampleReturn Example

Require lots of nops to avoid data hazards

demo-ret.ys

– 7 – Processor

Incorrect Return ExampleIncorrect Return Example0x023: ret F D E M

W0x024: irmovl $1,%eax # Oops! F D E M

W

0x02a: irmovl $2,%ecx # Oops! F D E M W0x030: irmovl $3,%edx # Oops! F D E M W0x00e: irmovl $5,%esi # Return F D E M W

# demo-ret

F D E M W

EvalE 2dstE = %ecx

MvalE = 1dstE = %eax

DvalC = 3dstE = %edx

FvalC 5rB %esi

W

valM = 0x0e

0x023: ret F D E M

W0x024: irmovl $1,%eax # Oops! F D E M

W

0x02a: irmovl $2,%ecx # Oops! F D E M W0x030: irmovl $3,%edx # Oops! F D E M W0x00e: irmovl $5,%esi # Return F D E M W

# demo-ret

F D E M W

EvalE 2dstE = %ecx

EvalE 2dstE = %ecx





FvalC 5rB %esi

FvalC 5rB %esi

W

valM = 0x0e

W

valM = 0x0e

Incorrectly execute 3 instructions following ret

– 8 – Processor

Handling MispredictionHandling Misprediction

Predict branch as takenPredict branch as taken Fetch 2 instructions at target

Cancel when mispredictedCancel when mispredicted Detect branch not-taken in execute stage On following cycle, replace instructions in execute and

decode by bubbles No side effects have occurred yet


1 2 3 4 5 6 7 8 9

F D E M WF D E M W0x002: jne target # Not taken F D E M WF D E M W

E M W

10# demo-j.ys

0x011: t: irmovl $2,%edx # Target

bubble

0x017: irmovl $3,%ebx # Target+1

F D

E M W

D

Fbubble

0x007: irmovl $1,%eax # Fall through

0x00d: nop

F D E M WF D E M W

F D E M WF D E M W

Figure 4.63 P346

– 9 – Processor

Detecting Mispredicted BranchDetecting Mispredicted Branch

ConditionCondition TriggerTrigger

Mispredicted BranchMispredicted Branch E_icode = IJXX & !e_BchE_icode = IJXX & !e_Bch

M

F

D

Instructionmemory

Instructionmemory

PCincrement

PCincrement

Registerfile

Registerfile

CCCC ALUALU

Datamemory

Datamemory

SelectPC

rB

dstE dstM

ALUA

ALUB

Mem.control

Addr

srcA srcB

read

write

ALUfun.

Fetch

Decode

Execute

Memory

Write back

data out

data in

A BM

E

M_valA

W_valE

W_valM

W_valE

M_valA

W_valM

f_PC

PredictPC

Bchicode valE valA dstE dstM

E icode ifun valC valA valB dstE dstM srcA srcB

valC valPicode ifun rA

predPC

d_srcBd_srcA

e_Bch

M_Bch

Sel+FwdA

FwdB

W icode valE valM dstE dstM

m_valM

W_valM

M_valE

e_valE

M

F

D

Instructionmemory

Instructionmemory

PCincrement

PCincrement

Registerfile

Registerfile

CCCC ALUALU

Datamemory

Datamemory

SelectPC

rB

dstE dstM

ALUA

ALUB

Mem.control

Addr

srcA srcB

read

write

ALUfun.

Fetch

Decode

Execute

Memory

Write back

data out

data in

A BM

E

M_valA

W_valE

W_valM

W_valE

M_valA

W_valM

f_PC

PredictPC




predPC

d_srcBd_srcA

e_Bch

M_Bch

Sel+FwdA

FwdB


m_valM

W_valM

M_valE

e_valE

m_valM

W_valM

M_valE

e_valE

M

F

D

Instructionmemory

Instructionmemory

PCincrement

PCincrement

Registerfile

Registerfile

CCCC ALUALU

Datamemory

Datamemory

SelectPC

rB

dstE dstM

ALUA

ALUB

Mem.control

Addr

srcA srcB

read

write

ALUfun.

Fetch

Decode

Execute

Memory

Write back

data out

data in

A BM

E

M_valA

W_valE

W_valM

W_valE

M_valA

W_valM

f_PC

PredictPC




predPC

d_srcBd_srcA

e_Bch

M_Bch

Sel+FwdA

FwdB


m_valM

W_valM

M_valE

e_valE

m_valM

W_valM

M_valE

e_valE

M

F

D

Instructionmemory

Instructionmemory

PCincrement

PCincrement

Registerfile

Registerfile

CCCC ALUALU

Datamemory

Datamemory

SelectPC

rB

dstE dstM

ALUA

ALUB

Mem.control

Addr

srcA srcB

read

write

ALUfun.

Fetch

Decode

Execute

Memory

Write back

data out

data in

A BM

E

M_valA

W_valE

W_valM

W_valE

M_valA

W_valM

f_PC

PredictPC




predPC

d_srcBd_srcA

e_Bch

M_Bch

Sel+FwdA

FwdB


m_valM

W_valM

M_valE

e_valE

m_valM

W_valM

M_valE

e_valE

m_valM

W_valM

M_valE

e_valE

m_valM

W_valM

M_valE

e_valE

Figure 4.64 P347

– 10 – Processor

Control for MispredictionControl for Misprediction


1 2 3 4 5 6 7 8 9

F D E M WF D E M W

0x002: jne t # Not taken F D E M WF D E M W

E M W

10# demo-j.ys

0x011: t: irmovl $2,%edx # Target

bubble

0x017: irmovl $3,%ebx # Target+1

F D

E M W

D

F

bubble

0x007: irmovl $1,%eax # Fall through

0x00d: nop

F D E M WF D E M W

F D E M WF D E M W

ConditionCondition FF DD EE MM WW

Mispredicted BranchMispredicted Branch normalnormal bubblebubble bubblebubble normalnormal normalnormal

Figure 4.63 P346

Figure 4.66 P348


0x000: irmovl Stack,%esp # Initialize stack pointer 0x006: call p # Procedure call 0x00b: irmovl $5,%esi # Return point 0x011: halt 0x020: .pos 0x20 0x020: p: irmovl $-1,%edi # procedure 0x026: ret 0x027: irmovl $1,%eax # Should not be executed 0x02d: irmovl $2,%ecx # Should not be executed 0x033: irmovl $3,%edx # Should not be executed 0x039: irmovl $4,%ebx # Should not be executed 0x100: .pos 0x100 0x100: Stack: # Stack: Stack pointer

Return ExampleReturn Example

Previously executed three additional instructions

demo-retb.ys


0x026: ret F D E M

Wbubble F D E M

W

bubble F D E M W

bubble F D E M W

0x00b: irmovl $5,%esi # Return F D E M W

# demo-retb

F D E M W

FvalC 5rB %esi

FvalC 5rB %esi

W

valM = 0x0b

W

valM = 0x0b

•••

Correct Return ExampleCorrect Return Example

As ret passes through pipeline, stall at fetch stage

While in decode, execute, and memory stage

fetch the same instruction after ret 3 times.

Inject bubble into decode stage

Release stall when reach write-back stage

Figure 4.61 P344


Detecting ReturnDetecting Return


Processing retProcessing ret IRET in { D_icode, E_icode, M_icode }IRET in { D_icode, E_icode, M_icode }

M

D

Registerfile

Registerfile

CCCC ALUALU

rB

dstE dstM

ALUA

ALUB

srcA srcB

ALUfun.

Decode

Execute

A BM

E

W_valM

W_valE




d_srcBd_srcA

e_Bch

M_Bch

Sel+FwdA

FwdB

M_valE

e_valE

Figure 4.64 P347


0x026: ret F D E M

Wbubble F D E M

W

bubble F D E M W

bubble F D E M W

0x00b: irmovl $5,%esi # Return F D E M W

# demo-retb

F D E M W

Control for ReturnControl for Return


Processing retProcessing ret stallstall bubblebubble normalnormal normalnormal normalnormal

Figure 4.66 P348


Special Control CasesSpecial Control CasesDetectionDetection

ActionAction


Processing retProcessing ret IRET in { D_icode, E_icode, M_icode }IRET in { D_icode, E_icode, M_icode }

Load/Use HazardLoad/Use Hazard E_icode in { IMRMOVL, IPOPL } && E_icode in { IMRMOVL, IPOPL } && E_dstM in { d_srcA, d_srcB }E_dstM in { d_srcA, d_srcB }

Mispredicted BranchMispredicted Branch E_icode = IJXX & !e_BchE_icode = IJXX & !e_Bch



Load/Use HazardLoad/Use Hazard stallstall stallstall bubblebubble normalnormal normalnormal


Figure 4.64 P347

Figure 4.66 P348


Implementing Pipeline ControlImplementing Pipeline Control

Combinational logic generates pipeline control signals Action occurs at start of following cycle

E

M

W

F

D

CCCC

rB

srcA

srcB

icode valE valM dstE dstM


icode ifun valC valA valB dstE dstM srcA srcB


predPC

d_srcB

d_srcA

e_Bch

D_icode

E_icode

M_icode

E_dstM

Pipecontrollogic

D_bubble

D_stall

E_bubble

F_stall

Figure 4.68 P351


Initial Version of Pipeline ControlInitial Version of Pipeline Controlbool F_stall =

# Conditions for a load/use hazardE_icode in { IMRMOVL, IPOPL } && E_dstM in { d_srcA, d_srcB } ||# Stalling at fetch while ret passes through pipelineIRET in { D_icode, E_icode, M_icode };

bool D_stall = # Conditions for a load/use hazardE_icode in { IMRMOVL, IPOPL } && E_dstM in { d_srcA, d_srcB };

bool D_bubble =# Mispredicted branch(E_icode == IJXX && !e_Bch) ||# Stalling at fetch while ret passes through pipeline IRET in { D_icode, E_icode, M_icode };

bool E_bubble =# Mispredicted branch(E_icode == IJXX && !e_Bch) ||# Load/use hazardE_icode in { IMRMOVL, IPOPL } && E_dstM in { d_srcA, d_srcB};


Control CombinationsControl Combinations

Special cases that can arise on same clock cycle

Combination ACombination A Not-taken branch ret instruction at branch target

Combination BCombination B Instruction that reads from memory to %esp Followed by ret instruction

LoadE

UseD

M

Load/use

JXXE

D

M

Mispredict

JXXE

D

M

Mispredict

EretD

M

ret 1

retE

bubbleD

M

ret 2

bubbleE

bubbleD

retM

ret 3

EretD

M

ret 1

EretD

M

ret 1

retE

bubbleD

M

ret 2

retE

bubbleD

M

ret 2

bubbleE

bubbleD

retM

ret 3

bubbleE

bubbleD

retM

ret 3

Combination B

Combination A

Figure 4.67 P349


Control Combination AControl Combination A

Should handle as mispredicted branch Stalls F pipeline register But PC selection logic will be using M_valA anyhow

JXXE

D

M

Mispredict

JXXE

D

M

Mispredict

E

retD

M

ret 1

E

retD

M

ret 1

E

retD

M

ret 1

Combination A




CombinationCombination stallstall bubblebubble bubblebubble normalnormal normalnormal

E

M

W

F

D

Instructionmemory

Instructionmemory

PCincrement

PCincrement

Registerfile

Registerfile

ALUALU

Datamemory

Datamemory

SelectPC

rB

dstE dstMSelectA

ALUA

ALUB

Mem.control

Addr

srcA srcB

read

write

ALUfun.

Fetch

Decode

Execute

Memory

Write back

icode

data out

data in

A BM

E

M_valA

W_valM

W_valE

M_valA

W_valM

d_rvalA

f_PC

PredictPC

valE valM dstE dstM


icode ifun valC valA valB dstE dstM srcA srcB


predPC

CCCC

d_srcBd_srcA

e_Bch

M_Bch

CCCC

d_srcBd_srcA

e_Bch

M_Bch


Control Combination BControl Combination B

Would attempt to bubble and stall pipeline register D Signaled by processor as pipeline error

LoadE

UseD

M

Load/use

E

retD

M

ret 1

E

retD

M

ret 1

E

retD

M

ret 1

Combination B




CombinationCombination stallstall bubble + bubble + stallstall

bubblebubble normalnormal normalnormal


Handling Control Combination BHandling Control Combination B

Load/use hazard should get priority ret instruction should be held in decode stage for additional

cycle

LoadE

UseD

M

Load/use

E

retD

M

ret 1

E

retD

M

ret 1

E

retD

M

ret 1

Combination B




CombinationCombination stallstall stallstall bubblebubble normalnormal normalnormal


Corrected Pipeline Control LogicCorrected Pipeline Control Logic

Load/use hazard should get priority ret instruction should be held in decode stage for additional

cycle




CombinationCombination stallstall stallstall bubblebubble normalnormal normalnormal

bool D_bubble =# Mispredicted branch(E_icode == IJXX && !e_Bch) ||# Stalling at fetch while ret passes through pipeline IRET in { D_icode, E_icode, M_icode } # but not condition for a load/use hazard && !(E_icode in { IMRMOVL, IPOPL }

&& E_dstM in { d_srcA, d_srcB });


Pipeline SummaryPipeline Summary

Data HazardsData Hazards Most handled by forwarding

No performance penalty

Load/use hazard requires one cycle stall

Control HazardsControl Hazards Cancel instructions when detect mispredicted branch

Two clock cycles wasted

Stall fetch stage while ret passes through pipelineThree clock cycles wasted

Control CombinationsControl Combinations Must analyze carefully First version had subtle bug

Only arises with unusual instruction combination


Performance MetricsPerformance Metrics

Clock rateClock rate Measured in Megahertz or Gigahertz Function of stage partitioning and circuit design

Keep amount of work per stage small

Rate at which instructions executedRate at which instructions executed CPI: cycles per instruction On average, how many clock cycles does each instruction

require? Function of pipeline design and benchmark programs

E.g., how frequently are branches mispredicted?

4.5.10


CPI for PIPECPI for PIPE

CPI CPI 1.0 1.0 Fetch instruction each clock cycle Effectively process new instruction almost every cycle

Although each individual instruction has latency of 5 cycles

CPI CPI >> 1.0 1.0 Sometimes must stall or cancel branches

Computing CPIComputing CPI C clock cycles I instructions executed to completion B bubbles injected (C = I + B)

CPI = C/I = (I+B)/I = 1.0 + B/I Factor B/I represents average penalty due to bubbles


CPI for PIPE (Cont.)CPI for PIPE (Cont.)B/I = LP + MP + RP

LP: Penalty due to load/use hazard stalling Fraction of instructions that are loads 0.25 Fraction of load instructions requiring stall 0.20 Number of bubbles injected each time 1

LP = 0.25 * 0.20 * 1 = 0.05

MP: Penalty due to mispredicted branches Fraction of instructions that are cond. jumps 0.20 Fraction of cond. jumps mispredicted 0.40 Number of bubbles injected each time 2

MP = 0.20 * 0.40 * 2 = 0.16

RP: Penalty due to ret instructions Fraction of instructions that are returns 0.02 Number of bubbles injected each time 3

RP = 0.02 * 3 = 0.06

Net effect of penalties 0.05 + 0.16 + 0.06 = 0.27 CPI = 1.27 (Not bad!)

Typical Values


Fetch Logic RevisitedFetch Logic Revisited

During Fetch CycleDuring Fetch Cycle1. Select PC

2. Read bytes from instruction memory

3. Examine icode to determine instruction length

4. Increment PC

TimingTiming Steps 2 & 4 require

significant amount of time F

D rB

M_icode

PredictPC


Instructionmemory

Instructionmemory

PCincrement

PCincrement

predPC

Needregids

NeedvalC

Instrvalid

AlignAlignSplitSplit

Bytes 1-5Byte 0

SelectPC

M_Bch

M_valA

W_icode

W_valM


Standard Fetch TimingStandard Fetch Timing

Must Perform Everything in Sequence Can’t compute incremented PC until know how much to

increment it by

Select PC

Mem. Read Increment

need_regids, need_valC

1 clock cycle


A Fast PC Increment Circuit A Fast PC Increment Circuit

3-bit adder

need_ValC

need_regids

0

29-bitincre-

menter

MUX

High-order 29 bits

Low-order 3 bits

High-order 29 bits Low-order 3 bits

0 1

PC

incrPC

Slow Fast

carry


Modified Fetch TimingModified Fetch Timing

29-Bit Incrementer29-Bit Incrementer Acts as soon as PC selected Output not needed until final MUX Works in parallel with memory read

Select PC

Mem. Read

Incrementer

need_regids, need_valC3-bit add

MUX

1 clock cycle

Standard cycle


More Realistic Fetch LogicMore Realistic Fetch Logic

Fetch BoxFetch Box Integrated into instruction cache Fetches entire cache block (16 or 32 bytes) Selects current instruction from current block Works ahead to fetch next block

As reaches end of current blockAt branch target

InstructionCache

InstructionCache

Bytes 1-5Byte 0

Current Block

Next Block

CurrentInstructionCurrent

InstructionInstr.

LengthInstr.

LengthFetch

ControlFetch

Control

Other PC Controls


ExceptionException

Condition: Instruction encounter an error conditionCondition: Instruction encounter an error condition

Deal flow:Deal flow: Break the program flow Invoke the exception hander provided by OS. (Maybe) continue the program flow

E.g. page fault exception


ExceptionsExceptions Conditions under which pipeline cannot continue normal

operation

CausesCauses Halt instruction (Current) Bad address for instruction or data (Previous) Invalid instruction (Previous) Pipeline control error (Previous)

Desired ActionDesired Action Complete some instructions

Either current or previous (depends on exception type)

Discard others Call exception handler

Like an unexpected procedure call


Exception ExamplesException Examples

Detect in Fetch StageDetect in Fetch Stage

irmovl $100,%eax rmmovl %eax,0x10000(%eax) # invalid address

jmp $-1 # Invalid jump target

.byte 0xFF # Invalid instruction code

halt # Halt instruction

Detect in Memory StageDetect in Memory Stage


Exceptions in Pipeline Processor #1Exceptions in Pipeline Processor #1

Desired BehaviorDesired Behavior rmmovl should cause exception

# demo-exc1.ys irmovl $100,%eax rmmovl %eax,0x10000(%eax) # Invalid address nop .byte 0xFF # Invalid instruction code

0x000: irmovl $100,%eax

1 2 3 4

F D E M

F D E0x006: rmmovl %eax,0x10000(%eax)

0x00c: nop

0x00d: .byte 0xFF

F D

F

W

5

M

E

D

Exception detected

Exception detected


Exceptions in Pipeline Processor #2Exceptions in Pipeline Processor #2

Desired BehaviorDesired Behavior No exception should occur

# demo-exc2.ys 0x000: xorl %eax,%eax # Set condition codes 0x002: jne t # Not taken 0x007: irmovl $1,%eax 0x00d: irmovl $2,%edx 0x013: halt 0x014: t: .byte 0xFF # Target


1 2 3

F D E

F D0x002: jne t

0x014: t: .byte 0xFF

0x???: (I’m lost!)

F

Exception detected


4

M

E

F

D

W

5

M

D

F

E

E

D

M

6

M

E

W

7

W

M

8

W

9


Maintaining Exception OrderingMaintaining Exception Ordering

Add exception status field to pipeline registers Fetch stage sets to either “AOK,” “ADR” (when bad fetch

address), or “INS” (illegal instruction) Decode & execute pass values through Memory either passes through or sets to “ADR” Exception triggered only when instruction hits write back

F predPC

W icode valE valM dstE dstMexc

M Bchicode valE valA dstE dstMexc

E icode ifun valC valA valB dstE dstM srcA srcBexc

D rB valC valPicode ifun rAexc


Side Effects in Pipeline ProcessorSide Effects in Pipeline Processor

Desired BehaviorDesired Behavior rmmovl should cause exception No following instruction should have any effect

# demo-exc3.ys irmovl $100,%eax rmmovl %eax,0x10000(%eax) # invalid address addl %eax,%eax # Sets condition codes


1 2 3 4

F D E M

F D E0x006: rmmovl %eax,0x10000(%eax)

0x00c: addl %eax,%eax F D

W

5

M

E

Exception detected

Condition code set


Avoiding Side EffectsAvoiding Side Effects

Presence of Exception Should Disable State UpdatePresence of Exception Should Disable State Update When detect exception in memory stage

Disable condition code setting in executeMust happen in same clock cycle

When exception passes to write-back stageDisable memory write in memory stageDisable condition code setting in execute stage

ImplementationImplementation Hardwired into the design of the PIPE simulator You have no control over this


Rest of Exception HandlingRest of Exception Handling

Calling Exception HandlerCalling Exception Handler Push PC onto stack

Either PC of faulting instruction or of next instructionUsually pass through pipeline along with exception status

Jump to handler addressUsually fixed addressDefined as part of ISA

ImplementationImplementation Haven’t tried it yet!


Modern CPU DesignModern CPU Design

ExecutionExecution

FunctionalUnits

Instruction ControlInstruction Control

Integer/Branch

FPAdd

FPMult/Div

Load Store

InstructionCache

DataCache

FetchControl

InstructionDecode

Address

Instructions

Operations

PredictionOK?

DataData

Addr. Addr.

GeneralInteger

Operation Results

RetirementUnit

RegisterFile

RegisterUpdates


Processor SummaryProcessor Summary

Design TechniqueDesign Technique Create uniform framework for all instructions

Want to share hardware among instructions

Connect standard logic blocks with bits of control logic

OperationOperation State held in memories and clocked registers Computation done by combinational logic Clocking of registers/memories sufficient to control overall

behavior

Enhancing PerformanceEnhancing Performance Pipelining increases throughput and improves resource

utilization Must make sure maintains ISA behavior

pipelined implementation part ii

Documents