computer architecture lecture 6: pipelining -...

Computer Architecture

Lecture 6: Pipelining

Dr. Ahmed SallamSuez Canal University

Based on original slides by Prof. Onur Mutlu

Agenda for Today & Next Few Lectures Single-cycle Microarchitectures

Multi-cycle and Microprogrammed Microarchitectures

Pipelining

Issues in Pipelining: Control & Data Dependence Handling, State Maintenance and Recovery, …

Out-of-Order Execution

Issues in OoO Execution: Load-Store Handling, …

2

Recap of Last Lecture Multi-cycle and Microprogrammed Microarchitectures

Benefits vs. Design Principles When to Generate Control Signals Microprogrammed Control: uInstruction, uSequencer, Control

Store LC-3b State Machine, Datapath, Control Structure An Exercise in Microprogramming Variable Latency Memory, Alignment, Memory Mapped I/O, …

Microprogramming Power of abstraction (for the HW designer) Advantages of uProgrammed Control Update of Machine Behavior

3

Review: A Simple LC-3b Control and Datapath

4

Microinstruction

R

Microsequencer

BEN

x2

Control Store6

IR[15:11]

6

(J, COND, IRD)

269

35

35

Simple Design of the Control Structure

A Simple DatapathCan Become Very Powerful

R

PC<! BaseR

To 18

12

To 18

To 18

RR

To 18

To 18

To 18

MDR<! SR[7:0]

MDR <! M

IR <! MDR

R

DR<! SR1+OP2*set CC

DR<! SR1&OP2*set CC

[BEN]

PC<! MDR

32

1

5

0

0

1To 18

To 18 To 18

R R

[IR[15:12]]

28

30

R7<! PCMDR<! M[MAR]

set CC

BEN<! IR[11] & N + IR[10] & Z + IR[9] & P

9DR<! SR1 XOR OP2*

4

22

To 111011

JSRJMP

BR

1010

To 10

21

200 1

LDB

MAR<! B+off6

set CC

To 18

MAR<! B+off6

DR<! MDRset CC

To 18

MDR<! M[MAR]

25

27

3762

STW STBLEASHF

TRAP

XOR

AND

ADD

RTI

To 8

set CC

set CCDR<! PC+LSHF(off9, 1)

14

LDW

MAR<! B+LSHF(off6,1) MAR<! B+LSHF(off6,1)

PC<! PC+LSHF(off9,1)

33

35

DR<! SHF(SR,A,D,amt4)

NOTESB+off6 : Base + SEXT[offset6]

R

MDR<! M[MAR[15:1]’0]

DR<! SEXT[BYTE.DATA]

R

29

31

18, 19

MDR<! SR

To 18

R R

M[MAR]<! MDR

16

23

R R

17

To 19

24

M[MAR]<! MDR**

MAR<! LSHF(ZEXT[IR[7:0]],1)

15To 18

PC+off9 : PC + SEXT[offset9]

MAR <! PCPC <! PC + 2

*OP2 may be SR2 or SEXT[imm5]** [15:8] or [7:0] depending on MAR[0]

[IR[11]]

PC<! BaseR

PC<! PC+LSHF(off11,1)

R7<! PC

R7<! PC

13

Review: The Power of Abstraction The concept of a control store of microinstructions enables

the hardware designer with a new abstraction: microprogramming

The designer can translate any desired operation to a sequence of microinstructions

All the designer needs to provide is The sequence of microinstructions needed to implement the

desired operation The ability for the control logic to correctly sequence through

the microinstructions Any additional datapath elements and control signals needed

(no need if the operation can be “translated” into existing control signals)

8

Review: Advantages of Microprogrammed Control

Allows a very simple design to do powerful computation by controlling the datapath (using a sequencer) High-level ISA translated into microcode (sequence of u-instructions) Microcode (u-code) enables a minimal datapath to emulate an ISA Microinstructions can be thought of as a user-invisible ISA (u-ISA)

Enables easy extensibility of the ISA Can support a new instruction by changing the microcode Can support complex instructions as a sequence of simple

microinstructions

Enables update of machine behavior A buggy implementation of an instruction can be fixed by changing the

microcode in the field

9

Multi-Cycle vs. Single-Cycle uArch Advantages

Disadvantages

You should be very familiar with this right now

10

Microprogrammed vs. Hardwired Control Advantages

Disadvantages

You should be very familiar with this right now

11

Can We Do Better? What limitations do you see with the multi-cycle design?

Limited concurrency Some hardware resources are idle during different phases of

instruction processing cycle “Fetch” logic is idle when an instruction is being “decoded” or

“executed” Most of the datapath is idle when a memory access is

happening

12

Can We Use the Idle Hardware to Improve Concurrency?

Goal: More concurrency Higher instruction throughput (i.e., more “work” completed in one cycle)

Idea: When an instruction is using some resources in its processing phase, process other instructions on idle resources not needed by that instruction E.g., when an instruction is being decoded, fetch the next

instruction E.g., when an instruction is being executed, decode another

instruction E.g., when an instruction is accessing data memory (ld/st),

execute the next instruction E.g., when an instruction is writing its result into the register

file, access data memory for the next instruction13

Pipelining

14

Pipelining: Basic Idea More systematically:

Pipeline the execution of multiple instructions Analogy: “Assembly line processing” of instructions

Idea: Divide the instruction processing cycle into distinct “stages” of

processing Ensure there are enough hardware resources to process one

instruction in each stage Process a different instruction in each stage

Instructions consecutive in program order are processed in consecutive stages

Benefit: Increases instruction processing throughput (1/CPI) Downside: Start thinking about this…

15

Example: Execution of Four Independent ADDs

Multi-cycle: 4 cycles per instruction

Pipelined: 4 cycles per 4 instructions (steady state)

16

Time

F D E WF D E W

F D E WF D E W

F D E WF D E W

F D E WF D E W

Time

Is life always this beautiful?

UNDERSTANDING PIPELINE

17

The Laundry Analogy

“place one dirty load of clothes in the washer” “when the washer is finished, place the wet load in the dryer” “when the dryer is finished, take out the dry load and fold” “when folding is finished, ask your roommate (??) to put the clothes

away”

18

- steps to do a load are sequentially dependent- no dependence between different loads- different steps do not share resources

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task order

Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]

Pipelining Multiple Loads of Laundry

19

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task order

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task order

- latency per load is the same- throughput increased by 4

- 4 loads of laundry in parallel- no additional resources


Pipelining Multiple Loads of Laundry: In Practice

20

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task order

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task order

the slowest step decides throughput


Pipelining Multiple Loads of Laundry: In Practice

21

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task order

A

BA

B

throughput restored (2 loads per hour) using 2 dryers

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task order


PERFORMING PIPELINE

22

An Ideal Pipeline Goal: Increase throughput with little increase in cost

(hardware cost, in case of instruction processing)

Repetition of identical operations The same operation is repeated on a large number of different

inputs (e.g., all laundry loads go through the same steps) Repetition of independent operations

No dependencies between repeated operations Uniformly partitionable suboperations

Processing can be evenly divided into uniform-latency suboperations (that do not share resources)

Fitting examples: automobile assembly line, doing laundry What about the instruction processing “cycle”?

23

Ideal Pipelining

24

combinational logic (F,D,E,M,W)T psec

BW=~(1/T)

BW=~(2/T)T/2 ps (F,D,E) T/2 ps (M,W)

BW=~(3/T)T/3ps (F,D)

T/3ps (E,M)

T/3ps (M,W)

More Realistic Pipeline: Throughput Nonpipelined version with delay T

BW = 1/(T+S) where S = latch delay

k‐stage pipelined versionBWk‐stage = 1 / (T/k +S )BWmax = 1 / (1 gate delay + S )

25

T ps

T/kps

T/kps

Latch delay reduces throughput(switching overhead b/w stages)

More Realistic Pipeline: Cost Nonpipelined version with combinational cost G

Cost = G+L where L = latch cost

k‐stage pipelined versionCostk‐stage = G + Lk

26

G gates

G/k G/k

Latches increase hardware cost

Pipelining Instruction Processing

27

Remember: The Instruction Processing Cycle

Fetch Decode Evaluate Address Fetch Operands Execute Store Result

28

1. Instruction fetch (IF)2. Instruction decode and

register operand fetch (ID/RF)3. Execute/Evaluate memory address (EX/AG)4. Memory operand fetch (MEM)5. Store/writeback result (WB)

Remember the Single-Cycle Uarch

29

Shift left 2

PC

Instruction memory

Read address

Instruction [31– 0]

Data memory

Read data

Write data

RegistersWrite register

Write data

Read data 1

Read data 2

Read register 1

Read register 2




Add

ALU result

Zero


MemtoRegALUOpMemWrite

RegWrite

MemReadBranchJumpRegDst

ALUSrc


4

M u x

Instruction [25– 0] Jump address [31– 0]

PC+4 [31– 28]

Sign extend

16 32Instruction [15– 0]

1

M u x

1

0

M u x

0

1

M u x

0

1

ALU control

Control

Add ALU result

M u x

0

1 0

ALU

Shift left 226 28

Address

PCSrc2=Br Taken

PCSrc1=Jump

ALU operation

bcond


T BW=~(1/T)

PIPELINE DATA PATH

30

Dividing Into Stages

31

200ps

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Instruction

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read dataAddress

Data memory

1

ALU result

M u x

ALUZero

IF: Instruction fetch ID: Instruction decode/ register file read

EX: Execute/ address calculation

MEM: Memory access WB: Write back100ps 200ps 200ps 100ps

RFwrite

ignorefor now


Instruction Pipeline Throughput

32

Instruction fetch Reg ALU Data

access Reg

8 ns Instruction fetch Reg ALU Data

access Reg

8 ns Instruction fetch

8 ns

Time

lw $1, 100($0)

lw $2, 200($0)

lw $3, 300($0)

2 4 6 8 10 12 14 16 18

2 4 6 8 10 12 14

...

Program execution order (in instructions)

Instruction fetch Reg ALU Data

access Reg

Time

lw $1, 100($0)

lw $2, 200($0)

lw $3, 300($0)


access Reg


access Reg

2 ns 2 ns 2 ns 2 ns 2 ns

Program execution order (in instructions)

200 400 600 800 1000 1200 1400 1600 1800

200 400 600 800 1000 1200 1400

800ps

800ps

800ps

200ps200ps200ps200ps200ps

200ps

200ps

5-stage speedup is 4, not 5 as predicted by the ideal model. Why?

Enabling Pipelined Processing: Pipeline Registers

33T

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Instruction

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read dataAddress

Data memory

1

ALU result

M u x

ALUZero

IF: Instruction fetch ID: Instruction decode/ register file read

EX: Execute/ address calculation

MEM: Memory access WB: Write back

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX

Data memory

Address

No resource is used by more than 1 stage!

IRD

PCF

PCD+4

PCE+4

nPC M

A EB E

Imm

E

Aout

MB M

MDR W

Aout

W


T/kps

T/kps

Pipeline performance impact

34

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX

Instruction fetchlw

Address

Data memory

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX MEM/WB

Instruction decodelw

Address

Data memory

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX MEM/WB

Executionlw

Address

Data memory

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read dataData

memory1

ALU result

M u x

ALUZero

ID/EX MEM/WB

Memorylw

Address

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write data

Read dataData

memory

1

ALU result

M u x

ALUZero

ID/EX MEM/WB

Write backlw

Write register

AddressInstruction

memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0

Address

Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

Data memory

1

ALU result

M u x

ALUZero

ID/EX


All instruction classes must follow the same pathand timing through the pipeline stages. (compare lw, add)

Any performance impact?

Pipelined Operation Example

35

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX

Instruction fetchlw $10, 20($1)

Address

Data memory

Clock 1

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX

Instruction decodelw $10, 20($1)

Instruction fetchsub $11, $2, $3

Address

Data memory

Clock 2

Instruction memory

Address

4

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX

Executionlw $10, 20($1)

Instruction decodesub $11, $2, $3

3216Sign

extend

Address

Data memory

Clock 3

Instruction memory

Address

4

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

3216Sign

extend

Write register

Write data

Memorylw $10, 20($1)

Read data

1

ALU result

M u x

ALUZero

ID/EX

Executionsub $11, $2, $3

Data memory

Address

Clock 4

Instruction memory

Address

4

32

0

Add Add result

1

ALU result

Zero

Shift left 2

Inst

ruct

ion

IF/ID EX/MEMID/EX MEM/WB

Write backM u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

lw $10, 20($1)

Memory

sub $11, $2, $3

Address

Data memory

Clock 5

Instruction memory

Address

4

32

0

Add Add result

1

ALU result

Zero

Shift left 2

Inst

ruct

ion


Write backM u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

sub $11, $2, $3

Address

Data memory

Clock 6


Is life always this beautiful?

Illustrating Pipeline Operation: Operation View

36

MEMEXIDIFInst4

WB

IF

MEM

IF

MEMEX

t0 t1 t2 t3 t4 t5

IDEXIF ID

IF ID

Inst0 IDIFInst1

EXIDIFInst2

MEMEXIDIFInst3

WB

WBMEMEX

WB

steady state(full pipeline)

Control Points in a Pipeline

37

PC

Instruction memory

Address

Inst

ruct

ion


MemtoReg

ALUOp

Branch

RegDst

ALUSrc

4


0

0Registers

Write register

Write data

Read data 1

Read data 2

Read register 1

Read register 2

Sign extend

M u x

1Write

data

Read

data M u x

1

ALU control

RegWrite

MemRead


6

IF/ID ID/EX EX/MEM MEM/WB

MemWrite

Address

Data memory

PCSrc

Zero

Add Add result

Shift left 2

ALU result

ALUZero

Add

0

1

M u x

0

1

M u x

Identical set of control points as the single-cycle datapath!!


PIPELINE CONTROL SIGNALS

38

Control Signals in a Pipeline For a given instruction

same control signals as single-cycle, but control signals required at different cycles, depending on stage Option 1: decode once using the same logic as single-cycle and

buffer signals until consumed

Option 2: carry relevant “instruction word/field” down the pipeline and decode locally within each or in a previous stage

Which one is better?

39

Control

EX

M

WB

M

WB

WB

IF/ID ID/EX EX/MEM MEM/WB

Instruction

Pipelined Control Signals

40

PC

Instruction memory

Inst

ruct

ion

Add


Mem

toR

eg

ALUOp

Branch

RegDst

ALUSrc

4


0

0

M u x

0

1

Add Add result

RegistersWrite register

Write data

Read data 1

Read data 2

Read register 1

Read register 2

Sign extend

M u x

1

ALU result

Zero

Write data

Read data

M u x

1

ALU control

Shift left 2

Reg

Writ

e

MemRead

Control

ALU


6

EX

M

WB

M

WB

WBIF/ID

PCSrc

ID/EX

EX/MEM

MEM/WB

M u x

0

1

Mem

Writ

e

AddressData

memory

Address


PIPELINE ISSUES

41

Remember: An Ideal Pipeline Goal: Increase throughput with little increase in cost

(hardware cost, in case of instruction processing)

Repetition of identical operations The same operation is repeated on a large number of different

inputs (e.g., all laundry loads go through the same steps) Repetition of independent operations

No dependencies between repeated operations Uniformly partitionable suboperations

Processing an be evenly divided into uniform-latency suboperations (that do not share resources)

Fitting examples: automobile assembly line, doing laundry What about the instruction processing “cycle”?

42

Instruction Pipeline: Not An Ideal Pipeline Identical operations ... NOT!

different instructions not all need the same stagesForcing different instructions to go through the same pipe stages

external fragmentation (some pipe stages idle for some instructions)

Uniform suboperations ... NOT! different pipeline stages not the same latency

Need to force each stage to be controlled by the same clock internal fragmentation (some pipe stages are too fast but all take the same clock cycle time)

Independent operations ... NOT! instructions are not independent of each other

Need to detect and resolve inter-instruction dependencies to ensure the pipeline provides correct results pipeline stalls (pipeline is not always moving)

43

Issues in Pipeline Design Balancing work in pipeline stages

How many stages and what is done in each stage

Keeping the pipeline correct, moving, and full in the presence of events that disrupt pipeline flow Handling dependences

Data Control

Handling resource contention Handling long-latency (multi-cycle) operations

Handling exceptions, interrupts

Advanced: Improving pipeline throughput Minimizing stalls

44

Pipeline Stalls Stall: A condition when the pipeline stops moving

Resource contention

Dependences (between instructions) Data Control

Long-latency (multi-cycle) operations

45

DEPENDENCES

46

A computer program Following the Von Neumann model, the program is a

sequence of instructions

L1: Mov AX, 3Add ax, bx….jmp l1

47

Dependences and Their Types Also called “dependency” or less desirably “hazard”

Dependences dictate ordering requirements between instructions

Two types Data dependence Control dependence

Resource contention is sometimes called resource dependence

48

Handling Resource Contention Happens when instructions in two pipeline stages need the

same resource

Solution 1: Eliminate the cause of contention Duplicate the resource or increase its throughput

E.g., use separate instruction and data memories (caches) E.g., use multiple ports for memory structures

Solution 2: Detect the resource contention and stall one of the contending stages Which stage do you stall? Example: What if you had a single read and write port for the

register file?

49

UNDERSTANDING DEPENDENCES

50

Data Dependences Types of data dependences

Flow dependence (true data dependence – read after write) Comes from the running program semantic

Anti dependence (write after read) Output dependence (write after write)

Which ones cause stalls in a pipelined machine? For all of them, we need to ensure semantics of the program

is correct Flow dependences always need to be obeyed because they

constitute true dependence on a value Anti and output dependences exist due to limited number of

architectural registers They are dependence on a name, not a value

51

Data Dependence Types

52

Flow dependencer3 r1 op r2 Read‐after‐Write r5 r3 op r4 (RAW)

Anti dependencer3 r1 op r2 Write‐after‐Read r1 r4 op r5 (WAR)

Output‐dependencer3 r1 op r2 Write‐after‐Write r5 r3 op r4 (WAW)r3 r6 op r7

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX


Address

Data memory

Clock 1

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX



Address

Data memory

Clock 2

Sub $10,$2,$3

Instruction memory

Address

4

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX



3216Sign

extend

Address

Data memory

Clock 3

Sub $10,$2,$3

Instruction memory

Address

4

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

3216Sign

extend

Write register

Write data


Read data

1

ALU result

M u x

ALUZero

ID/EX


Data memory

Address

Clock 4

Sub $10,$2,$3

Instruction memory

Address

4

32

0

Add Add result

1

ALU result

Zero

Shift left 2

Inst

ruct

ion


Write backM u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

lw $10, 20($1)

Memory

sub $11, $2, $3

Address

Data memory

Clock 5

Sub $10,$2,$3

Instruction memory

Address

4

32

0

Add Add result

1

ALU result

Zero

Shift left 2

Inst

ruct

ion


Write backM u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

sub $11, $2, $3

Address

Data memory

Clock 6

Sub $10,$2,$3

Example output dependence

53

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX


Address

Data memory

Clock 1

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX



Address

Data memory

Clock 2

Sub $11,$2,$10

Instruction memory

Address

4

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

Write register

Write data

Read data

1

ALU result

M u x

ALUZero

ID/EX



3216Sign

extend

Address

Data memory

Clock 3

Sub $11,$2,$10

Instruction memory

Address

4

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

3216Sign

extend

Write register

Write data


Read data

1

ALU result

M u x

ALUZero

ID/EX


Data memory

Address

Clock 4

Sub $11,$2,$10

Instruction memory

Address

4

32

0

Add Add result

1

ALU result

Zero

Shift left 2

Inst

ruct

ion


Write backM u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

lw $10, 20($1)

Memory

sub $11, $2, $3

Address

Data memory

Clock 5

Sub $11,$2,$10

Instruction memory

Address

4

32

0

Add Add result

1

ALU result

Zero

Shift left 2

Inst

ruct

ion


Write backM u x

0

1

Add

PC

0Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

sub $11, $2, $3

Address

Data memory

Clock 6

Sub $11,$2,$10

Example of flow dependence

54

Control Dependence Question: What should the fetch PC be in the next cycle? Answer: The address of the next instruction

All instructions are control dependent on previous ones. Why?

If the fetched instruction is a non-control-flow instruction: Next Fetch PC is the address of the next-sequential instruction Easy to determine if we know the size of the fetched instruction

If the instruction that is fetched is a control-flow instruction: How do we determine the next Fetch PC?

In fact, how do we know whether or not the fetched instruction is a control-flow instruction?

55

DEPENDENCES DETECTION

56

Interlocking Detection of dependence between instructions in a

pipelined processor to guarantee correct execution

Software based interlockingvs.

Hardware based interlocking

MIPS acronym? Microprocessor without Interlocked Pipeline Stages

57

Approaches to Dependence Detection (I) Scoreboarding

Each register in register file has a Valid bit associated with it An instruction that is writing to the register resets the Valid bit An instruction in Decode stage checks if all its source and

destination registers are Valid Yes: No need to stall… No dependence No: Stall the instruction

Advantage: Simple. 1 bit per register

Disadvantage: Need to stall for all types of dependences, not only flow dep.

58

Scoreboarding

59

Instruction memory

Address

4

32

0

Add Add result

Shift left 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0

Address

Write data

M u x

1Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

Data memory

1

ALU result

M u x

ALUZero

ID/EX

Not Stalling on Anti and Output Dependences What changes would you make to the scoreboard to enable

this? counter for writing operation, not just 1 and 0

60

Approaches to Dependence Detection (II) Combinational dependence check logic

Special logic that checks if any instruction in later stages is supposed to write to any source register of the instruction that is being decoded

Yes: stall the instruction/pipeline No: no need to stall… no flow dependence

Advantage: No need to stall on anti and output dependences

Disadvantage: Logic is more complex than a scoreboard Logic becomes more complex as we make the pipeline deeper

and wider (flash-forward: think superscalar execution)61

DATA DEPENDENCE HANDLING

62

Once You Detect the Dependence in Hardware What do you do afterwards?

Observation: Dependence between two instructions is detected before the communicated data value becomes available

63

How to Handle Data Dependences Anti and output dependences are easier to handle

write to the destination in one stage and in program order

Flow dependences are more interesting

Five fundamental ways of handling flow dependences 1. Detect and wait until value is available in register file 2. Detect and forward/bypass data to dependent instruction Detect and eliminate the dependence at the software level No need for the hardware to detect dependence (MIPS NOP) Do something else (same program “reorder”), (different

program “fine-grained multithreading”) and No need to detect. Predict the needed value(s), execute “speculatively”, and verify

64

Right place to eliminate dependency Which one of the following flow dependences lead to

conflicts in the 5-stage pipeline?

65

MEM

WBIF ID

IF

EX

ID

MEM

EX WB

addi r1 ‐ ‐

addi ‐ r1 ‐

MEMIF ID EX

IF ID EX

IF ID

IF

addi ‐ r1 ‐

addi ‐ r1 ‐

addi ‐ r1 ‐

addi ‐ r1‐

?

Safe and Unsafe Movement of Pipeline

66

i:rk_

j:_rk Reg Read

Reg Write

iFj

stage X

stage Y

dist(i,j) dist(X,Y) ??dist(i,j) > dist(X,Y) ??

RAW Dependence

i:_rk

j:rk_ Reg Write

Reg Read

iAj

WAR Dependence

i:rk_

j:rk_ Reg Write

Reg Write

iOj

WAW Dependence

dist(i,j) dist(X,Y) Unsafe to keep j movingdist(i,j) > dist(X,Y) Safe

RAW Dependence Analysis Example

Instructions IA and IB (where IA comes before IB) have RAW dependence iff IB (R/I, LW, SW, Br or JR) reads a register written by IA (R/I or LW) dist(IA, IB) dist(ID, WB) = 3

What about WAW and WAR dependence?What about memory data dependence?

67

R/I‐Type LW SW Br J Jr

IFID read RF read RF read RF read RF read RFEX

MEMWB write RF write RF

Pipeline Stall: Resolving Data Dependence

68

IF

WB

IF ID ALU MEMIF ID ALU MEM

IF ID ALU MEMIF ID ALU

t0 t1 t2 t3 t4 t5

IF ID MEMIF ID ALU

IF ID

InstiInstjInstkInstl

WBWB

i: rx _j: _ rx dist(i,j)=1

ij

Insth

WBMEMALU

i: rx _bubblej: _ rx dist(i,j)=2

WB


IF ID ALU MEMIF ID ALU

t0 t1 t2 t3 t4 t5

MEM


WBWBi

j

Insth

IDIF

IF

IF ID ALUIF ID

i: rx _bubblebubblej: _ rx dist(i,j)=3

IF


IF ID ALUIF ID

t0 t1 t2 t3 t4 t5

IF

MEMALUID


WBWBi

j

Insth

IDIF

IDIF

i: rx _bubblebubblebubblej: _ rx dist(i,j)=4

IF


IF IDIF

t0 t1 t2 t3 t4 t5

ALUID


WBWBi

j

Insth

IDIF

IDIF

IDIF

Stall = make the dependent instruction wait until its source data value is available

1. stop all up-stream stages2. drain all down-stream stages

Sample Assembly (P&H) for (j=i‐1; j>=0 && v[j] > v[j+1]; j‐=1) { ...... }

69

addi $s1, $s0, ‐1for2tst: slti $t0, $s1, 0

bne $t0, $zero, exit2sll $t1, $s1, 2add $t2, $a0, $t1lw $t3, 0($t2)lw $t4, 4($t2)slt $t0, $t4, $t3beq $t0, $zero, exit2.........addi $s1, $s1, ‐1j for2tst

exit2:

3 stalls3 stalls

3 stalls3 stalls

3 stalls3 stalls

Readings P&H Chapter 4.9-4.11

Smith and Sohi, “The Microarchitecture of Superscalar Processors,” Proceedings of the IEEE, 1995 More advanced pipelining Interrupt and exception handling Out-of-order and superscalar execution concepts

70

computer architecture lecture 6: pipelining -...

Documents