cs 152 lec 12.1 cs 152: computer architecture and engineering lecture 12 multicycle controller...

CS 152Lec 12.1

CS 152: Computer Architecture

and Engineering

Lecture 12

Multicycle Controller Design Pipelining

Randy H. Katz, InstructorSatrajit Chatterjee, Teaching Assistant

George Porter, Teaching Assistant

CS 152Lec12.2

Recap: Microprogramming Microprogramming is a convenient method for

implementing structured control state diagrams:• Random logic replaced by microPC sequencer and ROM

• Each line of ROM called a instruction: contains sequencer control + values for control

points

• limited state transitions: branch to zero, next sequential,branch to instruction address from dispatch ROM

Horizontal Code: one control bit in Instruction for every control line in datapath

Vertical Code: groups of control-lines coded together in Instruction (e.g., possible ALU dest)

Control design reduces to Microprogramming• Part of the design process is to develop a “language” that

describes control and is easy for humans to understand

CS 152Lec12.3

Recap: Microprogramming

Microprogramming is a fundamental concept• implement an instruction set by building a very simple

processor and interpreting the instructions

• essential for very complex instructions and when few register transfers are possible

• overkill when ISA matches datapath 1-1

sequencercontrol

datapath control

micro-PC-sequencer:fetch,dispatch,sequential

microinstruction ()

DispatchROMOpcode

-Code ROM

DecodeDecode

To DataPath

Decoders implement our -code language:

For instance:rt-ALUrd-ALUmem-ALU

CS 152Lec12.4

Recap: Exceptions

Exception = unprogrammed control transfer• system takes action to handle the exception

- must record the address of the offending instruction- record any other information necessary to return

afterwards• returns control to user• must save & restore user state

Allows constuction of a “user virtual machine”

normal control flow: sequential, jumps, branches, calls, returns

user program SystemExceptionHandlerException:

return fromexception

CS 152Lec12.5

Recap: Interrupts vs. Traps Interrupts

• Caused by external events: - Network, Keyboard, Disk I/O, Timer

• Asynchronous to program execution- Most interrupts can be disabled for brief periods of time- Some (like “Power Failing”) are non-maskable (NMI)

• May be handled between instructions• Simply suspend and resume user program

Traps• Caused by internal events

- Exceptional conditions (overflow)- Errors (parity)- Faults (non-resident page)

• Synchronous to program execution• Condition must be remedied by the handler• Instruction may be retried or simulated and program

continued or program may be aborted

CS 152Lec12.6

Recap: How Control Handles Traps in Our FSD Undefined Instruction–detected when no next state is defined from state 1 for the op value.

• We handle this exception by defining the next state value for all op values other than lw, sw, 0 (R-type), jmp, beq, and ori as new state 12.

• Shown symbolically using “other” to indicate that the op field does not match any of the opcodes that label arcs out of state 1.

Arithmetic overflow–detected on ALU ops such as signed add

• Used to save PC and enter exception handler

External Interrupt–flagged by asserted interrupt line• Again, must save PC and enter exception handler

Note: Challenge in designing control of a real machine is to handle different interactions between instructions and other exception-causing events such that control logic remains small and fast.

• Complex interactions makes the control unit the most challenging aspect of hardware design

CS 152Lec12.7

Recap: Adding Traps and Interrupts to State Diagram

IR <= MEM[PC]PC <= PC + 4

R-type

S <= A fun B

R[rd] <= S

S <= A op ZX

R[rt] <= S

ORi

S <= A + SX

R[rt] <= M

M <= MEM[S]

LW

S <= A + SX

MEM[S] <= B

SW

“instruction fetch”

“decode”

0000

0001

0100

0101

0110

0111

1000

1001

1010

1011

1100

BEQ

0010

If A = Bthen PC <= S

S<= PC+SX

undefined instruction

EPC <= PC - 4PC <= exp_addrcause <= 10 (RI)

other

overflow

EPC <= PC - 4PC <= exp_addrcause <= 12 (Ovf)

EPC <= PC - 4PC <= exp_addrcause <= 0(INT)

HandleInterrupt

Pending INT

CS 152Lec12.8

Recap: Non-Ideal Memory

IR <= MEM[PC]

R-type

A <= R[rs]B <= R[rt]

S <= A fun B

R[rd] <= SPC <= PC + 4

S <= A or ZX

R[rt] <= SPC <= PC + 4

ORi

S <= A + SX

R[rt] <= MPC <= PC + 4

M <= MEM[S]

LW

S <= A + SX

MEM[S] <= B

BEQPC <=

Next(PC)

SW

“instruction fetch”

“decode / operand fetch”

Execute

Memory

Write-back

~wait wait

~wait wait

PC <= PC + 4

~wait wait

CS 152Lec12.9

If simple instruction could execute at very high clock rate…

If you could even write compilers to produce microinstructions…

If most programs use simple instructions and addressing modes…

If microcode is kept in RAM instead of ROM so as to fix bugs …

If same memory used for control memory could be used instead as cache for “macroinstructions”…

Then why not skip instruction interpretation by a microprogram and simply compile directly into lowest language of machine? (microprogramming is overkill when ISA matches datapath 1-1)

Motivation for Microprogramming

CS 152Lec12.10

Recall: Performance Evaluation What is the average CPI?

• state diagram gives CPI for each instruction type

• workload gives frequency of each type

Type CPIi for type Frequency CPIi x freqIi

Arith/Logic 4 40% 1.6

Load 5 30% 1.5

Store 4 10% 0.4

branch 3 20% 0.6

Average CPI:4.1

CS 152Lec12.11

Can we get CPI < 4.1?

IdealMemoryWrAdrDin

RAdr

32

32

32Dout

MemWr32

ALU

3232

ALUOp

ALUControl

32

IRWr

Instru

ction R

eg

32

Reg File

Ra

Rw

busW

Rb5

5

32busA

32busB

RegWr

Rs

Rt

Mux

0

1

Rt

Rd

PCWr

ALUSelA

Mux 01

RegDst

Mux

0

1

32

PC

MemtoReg

Extend

ExtOp

Mux

0

132

0

1

23

4

16Imm 32

<< 2

ALUSelB

Mux

1

0

32

Zero

ZeroPCWrCond PCSrc

32

IorD

Mem

Data

Reg

ALU

Out

B

A

Seems to be lots of “idle” hardware• Why not overlap instructions???

CS 152Lec12.12

The Big Picture: Where are We Now?

The Five Classic Components of a Computer

Next Topics: • Pipelining by Analogy

• Pipeline hazards

Control

Datapath

Memory

Processor

Input

Output

CS 152Lec12.13

Pipelining is Natural!

Laundry Example

Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold

Washer takes 30 minutes

Dryer takes 40 minutes

“Folder” takes 20 minutes

A B C D

CS 152Lec12.14

Sequential Laundry

Sequential laundry takes 6 hours for 4 loads If they learned pipelining, how long would laundry

take?

A

B

C

D

30 40 20 30 40 20 30 40 20 30 40 20

6 PM 7 8 9 10 11 Midnight

Task

Order

Time

CS 152Lec12.15

Pipelined Laundry: Start Work ASAP

Pipelined laundry takes 3.5 hours for 4 loads

A

B

C

D

6 PM 7 8 9 10 11 Midnight

Task

Order

Time

30 40 40 40 40 20

CS 152Lec12.16

Pipelining Lessons

Pipelining doesn’t help latency of single task, it helps throughput of entire workload

Pipeline rate limited by slowest pipeline stage

Multiple tasks operating simultaneously using different resources

Potential speedup = Number pipe stages

Unbalanced lengths of pipe stages reduces speedup

Time to “fill” pipeline and time to “drain” it reduces speedup

Stall for Dependences

A

B

C

D

6 PM 7 8 9

Task

Order

Time

30 40 40 40 40 20

CS 152Lec12.17

The Five Stages of Load

Ifetch: Instruction Fetch• Fetch the instruction from the Instruction Memory

Reg/Dec: Registers Fetch and Instruction Decode

Exec: Calculate the memory address

Mem: Read the data from the Data Memory

Wr: Write the data back to the register file

Cycle 1Cycle 2 Cycle 3Cycle 4 Cycle 5

Ifetch Reg/Dec Exec Mem WrLoad

CS 152Lec12.18

Note: These 5 stages were there all along!

IR <= MEM[PC]PC <= PC + 4

R-type

ALUout <= A fun B

R[rd] <= ALUout

ALUout <= A op ZX

R[rt] <= ALUout

ORi

ALUout <= A + SX

R[rt] <= M

M <= MEM[ALUout]

LW

ALUout <= A + SX

MEM[ALUout] <= B

SW

0000

0001

0100

0101

0110

0111

1000

1001

1010

1011

1100

BEQ

0010

If A = B then PC <= ALUout

ALUout <= PC +SX

Execute

Memory

Write-back

DecodeFetch

CS 152Lec12.19

Pipelining Improve performance by increasing throughput

Ideal speedup is number of stages in the pipeline. Do we achieve this?

Instructionfetch

Reg ALUData

accessReg

8 nsInstruction

fetchReg ALU

Dataaccess

Reg

8 nsInstruction

fetch

8 ns

Time

lw $1, 100($0)

lw $2, 200($0)

lw $3, 300($0)

2 4 6 8 10 12 14 16 18

2 4 6 8 10 12 14

...

Programexecutionorder(in instructions)

Instructionfetch

Reg ALUData

accessReg

Time

lw $1, 100($0)

lw $2, 200($0)

lw $3, 300($0)

2 nsInstruction

fetchReg ALU

Dataaccess

Reg

2 nsInstruction

fetchReg ALU

Dataaccess

Reg

2 ns 2 ns 2 ns 2 ns 2 ns


CS 152Lec12.20

Basic Idea

What do we need to add to split the datapath into stages?

Instructionmemory

Address

4

32

0

Add Addresult

Shiftleft 2

Instruction

Mux

0

1

Add

PC

0Writedata

Mux

1Registers

Readdata 1

Readdata 2

Readregister 1

Readregister 2

16Sign

extend

Writeregister

Writedata

ReaddataAddress

Datamemory

1

ALUresult

Mux

ALUZero

IF: Instruction fetch ID: Instruction decode/register file read

EX: Execute/address calculation

MEM: Memory access WB: Write back

CS 152Lec12.21

Pipelined Datapath

Instructionmemory

Address

4

32

0

Add Addresult

Shiftleft 2

Inst

ruct

ion

IF/ID EX/MEM MEM/WB

Mux

0

1

Add

PC

0Writedata

Mux

1Registers

Readdata 1

Readdata 2

Readregister 1

Readregister 2

16Sign

extend

Writeregister

Writedata

Readdata

1

ALUresult

Mux

ALUZero

ID/EX

Datamemory

Address

CS 152Lec12.22

Graphically Representing Pipelines

Can help with answering questions like:• how many cycles does it take to execute this code?

• what is the ALU doing during cycle 4?

• use this representation to help understand datapaths

IM Reg DM Reg

IM Reg DM Reg

CC 1 CC 2 CC 3 CC 4 CC 5 CC 6

Time (in clock cycles)

lw $10, 20($1)


sub $11, $2, $3

ALU

ALU

CS 152Lec12.23

Conventional Pipelined Execution Representation

IFetch Dcd Exec Mem WB





IFetch Dcd Exec Mem WBProgram Flow

Time

CS 152Lec12.24

Single Cycle, Multiple Cycle, vs. Pipeline

Clk

Cycle 1

Multiple Cycle Implementation:

Ifetch Reg Exec Mem Wr

Cycle 2Cycle 3Cycle 4Cycle 5Cycle 6Cycle 7Cycle 8Cycle 9Cycle 10

Load Ifetch Reg Exec Mem Wr

Ifetch Reg Exec Mem

Load Store

Pipeline Implementation:

Ifetch Reg Exec Mem WrStore

Clk

Single Cycle Implementation:

Load Store Waste

Ifetch

R-type

Ifetch Reg Exec Mem WrR-type

Cycle 1 Cycle 2

CS 152Lec12.25

Why Pipeline?

Suppose we execute 100 instructions

Single Cycle Machine• 45 ns/cycle x 1 CPI x 100 inst = 4500 ns

Multicycle Machine• 10 ns/cycle x 4.6 CPI (due to inst mix) x 100 inst = 4600 ns

Ideal pipelined machine• 10 ns/cycle x (1 CPI x 100 inst + 4 cycle drain) = 1040 ns

CS 152Lec12.26

Why Pipeline? Because we can!

Instr.

Order

Time (clock cycles)

Inst 0

Inst 1

Inst 2

Inst 4

Inst 3

ALUIm Reg Dm Reg

ALUIm Reg Dm Reg

ALUIm Reg Dm Reg

ALUIm Reg Dm Reg

ALUIm Reg Dm Reg

CS 152Lec12.27

Can Pipelining Get Us Into Trouble? Yes: Pipeline Hazards

• Structural hazards: attempt to use the same resource two different ways at the same time

- Memory access (Instruction Fetch & data access)

• Control hazards: attempt to make a decision before condition is evaluated

- Branch instructions

• Data hazards: attempt to use item before it is ready

- Instruction depends on result of prior instruction still in the pipeline

Can always resolve hazards by waiting• Pipeline control must detect the hazard

• Take action (or delay action) to resolve hazards

CS 152Lec12.28

Mem

Single Memory is a Structural Hazard

Instr.

Order

Time (clock cycles)

Load

Instr 1

Instr 2

Instr 3

Instr 4A

LUMem Reg Mem Reg

ALUMem Reg Mem Reg

ALUMem Reg Mem Reg

ALUReg Mem Reg

ALUMem Reg Mem Reg

Detection is easy in this case! (right half highlight means read, left half write)

CS 152Lec12.29

Structural Hazards Limit Performance

Example: if 1.3 memory accesses per instruction and only one memory access per cycle then

• average CPI 1.3

• otherwise resource is more than 100% utilized

CS 152Lec12.30

Stall: wait until decision is clear Impact: 2 lost cycles (i.e. 3 clock cycles per

branch instruction) => slow Move decision to end of decode

• save 1 cycle per branch

Control Hazard Solution #1: Stall

Instr.

Order

Time (clock cycles)

Add

Beq

Load

ALUMem Reg Mem Reg

ALUMem Reg Mem Reg

ALUReg Mem RegMem

Lostpotential

CS 152Lec12.31

Predict: guess one direction then back up if wrong

Impact: 0 lost cycles per branch instruction if right, 1 if wrong (right ~ 50% of time)

• Need to “Squash” and restart following instruction if wrong• Produce CPI on branch of (1 *.5 + 2 * .5) = 1.5• Total CPI might then be: 1.5 * .2 + 1 * .8 = 1.1 (20% branch)

More dynamic scheme: history of 1 branch (~ 90%)

Control Hazard Solution #2: PredictInstr.

Order

Time (clock cycles)

Add

Beq

Load

ALUMem Reg Mem Reg

ALUMem Reg Mem Reg

Mem

ALUReg Mem Reg

CS 152Lec12.32

Delayed Branch: Redefine branch behavior (takes place after next instruction)

Impact: 0 clock cycles per branch instruction if can find instruction to put in “slot” (~ 50% of time)

Control Hazard Solution #3: Delayed Branch

Instr.

Order

Time (clock cycles)

Add

Beq

Misc

ALUMem Reg Mem Reg

ALUMem Reg Mem Reg

Mem

ALUReg Mem Reg

Load Mem

ALUReg Mem Reg

CS 152Lec12.33

Data Hazard on r1

add r1,r2,r3

sub r4,r1,r3

and r6,r1,r7

or r8,r1,r9

xor r10,r1,r11

CS 152Lec12.34

• Dependencies backwards in time are hazards

Data Hazard on r1:

Instr.

Order

Time (clock cycles)

add r1,r2,r3

sub r4,r1,r3

and r6,r1,r7

or r8,r1,r9

xor r10,r1,r11

IF ID/RF EX MEM WBALUIm Reg Dm Reg

ALUIm Reg Dm Reg

ALUIm Reg Dm Reg

Im

ALUReg Dm Reg

ALUIm Reg Dm Reg

CS 152Lec12.35

• “ Forward” result from one stage to another

• “or” OK if define read/write properly

Data Hazard Solution:

Instr.

Order

Time (clock cycles)

add r1,r2,r3

sub r4,r1,r3

and r6,r1,r7

or r8,r1,r9

xor r10,r1,r11

IF ID/RF EX MEM WBALUIm Reg Dm Reg

ALUIm Reg Dm Reg

ALUIm Reg Dm Reg

Im

ALUReg Dm Reg

ALUIm Reg Dm Reg

CS 152Lec12.36

Reg


• Can’t solve with forwarding: • Must delay/stall instruction dependent on loads

Forwarding (or Bypassing): What about Loads?

Time (clock cycles)

lw r1,0(r2)

sub r4,r1,r3

IF ID/RF EX MEM WBALUIm Reg Dm

ALUIm Reg Dm Reg

CS 152Lec12.37

Reg


• Can’t solve with forwarding: • Must delay/stall instruction dependent on loads

Forwarding (or Bypassing): What about Loads

Time (clock cycles)

lw r1,0(r2)

sub r4,r1,r3

IF ID/RF EX MEM WBALUIm Reg Dm

ALUIm Reg Dm RegStall

CS 152Lec12.38

Designing a Pipelined Processor

Go back and examine your datapath and control diagram

Associated resources with states

Ensure that flows do not conflict, or figure out how to resolve

Assert control in appropriate stage

CS 152Lec12.39

Control and Datapath: Split State Diagram into 5 Pieces

IR <- Mem[PC]; PC <– PC+4;

A <- R[rs]; B<– R[rt]

S <– A + B;

R[rd] <– S;

S <– A + SX;

M <– Mem[S]

R[rd] <– M;

S <– A or ZX;

R[rt] <– S;

S <– A + SX;

Mem[S] <- B

If CondPC < PC+SX;

Exec

Reg.

File

Mem

Acc

es

sD

ata

Mem

A

B

S

Reg

File

Equ

al

PC

Next

PC

IR

Inst

. M

em

D

M

CS 152Lec12.40

Summary: Pipelining Reduce CPI by overlapping many instructions

• Average throughput of approximately 1 CPI with fast clock

Utilize capabilities of the Datapath • Start next instruction while working on the current one• Limited by length of longest stage (plus fill/flush)• Detect and resolve hazards

What makes it easy• All instructions are the same length• Just a few instruction formats• Memory operands appear only in loads and stores

What makes it hard?• Structural hazards: suppose we had only one memory• Control hazards: need to worry about branch instructions• Data hazards: an instruction depends on a previous instruction

CS 152Lec12.41

Summary Microprogramming is a fundamental concept

• Implement an instruction set by building a very simple processor and interpreting the instructions

• Essential for very complex instructions and when few register transfers are possible

• Control design reduces to Microprogramming

Exceptions are the hard part of control• Need to find convenient place to detect exceptions and to branch

to state or microinstruction that saves PC and invokes the operating system

• Providing clean interrupt model gets hard with pipelining!

Precise Exception state of the machine is preserved as if program executed up to the offending instruction

• All previous instructions completed

• Offending instruction and all following instructions act as if they have not even started

CS 152Lec12.42

Summary: Where This Class is Going We’ll build a simple pipeline and look at these

issues• Lab 5 Pipelined Processor• Lab 6 With caches

We’ll talk about modern processors and what’s really hard:

• Exception handling

• Trying to improve performance with out-of-order execution, etc.

• Trying to get CPI < 1 (Superscalar execution)

cs 152 lec 12.1 cs 152: computer architecture and engineering lecture 12 multicycle controller...

Documents