fifth lecture: chapter 3: dataflow processors - hybrids

1

Fifth Lecture:Chapter 3: Dataflow Processors - Hybrids

Please recall: Dataflow model: the execution is driven only by the availability of operands! Static & dynamic dataflow Dynamic dataflow pipeline:

Token buffer Token Matching, Instruction Fetch, Execute & New Tag, Form New Tokens (max. of 2) Token buffer or network

A successor instruction can only be executed when the previous instruction finished (token recycling!) no data or control hazards in pipeline

Hazards in dataflow model Data hazards due to

true dependences dataflow principle name (false) dependences not present due to single assignment rule in

dataflow languages Control hazards transformed into data dependences Structural hazards: mostly ignored in dataflow literature

Explicit token store removes associative matching need

2

Augmenting Dataflow with Control-Flow

poor sequential code performance by dynamic dataflow computers Why? an instruction of the same thread is issued to the dataflow pipeline after the

completion of its predecessor instruction. In the case of an 8-stage pipeline, instructions of the same thread can be

issued at most every eight cycles. Low workload: the utilization of the dataflow processor drops to one eighth of

its maximum performance.

Another drawback: the overhead associated with token matching. before a dyadic instruction is issued to the execution stage, two result tokens

have to be present. The first token is stored in the waiting-matching store, thereby introducing a

bubble in the execution stage(s) of the dataflow processor pipeline. measured pipeline bubbles on Monsoon: up to 28.75 %

no use of registers possible!

3

Augmenting Dataflow with Control-Flow

Solution: combine dataflow with control-flow mechanisms:

threaded dataflow, large-grain dataflow, dataflow with complex machine operations, further hybrids

4

Threaded Dataflow

Threaded dataflow: the dataflow principle is modified so that instructions of certain instruction streams are processed in succeeding machine cycles.

A subgraph that exhibits a low degree of parallelism is transformed into a sequential thread.

The thread of instructions is issued consecutively by the matching unit without matching further tokens except for the first instruction of the thread.

Threaded dataflow coversthe repeat-on-input technique used in Epsilon-1 and

Epsilon-2 processors, the strongly connected arc model of EM-4, andthe direct recycling of tokens in Monsoon.

5

Threaded Dataflow (continued)

Data passed between instructions of the same thread is stored in registers instead of written back to memory.

Registers may be referenced by any succeeding instruction in the thread. Single-thread performance is improved. The total number of tokens needed to schedule program

instructions is reduced which in turn saves hardware resources. Pipeline bubbles are avoided for dyadic instructions within a

thread.Two threaded dataflow execution techniques can be distinguished:

direct token recycling (Monsoon), consecutive execution of the instructions of a single thread

(Epsilon & EM).

6

Direct token recycling of Monsoon

Cycle-by-cycle instruction interleaving of threads similar to multithreaded von Neumann computers!

8 register sets can be used by 8 different threads.Dyadic instructions within a thread (except for the start

instruction!) refer to at least one register, need only a single token to be enabled.

A result token of a particular thread is recycled ASAP in the 8-stage pipeline, i.e. every 8th cycle the next instruction of a thread is fired and executed.

This implies that at least 8 threads must be active for a full pipeline utilization.

Threads and fine-grain dataflow instructions can be mixed in the pipeline.

7

Epsilon and EM-4

Instructions of a thread are executed consecutively. The circular pipeline of fine-grain dataflow is retained. The matching unit is enhanced with a mechanism that, after

firing the first instruction of a thread, delays matching of further tokens in favor of consecutive issuing of all instructions of the started thread.

Problem: implementation of an efficient synchronization mechanism

8

Large-Grain (coarse-grain) Dataflow

A dataflow graph is enhanced to contain fine-grain (pure) dataflow nodes and macro dataflow nodes.A macro dataflow node contains a sequential block of

instructions.A macro dataflow node is activated in the dataflow manner,

its instruction sequence is executed in the von Neumann style!

Off-the-shelf microprocessors can be used to support the execution stage.

Large-grain dataflow machines typically decouple the matching stage (sometimes called signal stage, synchronization stage, etc.) from the execution stage by use of FIFO-buffers.

Pipeline bubbles are avoided by the decoupling and FIFO-buffering.

9

Dataflow with Complex Machine Operations

Use of complex machine instructions, e.g. vector instructions ability to exploit parallelism at the subinstruction level Instructions can be implemented by pipeline techniques

as in vector computers. The use of a complex machine operation may spare

several nested loops.

Structured data is referenced in block rather than element-wise and can be supplied in a burst mode.

10

Dataflow with Complex Machine Operationsand combined with LGDF

Often use of FIFO-buffers to decouple the firing stage and the execution stage bridges different execution times within a mixed stream of

simple and complex instructions. Major difference to pure dataflow: tokens do not carry data

(except for the values true or false). Data is only moved and transformed within the execution

stage. Applied in: Decoupled Graph/Computation Architecture, the

Stollmann Dataflow Machine, and the ASTOR architecture. These architectures combine complex machine instructions

with large-grain dataflow.

12

Lessons Learned from Dataflow

Superscalar microprocessors display an out-of-order dynamic execution that is referred to as local dataflow or micro dataflow.

Colwell and Steck 1995, in the first paper on the PentiumPro:“The flow of the Intel Architecture instructions is predicted and these instructions are decoded into micro-operations (ops), or series of ops, and these ops are register-renamed, placed into an out-of-order speculative pool of pending operations, executed in dataflow order (when operands are ready), and retired to permanent machine state in source program order.”

State-of-the-art microprocessors typically provide 32 (MIPS R10000), 40 (Intel PentiumPro) or 56 (HP PA-8000) instruction slots in the instruction window or reorder buffer.

Each instruction is ready to be executed as soon as all operands are available.

Dataflow and Superscalar

Pros and Cons

next

14

Comparing dataflow computers with superscalar microprocessors

Superscalar microprocessors are von Neumann based: (sequential) thread of instructions as input not enough fine-grained parallelism to feed the multiple functional units speculation

dataflow approach resolves any threads of control into separate instructions that are ready to execute as soon as all required operands become available.

The fine-grained parallelism generated by dataflow principle is far larger than the parallelism available for microprocessors.

However, locality is lost no caching, no registers

15

Lessons Learned from Dataflow (Pipeline Issues)

Microprocessors: Data and control dependences potentially cause pipeline hazards that are handled by complex forwarding logic.

Dataflow: Due to the continuous context switches, pipeline hazards are avoided; disadvantage: poor single thread performance.

Microprocessors: Antidependences and output dependences are removed by register renaming that maps the architectural registers to the physical registers.

Thereby the microprocessor internally generates an instruction stream that satisfies the single assignment rule of dataflow.

The main difference between the dependence graphs of dataflow and the code sequence in an instruction window of a microprocessor:branch prediction and speculative execution.

Microprocessors: rerolling execution in case of a wrongly predicted path is costly in terms of processor cycles.

16

Lessons Learned from Dataflow (Continued)

Dataflow: The idea of branch prediction and speculative execution has never been evaluated in the dataflow environment.

Dataflow was considered to produce an abundance of parallelism while speculation leads to speculative parallelism which is inferior to real parallelism.

Microprocessors: Due to the single thread of control, a high degree of data and instruction locality is present in the machine code.

Microprocessors: The locality allows to employ a storage hierarchy that stores the instructions and data potentially executed in the next cycles close to the executing processor.

Dataflow: Due to the lack of locality in a dataflow graph, a storage hierarchy is difficult to apply.

17


Microprocessors: The operand matching of executable instructions in the instruction window is restricted to a part of the instruction sequence.

Because of the serial program order, the instructions in this window are likely to become executable soon. The matching hardware can be restricted to a small number of slots.

Dataflow: the number of tokens waiting for a match can be very high. A large waiting-matching store is required.

Dataflow: Due to the lack of locality, the likelihood of the arrival of a matching token is difficult to estimate, caching of tokens to be matched soon is difficult.

18

Lessons Learned from Dataflow (Memory Latency)

Microprocessors: An unsolved problem is the memory latency caused by cache misses.

Example: SGI Origin 2000: latencies are 11 processor cycles for a L1 cache miss, 60 cycles for a L2 cache miss, and can be up to 180 cycles for a remote memory access. In principle, latencies should be multiplied by the degree of superscalar.

Microprocessors: Only a small part of the memory latency can be hidden by out-of-order execution, write buffer, cache preload hardware, lockup free caches, and a pipelined system bus.

Microprocessors often idle and are unable to exploit the high degree of internal parallelism provided by a wide superscalar approach.

Dataflow: The rapid context switching avoids idling by switching execution to another context.

19


Microprocessors: Finding enough fine-grain parallelism to fully exploit the processor will be the main problem for future superscalars.

Solution: enlarge the instruction window to several hundred instruction slots; two draw-backs

Most of the instructions in the window will be speculatively assigned with a very deep speculation level (today's depth is normally four at maximum). most of the instruction execution will be speculative. The principal problem here arises from the single instruction stream that feeds the instruction window.

If the instruction window is enlarged, the updating of the instruction states in the slots and matching of executable instructions lead to more complex hardware logic in the issue stage of the pipeline thus limiting the cycle rate.

20


Solutions: the decoupling of the instruction window with respect to different instruction

classes, the partitioning of the issue stage into several pipeline stages, and alternative instruction window organizations.

Alternative instruction window organization: the dependence-based microprocessor:

Instruction window is organized as multiple FIFOs. Only the instructions at the heads of a number of FIFO buffers can be issued

to the execution units in the next cycle. The total parallelism in the instruction window is restricted in favor of a less

costly issue that does not slow down processor cycle rate. Thereby the potential fine-grained parallelism is limited

somewhat similar to the threaded dataflow approach.

21

Lessons Learned from Dataflow (alternative instruction window organizations)

Look at dataflow matching store implementations Look into dataflow solutions like threaded dataflow

(e.g. repeat-on-input technique or strongly-connected arcs model) Repeat-on-input strategy issues compiler-generated code sequences

serially (in an otherwise fine-grained dataflow computer). Transferred to the local dataflow in an instruction window:

an issue string might be used; a serie of data dependent instructions is generated by a compiler and issued

serially after the issue of the leading instruction.

However, the high number of speculative instructions in the instruction window remains.

22

CSIDC 2001

The second annual Computer Society International Design Competition is now accepting applications from student teams.

Mobiles Gerät mit Bluetooth-Technologie

Vortreffen: geänderter Termin: Do 23.11. 16:00 Geb. 20.20 Raum 267

http://computer.org/ (xxx)

23

Chapter 3: CISC Processors

A brief look at CISC ProcessorsOut-of-order executionScoreboarding techniqueCDC 6600 (with scoreboarding)Alternate technique: Tomasulo scheduling

24

Prerequisites for CISC Processors

Technology in the 1960s and early 1970s was dominated by high hardware cost, in particular by high cost for memory.

Only a small main memory and slow memory access! Instruction fetch was done from main memory and could be overlapped with

decode and execution of previous instructions. Observation: the number of cycles per instruction was determined by the number

of cycles taken to fetch the instruction. CISC (complex instruction set computer) approach:

it is acceptable to increase the average number of cycles taken to decode and execute an instruction.

reduce the number of instructions and encode these instructions densely.

Multiple-cycle instructions reduce the overall number of instructions, and thus reduce the overall execution time because they reduce the instruction-fetch time.

25

A Brief Look at CISC Processors

All mainframes in the 1960s and 1970s were CISCs, namely e.g. CDC 6600, and the IBM 360/370 family

CISC microprocessor lines: Intel x86 family line from Intel 8088 to Pentium IIIMotorola from 6800 to 68060Zilog from Z80 to Z80000 (terminated)National Semiconductor NS320xx (terminated)

26

Scheduling

Scheduling: a process which determines when to start a particular instruction, when to read its operands, and when to write its result,

Target of scheduling: rearrange instructions to reduce stalls when data or control dependences are present

Static scheduling: the compiler does it Dynamic scheduling: the hardware does it Key idea: Allow instructions behind stall to proceed

DIVD F0,F2,F4

ADDD F10,F0,F8

SUBD F12,F8,F14SUBD is not data dependent on anything in the pipeline Enables out-of-order execution out-of-order completion ID stage checks for structural and data dependencies

27

Dynamic Scheduling

Dynamic scheduling works also when stalls arise that are unknown at compile-time, e.g. cache misses

Dynamic scheduling can be either: Control flow scheduling, when performed centrally at

the time of decodee.g. scoreboarding in CDC 6600

Dataflow scheduling, if performed in a distributed manner by the FUs themselves at execute time. Instructions are decoded and issued to reservation stations awaiting their operands. Tomasulo scheme in the IBM System/360 Model 91 processor

28

Scoreboarding

Introduced in 1963 by Thornton in the CDC6600 processor.Goal of scoreboarding is to maintain an execution rate of one

instruction per clock cycle by executing an instruction as early as possible.

Instructions execute out-of-order when there are sufficient resources and no data dependences.

A scoreboard is a hardware unit that keeps track of the instructions that are in the process of being executed, the functional units that are doing the executing, and the registers that will hold the results of those units.

A scoreboard centrally performs all hazard detection and resolution and thus controls the instruction progression from one step to the next.

29

Scoreboard Pipeline

IF IS WB

MEMEXINT

EXFUf

EXFU1

EXFUn

RO

Scoreboard

30

Scoreboarding

The ID stage of the standard pipeline is split into two stages, the issue (IS) stage:

decode instructions, check for structural and WAW hazards and

the read operands (RO) stage: wait until no data hazards, then read operands from registers.

EX and WB stages are augmented with additional bookkeeping tasks.

31

Scoreboarding (in more detail)

IS stage: if there is no structural hazard and no WAW hazard, the scoreboard issues the instruction to the FU and updates its internal data

structure; otherwise, the instruction issue stalls, and no further instruction is issued until

the hazard is cleared (==> single issue and in-order issue!).

RO stage: the scoreboard monitors the availability of the input operands, and when they are all available, tells the FU to read them from the register file (==> no forwarding!!) and to proceed to EX stage. RAW hazards are dynamically resolved; (==> instructions may be dispatched into EX stage out of order)

EX stage: the FU begins execution (which may take multiple cycles) and notifies the scoreboard when the result is ready (result ready flag set!).

WB stage: once the scoreboard is aware that the FU has completed execution, the scoreboard checks for WAR hazards and stalls the completing instruction, if necessary. Otherwise, the scoreboard tells the FU to write its result to the destination register.

32

Scoreboard Implementation

Register result status table (R): indicates which FU will produce a result in each register (if any).The number of entries in R is equal to the number m of registers.

Functional unit status table (F): indicates the phase of execution each instruction is in. Phase flags Busy, RO, EX, and WB for each FU.

Instruction status table (also F): one entry per FU, telling what operation the FU is scheduled to do (opcode), where its result goes (destination register), where its operands come from (source registers), and if those results are available (validity of sources). If an operand is not available, the table tells which FU will produce it

(FU that produces a source value).

On power up, the scoreboard is initialized by setting all its entries to zero.

33

Scoreboarding Example

mul Reg1, Reg3, Reg5R 1 2 3 4 5 6 sub Reg2, Reg4, Reg3

div Reg6, Reg1, Reg4add Reg4, Reg2, Reg3

F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add2 mul3 div

cycle 0op EX cycles

all fields are initialized with 0 add 1empty fields have the value 0 sub 1

mul 4div 4

instruction information of the FUs

registers

phase flags of the FUs

func

tiona

l un

its

34



2 div Reg6, Reg1, Reg4add Reg4, Reg2, Reg3

F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add2 mul 1 mul 1 3 1 5 13 div

cycle 1


registers


func

tiona

l un

its

35



2 1 div Reg6, Reg1, Reg4add Reg4, Reg2, Reg3

F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add 1 sub 2 4 1 3 12 mul 1 1 mul 1 3 1 5 13 div

cycle 2


registers


func

tiona

l un

its

36



2 1 3 div Reg6, Reg1, Reg4add Reg4, Reg2, Reg3

F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add 1 1 sub 2 4 1 3 12 mul 1 1 3 mul 1 3 1 5 13 div 1 div 6 1 2 4 1

cycle 3remaining cycles in EX

mul takes 4 cycles in EX


registers


func

tiona

l un

its

37




F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add 1 1 1 sub 2 4 1 3 12 mul 1 1 2 mul 1 3 1 5 13 div 1 div 6 1 2 4 1

cycle 4


registers


func

tiona

l un

its

38




F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add 1 1 12 mul 1 1 1 mul 1 3 1 5 13 div 1 div 6 1 2 4 1

cycle 5


registers


func

tiona

l un

its

39




F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add 1 add 4 2 1 3 12 mul 1 1 1 0 mul 1 3 1 5 13 div 1 div 6 1 2 4 1

cycle 6


registers


func

tiona

l un

its

40




F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add 1 1 add 4 2 1 3 12 mul 1 1 13 div 1 div 6 1 1 4 1

cycle 7


registers


func

tiona

l un

its

41




F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add 1 1 1 add 4 2 1 3 12 mul 1 1 13 div 1 1 div 6 1 1 4 1

cycle 8


registers


func

tiona

l un

its

42




F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add 1 1 12 mul 1 1 13 div 1 1 3 div 6 1 1 4 1

cycle 9


registers


func

tiona

l un

its

43





cycle 10


registers


func

tiona

l un

its

44





cycle 11


registers


func

tiona

l un

its

45




F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add 1 1 12 mul 1 1 13 div 1 1 1 0 div 6 1 1 4 1

cycle 12


registers


func

tiona

l un

its

46



div Reg6, Reg1, Reg4add Reg4, Reg2, Reg3

F Busy RO EX WB Op Dest Src1 Vld1 FU1 Src2 Vld2 FU21 add 1 1 12 mul 1 1 13 div 1 1 1

cycle 13


registers


func

tiona

l un

its

47

CDC 6600 Processor

delivered in 1964 by Control Data Corporation

pipelining register-register instruction set (load/store architecture) 3-address instruction format

==> several characteristics of a RISC processor

first processor to make extensive use of multiple functional units 10 FUs able to operate simultaneously on 24 registers

4 FUs for 60-bit floating-point operations among eight 60-bit operand registers,

6 FUs for logic, indexing, and program control on the eight 18-bit address registers and eight 18-bit increment/index registers.

scoreboarding scheme!

48

CDC 6600 Processor

PP0PP1

PP2PP3PP4PP5PP6

PP7

PP8PP9

X0X1

X2

X3X4X5X6

X7

A0A1

A2A3A4A5A6

A7

B0B1

B2

B3B4B5B6B7

Add

BranchShift

Boolean

Fixed addDivide

Multiply

Multiply

IncrementIncrement

12 P

erip

hera

l Cha

nnel

s

Cen

tral

Mem

ory

Operating Registers

PeripheralProcessors Functional Units

ScoreboardInstructionStack

49

Scoreboard Summary

Main advantage: managing multiple FUs out-of-order execution of multicycle operations maintaining all data dependences (RAW, WAW, WAR)

Scoreboard limitations: single issue scheme, however: scheme is extendable to multiple-issue in-order issue no renaming antidependences and output dependences may lead to WAR and

WAW stalls, no forwarding hardware all results go through the registers

General limitations (not only valid for scoreboarding) number and types of FUs since contention for FUs leads to structural hazards the amount of parallelism available in code (dependences lead to stalls)

Tomasulo scheme removes some of the scoreboard limitations by forwarding and renaming hardware, but is still single and in-order issue

50

Next Lecture

Tomasulo in detail Tomasulo animation

fifth lecture: chapter 3: dataflow processors - hybrids

Documents

dataflow modeldata

dataflow processor pipeline

thread of instructions

dynamic dataflow computerswhy

largegrain dataflow

token buffer token matching

succeeding instruction

sequential thread