lecture 5: eitf20 computer architecture · lecture 5: eitf20 computer architecture anders ardö eit...

logoonly

Lecture 5: EITF20 Computer Architecture

Anders Ardö

EIT – Electrical and Information Technology, Lund University

November 19, 2014

A. Ardö, EIT Lecture 5: EITF20 Computer Architecture November 19, 2014 1 / 59

logoonly

Outline

1 Reiteration

2 Dynamic scheduling - Tomasulo

3 Superscalar, VLIW

4 Speculation

5 ILP limitations

6 What we have done so far


logoonly

Instruction Level Parallelism - ILP

ILP: Overlap execution of unrelated instructions: PipeliningTwo main approaches:

DYNAMIC =⇒ hardware detects parallelismSTATIC =⇒ software detects parallelism

Often a mix between both.

Pipeline CPI = Ideal CPI + Structural stalls+ Data hazard stalls + Control stalls


logoonly

Why loop unrolling works

Longer sequences of straight code without branches (longer basicblocks) allows for easier compiler static reschedulingLonger basic blocks also facilitates dynamic rescheduling such asScoreboard and Tomasulo’s algorithm


logoonly

Dynamic Branch Prediction

Branches limit performance because:Branch penaltiesLimit to available Instruction Level Parallelism

Solution: Dynamic branch prediction to predict the outcome ofconditional branches.Benefits:

Reduce the time to when the branch condition is knownReduce the time to calculate the branch target address


logoonly

Dependencies

Two instructions must be independent in order to execute inparallelThere are three general types of dependencies that limitparallelism:

Data dependenciesName dependenciesControl dependencies

Dependencies are properties of the programWhether a dependency leads to a hazard or not is a property ofthe pipeline implementation


logoonly

Scoreboard pipeline

Goal of scoreboarding is to maintain an execution rate of oneinstruction per clock cycle by executing an instruction as early aspossible.Instructions execute out-of-order when there are sufficientresources and no data dependencies.A scoreboard is a hardware unit that keeps track of

the instructions that are in the process of being executed,the functional units that are doing the executing,and the registers that will hold the results of those units.

A scoreboard centrally performs all hazard detection andresolution and thus controls the instruction progression from onestep to the next.


logoonly

Summary

ILP:Rescheduling and loop unrolling are important to take advantage ofpotential Instruction Level Parallelism

Dynamic instruction schedulingAn alternative to compile-time schedulingDoes not need recompilation to increase performanceUsed in most new processor implementations

Dynamic Branch Predictionreduce branch penalties by early prediction of conditional branchoutcomes


logoonly

Questions!

QUESTIONS?

COMMENTS?


logoonly

Lecture 5 agenda

Chapters 2.4-2.8, 3.1-3.4 in "Computer Architecture"

1 Reiteration


3 Superscalar, VLIW

4 Speculation

5 ILP limitations



logoonly

Outline

1 Reiteration


3 Superscalar, VLIW

4 Speculation

5 ILP limitations



logoonly

Scoreboard pipeline

Issue: Decode and check for structural hazardsRead operands: wait until no data hazards, then read operandsAll data hazards are handled by the scoreboard


logoonly

Limitations with Scoreboard

The number of scoreboard entries (window size)The number and types of functional unitsNumber of datapaths to registersThe presence of name dependencies

Tomasulo’s algorithm addresses the last two limitations.


logoonly

Tomasulo’s Algorithm

Another dynamic instruction scheduling algorithmFor IBM 360/91, a few years after the CDC 6600 (Scoreboard)Goal: High performance without compiler supportDifferences between Tomasulo & Scoreboard:

Control & Buffers distributed with FUs (called reservationstations) vs. centralized in ScoreboardRegister names in instructions replaced by pointers to reservationstation buffer (HW register renaming)Common Data Bus broadcasts results to all FUsLoads and Stores treated as FUs as well


logoonly

Tomasulo Organization


logoonly

Three Stages of Tomasulo Alg.

1. Issue – get instruction from FP Op QueueIf reservation station free (no structural hazard), the instruction isissued together with its operands (renames registers)

2. Execution – operate on operands (EX)When both operands are ready, then execute; if not ready, watchCommon Data Bus (CDB) for operands (snooping)

3. Write result – finish execution (WB)Write on CDB to all awaiting functional units; mark reservationstation available

Normal bus: data + destinationCommon Data Bus: data + source (snooping)


logoonly

Tomasulo example, cycle 0


logoonly



logoonly



logoonly



logoonly



logoonly



logoonly



logoonly

Elimination of WAR hazards

Example:

LD F6, 34(R2)... ...DIVD F10,F0,F6ADDD F6,F8,F2

ADDD can safely finish before DIVD has read register F6because:

DIVD has renamed register F6 to point at the reservation stationLD broadcasts its result on the Common Data Bus

Register renaming can thus be done:statically by the compilerdynamically by the hardware


logoonly



logoonly



logoonly



logoonly

Benefits Tomasulo

distributed hazard detection logicdistributed reservation stationsCommon Data Bus (CDB) with snooping

elimination WAR,WAW hazards (renaming registers)


logoonly

Dynamic scheduling Tomasulo - summary

tolerates unpredictable delayscompile for one pipeline - run effectively on anothersignificant increase in HW complexityout-of-order execution, completionregister renaming


logoonly

Outline

1 Reiteration


3 Superscalar, VLIW

4 Speculation

5 ILP limitations



logoonly

Getting CPI < 1!

Issuing multiple instructions per clock cycleSuperscalar : varying number of instructions/cycle (1-8) scheduledby compiler or HW

IBM Power5, Pentium 4, Sun SuperSparc, DEC AlphaSimple hardware, complicated compiler or...Very complex hardware but simple for compiler

Very Long Instruction Word (VLIW): fixed number of instructions(3-5) scheduled by the compiler

HP/Intel IA-64, ItaniumSimple hardware, difficult for compilerhigh performance through extensive compiler optimization


logoonly

Approaches for multiple issue

Issue Hazard Scheduling Characteristicsdetection /examples

Superscalar dynamic HW static in-order executionARM

Superscalar dynamic HW dynamic out-of-orderexecution

Superscalar dynamic HW dynamic speculationPentium 4

IBM power5VLIW static compiler static TI C6xEPIC static compiler mostly static Itanium


logoonly

Very Long Instruction Word (VLIW)

A number of functional units that independently executeinstructions in parallel.The compiler decides which instructions can execute in parallelNo hazard detection needed


logoonly

Itanium instruction format


logoonly

Itanium architecture


logoonly

Limits of VLIW

Limited Instruction Level ParallelismWith n functional units and k pipeline stages we need n x kindependent instructions to utilize the hardware

Memory and register bandwidthWith increasing number of functional units, the number of portsneeded at the memory or register file must increase to preventstructural hazards

Code sizeCompiler scheduled pipeline “bubbles” take up space in theinstructionNeed more aggressive loop unrolling to work well which alsoincreases code size

No binary code compatibility


logoonly

Outline

1 Reiteration


3 Superscalar, VLIW

4 Speculation

5 ILP limitations



logoonly

HW supported speculation

A combination of three main ideas:Dynamic instruction scheduling; take advantage of ILPDynamic branch prediction; allows instruction scheduling acrossbranchesSpeculative execution; execute instructions before all controldependencies are resolved

Hardware based speculation uses data-flow execution:instructions execute when their operands are available


logoonly

HW vs. SW speculation

Advantages:Dynamic runtime disambiguation of memory addressesDynamic branch prediction is often better than static which limitsthe performance of SW speculationHW speculation can maintain a precise exception modelCan achieve higher performance on older code (withoutrecompilation)

Main disadvantage:Extremely complex implementation and extensive need forhardware resources


logoonly

Tomasulo extended to handle speculation


logoonly

Re-order buffer - ROB

Data structure

entry instruction type destination value ready12...n

supports speculative executioninstructions commit in orderprecise exceptions


logoonly

Four steps of Speculative Tomasulo

Issue – get instruction from FP Op QueueIf reservation station and reorder buffer slot free, issue instr &send operands & reorder buffer nr. for destinationExecution – operate on operands (EX)If both operands ready: execute; if not, watch CDB for result;when both operands are in reservation station: executeWrite result – complete executionWrite on Common Data Bus to all awaiting FUs & reorder buffer ;mark reservation station availableCommit – update register with reorder resultWhen instr. is at head of reorder buffer & result is present; updateregister with result (or store to memory) and remove instr. fromreorder buffer;(handle misspeculations and precise exceptions)


logoonly

Misspeculation!

Commit – branch prediction wrongWhen branch instr. is at head of reorder buffer & incorrect prediction:remove all instr. from reorder buffer (flush);restart execution at correct instruction

Expensive =⇒ try to recover as early as possiblePerformance sensitive to branch prediction/speculationmechanism


logoonly

Multiple issue and speculation

Possible to extend Tomasulo with both multiple issue andspeculation.Major issues – instruction issue and monitoring CDBMust be able to handle multiple commitsAlternative to Tomasulo is to use extra physical registers for botharchitecturally visible registers and temporary values with registerrenaming


logoonly

Tomasulo speculation - increased complexity


logoonly

Dynamic scheduling, speculation - summary

tolerates unpredictable delayscompile for one pipeline - run effectively on anotherallows speculation

multiple branchesin-order commitprecise exceptionstime, energy; recovery

significant increase in HW complexityout-of-order execution, completionregister renaming


logoonly

Sandy Bridge microarchitecture


logoonly

Outline

1 Reiteration


3 Superscalar, VLIW

4 Speculation

5 ILP limitations



logoonly

ILP

How much performance canwe get by utilizing ILP?


logoonly

A model of an ideal processor

Provides a base for ILP measurementsNo structural hazardsRegister renaming – infinite virtual registers and all WAW & WARhazards avoidedMachine with perfect speculation

Branch prediction – perfect; no mispredictionsJump prediction – all jumps perfectly predicted

Memory-address alias analysis – addresses are known & a storecan be moved before a load provided addresses not equalPerfect caches

There are only true data dependencies left!


logoonly

Upper Limit to ILP


logoonly

Impact window size


logoonly

More realistic HW: Branch impact


logoonly

More realistic HW: Register impact


logoonly

Summary

Software (compiler ) tricks:Loop unrollingStatic instructionscheduling (with registerrenaming)... and more

Hardware tricks:Dynamic instructionschedulingDynamic branchpredictionMultiple issue –Superscalar, VLIWSpeculative execution... and more


logoonly

Outline

1 Reiteration


3 Superscalar, VLIW

4 Speculation

5 ILP limitations



logoonly

AMD Phenom CPU


logoonly

Intel Core2


logoonly

Intel Core2 chip (Nehalem)


lecture 5: eitf20 computer architecture · lecture 5: eitf20 computer architecture anders ardö eit...

Documents