high performance computing: evolution or...

High Performance Computing:High Performance Computing:Evolution or ReEvolution or Re--Definition?Definition?

HPCS 2003Sherbrooke, Quebec, Canada

May 11-14, 2003

David BarkaiHPC Computational Architect

Intel

2

EMEA HPTC Virtual TeamNational Lab Series

*Other brands and names are the property of their respective owners© Copyright 2002 Intel Corporation. All Rights Reserved.

What this talk is about

Observations about the evolution of HPCChoices & MetricsChallengesFrom clusters to gridsConvergence of technologies?

3



HPC Evolution

Custom CPUsCustom memoryCustom packagingCustom interconnectsCustom OS

COTS CPUsCOTS memoryCustom packagingCustom interconnectsCustom OS

COTS CPUCOTS MemoryCOTS PackagingCOTS InterconnectsCOTS OS

1980

~$5 million/gigaflop

1980

~$5 million/gigaflop 1990

~$200K /gigaflop

1990

~$200K /gigaflop 2000

<$2K /gigaflop

2000

<$2K /gigaflopFrom ~1GF ~100’s GF >10TFFrom ~1GF ~100’s GF >10TF

4



HPC EvolutionThe Enabling Hardware Technologies

EUV Lithography 15nm

TeraHertz Transistor(Performance , Power )

SiliconDie

Bumpless Build-Up Layer (BBUL)

Packaging

52-megabit chipone square micronSRAM cell on 90nm

Manufacturing (130nm)

300mm Wafers

www.intel.com/research/silicon

5



HPC Evolution

Silicon by the end of the Decade

Transistors

40048008

80858086

286386

486

Pentium® Pro

Itanium®

1.8B

0.001

0.01

0.1

1

10

100

1,000

10,000

’70 ’80 ’90 ’00 ’10

~2B Transistors

8080

Xeon®

Itanium2®

1,000

10,000

100,000

Itanium® procPentium® Pro

Pentium® proc486

100

’00

386286

1

10

’70 ’90

80868085

80808008

4004

’800.1

30GHz

14GHz6.5GHz

3 GHz

’10

~30 GHz

Frequency

10,00

100,000

1,000,000

“30 gigahertz devices, 10 nanometer or less delivering “30 gigahertz devices, 10 nanometer or less delivering a tera instruction of performance by 2010a tera instruction of performance by 2010”(1)”(1)

1) Pat Gelsinger, Intel CTO, Spring 2002 IDF

6



HPC Evolution

Vertical Model

Workforce ProductivityWorkforce ProductivityTechnology RefreshTechnology Refresh

High Performance ComputingHigh Performance ComputingServer ConsolidationServer Consolidation

Gov

ernm

ent &

Educ

atio

n

Ret

ail &

CPG

Hea

lthca

re

Dig

ital M

edia

Com

mun

icat

ions

Man

ufac

turin

g

Ener

gy &

Util

ities

Fina

ncia

l Ser

vice

s

System Management CRM, SCM, EAI, ERP

Security, Mobility/Occasionally ConnectedG

over

nmen

t &Ed

ucat

ion

Ret

ail &

CPG

Hea

lthca

re

Dig

ital M

edia

Com

mun

icat

ions

Man

ufac

turin

g

Ener

gy

Fina

ncia

l Ser

vice

s

Life

Sci

ence

s

7



HPC Evolution

Mutual Interests and Objectives

Technology Leadership Technology Adopters

Market Maker Market Mover & Influencer

Early InnovatorsKey Strategic Influencer

Grand Challenge ProblemsSolution Enabler

Products Computational Needs

8



HPC Evolution

iA Solution Deployments over the last 18mosIndustry•Western GECO•GCC•Shell•Saudi Aramco•Tensor Geophysical •WETA•Pemex•Google•Inpharmatica•Syrxx•Immunex•MDS Proteomics•Pixar•British Petroleum•DaimlerChrysler•SAS•Ifineon•Volvo

Academic•SUNY Buffalo•Denmark Scientific•Mississippi State•Louisiana State•Brookhaven•Clemson•Utah•Cornell•Toronto•Virginia Polytech•Ohio State•Tsinghua Uni.•Imperial College•National U of Singapore•Swinburne•Stanford Linear Accelerator •South Hampton•Valencia•Oxford•Johns Hopkins•Cal Tech•Belfast•Zeijing•Princeton

Government•Tera-Grid•Los Alamos National Lab •Lawrence Livermore National Lab•NOAA•Pacific Northwest National Lab •Sandia National Lab•NCSA•Classified Defense Sites•China Atmospheric Center•CERN•China Atmosphere•Seoul National University Grid•Ohio Supercomputing Center•Inria

…and many others

9



The Growing Popularity of Cluster Computing

HPC Evolution

The number of clusters in the TOP500 has grown to nearly 20 percent, with a total of 93 systems, 56 of them are Intel Architecture based.

10



HPC EvolutionThe Community Triangle

HPC Community

Intel

HPC-FocusedComputational

Scientists

Fellow-travelers:OEM,IHV, ISV

World-wide HPC team chartered with beingthe conduit between Intel and the HPC user community

11



Choices, MetricsIntel® Architecture Server

Processor RoadmapPe

rfor

man

ce ,

RA

S, S

cala

bilit

y

2002 2004

. . .. . .

Intel® Itanium® 2 Processor

Intel® Itanium® Intel® Itanium® 2 Processor2 Processor Intel® Xeon™

ProcessorIntel® Xeon™Intel® Xeon™

ProcessorProcessor

2003 2005

Intel® Itanium® 2 Processor (“Madison”)

Intel® Itanium® Intel® Itanium® 2 Processor 2 Processor (“Madison”)(“Madison”)

. . .. . .

Intel® Xeon™Processor

Intel® Xeon™Intel® Xeon™ProcessorProcessor

Future IA-32Xeon Processor

Family

Future IAFuture IA--3232Xeon ProcessorXeon Processor

FamilyFamily

• Intel® Itanium® architecture

• 1 GHz• 0.18µ• 3MB IL3 cache

• 0.13µ• 6MB IL3 cache•Intel® Itanium® 2 processor platform compatible

• 130nm• 9MB IL3 cache• Intel® Itanium® 2

processor platform compatible

• Intel® NetBurst™ microarchitecture• Hyper-Threading Technology• 0 .13µ; 2MB iL3 cache

• Intel® NetBurst™ microarchitecture• Hyper-Threading Technology• Large iL3 cache• 90nm

2006-Beyond

Future Future DirectionsDirections

••Second half ofSecond half ofdecade could decade could see:see:<65nm process<65nm process~1 billion ~1 billion

transistorstransistors--MultiMulti--threading threading on Itanium on Itanium Processor FamilyProcessor Family

••Possible uses forPossible uses for1Bn transistors;1Bn transistors;--Very large L3Very large L3cachescaches

--Multiple Multiple processor cores processor cores with shared with shared cachecache

All dates specified are target dates, are provided for planning purposes only and are subject to change

Intel® Itanium® 2 Processor (“Madison+”)

Intel® Itanium® Intel® Itanium® 2 Processor 2 Processor (“Madison+”)(“Madison+”)

Intel® Itanium® Dual Core Processor

(“Montecito”)

Intel® Itanium® Intel® Itanium® Dual Core Dual Core Processor Processor

(“Montecito”)(“Montecito”)

Deerfield~60w

~1.5MB IL310Gb SerDesIntegrated on90nm process

PCI-Express

12



Choices, Metrics

14

19851985 20022002

SystemSystemCapabilityCapability

ProcessorProcessorFrequencyFrequency

Intel® 486Processor

Intel® 486Intel® 486ProcessorProcessori386

Processori386i386

ProcessorProcessor

Pentium®Processor

Pentium®Pentium®ProcessorProcessor

Pentium® ProProcessor

Pentium® ProPentium® ProProcessorProcessor

Pentium® IIXeon™Processor

Pentium® IIPentium® IIXeon™Xeon™ProcessorProcessor

Pentium® IIIXeon™Processor

Pentium® Pentium® IIIIIIXeon™Xeon™ProcessorProcessor

Itanium™Processor

Itanium™Itanium™ProcessorProcessor

Xeon™ProcessorXeon™Xeon™ProcessorProcessor

More than SiliconMore than Silicon

ProcessorProcessorCapabilitiesCapabilities

32-bit3232--bitbit

Mathc o pro c e s s o r

MathMathc o pro c e s s o rc o pro c e s s o r

MMXTe c hno lo g y

MMXMMXTe c hno lo g yTe c hno lo g y

S upe rs c alarS upe rs c alarS upe rs c alar

S MPS MPS MP

Ne tburs t™µ-arc h

Ne tburs t™Ne tburs t™µµ--arc harc h

L2 Cac he Bus

L2 Cac he L2 Cac he BusBus

Larg e L2Larg e L2Larg e L2

S tre am ing S IMDExte ns io ns

S tre am ing S IMDS tre am ing S IMDExte ns io nsExte ns io ns

Inte g rate d L2Inte g rate d L2Inte g rate d L2

ECCECCECC

EPICEPICEPIC

MCAMCAMCA

64-bitArc h.6464--bitbitArc h.Arc h.

Cartridg eCartridg eCartridg e

Hype r-Thre adingTe c hno lo g y

Hype rHype r--Thre adingThre adingTe c hno lo g yTe c hno lo g y

S S E2S S E2S S E2

(Paralle lis m )(Paralle lis m )(Paralle lis m )

S e am le s s4-way

S e am le s sS e am le s s44--w ayway

2-way22--w ayway

Pro fus io n8-way

Pro fus io nPro fus io n88--wayway

AGP2X

AGPAGP2X2X

AGP4X

AGPAGP4X4X

Larg e S MP16-64P

Larg e S MPLarg e S MP1616--64P64P

Platform & Platform & ChipsetChipset

CapabilitiesCapabilities

InfiniBand*Te c hno lo g yInfiniBand*InfiniBand*Te c hno lo g yTe c hno lo g y

AGP8X

AGPAGP8X8X

Ultra-de ns eFo rm fac to rsUltraUltra--de ns ede ns e

Fo rm fac to rsFo rm fac to rsCarrie r-g rade

S e rve rsCarrie rCarrie r--g radeg rade

S e rve rsS e rve rs

PCIPCIPCI

FaultTo le ranc e

FaultFaultTo le ranc eTo le ranc e

*Other names and brands are property of their respective owners.*Other names and brands are property of their respective owners.

PCI-xpre s sPCIPCI--xpre s sxpre s s

Multi-c o reMultiMulti--c o rec o re

Mo re GHzMo re GHzMo re GHz

FutureFutureFuture

Innovating at all levels of the platformInnovating at all levels of the platform

13



Choices, MetricsBlade Clusters – The Next Wave?

14



Some choices of cluster interconnectChoices, Metrics

805 MB/s7.5 usVariousInfiniBand* 4x

340 MB/s< 5 usFat TreeQuadrics*

320 MB/s< 4 us2D/3D TorusDolphin* SCI

243 MB/s7 usFat TreeMyrinet*

BandwidthMPI LatencyTopologyInterconnect

15



Choices, Metrics

Software Development Tools

Compilers

VTune™

Performance Analyzer

Performance Libraries

SW Products Developer Services

Intel®

ThreadingTools

www.intel.com/ids

16



Choices, Metrics

Processors Chipsets Platform Interconnect Software

17



More than Silicon –Broad Choice of Solutions and Services

Choices, Metrics

Rack SaverDP/1U

Scale-Up(SMP, ccNUMA)

SGI 64/512P

NEC 32P

Unisys 16P

Scale-Out(Cluster)

HPDP/2U HP

4P/4UIBM4P/8P/16P

18



Intel Confidential – Internal Use OnlyChoices, Metrics

Solutions Market Development

Solutions ChannelsSolutions Channels

Solutions EnablingSolutions Enabling

ProductsProducts

Architecture / StandardsArchitecture / Standards

*All names and brands are property of their respective owners

19



Choices, MetricsDifferentiation of HPC

Computationally intensiveLarge scale appsRandom, dynamic data access patterns

Pre-early adoptersExpert users

20



Choices, MetricsArchitectural implications

Fast computational units– Not just floating point

Large memory with high bandwidth & low latency– Larger caches don’t always help– Maintain bandwidth/latency within n-way node

Wide, low-latency interconnect for distributed applications

21



Choices, MetricsWe’d like to believe that...

The academic and public sector HPC serves as proving grounds for Industry / private sectorHigh-end technologies experimented by HPC community will go main-stream in 1-2 ‘technology generations’

The Industry needs to invest in HPC becauseits community is technology trend setter and innovator-adopter

22



Needs of apps drive selection metrics Choices, Metrics

Cannot satisfy every job – Step 1 = Determine the apps/datasets that matter!– Not necessarily the most demanding..

App “environment” factors; such as --– Distribution of workload by number and size of jobs– Distribution by required time to solution and size– Frequency of compilations, restarts, ..

App characteristics; such as --– Data access patterns and size– Granularity of parallelism– Computations – per data accessed; float/integer mix

23



The Metrics form a multidimensional Choices, Met

Space

rics

Memory- Bandwidth- Latency- Size

Computations- Flop rate- Integer rate- Float/Integer mix

Interconnect- Bandwidth from node- Latency- Bisection bandwidth

Operational Env- Size- Power / rack, total- File System*

* File system choice can be thought as derived from app needs, or as an arch choice that drives others..

24



Choices, MetricsDeriving a Metric

Find your part of a surface or a bounded sub-space in the universe of the metricsDetermine the relationships among parameters in your metrics spaceExamples:– Max B:F in node, but <10– B:F:I = 4:1:0.5 or 1:1:0.1 – Min Watt/Flop– Etc., etc.

25



Choices, MetricsA Paradox?

The choices in cluster components allow us to tailor a cluster to very specific needs– Examples: Vis, Storage clusters

COTS make HPC cluster affordable at the department or project levelAs a result, the cluster can be as specific as we wish..

Commodity components enable (drive?) custom solutions

26



Choices, MetricsPrediction

Special Purpose Clusters will be common– More correctly, “tailored” or “tunable” clusters may

become the norm– Is “SPC” or “TC” in our future?

27



ChallengesNot all is done..

A reminder that existing challenges only get worse as we march on to the implications of Moore’s LawThe next frontier(s)– I/O and interconnect keeping pace with processor

speed and memory size– Managing clusters – quality and performance of

monitoring, maintenance, effective use

28



®

9Copyright © 2002 Intel Corporation. All Rights Reserved.*All names and brands are property of their respective owners

Intel WW ServicesIntel WW Services--Dedicated HPC PersonnelDedicated HPC Personnel--End User Developer EnablingEnd User Developer Enabling--Infrastructure (WA, NM, VG) Infrastructure (WA, NM, VG) --Premier Technical SupportPremier Technical Support--HW Design SupportHW Design Support

Intel BuildingIntel BuildingBlocksBlocks--MicroprocessorsMicroprocessors--Chipsets Chipsets --InterconnectsInterconnects--Platform EnablingPlatform Enabling--Software SuiteSoftware Suite

Intel Portfolio Intel Portfolio InvestmentsInvestments--SCALI*, United Devices*SCALI*, United Devices*--Portfolio MatchmakingPortfolio Matchmaking

Strategic CollaborationsStrategic Collaborations--Cornell Theory Center* (CTC)Cornell Theory Center* (CTC)--CERNCERN--NCSANCSA--Western GECO*Western GECO*--European Virtual Grid Center* European Virtual Grid Center* --DaresburyDaresbury Benchmark Center*Benchmark Center*--Singapore Singapore BioITBioIT Grid*Grid*

Intel ProgramsIntel Programs-- 13 major hardware vendors13 major hardware vendors-- Regional LeadersRegional Leaders-- SI ProgramsSI Programs-- iSV iSV enabling enabling

Intel TrainingIntel Training--Worldwide Software CollegeWorldwide Software College--Worldwide TrainingWorldwide Training--Cluster RecipesCluster Recipes--Case Studies/Blue PrintsCase Studies/Blue Prints

IntelIntelManufacturingManufacturing--.13us, 90nm, 300mm.13us, 90nm, 300mm-- Platform ValidationPlatform Validation

Joint Joint Solutions CentersSolutions Centers-- ChannelChannel-- OilOil-- Finance Finance -- Life SciencesLife Sciences

HPC Program Office HPC Program Office More than SiliconMore than Silicon

* Other names and brands may be claimed as the property of others.

Finding Solutions to the Challenges requires an Ecosystem

Challenges

29



From clusters to GridsBeyond Clusters - Grids

*All names and brands are property of their respective owners.© Copyright 2002, Intel Corporation. All rights reserved. 9

GridGrid

Shared Shared ResourcesResources

Rich ClientRich Client

Cluster, SMP, BladesCluster, SMP, Blades

Distributed Distributed ApplicationsApplications

ApplicationsApplicationsOSOS

MiddlewareMiddlewareHardwareHardware

Traditional Traditional HPC ArchitectureHPC Architecture

Standard BasedStandard BasedClustersClusters

SMPSMP

Future HPC Future HPC ArchitectureArchitecture

Current Current HPC ArchitectureHPC Architecture

ProprietaryProprietaryRISCRISC

VectorVectorCustomCustom

The Changing Models ofThe Changing Models ofHigh Performance ComputingHigh Performance Computing

30



From clusters to GridsSolutions Adoption Curve

Indu

stry

Ado

ptio

nIn

dust

ry A

dopt

ion

GOAL = GOAL = EcosystemEcosystem

scalescale

GOAL = GOAL = Select the RightSelect the RightOpportunitiesOpportunities

GOAL = GOAL = Select the RightSelect the Right

PartnersPartners

GOAL = GOAL = Solid EUSolid EU

Proof PointsProof Points

GOAL = GOAL = Extend Ecosystem Extend Ecosystem

ParticipationParticipation

GridGrid ClustersClusters SMPSMP

MATURITYMATURITY TimeDEVELOPMENT RAMPRAMPDISCOVERY DISCOVERY DEVELOPMENT

31



From clusters to GridsThe duality of Grid Computing

Grid Computing as an networked ensemble of data centers– As originally understood by the people who coined the

term (and formed the Grid Forum)Grid computing as harnessing resources on desktops– Activity that was originally known as “distributed

computing” (.. On the Internet)– Proprietary software is being replaced by Globus

32



Technology ConvergenceIntersection of Technologies

Is desktop grid computing (DGC) HPC? Is it cluster computing?– DGC can, and is, employed to solve problems of the

size we call “HPC problems”– DGC employs, more and more, the same resource

sharing tools used for cluster-based gridsDGC is interesting – it builds on both P2P and clusters technologies– P2P: identification/authentication, intermittent

connectivity– Cluster: scheduling, monitoring, ..

33



Technology ConvergenceFodder for Thought

DGC, with P2P foundation and cluster-based traditional grid tools, can define a special class of clustersP2P technologies enable types of apps other than computations and data handling– Content distribution, team collaboration

Should the concept of cluster be extended to include these other P2P app types?– After all, they involved coordinated use of multiple

and networked collection of systemsShould “HPC” be extended to these app types– And then really be a superset of HPTC

34



Not mere Semantics

Defining the scope of “clusters” and “HPC”matters to their development– If technologies are shared it makes sense for all

relevant practitioners to collaborate in their development

– For that to happen we’d need some sort of a conceptual umbrella ..

Clusters and HPC can be thought as occupying a spectrum – from dedicated high-end clustered SMPs to loosely coupled Internet-connected desktops

35



Concluding RemarksThe economics of HPC have changed, and it drives exciting opportunitiesIn an interesting twist commoditization opens the door to specializationThe next big challenges are Interconnect (hardware) and Cluster Mgmt (mostly software)Intel works with the Industry and with the user community to enable HPC for discovery and scienceThe “beyond clusters” examination can get the computer industry to pay closer attention to HPC

36



Thank you

Questions?

high performance computing: evolution or...

Documents