The TOP500 Project:Lessons learned throughnine years of experience

Hans Werner MeuerUniversity of Mannheim

andPrometeus GmbH

TOP500 Promotion Tour in Singapore and New ZealandJanuary / February 2002

TOP500

� Project about:- Supercomputers- HPC-market analysis- Technology trends

� It�s about performance�The Need For Speed�

- Competition between vendors and between sites

� What is the TOP500?

Outlook

� TOP500 Approach

� HPC market as of 11/2001� Market Trends, Technology Trends,

Architecture Trends

� Customer Types

� WWW service for TOP500

Supercomputer and TOP500

� 1980�s (early)- Vector computers- One main application area

� �numerical�, scientific and engineering

� What is a �Supercomputer�?

� 1990�s- ?

! Mannheim Supercomputer Statistics

TOP500 Idea� Assemble list of the XXX most

powerful computers in the World anddefine them as �Supercomputer�

� Update as needed� The problem is now shifted from

defining �Supercomputer� to�powerful�

� Define �powerful� with a benchmarkresult

TOP500 Benefits� Basis for analysing the HCP market

� Quantification of observations

� Detection of trends- market,- architecture,- technology

- Listing of the 500 most powerful computers in the world - Yardstick: Rmax from Linpack

Ax=b, dense problem

- Updated twice a year:ISC�xy in Germany, June xySC�xy in USA, November xy

- All data available from www.top500.org

TOP500 Procedure

Size

Rat

e

TPP perform ance

TMCCM-2(2048)

Fujitsu VP-2600

Cray Y-MP (8)

05

101520253035404550

1990 1992 1994 1996 1998 2000

Year

GF

lop/

s

In 1980 a computation that took 1 full year to completecan now be done in 1 month!

Fastest Computer Over Time

In 1980 a computation that took 1 full year to completecan now be done in 4 days!

HitachiCP-PACS

(2040)IntelParagon

(6788)

FujitsuVPP-500

(140)TMC CM-5(1024)

NEC SX-3(4)

TMCCM-2(2048)

Fujitsu VP-2600

Cray Y-MP (8)

050

100150200250300350400450500

1990 1992 1994 1996 1998 2000

Year

GF

lop/

s

Fastest Computer Over Time

ASCI White Pacific(8192)

ASCI BluePacific SST

(5808)

SGI ASCIBlue

Mountain(5040)

Intel ASCI Red

(9152)Hitachi

CP-PACS(2040)

IntelParagon(6788)

FujitsuVPP-500

(140)

TMC CM-5(1024)

NEC SX-3

(4)

TMCCM-2(2048)

Fujitsu VP-2600

Cray Y-MP (8)

Intel ASCI Red Xeon

(9632)

010002000300040005000600070008000

1990 1992 1994 1996 1998 2000

Year

GF

lop/

s

In 1980 a computation that took 1 full year to completecan today be done in 1 hour!

Fastest Computer Over Time

TOP500 Status

� 1. List June 1993 ISC1993 in Mannheim

� 18. List November 2001 SC2001 in Denver

� 19. List June 2002 ISC2002 in Heidelberg

� Accepted by users and manufacturers

TOP500 Authors

- Authors since 06/1993 and 11/2000

Jack Dongarra Horst Simon

- Started in spring 1993 by:

Hans W. Meuer Erich Strohmaier

TOP500 List � Data Shown� Manufacturer Manufacturer or vendor� Computer Type Indicated by manufacturer or vendor� Installation Site Customer� Location Location and country� Year Year of installation / last major update� Customer Segment Academic,Research,Industry,Vendor,Class.� # Processors Number of processors� Rmax Maxmimal LINPACK performance achieved� Rpeak Theoretical peak performance� Nmax Problemsize for achieving Rmax

� N1/2 Problemsize for achieving half of Rmax

� Nworld Position within the TOP500 ranking

Rank Manufacturer Computer Rmax

[TF/s] Installation Site Country Year Area of

Installation # Proc

1 IBM ASCI White SP Power3

7.23 Lawrence Livermore National Laboratory

USA 2000 Research 8192

2 Compaq AlphaServer SC ES45 1 GHz 4.06 Pittsburgh

Supercomputing Center USA 2001 Academic 3024

3 IBM SP Power3 375 MHz 3.05 NERSC/LBNL USA 2001 Research 3328

4 Intel ASCI Red 2.38 Sandia National Laboratory USA 1999 Research 9632 5 IBM ASCI Blue Pacific

SST, IBM SP 604E

2.14 Lawrence Livermore National Laboratory

USA 1999 Research 5808

6 Compaq AlphaServer SC ES45 1 GHz

2.10 Los Alamos National Laboratory

USA 2001 Research 1536

7 Hitachi SR8000/MPP 1.71 University of Tokyo Japan 2001 Academic 1152 8 SGI ASCI Blue

Mountain 1.61 Los Alamos

National Laboratory USA 1998 Research 6144

9 IBM SP Power3 375Mhz

1.42 Naval Oceanographic Office (NAVOCEANO)

USA 2000 Research 1336

10 IBM SP Power3 375Mhz 1.29 Deutscher Wetterdienst Germany 2001 Research 1280

TOP 1011/2001

Outlook

� TOP500 Approach

� HPC market as of 11/2001� Market Trends, Technology Trends,

Architecture Trends

� Customer Types

� WWW service for TOP500

Systems installed

Japan11%

Europe32%

Others8% USA

49%

Total: 500

11/2001

Others5%

USA59%

Europe22%

Japan14%

Total: 134,9 Tflops

Rmax installed11/2001

0%

5%

10%

15%

20%

25%

30%

35%

40%Au

stria

/Sw

itzer

land

Bene

lux

Ger

man

y

Fran

ce UK Italy

Scan

dina

via

Total: 158

Number of systems installed inEurope11/2001

Rank Manufacturer Computer Rmax [TF/s]

Installation Site Country Year Area of Installation

# Proc

31 Compaq AlphaServer SC ES45 1

GHz

0.71 Australian Partnership for Advanced Computing

Australia 2001 Academic 480

122 NEC SX-5/32M2 0.24 Bureau of Meteorology / CSIRO HPCCC

Australia 2000 Research 32

225 Compaq AlphaServer SC ES40 833 MHz

0.15 Victorian Partnership for Advanced Computing

Australia 2001 Academic 128

349 Cray Inc. T3E1200 0.12 National Institute of Water and Atmospheric

Research

New Zealand

1999 Research 140

361 Sun HPC 420 450 MHz Cluster

0.11 University of Adelaide Australia 2000 Academic 160

463 Hewlett-Packard SuperDome/ HyperPlex

0.09 Citibank Singapore 2001 Industry 64

Number of systems installed inAustralia, New Zealand and Singapore

11/2001

Rank Manufacturer Computer Rmax [TF/s]

Installation Site Country Year Area of Installation

# Proc

142 NEC SX-5/28M2 0.21 Korea Meteorological Administration (KMA)

Korea 2000 Research 28

156 Hewlett-Packard

SuperDome/

HyperPlex

0.20 NewSky Korea 2001 Industry 128

161 Hewlett-Packard

SuperDome/

HyperPlex

0.20 POSDATA Korea 2001 Industry 128

177 Hewlett-Packard

SuperDome/

HyperPlex

0.19 LG-EDS Systems Korea 2001 Industry 128

179 Hewlett-Packard

SuperDome/

HyperPlex

0.19 Supreme Court of Korea Korea 2001 Industry 128

184 Self-made Netfinity Cluster PIII

0.18 POSDATA Korea 2001 Industry 320

220 IBM SP Power3 375 MHz 16 way

0.16 Seoul National University Korea 2000 Academic 144

231 Hewlett-Packard

SuperDome/

HyperPlex

0.15 POSCO Korea 2001 Industry 96

11/2001

Number of systems installed inKorea: 16

Rank Manufacturer Computer Rmax [TF/s]

Installation Site Country Year Area of Installation

# Proc

436 Hewlett-Packard

SuperDome 0.10 POSDATA Korea 2001 Industry 64

437 Hewlett-Packard

SuperDome 0.10 POSDATA Korea 2001 Industry 64

440 Hewlett-Packard

SuperDome 0.10 Samsung Fire & Insurance Korea 2001 Industry 64

459 Hewlett-Packard

SuperDome/

HyperPlex

0.10 CTI/ShanDong Common Korea 2001 Industry 64

472 Hewlett-Packard

SuperDome/

HyperPlex

0.10 Korea Investment Trust Korea 2001 Industry 64

483 Hewlett-Packard

SuperDome/

HyperPlex

0.10 Samsung Korea 2001 Industry 64

484 Hewlett-Packard

SuperDome/

HyperPlex

0.10 Samsung Korea 2001 Industry 64

485 Hewlett-Packard

SuperDome/

HyperPlex

0.10 Shinhan Bank Korea 2001 Industry 64

11/2001

Number of systems installed inKorea: 16

Performance Statistics � CountriesKflops per inhabitant

11/2001

281

147 142123 113

66

33 24

322

0

50100

150200250

300350

USAGerm

any

Japa

n UKSca

ndinav

iaFra

nceSwitz

erlan

d

Italy

Luxe

mbourg

Outlook

� TOP500 Approach

� HPC market as of 11/2001� Market Trends, Technology Trends,

Architecture Trends

� Customer Types

� WWW service for TOP500

Performance Development135 TF/s

1.167 TF/s

59.7 GF/s

7.23 TF/s

0.4 GF/s

94.3 GF/s

Jun-9

3Nov

-93Ju

n-94

Nov-94

Jun-9

5Nov

-95Ju

n-96

Nov-96

Jun-9

7Nov

-97Ju

n-98

Nov-98

Jun-9

9Nov

-99Ju

n-00

Nov-00

Jun-0

1Nov

-01

Intel XP/S140Sandia

Fujitsu'NWT' NAL

SNI VP200EXUni Dresden

Hitachi/TsukubaCP-PACS/2048

Intel ASCI RedSandia

IBM ASCI WhiteLLNL

N=1

N=500

SUM

Cray T3E1200Environmental

Protection Agency 1 Gflop/s

1 Tflop/s

100 Mflop/s

100 Gflop/s

100 Tflop/s

10 Gflop/s

10 Tflop/s

1 Pflop/s

Performance Developmentprojection

ASCIEarth Simulator

Jun 93

Jun 94

Jun 95

Jun 96

Jun 97

Jun 98

Jun 99

Jun 00

Jun 01

Jun 02

Jun 03

Jun 04

Jun 05

N=1

N=500

N=100

Sum

N=10

1 GFlop/s

1 TFlop/s

1 PFlop/s

100 MFlop/s

100 GFlop/s

100 TFlop/s

10 GFlop/s

10 TFlop/s

Continents - Systems

USA/Canada

Europe

Japanothers

0

100

200

300

400

500

Jun 93

Nov 93Ju

n 94Nov 94Ju

n 95Nov 95Ju

n 96Nov 96Ju

n 97Nov 97Ju

n 98Nov 98Ju

n 99Nov 99Ju

n 00Nov 00Ju

n 01Nov 01

Continents - Performance

0%10%20%30%40%50%60%70%80%90%

100%Ju

n 93

Nov

93Ju

n 94

Nov

94Ju

n 95

Nov

95Ju

n 96

Nov

96Ju

n 97

Nov

97Ju

n 98

Nov

98Ju

n 99

Nov

99Ju

n 00

Nov

00Ju

n 01

Nov

01

Perfo

rman

ce

USA/Canada

other

Japan

Europe

Europe- Countries

Germany

UKFrance

ScandinaviaBenelux

Switzerlandothers

0

50

100

150

Jun-9

3Nov-9

3Ju

n-94

Nov-94

Jun-9

5Nov-9

5Ju

n-96

Nov-96

Jun-9

7Nov-9

7Ju

n-98

Nov-98

Jun-9

9Nov-9

9Ju

n-00

Nov-00

Jun-0

1Nov-0

1

Producers

USA

JapanEurope

0

100

200

300

400

500

Jun-93

Nov-93Ju

n-94Nov-94Ju

n-95Nov-95Ju

n-96Nov-96Ju

n-97Nov-97Ju

n-98Nov-98Ju

n-99Nov-99Ju

n-00Nov-00Ju

n-01Nov-01

Manufacturers - Systems

Cray

SGI

IBM

Sun

HP

TMC

Intel

FujitsuNEC

Hitachiothers

0

100

200

300

400

500

Jun-93

Nov-93

Jun-94

Nov-94

Jun-95

Nov-95

Jun-96

Nov-96

Jun-97

Nov-97

Jun-98

Nov-98

Jun-99

Nov-99

Jun-00

Nov-00

Jun-01

Nov-01

Manufacturers - Performance

Cray

SGI

IBM

Sun HP

TMC

Intel

Fujitsu

NECHitachiothers

0%10%20%30%40%50%60%70%80%90%

100%

Jun

93No

v 93

Jun

94No

v 94

Jun

95No

v 95

Jun

96No

v 96

Jun

97No

v 97

Jun

98No

v 98

Jun

99No

v 99

Jun

00No

v 00

Jun

01No

v 01

Perfo

rman

ce

HPC Manufacturers over 20 Years

�������������������������������������������

��������������������������������������������������������������������������������������

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

Cray

Cray Computer

SGI

CDC/ETA

Fujitsu

NEC

Hitachi

Convex/HP

TMC

Intel

nCUBE

Alliant

FPS

Meiko

Parsytec

MasPar

DEC/Compaq

KSR

IBM

Sun

/

/

Processor Type

Scalar

VectorSIMD

0

100

200

300

400

500

Jun 93

Nov 93

Jun 94

Nov 94

Jun 95

Nov 95

Jun 96

Nov 96

Jun 97

Nov 97

Jun 98

Nov 98

Jun 99

Nov 99

Jun 00

Nov 00

Jun 01

Nov 01

Chip Technology

CMOS/off the shelf

CMOS/ proprietaryECL

0

100

200

300

400

500

Jun-93

Nov-93

Jun-94

Nov-94

Jun-95

Nov-95

Jun-96

Nov-96

Jun-97

Nov-97

Jun-98

Nov-98

Jun-99

Nov-99

Jun-00

Nov-00

Jun-01

Nov-01

Chip Technology

Alpha

PowerPA

intel

MIPS

Sparcother COTSproprietary

0

100

200

300

400

500

Jun 93Nov 93Jun 94Nov 94Jun 95Nov 95Jun 96Nov 96Jun 97Nov 97Jun 98Nov 98Jun 99Nov 99Jun 00Nov 00Jun 01Nov 01

Architectures

Single Processor

SMP

MPP

SIMD

Constellation

Cluster - NOW

0

100

200

300

400

500

Jun-9

3Nov-9

3Ju

n-94

Nov-94

Jun-9

5Nov-9

5Ju

n-96

Nov-96

Jun-9

7Nov-9

7Ju

n-98

Nov-98

Jun-9

9Nov-9

9Ju

n-00

Nov-00

Jun-0

1Nov-0

1

Y-MP C90

Sun HPC

Paragon

CM5T3D

T3E

SP2

Cluster of Sun HPC

ASCI Red

CM2

VP500

SX3

Outlook

� TOP500 Approach

� HPC market as of 11/2001� Market Trends, Technology Trends,

Architecture Trends

� Customer Types

� WWW service for TOP500

Customer Types

0

100

200

300

400

500

Jun 93

Nov 93

Jun 94

Nov 94

Jun 95

Nov 95

Jun 96

Nov 96

Jun 97

Nov 97

Jun 98

Nov 98

Jun 99

Nov 99

Jun 00

Nov 00

Jun 01

Nov 01

Research

Academic

Vendor

Industrial/Commercial

Classified

0%10%20%30%40%50%60%70%80%90%

100%

Jun

93No

v 93

Jun

94No

v 94

Jun

95No

v 95

Jun

96No

v 96

Jun

97No

v 97

Jun

98No

v 98

Jun

99No

v 99

Jun

00No

v 00

Jun

01No

v 01

Research

Academic

Vendor

Industrial/Commercial

Classified

Customer Types - Performance

Industrial/Commercial Applications

Engineering

Commercial

Unknown

0

50

100

150

200

250

Jun-93

Nov-93

Jun-94

Nov-94

Jun-95

Nov-95

Jun-96

Nov-96

Jun-97

Nov-97

Jun-98

Nov-98

Jun-99

Nov-99

Jun-00

Nov-00

Jun-01

Nov-01

Customer Types - USA

0%10%20%30%40%50%60%70%80%90%

100%

Jun 93

Nov 93Ju

n 94Nov 94Ju

n 95Nov 95Ju

n 96Nov 96Ju

n 97Nov 97Ju

n 98Nov 98Ju

n 99Nov 99Ju

n 00Nov 00Ju

n 01Nov 01

Research

Academic

Other

Industrial/Commercial

Customer Types - Europe

0%10%20%30%40%50%60%70%80%90%

100%

Jun 93

Nov 93Ju

n 94Nov 94Ju

n 95Nov 95Ju

n 96Nov 96Ju

n 97Nov 97Ju

n 98Nov 98Ju

n 99Nov 99Ju

n 00Nov 00Ju

n 01Nov 01

Research

Academic

Other

Industrial/Commercial

Customer Types - Japan

0%10%20%30%40%50%60%70%80%90%

100%

Jun 9

3Nov

93Ju

n 94

Nov 94

Jun 9

5Nov

95Ju

n 96

Nov 96

Jun 9

7Nov

97Ju

n 98

Nov 98

Jun 9

9Nov

99Ju

n 00

Nov 00

Jun 0

1Nov

01

Research

AcademicOther

Industrial/Commercial

Manufacturers

0%

5%

10%

15%

20%

25%

30%

35%

Cray In

c.

Fujits

u HP

Hitach

i

IBM

NEC SGI

SUNSelf

Mad

e

Others

Total 500

11/2001

Share of customer types

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Cray

Inc.

Fujit

su HP

Hita

chi

IBM

NEC

SGI

SUN

OtherAcademicResearchIndustrial/Commercial

39 19 153 14 160 16 40 30 Σ 471

11/2001

Outlook

� TOP500 Approach

� HPC market as of 11/2001� Market Trends, Technology Trends,

Architecture Trends

� Customer Types

� WWW service for TOP500

www.supercomp.de

Thank you!

TheThe slidesslides of of thisthis presentationpresentation

cancan bebe downloadeddownloaded at: at:

www.uni-mannheim.de/rum/dokus/talks/