Workload Characterization Team•Leader: Pietro Cicotti - SDSC

•Members:

Ilene Carpenter – NREL

Devesh Tiwari - ORNL

Chadha, Narjit - NetApp

Rick Mohr - UTK

Mike Booth – HPC Results

Ben Evans – Terascala

Workload Characterization

•Goals

Understand and characterize workloads

Create a representative synthetic workload

•Synergies

Kernel extraction/creation

Monitoring

Responses

NICSNCRC OLCFARSC RIKEN SDSCcenters:

storage:

FS:

apps:

K OasisW2/3LTFSFS D/CD/HP/S

WRF,PISM

360TB

W22.2PB

W32.2PB

FS2.2PB

LTFS2.2PB

K30PB

3.4PB

600TB

Me576TB

Do1.2PB

Mo2.3PB

Pu576TB

mixed

Some stats

OLCF(widow2) OLCF (widow3) NCRC FS

NCRCLTFS

NICS(Kraken)

NICS(Medusa) SDSC ARSC RIKEN

# users ?? ?? ?? ?? 1650 1650100+ 345??server Version 1.8.8 1.8.8 1.8.8 1.8.8 1.8.4 1.8.6 1.8.7 2.1.2 ??

Clientversion ?? ?? ?? ?? 1.8.4 1.8.6, 1.8.8 1.8.7 >=1.8.6 ??

# clients 19042 19042 3908 40 9440 400 1638 500 88000

Interconnects(server-client)

DDR IB,Cray Gemini

DDR IB,Cray Gemini

QDR IB,Cray Gemini

QDR IB,Cray Gemini Cray SeaStar QDR IB

10 GigE, IB,Myrinet

IB,Ethernet IB, Tofu

size (raw) 2.2 PB 2.2 PB 900 TB 3.1 PB 3.36 PB 600 TB 4608 TB 360 TB 10 PB (?)

# files 107M 117M 65M 38M 256M 18.3M 141M 7.1M ??

File Size Distribution

2KB4KB

8KB16KB

32KB64KB

128KB256KB

512KB1MB

2MB4MB

8MB16MB

32MB64MB

128MB

256MB

512MB1GB

2GB4GB

8GB16GB

32GB64GB

128GB

256GB

512GB0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

File Size Distribution

NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average

File Size

File

s (%

)

2KB4KB

8KB16KB

32KB64KB

128KB256KB

512KB1MB

2MB4MB

8MB16MB

32MB64MB

128MB

256MB

512MB1GB

2GB4GB

8GB16GB

32GB64GB

128GB

256GB

512GB0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

100.00

File Size Cumulative Distribution

NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average

File Size

File

s (%

)

File Capacity Distribution

2KB8KB

16KB32KB

64KB128KB

256KB512KB

1MB2MB

4MB8MB

16MB32MB

64MB

128MB

256MB

512MB1GB

2GB4GB

8GB16GB

32GB64GB

128GB

256GB

512GB1TB 2TB 4TB

0

10

20

30

40

50

60

70

80

90

100

File Capacity Distribution

NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average

File Size

Capa

city

(%)

2KB8KB

16KB32KB

64KB128KB

256KB512KB

1MB2MB

4MB8MB

16MB32MB

64MB

128MB

256MB

512MB1GB

2GB4GB

8GB16GB

32GB64GB

128GB

256GB

512GB1TB 2TB 4TB

0

10

20

30

40

50

60

70

80

90

100

File Capacity Cumulative Distribution

NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average

File Size

Capa

city

(%)

Dir Size Distribution

<2 <4 <8 <16 <32 <64 <128 <256 <512 <1k <2K <4K <8K0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

Dir Size Distribution

NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average

DirectorySize

Dirs

(%)

<2 <4 <8 <16 <32 <64 <128 <256 <512 <1k <2K <4K <8K0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

100.00

Dir Size Cumulative Distribution

NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average

DirectorySize

Dirs

(%)

Dir Capacity Distribution

2KB 8KB 16KB 32KB 64KB 128KB 256KB 512KB 1MB 2MB 4MB 8MB 16MB 32MB 64MB0

10

20

30

40

50

60

70

80

90

100

Dir Capacity Distribution

NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average

Dir Size

Capa

city

(%)

2KB8KB

16KB32KB

64KB128KB

256KB512KB

1MB2MB

4MB8MB

16MB32MB

64MB

128MB

256MB

512MB1GB

2GB4GB

8GB16GB

32GB64GB

128GB

256GB

512GB1TB 2TB 4TB

0

10

20

30

40

50

60

70

80

90

100

File Capacity Cumulative Distribution

NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average

Dir Size

Capa

city

(%)

Cumulative Distributions

2KB 4KB 8KB 16KB 32KB 64KB 128KB 256KB 512KB 1MB 2MB 4MB 8MB 16MB 32MB 64MB 128MB 256MB 512MB 1GB0

10

20

30

40

50

60

70

80

90

100

Cumulative Distributions

file_size_cfile_capacicity_cdir_size_cdir_capacicity_c%

Creation Time Distribution

Modification Time Distribution

Access Time Distribution