state of the practice: surveys results opensfs bwg april 17, 2013
DESCRIPTION
Workload Characterization Goals Understand and characterize workloads Create a representative synthetic workload Synergies Kernel extraction/creation MonitoringTRANSCRIPT
State of the practice:surveys results
OpenSFS BWGApril 17, 2013
Workload Characterization Team
•Leader: Pietro Cicotti - SDSC•Members:
Ilene Carpenter - NREL Rick Mohr - UTK Mike Booth – HPC Results Ben Evans – Terascala
Workload Characterization
•Goals Understand and characterize workloads Create a representative synthetic workload
•Synergies Kernel extraction/creation Monitoring
Responses
NICSNCRC OLCFARSC RIKEN SDSCcenters:
storage:
FS:
apps:
K OasisW2/3LTFSFS D/CD/HP/S
WRF,PISM
360TB
W22.2PB
W32.2PB
FS2.2PB
LTFS2.2PB
K30PB
3.4PB
600TB
Me576TB
Do1.2PB
Mo2.3PB
Pu576TB
mixed
Some statsOLCF(widow2) OLCF (widow3) NCRC FS
NCRCLTFS
NICS(Kraken)
NICS(Medusa) SDSC ARSC RIKEN
# users ?? ?? ?? ?? 1650 1650100+ 345??server Version 1.8.8 1.8.8 1.8.8 1.8.8 1.8.4 1.8.6 1.8.7 2.1.2 ??
Clientversion ?? ?? ?? ?? 1.8.4 1.8.6, 1.8.8 1.8.7 >=1.8.6 ??
# clients 19042 19042 3908 40 9440 400 1638 500 88000
Interconnects(server-client)
DDR IB,Cray Gemini
DDR IB,Cray Gemini
QDR IB,Cray Gemini
QDR IB,Cray Gemini Cray SeaStar QDR IB
10 GigE, IB,Myrinet
IB,Ethernet IB, Tofu
size (raw) 2.2 PB 2.2 PB 900 TB 3.1 PB 3.36 PB 600 TB 4608 TB 360 TB 10 PB (?)
# files 107M 117M 65M 38M 256M 18.3M 141M 7.1M ??
File Size Distribution
2KB4KB
8KB16KB
32KB64KB
128KB256KB
512KB1MB
2MB4MB
8MB16MB
32MB64MB
128MB
256MB
512MB1GB
2GB4GB
8GB16GB
32GB64GB
128GB256GB
512GB0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
File Size Distribution
NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average
File Size
File
s (%
)
2KB4KB
8KB16KB
32KB64KB
128KB256KB
512KB1MB
2MB4MB
8MB16MB
32MB64MB
128MB
256MB
512MB1GB
2GB4GB
8GB16GB
32GB64GB
128GB256GB
512GB0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
100.00
File Size Cumulative Distribution
NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average
File Size
File
s (%
)
File Capacity Distribution
2KB8KB
16KB32KB
64KB128KB
256KB512KB
1MB2MB
4MB8MB
16MB32MB
64MB
128MB
256MB
512MB1GB
2GB4GB
8GB16GB
32GB64GB
128GB256GB
512GB1TB 2TB 4TB
0
10
20
30
40
50
60
70
80
90
100
File Capacity Distribution
NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average
File Size
Capa
city
(%)
2KB8KB
16KB32KB
64KB128KB
256KB512KB
1MB2MB
4MB8MB
16MB32MB
64MB
128MB
256MB
512MB1GB
2GB4GB
8GB16GB
32GB64GB
128GB256GB
512GB1TB 2TB 4TB
0
10
20
30
40
50
60
70
80
90
100
File Capacity Cumulative Distribution
NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average
File Size
Capa
city
(%)
Dir Size Distribution
<2 <4 <8 <16 <32 <64 <128 <256 <512 <1k <2K <4K <8K0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
Dir Size Distribution
NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average
DirectorySize
Dirs
(%)
<2 <4 <8 <16 <32 <64 <128 <256 <512 <1k <2K <4K <8K0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
100.00
Dir Size Cumulative Distribution
NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average
DirectorySize
Dirs
(%)
Dir Capacity Distribution
2KB 8KB 16KB 32KB 64KB 128KB 256KB 512KB 1MB 2MB 4MB 8MB 16MB 32MB 64MB0
10
20
30
40
50
60
70
80
90
100
Dir Capacity Distribution
NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average
Dir Size
Capa
city
(%)
2KB8KB
16KB32KB
64KB128KB
256KB512KB
1MB2MB
4MB8MB
16MB32MB
64MB
128MB
256MB
512MB1GB
2GB4GB
8GB16GB
32GB64GB
128GB256GB
512GB1TB 2TB 4TB
0
10
20
30
40
50
60
70
80
90
100
File Capacity Cumulative Distribution
NICS (Kraken)NICS (Medusa)OLCF (widow3)ARSCSDSC (Dolphin)Average
Dir Size
Capa
city
(%)
Cumulative Distributions
2KB 4KB 8KB 16KB 32KB 64KB 128KB 256KB 512KB 1MB 2MB 4MB 8MB 16MB 32MB 64MB 128MB 256MB 512MB 1GB0
10
20
30
40
50
60
70
80
90
100
Cumulative Distributions
file_size_cfile_capacicity_cdir_size_cdir_capacicity_c%
Next
•
Thank you!