MFE Simulation Data Management

SLAC DMW 2004March 16, 2004

W. W. Lee and S. Klasky

Princeton Plasma Physics Laboratory

Princeton, NJ

Spatial & Temporal Scales Present Major Challenge to Theory & Simulations

• Huge range of spatial and temporal scales.

• Overlap in scales often means strong (simplified) ordering not possible

• Different codes/theory for different scales.

• 5+years: Integration of physics into Fusion Simulation Project

10-6 10-4 10-2 100 102

Spatial Scales (m)electron gyroradius

Debye length

ion gyroradius

tearing length

skin depth system size

atomic mfp electron-ion mfp

10-10 10-5 100 105

Temporal Scales (s)

electron gyroperiod electron collision

ion gyroperiod Ion collision

inverse electron plasma frequency confinement

Inverse ion plasma frequency current diffusion

pulse length

Major Fusion Codes

Data Rates of Major Fusion CodesCode (GB)

now / 5yr

Runtimenow/5yr (hr)

Processors

Now/5yr

Mbs

Now/5yr

GTC 4,000 / 100,000 300/150 2048 80/ 1600

Gyro 10 / 100 30/30 512/2048 .8/ 8

GS2 10 / 100 30/30 512/2048 .8 / 8

Degas2 .1 1 10 .2

Transp .05 3 1 .04

Nimrod 5/ 50 20/20 128 .6/ 6

M3D 10 / 100 20/20 128 1.1/ 11

NSTX .25/shot

1/ 40.25 * 40 9, 36

Total (TB) 4.3 / 101

Plasma Turbulence Simulation

• Gyrokinetic Particle-In-Cell Simulation -- Reduced Vlasov-Maxwell Equations

• Simulations on MPP Platforms -- Cray T3E & IBM SP (NERSC), Cray-X1 (ORNL), SX6 (Earth Simulator, Japan)

• Simulation of Burning Plasmas -- International Tokamak Experimental Reactor (ITER) • Integrated Fusion Simulation Project

(MFE)• Visualization -- turbulence evolution & particle

orbits

Gyrokinetic Approximation

• Gyromotion

• Polarization provides quasineutrality

[W. W. Lee, PF ‘83; JCP ‘87]

Earth Simulator 18% 10 (Ethier)

QuickTime™ and aCinepak decompressor

are needed to see this picture.

QuickTime™ and aVideo decompressor

are needed to see this picture.

Ion Temperature Gradient Driven Turbulence

Electrostatic PotentialParticle Trajectories

Data Management challenges• GTC is producing TBs of data

– Data rates: 80Mbs now, 1.6Gbs 5 years.– Need QOS to stream data.

• This data needs to be post-processed– Essential to parallelize the post-processing routines to handle

our larger datasets.– We need a cluster to post process this data.

• M (supercomputer processors) x N (cluster processors) problem.• QOS becomes more important to sustain this post-processing.

• The post-processed data needs to be shared among collaborators– Different sections of the post-processed data may go to different

users .– Post-processed data, along with other metadata should be

archived into a relational database.

Post processing of GTC Data.• Particle Data

– No compression possible.– Sent to 1 cluster for visualization/analysis.– Work being done with K. Ma, U.C. Davis: Visualize a million

particles.– Gain new insights into the theory.

• Field Data– Geometric/Temporal compression of the data is possible.– Data needs to be streamed to a local cluster at PPPL.– Reduced subset needs to be sent to PPPL + collaborators.

• Use Logistic Network. [Beck, UT-K]• Data transfer needs to be automatic, and integrated into a

dataflow/webflow for use with parallel analysis routines.

– We desire to see post-processed data during the simulation.

After the analysis

• Post-processed data needs to be saved into a relational database– How do we query this abstract data to

compare it with experiments?– 3D correlation functions– Processing of TBs of data/run now, 100’s of

TBs of data/run in 5 years.– Data mining techniques will be necessary to

understand this data.