SeversAIST Cluster (50 CPU)
Titech Cluster (200 CPU)KISTI Cluster (25 CPU)
Climate Simulation on ApGrid/TeraGrid at SC2003
Client(AIST)
Ninf-G
SeversNCSA Cluster (225 CPU)
National Institute of Advanced Industrial Science and Technology
Example- Hybrid QM/MD Simulation -
QM/MD simulation over the Pacific at SC2004
QM Server
QM Server
MD Client
TCS (512 CPU) @ PSCTotal number of CPUs: 1792
Ninf-G
Close-up view
corrosion of Sillicon under stress
P32 (512 CPU)
P32 (512 CPU)
F32 (256 CPU)
1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 10
•Total number of CPUs: 1793•Total Simulation Time: 10 hour 20 min•# steps: 10 (= 7fs)•Average time / step: 1 hour•Size of generated files / step: 4.5GB
(some of) Lessons Learned
Practically impossible to occupy a Practically impossible to occupy a large-scale single system for few large-scale single system for few weeks.weeks.
How can we long-run the simulation?
Faults (e.g. HDD crush, network down) Faults (e.g. HDD crush, network down) cannot be avoided.cannot be avoided.
We don’t prefer manual restart. The simulation should be capable of automatic recovery from faults.How can the simulation recover from faults?
Objectives
Develop flexible, robust, and efficient Grid-enabled sDevelop flexible, robust, and efficient Grid-enabled simulation.imulation.
Flexible -- allow dynamic resource allocation/migration,robust -- detect errors and recover from faults automatically for long runs, andefficient -- manage thousands of CPUs.
Verify our strategy through large-scale experiments.Verify our strategy through large-scale experiments.Implemented Grid-enabled SIMOX (Separation by Implanted Oxygen) simulationRun the simulation on Japan-US Grid testbed for few weeks.
Hybrid QM/CL Simulation (1)
Enabling large scale simulation with Enabling large scale simulation with quantum accuracyquantum accuracy
Combining classical MD Simulation with quantum simulation
CL simulationSimulating the behavior of atoms in the entire regionBased on the classical MD using an empirical inter-atomic potential
QM simulationModifying energy calculated by MD simulation only in the interesting regionsBased on the density functional theory (DFT)
MD Simulation
QM simulationbased on DFT
simulation algorithmsimulation algorithm
Each QM computation isEach QM computation isindependent with each othercompute intensiveusually implemented as a MPI program
Hybrid QM/CL Simulation (2)
MD part QM part
initial set-up
Calculate MD forces of QM+MD regions
Update atomic positions and velocities
Calculate QM force of the QM region
Data of QM atoms
QM forces
Calculate QM force of the QM regionCalculate QM force of the QM region
Calculate MD forces of QM region
National Institute of Advanced Industrial Science and Technology
Implementation of Grid-enabled Simulation
- multi-scale QM/MD simulation using GridRPC and MPI -
Approach to “gridify” applications
Grid RPC enhances the flexibility and robustness by;dynamic allocation of server programs, anddetection of network/cluster trouble.MPI enhances the efficiency by; highly parallel computing on a cluster for both client and server programs.
The new programming approach, combining GridRPC with MPI, takes advantages of both programming models complementarily to run large-scale applications on the Grid for a long time. Client
Server
GridRPC
MPI
MD
MD
MD
MD
MPI
QM
QM
QM
QM
MPI
QM
QM
QM
QM