acemd2: high-performance molecular dynamics on gpus · 2011. 11. 22. · namd. nvidia fermi amd...

ACEMD2: High-performance molecular dynamics on GPUs

Gianni De FabritiisComputational Biochemistry and Biophysics Lab

Research Unit on Biomedical Informatics

• Introduction to MD

• ACEMD

• Parallel MD

• Computation of free energies of

binding

• Characterisation of binding pathways

•

Methods Ongoing work

• Introduction

Methods Ongoing work Prospective work

Force field

Molecular Dynamics (MD)

“Molecular simulation will mature within the next 5 years to allow simulations at temporal scales of biological interest, thus achieving its full potential for biological discovery”

“Molecular simulation will mature within the next 5 years to allow simulations at temporal scales of biological interest, thus achieving its full potential for biological discovery”

Why in 5 years?NAMD2.7b performance for

DHFR, dihydrofolate reductase, solvated in water, 23558 atoms

Hardware: X5570 CPUs 2.93 GHz and IB dual rail DDR.

20 ns/day -> 8.6 ms/step for NAMD

NVIDIA Fermi AMD Evergreen Intel Xeon

Compute Units 15 20 6

CE/CU (warp size) 32 (32) 16 (64) 1 or SSE vec width*

CE arch GF100: In-order, scalarGF104: OoO scalar

5-way VLIW OoO scalar+ explicit vectororOoO scalar*

Clock 1.4GHz 0.85GHz 2.9GHz

Peak GFLOPS sp (dp) 1344 (672) 2720 (544) 140 (70) (No FFA)

Peak Memory BW GB/s

177 155 32

Current AMD OpenCL for CPU maps a CPU core as a CE, so explicit vector code required.Forthcoming Intel OpenCL will use horizontal vectorisation: each

SSE lane mapped as a separate CE. OoO/VLIW means (high) ILP needed for optimal use of all but

CPU vs CELL vs GPU

CUDA/OpenCL programming model based on SIMT

• ACEMD

Methods Ongoing work Prospective work

ACEMDACEMD

Publications on ACEMDM. J. Harvey and G. De Fabritiis, An

implementation of the smooth particle-mesh Ewald (PME) method on GPU hardware, J. Chem. Theory

Comput., 5, 2371–2377 (2009).

M. Harvey, G. Giupponi and G. De Fabritiis, ACEMD: Accelerated molecular dynamics

simulations in the microseconds timescale, J.

• J. Selent, F. Sanz, M. Pastor and G. De Fabritiis, Induced Effects of Sodium Ions on Dopaminergic G-Protein Coupled Receptors, PLOS computational biology, 6, e1000884 (2010)

• I. Buch, M. J. Harvey, T. Giorgino, D. P. Anderson and G. De Fabritiis, High-throughput all-atom molecular dynamics simulations using distributed computing, J. Chem. Inf. and Mod. 50, 397 (2010).

• K. Sadiq and G. De Fabritiis,Explicit solvent dynamics and energetics of HIV-1 protease flap-opening and closing, Proteins 78, 2873 (2010)

• Benedict M. Sattelle, Steen U. Hansen, John Gardiner and Andrew Almond, Free Energy Landscapes of Iduronic Acid and Related Monosaccharides, J. Am. Chem. Soc., Article ASAPDOI: 10.1021/ja1054143

• Jeremy Fidelak, Jarek Juraszek, Davide Branduardi, Marc Bianciotto and Francesco Luigi Gervasio, Free-Energy-Based Methods for Binding Profile Determination in a Congeneric Series of CDK2 Inhibitors, The Journal of Physical Chemistry B2010 114 (29), 9516-9524

Evergreen GPUs

ILP. Evergreen CE is a 5-issue VLIW unit. Needs a code with plenty of ILP to fill it up. Warp is 64. Compiler optimisation. Inspection of the assembly emitted by the compiler (R800 ISA) reveals inefficient code generation.Compiler immaturity. SDK Version 2.2 is the first of 3 public releases to correctly compile and run the ACEMD code.Implementation immaturity. Global memory accesses perform poorly, high kernel launch overhead (>120us cf <10us with CUDA), no runtime profiling/performance counters available (c.f. CUDA_PROFILE)

G200/Fermi GPUs

ILP. Much less that ATI required but look at GF104. Nice warp at 32.G200/Fermi. G200 seemed to perform relatively better than Fermi for MD. We need more threads/MP. Registers are limited and very differently used by Fermi. Compiler optimisation. Quite good already. OpenCL is quite immature in terms of performance.Compiler maturity. Nvcc is by now quite stable. OpenCL is improving.

8 GPU system

• High throughput molecular dynamics

Methods Ongoing work Prospective work

Description ofbinding pathways

Prediction ofbinding sites

Prediction ofbinding modes

Calculation ofbinding affinities

& kinetics

Understandingmechanisms of

allosteric modulation

Hot topics ofprotein-ligand modelling

µsper day

System Atoms Data*

SH2 domain/ligand 39,000 905 µs

HIV-1 protease 56,000 230 µs

GPCR D2 receptor 60,000 69 µs

hERG K+ channel 44,000 31 µs

ß-Trypsin/ligand 35,000 30 µs

• Computation of free energies of

binding

Methods Ongoing work Prospective work

OBJECTIVE

Development of a protocol to reliablycompute accurate binding affinities of protein-ligand complexes.

High-throughput all-atom molecular dynamics simulations using distributed computing, I. Buch, M. J. Harvey, T. Giorgino, D. P. Anderson and G. De Fabritiis.J Chem Inf Model 50, 397 (2010)

METHODOLOGYWe use a one-dimensional potential of mean force protocol reconstructed from umbrella sampling simulations with weighted histogram analysis method.

3. Potential of mean force reconstruction*

2. Umbrella sampling

4. Standard free energy of binding**

1. Generation of initial configurations

p56lck SH2 domain/phosphotyrosine-Glu-Glu-Ile (pYEEI)

PDB 1LKK

CHARMM27 ff.Explicit solvent0.15 M [NaCl]temp. 298 K38,655 atoms

Schematic representation

• Characterisation of binding pathways

Methods Ongoing work Prospective work

OBJECTIVEFull description of binding events. Extraction of quantitative information (thermodynamics and kinetics) as well as qualitative about the binding pathway.

METHODOLOGY

Execution of thousands of long MD trajectories of free ligands in diffusion and building a Markov State Model

−20 −10 0 10 20 30−20

−10

0

10

20

30

z [A]

x[A

]

6.5

6

5.5

54.54

3.5

3

2.5

2

1

0 kcal/mol

−1

−2

−3

−4

−5

−6

beta-Trypsin/Benzamidine

Crystal structure PDB 3PTBAMBER99SB ff.

SH2 domain/pYEEI

PDB 1LKKCHARMM27 ff.

docked ligandcrystal ligand

Comparison of RMSD

unbound ligand

SH2 domain/pYEEI

De-ionization of charged groups

Reduction in orientational freedom

Electrostatic interactions

Reduction in conformational freedom

Dewettening of surfaces

Hydrophobic collapse

• Understanding allosteric modulation

Methods Ongoing work Prospective work

OBJECTIVE

Study of the effects of Na+ mediatedallosteric modulation upon ligand binding to D2 Dopaminergic receptors.

GPCRs are subject to allosteric modulation

Dopaminergic receptors modulation by Na+

GPCRs activated by orthosteric ligand binding

Na+ modulation effect to agonist/antagonist binding

Activation mechanism upon agonist binding

D2 Dopaminergic receptor, 1.1 usNa+ ionsMembrane and water not shown for simplicity

Sodium binding to GPCR

38

Research teamIgnasi BuchS. Kashif Sadiq, Toni Giorgino, Matt Harvey (Imperial College)Gianni De Fabritiis, (PI)

The GPUGRID volunteers

Funding