acemd: high-throughput molecular dynamics with nvidia kepler gpus
DESCRIPTION
Acellera Founder Gianni De Fabritiis, and CTO Matt Harvey talk about the latest developments of high-throughput molecular dynamics both in terms of applications and methodological advances. Examples are in the context of ACEMD, a highly efficient, best-in-class graphical processing units (GPUs) centric code for running MD simulations, and its protocols. In particular, attendees will learn how the high arithmetic performance and intrinsic parallelism of the latest NVIDIA Kepler GPUs can offer a technological edge for molecular dynamics simulations. Try GPUs for free via: www.Nvidia.com/GPUTestDriveTRANSCRIPT
www.acellera.com
ACEMD: HIGH-THROUGHPUT MOLECULAR DYNAMICS WITH NVIDIA KEPLER GPUS
www.acellera.com
- M. Harvey, G. Giupponi and G. De Fabritiis, ACEMD: Accelerating biomolecular dynamics in the microsecond time scale, J. Chem. Theory and Comput. 5, 1632 (2009).
- M. J. Harvey and G. De Fabritiis, An implementation of the smooth particle-mesh Ewald (PME) method on GPU hardware, J. Chem. Theory Comput., 5, 2371–2377 (2009).
www.acellera.com
Paradigms of molecular dynamics
High performance!• A single or few simulations
run for very long • Reached simulations time
of several milliseconds • Best systems: Anton,
Desmond • A bit easier to analyze
High-throughput!• Very many runs of
reasonable length (hundreds of ns)
• Reached simulations time of several milliseconds
• Best systems: GPUs clusters, GPUGRID.net, Folding@home
• Complex analysis
www.acellera.com
Binding processes
Direct! Intermediate!
A+B AB A+B AB* AB
www.acellera.com
SH2-pYEII
• T. Giorgino, I. Buch and G. De Fabritiis, J. Chem. Theory Comput.,8, 1171–1175 (2012).
www.acellera.com
FAAH-AEA
• E. Dainese, G. De Fabritiis et al. submitted (2013).
www.acellera.com
IN-SILICO LIGAND BINDING Trypsin-Benzamidine I. Buch, T. Giorgino and G. De Fabritiis,Complete reconstruction of an enzyme-inhibitor binding process by molecular dynamics simulations, PNAS 108, 10184-10189 (2011).
www.acellera.com
35,000 500
100 50
atoms
trajectories
µs of data
Beta-Trypsin/Benzamidine (3PTB) ACEMD software AMBER99SB ff. Explicit solvent
Free ligand binding simulations!
ns/each
www.acellera.com
Calculating kinetics of binding Assuming first order kinetics
Singhal N et al. J Chem Phys (2004) Guilliain F and Thusius D. J Am Chem Soc (1970)
www.acellera.com
Characteristic transition modes
www.acellera.com
a) Poses of Benzamidine on Trypsin detected through high pressure x-ray crystallography b) and c) The native binding pose of Benzamidine on Trypsin d) Benzamidine poses labeled from X0-X8 on the front and back side of Trypsin
Trypsin-benzamidine from X-ray
www.acellera.com
FBDD ON FACTOR XA With Noelia Ferruz Capapey (Universitat Pompeu Fabra) Matt Harvey (ACELLERA), Jordi Mestres (IMIM)
www.acellera.com
[1] Lee Fielding , Dan Fletcher , Samantha Rutherford , Jasmit Kaurand, Jordi Mestres. Exploring the active site of human factor Xa protein by NMR screening of small molecule probes. Royal Society of Chemistry 2003.
S1
S4
www.acellera.com
Library of compounds 34 compounds screened by STD-NMR 9 ligands bound to factor Xa 4 ligands selected for further studies
www.acellera.com
Three derived by known inhibitors
www.acellera.com
Experimental competition assays with TPAM
• Binding sites positions were hypothesized from similarity to known ligands:
• Ligands 29, 31 at site S1, • Ligands 10, 27 at site S4. • Displacement for ligands
10 and 29, but only partially to 27 and no displacement for 31.
Ligand Predicted Pose Displacement Further comments
31 S4 No Highest affinity but not
displacement
29 S1 Yes -
27 S4/core Yes Only partially displaced
10 S4 Yes Bad fit in experimental
curves
www.acellera.com
METHODS • 34 ligands built, simulated by
MD and analyzed b means of Markov State Models
• Protein structure from human Factor Xa (2BOK[2]) was established as the initial protein conformation
• Each box contained only one randomly placed ligand giving a final concentration of 0.0038M
• 1000 replicas of 50 ns were run for each system for an aggregate of 1.8 ms simulation data.
www.acellera.com
Ligand Kd (µM) Residence Time (ns.) ∆G (kcal/mol) kon(s-1·M) koff (s-1)
29 126.9 ± 56.9 74786 ± 15914 -5.35 ± 0.2 (1.17 ± 0.07) ·108 (1.46 ± 0.63) ·104
15 363.1 ± 12.6 113445 ± 3637 -4.69 ± 0.0 (2.43 ± 0.00) ·107 (8.82 ± 0.23) ·103
16 1089.9± 659.1 87491 ± 12875 -4.10 ± 0.2 (1.23 ± 2.81) ·107 (1.19 ± 0.34) ·104
31 1543.5 ± 432.3 11165 ± 2623 -3.86 ± 0.2 (6.33 ± 1.09) ·107 (9.38 ± 1.83) ·104
27 1736.7 ± 119.3 78895 ± 3783 -3.76 ± 0.0 (7.33 ± 0.40) ·106 (1.27 ± 0.07) ·104
10 2047.3 ± 24.1 52773 ± 50 -3.67 ± 0.0 (9.26 ± 0.11) ·106 (1.90 ± 0.00) ·104
3 2056.7 ± 699.5 19753 ± 26094 -3.72 ± 0.3 (3.68 ± 0.35) ·107 (7.49 ± 2.09) ·104
13 3045.3 ± 1375.8 48332 ± 57242 -3.51 ± 0.3 (9.38 ± 7.20) ·107 (3.80 ± 3.04) ·105
23 5404.1 ± 2531.3 9673 ± 21216 -3.34 ± 0.8 (1.35 ± 4.27) ·108 (7.23 ± 3.88) ·105
12 9199.3 ± 1026.4 1959 ± 313 -2.78 ± 0.1 (5.65 ± 0.18) ·107 (5.20 ± 0.64) ·105
21 12126 ± 10080 162406 ± 117707 -2.61 ± 1.4 (1.21 ± 1.89) ·107 (9.43 ± 4.53) ·103
11 27013 ± 2511 340 ± 7 -2.14± 0.1 (1.10 ± 0.11) ·107 (2.94 ± 0.06) ·106
28 28500 ± 1844 1092 ± 67 -2.11 ± 0.1 (3.23 ± 0.03) ·107 (9.19 ± 0.54) ·105
7 29433 ± 1331 294 ± 11 -2.09 ± 0.1 (1.16 ± 0.02) ·108 (3.40 ± 0.13) ·106
20 34626 ± 5960 1625 ± 274 -2.00 ± 0.0 (1.83 ± 0.01) ·107 (6.33 ± 1.15) ·105
8 37980 ± 1505 261 ± 3 -1.94 ± 0.0 (1.01 ± 0.05) ·108 (3.82 ± 0.06) ·106
32 42040 ± 11404 159 ± 3 -1.89 ± 0.1 (1.56 ± 0.27) ·108 (6.28 ± 0.15) ·106
26 60353 ± 11699 478 ± 89 -1.67 ± 0.1 (3.59 ± 0.09) ·107 (2.16 ± 0.38) ·106
2 92553 ± 27598 1697 ± 923 -1.44 ± 0.2 (1.52 ± 1.79) ·107 (1.78 ± 2.52) ·106
33 102180 ± 93464 1142 ± 365 -1.48 ± 0.3 (1.21 ± 0.07) ·107 (1.30 ± 1.35) ·106
25 123000 ± 5773 544 ± 3 -1.24 ± 0.0 (1.50 ± 0.06) ·107 (1.84 ± 0.01) ·106
6 136133 ± 4224 165 ± 6 -1.18 ± 0.0 (4.44 ± 0.11) ·107 (6.04 ± 0.22) ·106
5 137467 ± 4224 540 ± 5 -1.17 ± 0.0 (1.35 ± 0.03) ·107 (1.85 ± 0.02) ·106
1 157133 ± 3930 627 ± 0 -1.10 ± 0.0 (1.01 ± 0.00) ·107 (1.59 ± 0.00 ) ·106
19 157333 ± 805 673 ± 28 -1.10 ± 0.0 (9.44 ± 0.02) ·106 (1.49 ± 0.06) ·106
4 210333 ± 6609 169 ± 24 -0.92 ± 0.1 (2.84 ± 0.06) ·107 (5.98 ± 0.70) ·106
17 238333 ± 23561 376 ± 21 -0.88 ± 0.2 (1.21 ± 0.26) ·107 (2.67 ± 0.15) ·106
18 270400 ± 93196 353± 8 -0.77 ± 0.0 (1.05 ± 0.00) ·107 (2.83 ± 0.07) ·106
30 345600 ± 8073 217 ± 0 -0.63 ± 0.0 (1.33 ± 0.00 ) ·107 (4.60 ± 0.01) ·106
22 464667 ± 1143 605 ± 0 -0.45 ± 0.0 (3.56 ± 0.03) ·106 (1.65 ± 0.00) ·106
24 539400 ± 3960 144 ± 3 -0.37 ± 0.0 (1.28 ± 0.02) ·107 (6.90 ± 0.18) ·106
14 658867 ± 105204 90 ± 1 -0.26 ± 0.1 (1.73 ± 0.32) ·107 (1.10 ± 0.02) ·107
9 887000 ± 22518 140 ± 2 -0.07 ± 0.0 (8.03 ± 0.32) ·106 (7.12 ± 0.12) ·106
34 1172000 ± 33704 99 ± 2 0.09 ± 0.0 (8.59 ± 0.11) ·106 (1.00 ± 0.02) ·107
Kinetics and thermodynamics
www.acellera.com
www.acellera.com
LIGAND 27
Experimental results!
• Weakly displaced by TPAM. • Hypothesized to bind at S4 pocket
by similarity with Berlex compound
Computational results!
• Binds at the core part of the cavity and entrance of S1
www.acellera.com
www.acellera.com
LIGAND 10
Experimental results!
• Clearly displaced by TPAM. • Expected to bind at pocket S4 by
similarity with Rhone-Poulec Rorer compound.
Computational results!
• Binds at pocket S4 • Sixth in ranking by KD
www.acellera.com
LIGAND 29
Experimental results!
• Clearly displaced by TPAM. • Hypothesized to bind at S1 pocket
by similarity with DuPont compound
Computational results!
• Binds at pocket S1
www.acellera.com
LIGAND 31
Experimental results!
• KD = 30µM • Failure to be displaced by TPAM. • Expected to bind at S1 pocket
becouse of the known high affinity of the S1 pocket for the amidine fragment
Computational results!
• Binds beneath the loop between S1 & S4
www.acellera.com
Ensemble view with TPAM
www.acellera.com
METHODS
From hardware to software
www.acellera.com 27
“Molecular simulation will mature within the next 5 years to allow simulations at temporal scales of biological interest, thus achieving its full potential for biological discovery”
www.acellera.com
ACEMD - History • Developed from CellMD (2006), 19 times a CPU
• First CUDA GPUs released 2006
• ACEMD released 2007
• First fully-GPU accelerated MD application
• First GPU implementation of Particle Mesh Ewald doi:10.1021/ct900275y
• Presented in ACEMD: Accelerating Biomolecular Dynamics in the microsecond time scale, JCTC 2009 doi:10.1021/ct9000685
www.acellera.com
ACEMD - Capabilities ACEMD has all the features required for production simulations of biomolecules: • Major force fields: CHARMM, Amber, OPLS and Martini • Common file formats: PDB, Bincoor, PRMTOP, PSF, DCD, XTC
• PME or GRF electrostatics • NVE, NVT, NPT ensembles
– Langevin thermostat – Berendsen barostat
• Constraints, restraints • Powerful scripting and extension capability • Multi-host execution for replica-exchange methods • Binary distribution – no compilation necessary
www.acellera.com
ACEMD - Resources • User manual • Extensions developer manual • Protocols manuals • Support forum for everybody • [email protected] for paying users • Develop for you to allow to interface your methods
as plugins (almost always free) • Acecloud – acemd cloud • Metrocubo – acemd special patented hardware
www.acellera.com
ACEMD - Performance
• Benchmarking conditions: DHFR model (23558 atoms) NVT cutoff 9A, PME enabled (frequency 2), dt=4fs. Langevin thermostat • System: 4 GPUs, X79 chipset, CUDA 4.2, driver 310.44, CentOS 6
0 50 100 150 200 250
Tesla M2090, GTX 580
Tesla K20, GTX680
GTX780
Titan OC
DHFR
ns/day
Exceptional single GPU performance Parallel scaling up to 1.4x on 3 GPUs (single host) System sizes up to ~1M atoms Performance scales ~linearly with system size and GPU speed Does not need a GPU with fast double-precision arithmetic Does not need a fast CPU; performance normally dependent on GPU
www.acellera.com
ACEMD – Free basic download
• Optimal if you do little use of MD • Have only single GPU machines in your lab • Fully functional version of ACEMD on a single
GPU • Ideal for small groups or to start on MD • http://www.acellera.com/acemd
www.acellera.com
ACEMD - Extensions
• ACEMD can be extended by the user • Plugins – separate binary library
• Pros: – Written in C or C++ – Fast for numerically intensive work – More advanced features than TCL
interface – Have multiple plugins active
simultaneously • Cons:
– Written in C or C++ • Suitable for:
– Writing complex, intensive plugins – Interfacing with existing, third-party code
• TCL – coded directly in input file
• Pros: – Fast to develop – Very familiar for NAMD users
• Cons: – Slow for numerically intensive work – Not all features exposed – Only one TCL extension at a time
• Suitable for: – Applying point restraints – Modification of simulation parameters
(eg temperature annealing)
www.acellera.com
ACEMD – Extensions • Simple event-based programming model • Clean separation between ACEMD and extension
– Much easier and safer to develop for than direct source-code modification
• Documented API with examples
• Events: – Initialise called once at the beginning of the simulation – Calcforces called every iteration during force evaluation – Endstep called at the end of every iteration – Terminate called once at the end of the simulation
www.acellera.com
TCL Extension Example • Set thermostat parameters
• Enable tclforces • Set annealing parameters
• Frequency of calling extension
• Calcforces – called every* iteration – Calculate new target temperature
– Apply new target temperature
– Disable extension when target temperature reached
www.acellera.com
Plugin Example (1) • Include API definition
• Set default values
• Initialisation function: – Parse vlaues from arguments
passed in the input file
• Calcforces called every* iteration
– Apply new target temperature
www.acellera.com
Plugin Example (2) Compile:
Configure in input file:
www.acellera.com
Metrocubo • 4 GPU workstation designed for ACEMD • Compact, quiet chassis • E3 Xeon CPU • Operating System and ACEMD installed • Best price/performance for MD
Patent pending
www.acellera.com
ACEMD Test Drive
• One week of access to a 4-GPU Metrocubo
• Test ACEMD with your current models
• Expert support for system setup and testing
• http://www.acellera.com/products/metrocubo/metrocubo-test-drive/
www.acellera.com
ACECloud
• Run ACEMD easily on Cloud resources – No need to deal with queuing systems – All files copies transparently
• Simple command-line interface – optimised for managing large numbers of
simulations – Supports many users
• Test drive now: [email protected]
Patent pending
www.acellera.com
ACECloud Run a simulation on the cloud:
Patent pending
See the progress of all simulations:
www.acellera.com
In-silico binding assays @ Acellera
• We performed the calculations on 30 ligands in 45 days (1600 GPU days using acecloud)
• Determined 4 strong fragments by residence time and another small group as intermediates, the others discarded
• Poses available for follow-up • Pathway of binding • Currently performing NMR on top molecules
www.acellera.com
http://htmdworkshop.wordpress.com
GPU Accelerated Apps Momentum Key codes are GPU Accelerated!
" Abalone – GPU only code " ACEMD – GPU only code " AMBER " CHARMM " DL_POLY " GROMACS " HOOMD-Blue – GPU only code " LAMMPS " NAMD
Molecular Dynamics Quantum Chemistry
" ABINIT " BigDFT " CP2K " GAMESS " Gaussian – in development " NWChem " Quantum Espresso " TeraChem – GPU only code " VASP
Check many more apps at www.nvidia.com/teslaapps
Sign up for FREE GPU Test Drive on remotely hosted clusters www.nvidia.com/GPUTestDrive
Run ACEMD on Tesla K20 GPU today
Test Drive K20 GPUs! Experience The Acceleration
Test Drive K20 GPUs! Experience The Acceleration
Questions? Contact us
" Devang Sachdev - NVIDIA
" [email protected] " @DevangSachdev
" Acellera ltd " [email protected]
Stream other webinars from GTC Express: http://www.gputechconf.com/page/gtc-express-webinar.html
Run ACEMD on Tesla K20 GPU today
Sign up for FREE GPU Test Drive on remotely hosted clusters www.nvidia.com/GPUTestDrive
Upcoming GTC Express Webinars
Register at www.gputechconf.com/gtcexpress
July 30 - Getting Started with GPU-accelerated Computer Vision using OpenCV and CUDA
July 31 - NMath Premium: GPU-accelerated Math Libraries for .NET
August 7 - Accelerating High Performance Computing with GPUDirect RDMA
GTC 2014 Call for Submissions
Looking for submissions in the fields of
§ Science and research
§ Professional graphics
§ Mobile computing
§ Automotive applications
§ Game development
§ Cloud computing
Submit at www.gputechconf.com