![Page 1: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/1.jpg)
Large-Scale First-Principles Molecular Dynamics Simulations
François Gygi University of California, Davis
[email protected] http://eslab.ucdavis.edu
MASP2012 Workshop ISSP, Jul 11, 2012
1
![Page 2: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/2.jpg)
Acknowledgements
• Giulia Galli (Chemistry, UCDavis)
• Davide Donadio (MPI Mainz) • Zhaojun Bai (CS, UCDavis) • $$:
– NSF-OCI PetaApps 0749217 – DOE SciDAC
• cycles – NSF Teragrid (XSEDE) – DOE/ANL-ALCF – UC ShaRCS
• Ivan Duchemin (post-doc, Appl Sc) • Gary Yuan (grad st Appl. Sc) • Jun Wu (grad st, Appl. Sc) • Yuji Nakatsukasa (grad st, CS) • William Dawson (grad st, CS)
• Cui Zhang (grad st, Chemistry)
2
![Page 3: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/3.jpg)
Outline
• First-Principles Simulations • Hybrid density functionals • van der Waals density functionals • IR spectra in liquids • Verification and validation of
simulation results
3
![Page 4: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/4.jpg)
First-Principles Molecular Dynamics
Statistical Mechanics
Electronic Structure Theory
Quantum Chemistry
Density Functional Theory
…
Monte Carlo
Molecular Dynamics
…
FPMD
4
![Page 5: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/5.jpg)
First-Principles Molecular Dynamics
Density Functional Theory Molecular Dynamics
Kohn-Sham equations Newton equations
FPMD
R. Car and M. Parrinello (1985) 5
![Page 6: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/6.jpg)
Plane waves and pseudopotentials
• Plane wave basis for electronic wave functions
• Solutions represented by an (orthogonal) (m x n) matrix of complex Fourier coefficients
• No basis set superposition errors (BSSE) • Dimensions of Y: 106x104
• Pseudopotentials: only valence electrons represented 6
![Page 7: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/7.jpg)
Water in confined geometries
• Simulations of H2O confined in carbon nanotubes and between graphene sheets
• Study the effect of the interface on the water hydrogen bond network
G. Cicero, J.C. Grossman, E. Schwegler, F. Gygi, and G. Galli, JACS 130, 1871 (2008).
1.5 nm
7
![Page 8: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/8.jpg)
Hydrocarbons under pressure
• Investigation of the stability of hydrocarbons at deep Earth pressures and temperatures
• Study the possible creation of hydrocarbons under pressure
L. Spanu, D. Donadio, D. Hohl, E. Schwegler, and G. Galli, Proc. Nat. Acad. Sci. 108, 6843 (2011). 8
![Page 9: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/9.jpg)
Nanostructures in semiconductors
• Embedded silicon nanoparticle in silicon suboxide (SiOx)
• 1944 atoms, 9076 e-
• Investigation of relaxation of local defect structures, electronic structure, optical properties
Si1294O650
T. Li, F. Gygi and G. Galli, Phys. Rev. Lett. 107, 206805 (2011) 9
![Page 10: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/10.jpg)
FPMD simulations: goals and requirements • Goals
– simulate larger systems (3-5 nm)^3 – use more accurate models (hybrid DFT, vdW DFT) – include more features (spectroscopy: IR, Raman) – achieve O(N) scaling (with controlled accuracy)
• Needs – new scalable algorithms (strong scaling)
• ANL Intrepid, NICS Kraken, NSF Keeneland, TACC Ranger, NERSC Hopper
– flexible software infrastructure • HPC hardware is a moving target
10
![Page 11: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/11.jpg)
Qbox: our scalable implementation of FPMD
• C++/MPI/OpenMP Implementation of FPMD • Designed for large-scale parallelism • Built on
– ScaLAPACK/BLACS library – FFTW library – BLAS/ATLAS library – Apache Xerces-C library
• Client-server capability • Available under GPL license
http://eslab.ucdavis.edu/software/qbox
11
![Page 12: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/12.jpg)
Qbox code structure
Qbox
ScaLAPACK/PBLAS
BLACS
MPI
BLAS/MASSV
Apache XercesC XML lib
FFTW lib
D/ZGEMM lib
![Page 13: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/13.jpg)
The cost of FPMD simulations
• The current cost of FPMD is O(N3) for N atoms – compare with O(N log N) for classical MD – Many attempts to reduce to O(N) or O(N log N)
• Not feasible without approximations
• Origin of O(N3) – orthogonalization of solutions of the KS equations – non-local pseudopotentials
• Memory footprint: O(N2) – data location is critical
13
![Page 14: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/14.jpg)
Qbox communication patterns
• Distributed plane-wave coefficients
Data layout
![Page 15: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/15.jpg)
• Single k-point wavefunction: – ScaLAPACK matrix block distribution
• Dimensions of C: 106x104 • Typical process grid: 512x16
Qbox communication patterns Data layout
![Page 16: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/16.jpg)
MPI_Alltoallv
Qbox communication patterns
• 3-D Fourier transforms
Communication patterns
![Page 17: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/17.jpg)
MPI_Allreduce
Qbox communication patterns
• Accumulation of electronic charge density
Communication patterns
![Page 18: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/18.jpg)
PDGEMM/PZGEMM
Qbox communication patterns
• Other operations: (orthogonalization, non-local potential energy, Ritz diagonalization) – use the ScaLAPACK linear algebra library
PDSYRK
PDTRSM
Communication patterns
![Page 19: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/19.jpg)
Mapping tasks to physical nodes
• Mapping a 2-D process grid to a 3-D torus
?"
![Page 20: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/20.jpg)
Mapping tasks to physical nodes
• Mapping a 2-D process grid to a 3-D torus
![Page 21: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/21.jpg)
Mapping tasks to physical nodes
• Mapping a 2-D process grid to a 3-D torus
![Page 22: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/22.jpg)
History of FPMD performance
207.3 TFlops
22
Hasegawa et al 3.08 PFlops
The performance of FPMD codes has doubled every 8 months
![Page 23: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/23.jpg)
Designing scalable algorithms
• Reevaluate algorithms in terms of their communication patterns
• Example: Jacobi simultaneous diagonalization – involves two distinct communication patterns resulting in
communication bottlenecks beyond 512 cores
• Using a one-sided Jacobi implementation – arithmetic cost is increased – simple communication pattern (Eberlein, Park (1990)) – overall performance improved
23
![Page 24: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/24.jpg)
Limits of computational efficiency
• Time scales – 1 floating point operation (add/mult): 125 ps – fetch a double precision number from DRAM: 4 ns – fetch a double precision number from another node: >1 µs
• The only viable strategy is to overlap communication and computation – reuse data in processor cache – use asynchronous communication
• Try to achieve tcompute > tcommunicate
24
![Page 25: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/25.jpg)
Limits of computational efficiency
• Fundamental characteristics – F = Flop rate (e.g. AMD Opteron: 8 GFlop/s) – B = Bandwidth (e.g. Infiniband: 1 GB/s) – S = local data Size (e.g. matrix block size: 8 MB) – N(S) = number of operations (e.g. matrix multiply: N(S) = S3/2)
25
![Page 26: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/26.jpg)
Limits of computational efficiency
• Trends of new CPU architectures – Multicore processors: Flop rate (F) increases – Bandwidth (B) ~ constant
– Only possible for very large problems (large S) • strong scaling is likely unachievable
– N(S) must be superlinear
26
![Page 27: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/27.jpg)
Structure of DFT water
27
• PBE gOO(r) • 32-mol • 96 mol
• T=408 K
• A shift of ~100K approximately restores the main peak height
![Page 28: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/28.jpg)
Hybrid density functionals
• Conventional density functionals may not be sufficiently accurate to describe weak bonds or optical properties
• Hybrid density functionals include a fraction of the Hartree-Fock exchange energy
• The Hartree-Fock exchange energy involves N(N+1)/2 exchange integrals (for all e- pairs)
Ex = −12
ϕ i∗ r1( )ϕ i∗ r2( )ϕ j r1( )ϕ j r2( )
r1 − r2∫ dr1dr2
i, j
N
∑
• Cost: O(N3 log N) (with large prefactor)
28
![Page 29: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/29.jpg)
Scalability of Hartree-Fock calculations • The high cost of Hartree-Fock
exchange calculations can be mitigated using parallel processes
• Compute exchange integrals for all pairs of wave functions using cyclic permutations of data
• Overlap communication and computation (non-blocking MPI calls)
I. Duchemin, F. Gygi. Comput. Phys. Comm. 181, 855 (2010)
E. Bylaska et al. J. Comput. Chem. 32, 54 (2011)
2-d process grid
cyclic “laminar” data flow
29
![Page 30: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/30.jpg)
Scalability of Hartree-Fock calculations • Load-balancing of the computation of exchange
integrals
I. Duchemin, F. Gygi. Comput. Phys. Comm. 181, 855 (2010) 30
![Page 31: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/31.jpg)
Scalable Hartree-Fock and hybrid DFT implementation
• (H2O)64 PBE0 hybrid DFT
Spe
edup
I. Duchemin, F.Gygi. Comput. Phys. Comm. 181, 855 (2010) 31
![Page 32: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/32.jpg)
PBE0 water molecule & dimer
• Norm-conserving Hamann pseudopotentials, 85 - 200 Ry pw cutoff
• PBE-PBE0 comparison • vibrational frequencies
– dynamical matrix computed by finite differences of forces
32
C. Zhang, D. Donadio, F. Gygi, G. Galli, JCTC 7, 1443 (2011).
B. Santra, A. Michaelides, M. Scheffler, JCP 131, 124509 (2009).
Xu, X.; Goddard, W. A. J. Phys. Chem. A 108, 2305 (2004).
![Page 33: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/33.jpg)
PBE0 water molecule & dimer
• Note: Exact definitions of functionals vary
33
from: Xu, X.; Goddard, W. A. J. Phys. Chem. A 108, 2305 (2004).
![Page 34: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/34.jpg)
Water monomer (H2O)
34
aug-cc-pVTZ aug-cc-pVTZ(-f)
Ref 1: Xu et al, Ref 2: cccbdb.org, Ref3: Sit et al, Ref 4: Santra et al.
aug-cc-pV5Z aug-cc-pVTZ(-f)
![Page 35: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/35.jpg)
PBE0 water dimer
• The dimer binding energy is reduced when using PBE0 (compared to PBE)
• Ebind(PBE0)=5.12 kcal/mol • Ebind(PBE)=5.24 kcal/mol • Ebind(PBE)-Ebind(PBE0)=0.12 kcal/mol
– Santra et al: 0.15 kcal/mol – Xu et al: 0.13 kcal/mol
• Note: – BSSE corrections ~ 0.02 – 0.09 kcal/mol (Xu et al) – BSSE corrections ~ 0.33 – 0.36 kcal/mol (Todorova et al)
35
B. Santra, A. Michaelides, M. Scheffler, JCP 131, 124509 (2009).
Xu, X.; Goddard, W. A. J. Phys. Chem. A 108, 2305 (2004).
T. Todorova, et al, JPC B, 110, 3685 (2006)
![Page 36: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/36.jpg)
Water dimer
36 Ref 1: Xu et al, Ref 2: cccbdb.org, Ref3: Sit et al, Ref 4: Santra et al.
![Page 37: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/37.jpg)
Water dimer
37 Ref 1: Xu et al, Ref 2: cccbdb.org, Ref3: Sit et al, Ref 4: Santra et al.
![Page 38: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/38.jpg)
PBE0 liquid water (D2O)
• PBE0 liquid water simulations • 32 molecules, norm-conserving PP, 85 Ry • 17 ps simulations
38
T. Todorova, A.P. Seitsonen, J. Hutter, I.F.W. Kuo, C.J.Mundy, JPC B, 110, 3685 (2006).
M. Guidon, F. Schiffmann,J. Hutter, J. VandeVondele, JCP 128, 214104 (2008)
![Page 39: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/39.jpg)
Computation of Hartree-Fock exchange in extended systems • The long range of the Coulomb potential leads to an
integrable divergence in Brillouin zone integrals • Neglecting this singularity leads to significant errors in
computed band gaps
39
I. Duchemin, F. Gygi, Comp. Phys. Comm. 181, 855 (2010) 855–860
![Page 40: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/40.jpg)
Computation of Hartree-Fock exchange in extended systems • The divergent term can
be accounted for using a simple analytic regularization of the divergence
• Quadratic convergence of Brillouin zone sampling is achieved
40 I. Duchemin, F. Gygi, Comp. Phys. Comm. 181, 855 (2010) 855–860
Diamond, 8-atom cell
![Page 41: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/41.jpg)
PBE0 water (D2O)
PBE (439±29K)
PBE (367±25K)
PBE0 (438±29K)
PBE0 (374±27K)
Exp (300K)
C.Zhang, D.Donadio, F.Gygi, G.Galli, JCTC 7, 1443 (2011) 41
• PBE0 water is less structured than PBE water
![Page 42: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/42.jpg)
Computation of IR spectra in liquids • Total dipole in the condensed phase is not well
defined • Molecular dipole defined in terms of Maximally
Localized Wannier functions (MLWFs) – Wannier “centers” are the center of charge of each
MLWF
• IR absorption coefficient
42
![Page 43: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/43.jpg)
• Maximally Localized Wannier Functions (MLWF): linear combinations of electronic orbitals that minimize the spatial spread
• MLWFs can be used to compute the electronic polarization in crystals
• Computing MLWFs during a molecular dynamics simulation yields the infrared absorption spectrum
Maximally Localized Wannier Functions (MLWFs)
N. Marzari and D. Vanderbilt, Phys. Rev. B56, 12847 (1997)
R. Resta, Phys. Rev. Lett. 80, 1800 (1998) 43
![Page 44: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/44.jpg)
• Spread of a wavefunction associated with an operator Â
• Spread of a set of wavefunctions associated with an operator Â
Spread Functionals
44
![Page 45: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/45.jpg)
• The spread is not invariant under orthogonal transformations
• There exists a matrix X that minimizes the spread
Spread Functionals
45
![Page 46: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/46.jpg)
• Let
• Minimize the spread = maximize = diagonalize A
Spread Functionals
46
![Page 47: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/47.jpg)
• Case of multiple operators
• Minimize the spread = maximize = joint approximate diagonalization of the matrices A(k)
Spread Functionals
47
![Page 48: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/48.jpg)
• Example of multiple operators
• The matrices A(k) do not necessarily commute, even if the operators Â(k) do commute
Spread Functionals
48
![Page 49: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/49.jpg)
• In periodic systems
Calculation of MLWFs
A(1) = Cx ≡ cos2πLxx A(2) = Sx ≡ sin
2πLxx
A(3) = Cy ≡ cos2πLyy A(4) = S y ≡ sin
2πLyy
A(5) = Cz ≡ cos2πLzz A(6) = Sz ≡ sin
2πLzz
F.Gygi, J.L.Fattebert, E.Schwegler, Comput.. Phys. Comm. 155, 1 (2003) 49
![Page 50: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/50.jpg)
• The spread is minimized by simultaneous diagonalization of the matrices Cx,Sx,Cy,Sy,Cz,Sz
• Positions of the center of mass of the localized solutions (“Wannier centers”)
• Spreads
Calculation of MLWFs
F.Gygi, J.L.Fattebert, E.Schwegler, Comput.. Phys. Comm. 155, 1 (2003) 50
![Page 51: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/51.jpg)
• Jacobi algorithm for simultaneous diagonalization of symmetric matrices
Simultaneous diagonalization
A. Bunse-Gerstner, R. Byers, V. Mehrmann, SIAM J. Mat. Anal. Appl. 14, 927 (1993)
J.F.Cardoso and A. Souloumiac, SIAM J. Mat. Anal. Appl. 17, 161 (1996).
51
![Page 52: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/52.jpg)
Simultaneous diagonalization
J.F.Cardoso and A. Souloumiac, SIAM J. Mat. Anal. Appl. 17, 161 (1996). 52
![Page 53: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/53.jpg)
Simultaneous diagonalization
J.F.Cardoso and A. Souloumiac, SIAM J. Mat. Anal. Appl. 17, 161 (1996).
53
![Page 54: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/54.jpg)
Simultaneous diagonalization algorithm • The cost of simultaneous diagonalization is O(N3)
– a scalable algorithm is desirable
• Conventional implementations scale up to NCPU~N/2 • scalable algorithm based on the one-sided Jacobi
algorithm (P.J. Eberlein SIAM J. Alg. Disc. Meth. (1987))
54
![Page 55: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/55.jpg)
Conventional Parallel Jacobi algorithm
G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., (1996)
1 2 3 4
8 7 6 5
1 2 3 4
8 7 6 5
Each processor holds a pair of columns
Limited to n/2 processors for n columns 55
![Page 56: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/56.jpg)
One-sided Jacobi algorithms
P.J. Eberlein, H. Park, J. Parallel and Distrib. Comp. 8 (1990), 358–366.
Conventional Jacobi One-sided Jacobi
The one-sided Jacobi algorithm reduces the amount of global communication
56
![Page 57: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/57.jpg)
Scalability of simultaneous diagonalization
• One-sided Jacobi simultaneous diagonalization algorithm:
57
![Page 58: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/58.jpg)
Examples of MLWF calculations
• One-sided Jacobi simultaneous diagonalization algorithm included in Qbox
• Performance data: time to compute MLWFs starting from extended Kohn-Sham eigenstates (worst case) – 256 AMD 2.0 GHz cores
System Nel Ecut (Ry) MLWF time (s)
(HfSiO4)32 1216 70 38
(H2O)512 4096 72 440
58
![Page 59: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/59.jpg)
Water IR spectra (D2O)
59
• PBE IR spectrum from MLWFs at T=407 K
32 mol
96 mol
![Page 60: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/60.jpg)
Water IR spectra (D2O)
60 PBE0: 438 K (solid red), 374 K (solid blue)
PBE: 439 K (dash red), 367 K (dash blue).
![Page 61: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/61.jpg)
PBE0 water vs PBE water
• PBE0 yields a less structured liquid • PBE0 yields smaller dipole moments • PBE0 IR spectrum in better agreement with
experiment than PBE
61
![Page 62: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/62.jpg)
Van der Waals density functionals
• Dion-Rydberg-Schroeder-Langreth-Lundqvist (DRSLL) • Lee-Murray-Kong-Lundqvist-Langreth (LMKLL) • Klimes-Bowler-Michaelides (optB88) • Wang-Roman-Perez-Soler-Artacho-Fernandez-Serra
(DRSLLPBE)
62
![Page 63: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/63.jpg)
Dion et al. (2004) (DRSLL)
63
q = q0 (r)q ' = q0 ( ′r )
![Page 64: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/64.jpg)
Reduction of the cost to O(N) (Soler)
64 G. Román-Pérez and J. M. Soler, Phys. Rev. Lett. 103, 096102 (2009).
![Page 65: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/65.jpg)
Reduction of the cost to O(N) (Soler)
65 G. Román-Pérez and J. M. Soler, Phys. Rev. Lett. 103, 096102 (2009).
![Page 66: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/66.jpg)
vdW implementation
66 J. Wu, F. Gygi, JCP 136, 224107 (2012)
![Page 67: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/67.jpg)
vdW implementation
67
![Page 68: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/68.jpg)
vdW implementation
68
![Page 69: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/69.jpg)
vdW implementation
69
![Page 70: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/70.jpg)
vdW implementation
70
![Page 71: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/71.jpg)
vdW implementation
71
![Page 72: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/72.jpg)
vdW implementation
72
![Page 73: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/73.jpg)
Water-benzene complex
73
Ref 5: Lee et al. (2010)
Ref 21: J. Klimes (private comm.)
![Page 74: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/74.jpg)
Water-benzene complex
74
![Page 75: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/75.jpg)
Pbca benzene
75
J. Wu and F. Gygi, JCP 136, 224107 (2012)
E.J.Meijer and M. Sprik, JCP 105, 8684 (1996).
![Page 76: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/76.jpg)
Aspirin
• Crystal structure involves both hydrogen bonds and vdW interactions
• Monoclinic structure P21/c • New polymorph discovered
in 2005
76
C. Ouvrard and S. L. Price, Cryst. Growth Des. 4, 1119 (2004).
P. Vishweshwar, J. A. McMahon, M. Oliveira, M. L. Peterson, and M. J.
Zaworotko, J. Am. Chem. Soc. 127, 16802 (2005).
![Page 77: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/77.jpg)
Aspirin structure (P21/c)
77
![Page 78: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/78.jpg)
Aspirin internal angles
78
![Page 79: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/79.jpg)
vdW water dimer
79
Water dimer: Comparison of vdW functionals
CCSD(T) results by D. Lu (BNL)
![Page 80: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/80.jpg)
vdW liquid water
80
• vdW gOO(r) improved w.r.t. PBE • vdW effect differs from PBE0 effect
C.Zhang, J.Wu, G.Galli, F.Gygi, JCTC 7, 3054 (2011).
![Page 81: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/81.jpg)
DRSLLPBE: a modified vdW DF
81
• Wang et al (DRSLLPBE) • revPBE replaced by
PBE in DRSLL • Note: basis set
differences (SIESTA vs PW)
C.Zhang, J.Wu, G.Galli, F.Gygi, JCTC 7, 3054 (2011).
![Page 82: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/82.jpg)
vdW vibrational spectra
82 C.Zhang, J.Wu, G.Galli, F.Gygi, JCTC 7, 3054 (2011).
![Page 83: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/83.jpg)
vdW liquid water: dipole moment
83 C.Zhang, J.Wu, G.Galli, F.Gygi, JCTC 7, 3054 (2011).
![Page 84: Large-Scale First-Principles Molecular Dynamics Simulations€¦ · polarization in crystals • Computing MLWFs during a molecular dynamics simulation yields the infrared absorption](https://reader036.vdocuments.us/reader036/viewer/2022070811/5f09d1517e708231d428a15e/html5/thumbnails/84.jpg)
Summary
• Large-scale plane-wave pseudopotential FPMD simulations provide an accurate model of structure and electronic properties, with controlled accuracy
• Hybrid and vdW functionals improve the description of liquid water over PBE
• New software tools for verification and validation of simulations are needed
84