!1

Speaker: J. Hickman*

LAMMPS Workshop and Symposium

August 13-15, 2019

* National Institute of Standards and Technology (NIST) Materials measurement laboratory (MML)

Supported by NRC postdoctoral fellowship

Development of physically informed neural network (PINN) interatomic potentials

G. Purja Pun**, F. Tavazza*, Y. Mishin**

** George Mason University: Department of Physics

!2

Motivation

Ab-initio/DFT Classical (MD/MC)

• Slow • Size limited (N~ ) • 0K or short timescales • Very accurate

Better potential models

image source: http://evolution.skf.com/us/bearing-research-going-to-the-atomic-scale

Speed Accuracy Scalability

Transferability

Compromises

Classical MD/MC

102

i

• Computationally Fast • Larger systems (N~ ) • Kinetic phenomena/long simulations • Accuracy depends on approximation of

the potential energy surface (PES)

107

!3

Interatomic potential model typesTraditional interatomic potential:

“Mathematical” or “straight” NN potentials:

Physically informed neural network (PINN):

Pros Cons

• Fast • Decent extrapolation • Physically inspired

• Fast relative to DFT • DFT level accuracy

(~1-5 mEv) within training set

• Relatively straight forward/routine to train/fit

• Systematic improvement (add more data)

• Slower than traditional potentials

• Bad extrapolation

• Difficult to train/fit • Hard to improve

upon once finalized

• Accuracy limitations

• Slower than traditional potentials

• Same as straight NN • Decent

extrapolation • Physically inspired

• LJ, EAM, ADP, Tersoff, REAX … etc

• Machine learning potentials • Gaussian process regression • Interpolating moving least squares

• Kernel ridge regression • Compressed sensing • ANN potentials

local parameterization Physically informed artificial neural networks for atomistic modeling of materials GPP Pun, R Batra, R Ramprasad, Y Mishin - Nature communications, 2019

!4

Artificial neural networks

sigmoid

transfer functions:

Artificial neural network

p2⋮

input layerhidden layers output

layer

⋮ ⋮ ⋮⋮

G1

G2

GN

p1

pM

w11

w12

w13

b1

b2

wN1 bk

inpu

t vec

tor

outp

ut v

ecto

r 60 16 16 8

The weights and bias are the NN’s fitting parameters (~1500 parameters)

data:

test

validationtraining

Activation function

Activation function

Artificial neural networks

!5

training data

Neural network prediction

-sin(x)

Training region unable to extrapolate

excellent interpolation

Train NN to reproduce provided data

Toy example:

!6

Physically informed neural network potential (PINN)

eV

!7

Training/test set generation

!8

Traditional BOP potential cutoff function

bond order parameterpromotional

energy

Ep = − σo (Σj≠iSijbij fc(rij))12

Model can be used for Metallic

or covalent systems

Physically informed artificial neural networks for atomistic modeling of materials GPP Pun, R Batra, R Ramprasad, Y Mishin - Nature communications, 2019

Ei =12 (Σj≠ieAo−αorij − SijbijeBo−βorij) fc(rij) + Ep

bij = (1 + zij)− 12

Sijk = 1 − fc(rik + rjk − rij)e−λo(rik+rjk−rij) screening

zij = aoΣk≠i,jSik(cos(θijk + ho)2 fc(rik)Sij = ∏

k≠i,j

Sijk

(Ao, Bo, αo, βo, ao, ho, λo, σo)Fit a baseline traditional potential via 8 adjustable

step-0:

i

j

krik

rij✓ijk

E1E2

EN

⋮

Structures DFT energies Model energiesE1E2

EN

⋮

structure-1structure-2

structure-N⋮

“Training set”

RMSE = ( Σs(Es − Es)2

N )12

Model errorTraining process:pytorch

!9

Local structure parameters

Gm,ni = ∑

j,k

Pm(cos(θijk))f(rij)f(rik)

f(r) =1rn

e− (r − rn)2

σ2 fc(r)

i

i

j

k

rik

rij✓ijk

Structure parameters:

Gi’s act as “fingerprints” of the local atomic

environment

Angular term:Pm(cos(θ)) m = 0,1,2,4,6 (Legendre polynomials)

Radial term: fc(r) = Cutoff function

r1 r2rr12. . .rmin rc

choose 12 values for rn

8 rn with 5Pm

for each atom

concentric shells

Gm,ni = [40 × 1]

Physically informed artificial neural networks for atomistic modeling of materials GPP Pun, R Batra, R Ramprasad, Y Mishin - Nature communications, 2019

!10

PINN potential model

i

(r1, r2 . . . rn)

Energy of atom-i neighboring atom

positions

Local structure

parameter calculation

G1 = G1(rij, θikj)G2 = G2(rij, θikj)

(G1, G2 . . . GM)

structural “fingerprints”

Structure’s energy

BOP Model Ei

atomic energy

A(x) =1

e−x + 1−

12

δA

δα

δλ

perturbations to baseline parameters

RMSE = ( Σs(Es − Es)2

N )12

= NN(w, b)

(Ao, Bo, αo, βo, ao, ho, λo, σo)

(Ai, Bi, αi, βi, ai, hi, λi, σi) atom-i parameterization

=

baseline parameters+

NN perturbation(δA, δB, δα, δβ, δa, δh, δλ, δσ)

Training:

Find the NN weights and bias’s which minimize the RMSE

!11

Results

!12

Volume-energy curves

FCC

SC

BC8

diamond

wurtzite

diamond

thermally perturbed structures

ZOOMSingle layer silicene:Tetrameter

Bulk phases:Combined

NN: 60x32x8

!13

Reproduction of DFT energy landscape

Distribution of training error Distribution of validation error

(untrained)

y=x Equality with DFT

Training data Validation data

RMSE~3.5 meV

RMSE~ 4 meV

!14

Si PINN comparison with traditional potentialPINN:

RMSE (meV/atom)~ 3.5PINN

700X and 300X improvement respectively

zoom inPINN

SW

MOD

y=x Equality with DFT

±200 meV

Modified Tersoff

RMSE (meV/atom)

12812591Stillinger-Weber (1985)

Traditional potentials:

Pun, GP Purja Physical Review B 95.22 (2017): 224103.

!15

unphysical extrapolation!!

Excellent interpolation

physical extrapolation

physical extrapolation

zoom in

Training region

Diamond

ANN vs PINN Extrapolation

unphysical extrapolation!!

!16

ANN vs PINN Extrapolation

BCC (Si)

Simple Cubic (Si)Wurtzite (Si)

Simple Cubic (Al)

Physically informed artificial neural networks for atomistic modeling of materials GPP Pun, R Batra, R Ramprasad, Y Mishin - Nature communications, 2019

!17

Computational efficient

Paragrand MC:https://software.nasa.gov/software/LAR-18773-1Source: Vesselin Yamakov (NASA)

100

2.5

50

NOTE: results sensitive to hyper parameter choice

4500

273250

PINN vs EAM

NN vs EAM serial: 120X slower 40 CPU: ~30X slower GPU: ~15X slower

serial: 166X slower 40 CPU: ~40X slower GPU: ~20X slower

!18

Thermal properties (preliminary)

Paragrand MC:https://software.nasa.gov/software/LAR-18773-1

PINN-A PINN-B

Thermal expansion

Liquid RDF

PINN-A

Traditional potentials

MODC Pun, GP Purja Physical Review B 95.22 (2017): 224103.

syst

emat

ic im

prov

emen

t

!19

Future work•Get PINN_BOP working in LAMMPS •Hyper-parameter tune (speed/accuracy) •Explore other Chemical systems: •Ge, Pt, Cu

•Binary: SiGe, SiAl, …. etc •Possibly explore other traditional potentials formats •(PINN_EAM, PINN_ADP, etc )

•Applications •study thermal properties of 2D structures

•Si ,Ge, SiGe

!20

Conclusions•Developed a new silicon interatomic potential using the new PINN potential format •Even in preliminary stage we are obtaining excellent agreement with the DFT energies •Current potential reproduces DFT data around ~500x better than current traditional potentials •Better transferability than ANN potentials •Investigating methodological considerations to streamlining the fitting procedure for faster future development

!21

Acknowledgements

•Ganga Purja Pun •Vesselin Yamakov •Francesca Tavazza •Yuri Mishin •Adam Robinson •GMU,NRC,NIST

• https://www.tf.uni-kiel.de/matwis/amat/iss/kap_5/backbone/r5_3_3.html • https://www.wordstream.com/blog/ws/2017/07/28/machine-learning-applications • https://www.researchgate.net/figure/268158499_fig1_Figure-1-Phase-diagram-of-SiGe-alloys-showing-separation-of-the-solidus-and-liquidus • http://evolution.skf.com/us/bearing-research-going-to-the-atomic-scale • /https://www.chegg.com/homework-help/questions-and-answers/consider-concentric-metal-sphere-spherical-shells-shown-figure--innermost-solid-sphere-rad-

q4808250

Image references:

• Stillinger, Frank H., and Thomas A. Weber. "Computer simulation of local order in condensed phases of silicon." Physical review B 31.8 (1985): 5262. • (“In review”) G. P. Purja Pun(1), R. Batra(2), R. Ramprasad (3) and Y. Mishin(1): (1) George Mason Univ., (2) Univ. Connecticut, (3) Georgia Tech • Pun, GP Purja, and Y. Mishin. "Optimized interatomic potential for silicon and its application to thermal stability of silicene." Physical Review B 95.22 (2017): 224103.

References: