Part 3: Application Interfaces

Bořek Patzák, Vít Šmilauer, Martin Horák

Czech Technical University, Faculty of Civil Engineering, Prague

1

Content of Part 3• Model/Application interface overview• How to implement API• Workflow templates• Example: Thermo-mechanical problem

2

Service Description__init__ (self, file) Constructor. Initializes the application.

ARGS:• file (str): path to application initialization file.

getField(self, fieldID, time) Returns the requested field at given time. Field is identified by fieldID.ARGS:• fieldID (FieldID): identifier• time (double): target timeReturns: Returns requested field (Field).

setField(self, field) Registers the given (remote) field in application.ARGS:• field (Field): remote field to be registered by the applicationReturns: None

getProperty(self, propID, time, objectID=0)

Returns property identified by its ID evaluated at given time.ARGS:• propID (PropertyID): property ID• time (double): time when property to be evaluated• objectID (int): identifies object/submesh on which property is evaluated

(optional)Returns: Returns representation of requested property (Property).

setProperty(self, property, objectID=0) Register given property in the applicationARGS:• property (Property): the property class• objectID (int): identifies object/submesh on which property is evaluated

(optional)Returns: None

Overview of Application Interface

Service DescriptionsolveStep(self, tstep, stageID=0, runInBackground=False)

Solves the problem for a given time step. Evaluates the solution from actual state to given time. ARGS:• tstep(TimeStep): solution step• stageID(int): optional argument identifying solution stage• runInBackground(bool): if set to True, the solution will run in background (in

separate thread), if supported.

Returns: None

finishStep(self, tstep) Called after a global convergence within a time step.ARGS:• tstep(TimeStep): solution stepReturns: None

getCriticalTimeStep(self) Returns the actual (related to the current state) critical time step increment (double).Returns: critical time step (double)

getAssemblyTime(self, tstep) Returns the assembly time related to a given time step. The registered fields (inputs) should be evaluated in this time.ARGS:• tstep (TimeStep): solution stepReturns: assembly time (double)

terminate(self) Terminates the application.Returns: None

getAPIVersion(self) Returns the supported API version.Returns: API version (int)

Overview of Application Interface

Native API implementation • Direct calls to application's

function• Python interface (python code,

wrapper-Swig, Boost)

Indirect API implementation• Wrapper around I/O streams or

files, more common

MuPIF MuPIF

Application 1

API

Application.lib

Application 1

API

Application.exe

Application interface implementation

Direct Interface Implementation

● Communicate directly with the application (in memory).

● Interface implemented using direct communication with the application

● Requires python interface

– Existing python interface (LAMMPS)

– Calling application functions/procedures (linked as library) from python. Manual data conversion required.

– Python interface to existing code can be generated using wrapping tools (swig, boost python, f2py, etc.). Requires access to application source code.

Calling C/Fortran from Python#include <Python.h>

// Function to be called from Pythonstatic PyObject* py_myFunction(PyObject* self, PyObject* args){ char *s = "Hello from C!"; return Py_BuildValue("s", s);}

// Another function to be called from Pythonstatic PyObject* py_myOtherFunction(PyObject* self, PyObject* args){ double x, y; PyArg_ParseTuple(args, "dd", &x, &y); return Py_BuildValue("d", x*y);}

// Bind Python function names to our C functionsstatic PyMethodDef myModule_methods[] = { {"myFunction", py_myFunction, METH_VARARGS}, {"myOtherFunction", py_myOtherFunction, METH_VARARGS}, {NULL, NULL}};

// Python calls this to let us initialize our modulevoid initmyModule(){ (void) Py_InitModule("myModule", myModule_methods);}

Calling C/Fortran from Python

from myModule import *

print "Result from myFunction:", myFunction()print "Result from myOtherFunction(4.0, 5.0):", myOtherFunction(4.0, 5.0)

The output is:

Result from myFunction(): Hello from C!Result from myOtherFunction(4.0, 5.0): 20.0

● Compile myModule as shared library

● Using myModule from Python

● If you are going to make bigger libraries available from Python we strongly suggest you check out SWIG, f2py or Boost Python.

# Compile myModule.c as a shared library.> gcc -shared -I/usr/include/python2.7/ -fPIC -o myModule.so myModule.c

SWIG Wrapping example

double My_variable = 3.0; int fact(int n) { if (n <= 1) return 1; else return n*fact(n-1);} double dotProduct (double a[3], double b[3]) { double ans = 0.0; for (i=0; i<3; i++) ans+=a[i]*b[i]; return ans;}

/* example.i */%module example%include "carrays.i"%array_functions (double, doubleArray);%{ /* Put header files here or function declarations like below */ extern double My_variable; extern int fact(int n); extern double dotProduct(double a[3], double b[3]);%} extern double My_variable; extern int fact(int n); extern double dotProduct(double a[3], double b[3]);

C file with example

Swig interface file

SWIG Wrapping example

> swig -python example.icc -c example.c example_wrap.c -fPIC -I/usr/include/python2.7ld -shared example.o example_wrap.o -o _example.so

Build a Python module (linux example, win platform similar)

import _example as example

print "result of fact(5) is ", example.fact(5);

a=example.new_doubleArray(3)for i in range(3):

example.doubleArray_setitem(a,i,i)print "a.a is ", example.dotProduct(a,a)

Use compiled module from Python

result of fact(5) is 120a.a is 5.0

Output from Python

Indirect Interface Implementation

● Do not communicate directly with the application, but rather parse its input/output.

● Requires only external code– Should restart solution from stored state– Should be able to perform a single time step MuPIF

Application Interface

EXE

Typical procedure (for each time step):

● Collect and remember all external inputs set up using set methods ● Produce updated input file, based on external inputs● Restart the application and solve one step, store the application state

and produce output● Parse output file and respond to the get methods

Python

Indirect Implementation Example

Set Remember

Solve Produce input file execute

Get Parse output

Method Typical action

For an example, see examples/Example03 in MuPIF installation

Example: simple LAMMPS API● LAMMPS is a molecular dynamics code, and an acronym for Large-scale Atomic/Molecular Massively

Parallel Simulator● LAMMPS can work together with Python in different ways. In this example, we utilize the capability of

LAMMPS, where Python can wrap LAMMPS through the LAMMPS library interface, so that a Python script can create one or more instances of LAMMPS and launch one or more simulations. In Python lingo, this is “extending” Python with LAMMPS.

● Alternatively, there are PyLammps and IPyLammps wrapper classes in Python. These wrappers try to simplify the usage of LAMMPS in Python by providing an object-based interface to common LAMMPS functionality. They reduces the amount of code necessary to parameterize LAMMPS scripts through Python and makes variables and computes directly accessible.

● This example shows how to run an atomistic simulation of uniaxial compressive loading of an aluminum single crystal – simulation cell with fcc atoms with <100> orientations in the x, y, and z-directions. For this example, the simulation

cell size is 10 lattice units in each direction for 4,000 total atoms

– The potential used in the Mishin et al. (1999) aluminum potential

– The equilibration step allowing the lattice to expand to a temperature of 300 K with a pressure of 0 bar at each simulation cell boundary.

– Then, the simulation cell is deformed in the x-direction at a strain rate of 1010 s -1, while the lateral boundaries are controlled using the NPT equations of motion to zero pressure.

Tutorial taken from Mark Tschopp, https://icme.hpc.msstate.edu/mediawiki/index.php/Uniaxial_Compression

LAMMPS input

####################################### DEFORMATIONreset_timestep 0

fix 1 all npt temp 300 300 1 y 0 0 1 z 0 0 1 drag 1variable srate equal 1.0e10variable srate1 equal "v_srate / 1.0e12"fix 2 all deform 1 x erate ${srate1} units box remap x

# Output strain and stress info to file# for units metal, pressure is in [bars] = 100 [kPa] = 1/10000 [GPa]# p2, p3, p4 are in GPavariable strain equal "(lx - v_L0)/v_L0"variable p1 equal "v_strain"variable p2 equal "-pxx/10000"variable p3 equal "-pyy/10000"variable p4 equal "-pzz/10000"

#fix def1 all print 100 "${p1} ${p2} ${p3} ${p4}" file Al_SC_100.def1.txt screen no#run 20000

####################################### SIMULATION DONE#print "All done"

# Input file for uniaxial tensile loading of single crystal aluminum# Mark Tschopp, November 2010

# ------------------------ INITIALIZATION ----------------------------units metaldimension 3boundary p p patom_style atomicvariable latparam equal 4.05

# ----------------------- ATOM DEFINITION ----------------------------lattice fcc ${latparam}region whole block 0 10 0 10 0 10create_box 1 wholelattice fcc ${latparam} orient x 1 0 0 orient y 0 1 0 orient z 0 0 1create_atoms 1 region whole

# ------------------------ FORCE FIELDS ------------------------------pair_style eam/alloypair_coeff* * /home/bp/apps/lammps-17Nov16/potentials/Al99.eam.alloy Al

# ------------------------- SETTINGS ---------------------------------compute csym all centro/atom fcccompute peratom all pe/atom

####################################### EQUILIBRATIONreset_timestep 0timestep 0.001velocity all create 300 12345 mom yes rot nofix 1 all npt temp 300 300 1 iso 0 0 1 drag 1

# Set thermo outputthermo 1000thermo_style custom step lx ly lz press pxx pyy pzz pe temp

# Run for at least 10 picosecond (assuming 1 fs timestep)run 20000unfix 1

# Store final cell length for strain calculationsvariable tmp equal "lx"variable L0 equal ${tmp}print "Initial Length, L0: ${L0}"

LAMMPS model can be set up● via input file● from the Python interface (PyLammps recommended)

Example: LAMMPS API● The model described in lammps input file

● The stress-strain curve (for monotonic loading) computed once when first needed and recorded

● The stress for given strain then evaluated from stored history.

from mupif import *from lammps import lammps

class LAMMPS_MUPIF(Application.Application): """ Implementation of LAMMPS MuPIF API. .. automethod:: __init__ """ def __init__ (self, file, workdir=''): """ Constructor. Initializes the application.

:param str file: Name of file :param str workdir: Optional parameter for working directory """ super(LAMMPS_MUPIF, self).__init__(file, workdir) self.lmp=lammps() self.eps=[] self.sig=[] self.lastStrain = 0.0 self.init = False

def setProperty(self, property, objectID=0): if (property.getPropertyID() == PropertyID.PID_effective_strain): # remember the mapped value self.lastStrain = property.getValue() else: raise APIError.APIError ('Unknown property ID')

def getProperty(self, propID, time, objectID=0): if (propID == PropertyID.PID_effective_stress): help = next(obj for obj in enumerate(self.eps) if obj[1]>=self.lastStrain) return mupif.Property.Property(self.sig[help[0]], propID, ValueType.Scalar, time, None) else: raise APIError.APIError ('Unknown property ID') def solveStep(self, tstep, stageID=0, runInBackground=False): # run input file if not done before if not self.init: self.lmp.file(self.file)

self.eps=[] self.sig=[] for i in range(200): self.lmp.command('run 100') e=self.lmp.extract_variable("p1", None, 0) s=self.lmp.extract_variable("p2", None, 0) self.eps.append(e) self.sig.append(s)

self.init=True

import lammpsmupif as lmimport mupif as mupifapi=lm.LAMMPS_MUPIF('al2.lammps')

epsVal = [x*0.01 for x in range(18)]for eps in epsVal: prop = mupif.Property.Property(eps, mupif.PropertyID.PID_effective_strain, mupif.ValueType.Scalar, 0.0, None) api.setProperty(prop) api.solveStep(None) sig=api.getProperty(mupif.PropertyID.PID_effective_stress, 0.0).getValue()

print eps, sig

print "Demo done"

LAMMPS API

Stress-strain curve for uniaxial compressive loading of single crystal aluminum in the <100> loading direction.

Movie showing compressive deformation of single crystal of 108,000 aluminum atoms loaded in the <100> direction at a strain rate of 1010 s-1 and a temperature of 300 K.

Tutorial taken from Mark Tschopp, https://icme.hpc.msstate.edu/mediawiki/index.php/Uniaxial_Compression

Workflow templatesSequential workflow

time = 0timeStepNumber = 0targetTime = 10

while (abs(time-targetTime) > 1.e-6): dt=min(m1.getCriticalTimeStep(), m2.getCriticalStep(), m3.getCriticalStep()) time = time+dt if (time>targetTime): time=targetTime

timeStepNumber = timeStepNumber+1 istep=TimeStep.TimeStep(time, td, timeStepNumber) try: m1.solveStep(istep) m1.finishStep(istep); p = m1.getProperty() m2.setProperty(p) m2.solveStep(istep) ... m3.solveStep(istep) m3.finishStep(istep) except APIError.APIError as e: print ("Appi Error occured:",e) break

m1.terminate()m2.terminate()m3.terminate()

Geometry, mesh, conductivity

M 1

M 3

M 2

Tim

e lo

op

Workflow templatesloosely coupled workflow

time = 0timeStepNumber = 0targetTime = 10

while (abs(time-targetTime) > 1.e-6): dt=min(m1.getCriticalTimeStep(), m2.getCriticalStep(), m3.getCriticalStep()) time = time+dt if (time>targetTime): time=targetTime timeStepNumber = timeStepNumber+1 istep=TimeStep.TimeStep(time, td, timestep)

try: convergedFlag = False while not covergedFlag: m1.solveStep(istep) m2.setProperty(m1.getProperty(id)) m2.solveStep(istep) m1.setProperty(m2.getProperty(id2))

#check for convergence convergedFlag=checkConvergence()

m1.finishStep(istep) m2.finishStep(istep) m3.solveStep() m3.finishStep(istep) except APIError.APIError as e: print ("Appi Error occured:",e) break

m1.terminate()m2.terminate()m3.terminate()

Geometry, mesh, conductivity

M 1

M 3

M 2

Geometry, mesh, conductivity

Tim

e lo

op

Cou

plin

g lo

op

Example: Coupled Thermo-Mechanical problemconnecting applications

Demo04.pyDemo04.py

19

Thermal problem

Mechanical problem

Geometry, mesh, conductivity Continuum model for heat conduction Temperature

Geometry, mesh, conductivityContinuum model for

thermo-elasticity Displacement

Stress field

Interactive session: connecting applications

import mupifimport demoapp

# initialize thermal modelapp = demoapp.thermal('inputT.in','.')print(app.getApplicationSignature())

# solve thermal problemsol = app.solveStep(TimeStep.TimeStep(0,1)) f = app.getField(FieldID.FID_Temperature, 0.0)# vtk output of temperature fielddata = f.field2VTKData().tofile('example2')

# initialize mechanical modelapp2 = demoapp.mechanical('inputM.in', '.')print(app2.getApplicationSignature())# register external temperature fieldapp2.setField(f)# solve mechanical problemsol = app2.solveStep(TimeStep.TimeStep(0,1)) # get displacement fieldf = app2.getField(FieldID.FID_Displacement, 0.0)data = f.field2VTKData().tofile('example3')

Demo04.pyDemo04.py

20

# Model edges# ----------3----------# | |# 4 2# | | # ----------1---------#5 1 #Domain size x and y10 5 #Number of elements in x and y1 C 10. 1. #Domain edge number 1 with boundary condition (D-Dirichlet, C-convection, other-None), prescribed temperature/ambient temperature2 D 0.3 D 0.4 D 0.

InputT.inInputT.in

# Model edges# ----------3----------# | |# 4 2# | | # ----------1---------#5 1 # Domain size x and y20 8 # Number of elements in x and y1 N2 N 3 N4 D # Dirichlet condition on edge 4

InputM.inInputM.in

TODO

● Implement MuPIF representation of missing components, algorithms, and tools → Template for simulation chains

● Add support for component metadata (e-CUBA - common universal basic attributes, e-CUDS common universal data structure) – cooperation with FORCE project

● How to generate metadata? Defined by workflow/models, some generated automatically

● Integration with Granta data storage, serialization support for each component. Need to identify data (case id, workflow id, job id, …). Need an authority generating these unique IDs.

● Develop interfaces for individual models● Develop user case simulation workflows, MODA overview

Simulation chain templates

Purpose: ● Description of simulation workflow (MODA)

● Identification of components (models and data), their description

● Identification of services provided by components (Interface requirements)