Creating Profiling Tools to Analyze and

Optimize FiPy

Danya D. MuraliAdvisors: Jonathan E. Guyer and Daniel Wheeler

Materials Measurement Laboratory

Material Science and Engineering Division

Center for Theoretical and Computational Material Science

Outline

● FiPy Introduction

● How it works

● Examples

● Problems with FiPy

● Profiling Tools

● What are the they?

● How our tools work

● Results

● Conclusion

What is FiPy?

● An open source Python-based program that

uses the Finite Volume method to numerically

solve Partial Differential Equations (PDEs)

● Python has many powerful numerical libraries

● Designed for material scientists by material

scientists

What is a PDE?

Finite Volume Method

● Solve a general PDE on a given domain for a field

Finite Volume Method

● Solve a general PDE on a given domain for a field

● Integrate PDE over general control volumes

Finite Volume Method

● Solve a general PDE on a given domain for a field

● Integrate PDE over general control volumes

● Integrate PDE over polyhedral control volumes

Finite Volume Method

● Solve a general PDE on a given domain for a field

● Integrate PDE over general control volumes

● Integrate PDE over polyhedral control volumes

● Obtain a set of linear equations

How FiPy Works

import fipy as fp

L = 1.

N = 100

m = fp.Grid2D(Lx=L, Ly=L, nx=N, ny=N)

v = fp.CellVariable(mesh=m)

x, y = m.cellCenters

v[x > L / 2] = 1.

v.constrain(0., where=m.facesLeft |

m.facesRight)

v.constrain(1., where=m.facesTop |

m.facesBottom)

e = fp.TransientTerm() == fp.DiffusionTerm()

for i in range(10):

e.solve(v, dt=0.001)

fp.Viewer(v).plot()

How FiPy Works

import fipy as fp

L = 1.

N = 100

m = fp.Grid2D(Lx=L, Ly=L, nx=N, ny=N)

v = fp.CellVariable(mesh=m)

x, y = m.cellCenters

v[x > L / 2] = 1.

v.constrain(0., where=m.facesLeft |

m.facesRight)

v.constrain(1., where=m.facesTop |

m.facesBottom)

e = fp.TransientTerm() == fp.DiffusionTerm()

for i in range(10):

e.solve(v, dt=0.001)

fp.Viewer(v).plot()

Examples of FiPy: Polycrystal and Phase Field

courtesy S. A. David, ORNL

courtesy S. A. David, ORNL

Examples of FiPy: Polycrystal and Phase Field

courtesy S. A. David, ORNL

Examples of FiPy: Polycrystal and Phase Field

heatEq = (TransientTerm(var=dT)

== DiffusionTerm(coeff=Dt, var=dT)

+ TransientTerm(var=phase))

psi = theta + arctan2(phase.faceGrad[1], phase.faceGrad[0])

Phi = tan(N * psi / 2)

PhiSq = Phi**2

beta = (1. - PhiSq) / (1. + PhiSq)

DbetaDpsi = -N * 2 * Phi / (1 + PhiSq)

Ddia = (1.+ c * beta)

Doff = c * DbetaDpsi

D = alpha**2 * (1.+ c * beta) * (Ddia * (( 1, 0), ( 0, 1)) +

Doff * (( 0,-1), ( 1, 0)))

phaseEq = (TransientTerm(coeff=tau, var=phase)

== DiffusionTerm(coeff=D, var=phase)

+ ImplicitSourceTerm(coeff=(phase -

0.5 - kappa1 / pi * arctan(kappa2 * dT))

* (1 - phase)), var=phase)

eq = heatEq & phaseEq

courtesy S. A. David, ORNL

Examples of FiPy: Polycrystal and Phase Field

Examples Of FiPy: Extreme Fill

Examples Of FiPy: Extreme Fill

adsorptionCoeff = dt * suppressor * kPlus

thetaEq = fp.TransientTerm() ==

fp.ExplicitUpwindConvectionTerm(fp.SurfactantConvectionVariable(di

stance)) \

+ adsorptionCoeff * surface \

- fp.ImplicitSourceTerm(adsorptionCoeff *

distance._cellInterfaceFlag) \

- fp.ImplicitSourceTerm(kMinus * depositionRate * dt)

Examples Of FiPy: Extreme Fill

Problems with FiPy

● Potentially time

inefficient and excessive

in memory usage

● But how do we measure

that?

● Why do we even care?

● How do we find the

bottlenecks?

● Need profiling tools!

Profiling Tools

● What is profiling?

● Tool used to identify and quantify what resources

are being used by certain parts of a program

Profiling Tools

● What is profiling?

● Tool used to identify and quantify what resources

are being used by certain parts of a program

● Our profiler needs to:

Profiling Tools

● What is profiling?

● Tool used to identify and quantify what resources

are being used by certain parts of a program

● Our profiler needs to:

● Profile multiple functions at once

Profiling Tools

● What is profiling?

● Tool used to identify and quantify what resources

are being used by certain parts of a program

● Our profiler needs to:

● Profile multiple functions at once

● Cache profiling data for many simulations

Profiling Tools

● What is profiling?

● Tool used to identify and quantify what resources

are being used by certain parts of a program

● Our profiler needs to:

● Profile multiple functions at once

● Cache profiling data for many simulations

● Produce graphs of performance scaling against

system size

Speed Profilingclass FiPyProfileTime(FiPyProfile):

def __init__(self, profileFunc, ncells, regenerate=False):

...

def profile(self, ncell):

...

def get_time_for_function(self, function_key):

return stats[function_key]

def get_key_from_function_pointer(function_pointer):

return inspect.getfile(function_pointer)

def plot(self, keys, field="cumulative"):

...

Memory Profilingclass MemoryProfiler(object):

def __init__(self, profileMethod, runfunc):

...

def decorate(self, func):

def wrapper(*args, **kwargs):

...

return self.codeMap

def getLineMemory(self, line):

...

class MemoryViewer(object):

def generateData(self):

def worker(ncell, resultQ, profileMethod, runfunc, lines):

process = multiprocessing.Process(...)

Results: Profiling Polycrystal - Memory

Results: Profiling Polycrystal - Speed

Results: Profiling Extreme Fill - Memory

Results: Profiling Extreme Fill - Speed

Results: Profiling Different Solvers

So What?

● Identified that FiPy has memory issues that need

to be addressed

● Limits the size of simulations that we can run

● Located the classic memory for speed trade off

with Gmsh

● Determined that using inline was faster

● Identified that Trilinos is much slower than

Pysparse but has the option to run in parallel

● Next Step: Analyze algorithms to figure out how

to address these issues

So What?

● Identified that FiPy has memory issues that need

to be addressed

● Limits the size of simulations that we can run

● Located the classic memory for speed trade off

with Gmsh

● Determined that using inline was faster

● Identified that Trilinos is much slower than

Pysparse but has the option to run in parallel

● Next Step: Analyze algorithms to figure out how

to address these issues

So What?

● Identified that FiPy has memory issues that need

to be addressed

● Limits the size of simulations that we can run

● Located the classic memory for speed trade off

with Gmsh

● Determined that using inline was faster

● Identified that Trilinos is much slower than

Pysparse but has the option to run in parallel

● Next Step: Analyze algorithms to figure out how

to address these issues

So What?

● Identified that FiPy has memory issues that need

to be addressed

● Limits the size of simulations that we can run

● Located the classic memory for speed trade off

with Gmsh

● Determined that using inline was faster

● Identified that Trilinos is much slower than

Pysparse but has the option to run in parallel

● Next Step: Analyze algorithms to figure out how

to address these issues

So What?

● Identified that FiPy has memory issues that need

to be addressed

● Limits the size of simulations that we can run

● Located the classic memory for speed trade off

with Gmsh

● Determined that using inline was faster

● Identified that Trilinos is much slower than

Pysparse but has the option to run in parallel

● Next Step: Analyze algorithms to figure out how

to address these issues

Acknowledgements

● Mentors: Dr. Jon Guyer and Dr. Daniel Wheeler

● SHIP Student: Mira Holford

● SURF Program Staff and Peers

Questions?