Derek C. Richardson (U Maryland)Derek C. Richardson (U Maryland)

PKDGRAVPKDGRAV: A Parallel : A Parallel kk-D Tree -D Tree Gravity Solver for Gravity Solver for NN-Body Problems-Body Problems

FMM 2004FMM 2004

OverviewOverview

High-performance cosmology code.High-performance cosmology code.Written at U Washington Written at U Washington NN-body shop by -body shop by

Joachim Stadel (U Zurich) & Tom Quinn.Joachim Stadel (U Zurich) & Tom Quinn.Now adapted for a variety of applications, Now adapted for a variety of applications,

including planet formation.including planet formation.Lastest version has implementation of SPH Lastest version has implementation of SPH

(smoothed particle hydrodynamics).(smoothed particle hydrodynamics).

Sample MovieSample Movie

Spatial Binary TreeSpatial Binary Tree

Ironically, Ironically, PKDGRAVPKDGRAV no longer uses a no longer uses a kk-D -D tree for gravity…tree for gravity…

The problem: elongated cells can give large The problem: elongated cells can give large multipole moments and other pathologies.multipole moments and other pathologies.

The solution: spatially bisect cells and The solution: spatially bisect cells and squeeze the cell boundaries.squeeze the cell boundaries.

Also good for neighbour searching.Also good for neighbour searching.

Spatial Binary TreeSpatial Binary Tree

k-D Tree k-D with Squeeze

Tree WalkingTree Walking

Construct particle-particle and particle-cell Construct particle-particle and particle-cell interaction lists from top down.interaction lists from top down.

Define opening ball (based on Define opening ball (based on ) to test for ) to test for ball-bucket intersection.ball-bucket intersection.– If bucket outside ball, apply multipole (c-list).If bucket outside ball, apply multipole (c-list).– Otherwise open cell and test its children, etc., Otherwise open cell and test its children, etc.,

until leaves reached (which go on p-list).until leaves reached (which go on p-list).Nearby cells have similar lists: amortize.Nearby cells have similar lists: amortize.

Tree WalkingTree Walking

Note multipole Q acceptable to all particles in cell d.Note multipole Q acceptable to all particles in cell d.

Other IssuesOther Issues

Multipole expansion order.Multipole expansion order.– Use hexadecapole (best bang for buck).Use hexadecapole (best bang for buck).

Force softening.Force softening.– Use spline-softened gravity kernel.Use spline-softened gravity kernel.

Periodic boundary conditions.Periodic boundary conditions.– Use Ewald summation technique.Use Ewald summation technique.

Time steps.Time steps.– Use hierarchical steps (adaptive leapfrog).Use hierarchical steps (adaptive leapfrog).

Parallel ImplementationParallel Implementation

1.1. Master layer (serial).Master layer (serial).– Controls overall flow of program.Controls overall flow of program.

2.2. Processor Set Tree (PST) layer (parallel).Processor Set Tree (PST) layer (parallel).– Assigns tasks to processors.Assigns tasks to processors.

3.3. Parallel Parallel kk-D (PKD) layer (serial).-D (PKD) layer (serial).– MIMD execution of tasks.MIMD execution of tasks.

4.4. Machine-dependent Layer (MDL, separate).Machine-dependent Layer (MDL, separate).– Interface to parallel primitives.Interface to parallel primitives.

Domain DecompositionDomain Decomposition

Binary tree balanced by work factors. Nodes construct local trees.Binary tree balanced by work factors. Nodes construct local trees.

PST

Scaling at Fixed AccuracyScaling at Fixed Accuracy

Clustered Clustered cosmology cosmology simulationsimulation

((NN = 3 = 3··101066))

(( = 0.8) = 0.8)

Some Science: PlanetesimalsSome Science: Planetesimals

One of the recent additions to One of the recent additions to PKDGRAVPKDGRAV is is the ability to treat particle collisions.the ability to treat particle collisions.

Applications:Applications:– Planet formation (planetesimal growth).Planet formation (planetesimal growth).– Asteroid disruption.Asteroid disruption.– Planetary rings.Planetary rings.

Technique: predict collision during leapfrog Technique: predict collision during leapfrog drift step using fast neighbour search.drift step using fast neighbour search.

Planetesimal EvolutionPlanetesimal Evolution

Evolutionary models based on laboratory Evolutionary models based on laboratory experiments may be inadequateexperiments may be inadequate ..– Outcomes determined by gravitational reaccumulation.Outcomes determined by gravitational reaccumulation.

Leinhardt Leinhardt et alet al. 2000. 2000

Incorporating Collision ModelsIncorporating Collision Models

Based on individual Based on individual test cases, collision test cases, collision outcomes can be outcomes can be incorporated into incorporated into large-scale sims of large-scale sims of planet formation.planet formation.

Near-Earth Asteroid BinariesNear-Earth Asteroid Binaries

Reaccumulation Reaccumulation following tidal following tidal disruption could disruption could form NEA binaries.form NEA binaries.

Need reaccumulated Need reaccumulated (or shattered/fragile) (or shattered/fragile) body to start with…body to start with…

New Code: Rigid AggregatesNew Code: Rigid Aggregates

Recently the ability to treat collections of Recently the ability to treat collections of particles as rigid aggregates has been added.particles as rigid aggregates has been added.

Applications include realistic asteroid shape Applications include realistic asteroid shape models and more complex granular shapes.models and more complex granular shapes.

Technique: solve Euler rigid body equations Technique: solve Euler rigid body equations of motion with torque and allow for non-of motion with torque and allow for non-central impacts.central impacts.

Rigid Aggs: Movie 1Rigid Aggs: Movie 1

Rigid Aggs: Movie 2Rigid Aggs: Movie 2

Bouncing AllowedBouncing Allowed No BouncingNo Bouncing

Rigid Aggs: Movie 3Rigid Aggs: Movie 3

Rigid Aggs: Movie 4Rigid Aggs: Movie 4

Future WorkFuture Work

PKDGRAVPKDGRAV continues to be improved. continues to be improved.Algorithms currently being considered:Algorithms currently being considered:

– Local expansions.Local expansions.– Tree repair.Tree repair.– Improved collision detection.Improved collision detection.

Other possibilities include combining Other possibilities include combining PKDGRAVPKDGRAV with a Grape board!… with a Grape board!…