particle advection defined & usage
DESCRIPTION
Particle Advection Defined & Usage. Particle advection is a foundational visualization algorithm. Advecting particles creates integral curves. Particle advection is a foundational visualization algorithm. Courtesy C. Garth, Kaiserslautern. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/1.jpg)
Particle Advection Defined & Usage
![Page 2: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/2.jpg)
Particle advection is a foundational visualization algorithm
• Advecting particles creates integral curves
![Page 3: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/3.jpg)
Particle advection is a foundational visualization algorithm
Courtesy C. Garth, Kaiserslautern
![Page 4: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/4.jpg)
Particle advection is a foundational visualization algorithm
Streamlines in the “fish tank”
![Page 5: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/5.jpg)
Image credit:A. Sanderson, Univ. Utah
Particle advection is a foundational visualization algorithmPoincaré analysis
of a tokamak
![Page 6: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/6.jpg)
Particle advection is a foundational visualization algorithm
Stream surface
around a tokamak
![Page 7: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/7.jpg)
Lagrangian Methods
Visualize manifolds of maximal stretching in a flow, as indicated by dense particles
Finite-Time Lyapunov Exponent (FTLE)
Courtesy Garth
![Page 8: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/8.jpg)
www.vacet.org
Lagrangian Methods Visualize manifolds of maximal
stretching in a flow, as indicated by dense particles
Forward in time: FTLE+ indicates divergence
Backward in time: FTLE- indicates convergence
Courtesy Garth
![Page 9: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/9.jpg)
Particle Advection In A Parallel Setting
![Page 10: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/10.jpg)
Parallel Particle Advection: How To Load Balance?
N particles (P1, P2, … Pn), M Processing Elements/PEs (PE1, …, PEm) (Processing Element = instance of your program)
Each particle takes a variable number of steps, S1, S2, … Sn
Total number of steps is ΣSi We cannot do less work than this (ΣSi)
Goal: Distribute the ΣSi steps over M PEs such that problem finishes in minimal time
![Page 11: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/11.jpg)
Goal: Distribute the ΣSi steps over M PEs such that problem finishes in minimal time
Sounds sort of like a bin-packing problem, but… particles can move from PE to PE (i.e. work
can move from bin to bin) path of particle is data dependent and
unknown a priori (i.e. we don’t know Si beforehand)
big data significantly complicates this picture…. … data may not be readily available, introducing
starvation
Parallel Particle Advection: How To Load Balance?
![Page 12: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/12.jpg)
Advecting particles
Decomposition of large data set into blocks on filesystem
?
What is the right strategy for getting particle and data together?
![Page 13: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/13.jpg)
Strategy: load blocks necessary for advection
Decomposition of large data set into blocks on filesystem
Go to filesystem and read block
![Page 14: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/14.jpg)
Decomposition of large data set into blocks on filesystem
Strategy: load blocks necessary for advection
![Page 15: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/15.jpg)
“Parallelize over Particles” Basic idea: particles are partitioned over
PEs, blocks of data are loaded as needed. Positives:
Indifferent to data size: a serial program can process data of any size
Trivial parallelization (partition particles over processors)
Negative: Redundant I/O (both across PEs and within a
PE) is a significant problem.
![Page 16: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/16.jpg)
“Parallelize over data” strategy:parallelize over blocks and pass particles
PE1 PE2
PE4PE3
![Page 17: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/17.jpg)
“Parallelize over Data” Basic idea: data is partitioned over PEs,
particles are communicated as needed. Positives:
Only load data one time Great for in situ processing
Negative: Starvation!
![Page 18: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/18.jpg)
Contracts are needed to enable these different processing techniques
Parallelize-over-seeds One execution per block Only limited data available at one time
Parallelize-over-data One execution total Entire data set available at one time
![Page 19: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/19.jpg)
Both parallelization schemes have serious flaws.
Two approaches:
Parallelizing Over I/O EfficiencyData Good BadParticles Bad Good
Parallelizeover particles
Parallelizeover dataHybrid algorithms
![Page 20: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/20.jpg)
The master-slave algorithm is an example of a hybrid technique.
Uses “master-slave” model of communication One process has unidirectional control over
other processes Algorithm adapts during runtime to avoid
pitfalls of parallelize-over-data and parallelize-over-particles. Nice property for production visualization tools. (Implemented in VisIt)
D. Pugmire, H. Childs, C. Garth, S. Ahern, G. Weber, “Scalable Computation of
Streamlines on Very Large Datasets.” SC09, Portland, OR, November, 2009
![Page 21: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/21.jpg)
Master-Slave Hybrid Algorithm• Divide processors into groups of N
• Uniformly distribute seed points to each group
Master:- Monitor workload- Make decisions to optimize resource
utilization
Slaves:- Respond to commands from
Master- Report status when work
complete
SlaveSlaveSlave
Master
SlaveSlaveSlave
Master
SlaveSlaveSlave
Master
SlaveSlaveSlave
MasterP0P1P2P3
P4P5P6P7
P8P9P10P11
P12P13P14P15
![Page 22: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/22.jpg)
Master Process Pseudocode
Master(){ while ( ! done ) { if ( NewStatusFromAnySlave() ) { commands = DetermineMostEfficientCommand()
for cmd in commands SendCommandToSlaves( cmd ) } }}
What are the possible commands?
![Page 23: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/23.jpg)
Commands that can be issued by master
Master Slave
Slave is given a particle that is contained in a block that is already loaded
1. Assign / Loaded Block2. Assign / Unloaded Block3. Handle OOB / Load4. Handle OOB / Send
OOB = out of bounds
![Page 24: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/24.jpg)
Master Slave
Slave is given a particle and loads the block
Commands that can be issued by master
1. Assign / Loaded Block2. Assign / Unloaded Block3. Handle OOB / Load4. Handle OOB / Send
OOB = out of bounds
![Page 25: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/25.jpg)
Master Slave
Load
Slave is instructed to load a block. The particle advection in that block can then proceed.
Commands that can be issued by master
1. Assign / Loaded Block2. Assign / Unloaded Block3. Handle OOB / Load4. Handle OOB / Send
OOB = out of bounds
![Page 26: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/26.jpg)
Master Slave
Send to J
Slave J
Slave is instructed to send the particle to another slave that has loaded the block
Commands that can be issued by master
1. Assign / Loaded Block2. Assign / Unloaded Block3. Handle OOB / Load4. Handle OOB / Send
OOB = out of bounds
![Page 27: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/27.jpg)
Master Process Pseudocode
Master(){ while ( ! done ) { if ( NewStatusFromAnySlave() ) { commands = DetermineMostEfficientCommand()
for cmd in commands SendCommandToSlaves( cmd ) } }}
![Page 28: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/28.jpg)
Driving factors… You don’t want PEs to sit idle
(The problem with parallelize-over-data) You don’t want PEs to spend all their
time doing I/O (The problem with parallelize-over-seeds)
you want to do “least amount” of I/O that will prevent “excessive” idleness.
![Page 29: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/29.jpg)
The heuristics behind the heuristics Rules:
If no PE has the block, then load it! If a PE does have the block…
If it is not busy, then give it the particle If it is busy, then wait “a while”. If you’ve waited a “long time”, then load it
yourself.
![Page 30: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/30.jpg)
Master-slave in action
S0
S0
S1
S1S2
S3
S4
Iteration
Action
0 S0 reads B0,S3 reads B1
1 S1 passes points to S0,S4 passes points to S3,S2 reads B0
0: Read
0: Read1: Pass
1: Pass1: Read
![Page 31: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/31.jpg)
Algorithm Test Cases
- Core collapse supernova simulation- Magnetic confinement fusion simulation- Hydraulic flow simulation
![Page 32: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/32.jpg)
Workload distribution in parallelize-over-data
Starvation
![Page 33: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/33.jpg)
Workload distribution in parallelize-over-particles
Too much I/O
![Page 34: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/34.jpg)
Workload distribution in master-slave algorithm
Just right
![Page 35: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/35.jpg)
ParticlesData Hybrid
Workload distribution in supernova simulation
Parallelization by:
Colored by PE doing integration
![Page 36: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/36.jpg)
Astrophysics Test Case: Total time to compute 20,000 Streamlines
Sec
onds
Sec
onds
Number of procs Number of procs
Uniform Seeding
Non-uniform Seeding
Data HybridParticles
![Page 37: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/37.jpg)
Astrophysics Test Case: Number of blocks loaded
Blo
cks
load
ed
Blo
cks
load
ed
Number of procs Number of procs
Data Hybrid
Uniform Seeding
Non-uniform Seeding
Particles
![Page 38: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/38.jpg)
Summary: Master-Slave Algorithm
First ever attempt at a hybrid algorithm for particle advection
Algorithm adapts during runtime to avoid pitfalls of parallelize-over-data and parallelize-over-particles. Nice property for production visualization tools.
Implemented inside VisIt visualization and analysis package.
![Page 39: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/39.jpg)
Benefits of an Extended Memory Hierarchy
![Page 40: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/40.jpg)
Emerging I/O Configuration
Emerging I/O ConfigurationCommon I/O Configuration
![Page 41: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/41.jpg)
Simulation Codes
SimulationCode
GlobalFile System
Node
Checkpoint/Restart
SimulationCode
Node
Checkpoint/Restart
SSD
![Page 42: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/42.jpg)
Parallelize-over-Seeds Algorithm
IntegrationI/O
Memory Cache
GlobalFile System
Streamlines
Node
![Page 43: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/43.jpg)
Parallelize-over-Seeds Algorithm with the Extended Memory Hierarchy
IntegrationI/O
Memory Cache
GlobalFile System
Streamlines
Local File SystemNode
![Page 44: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/44.jpg)
I/O Configurations
![Page 45: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/45.jpg)
![Page 46: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/46.jpg)
![Page 47: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/47.jpg)
![Page 48: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/48.jpg)
A Real World Particle Advection System
![Page 49: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/49.jpg)
Goal Efficient code for a variety of particle
advection based techniques Cognizant of use cases with >>10K
particles. Need handling of every particle, every
evaluation to be efficient. Want to support diverse flow techniques:
flexibility/extensibility is key. Fit within data flow network design (i.e. a
filter)
![Page 50: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/50.jpg)
Motivating examples of system FTLE Stream surfaces Streamline Dynamical Systems (e.g. Poincaré Maps) Statistics based analysis + more
![Page 51: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/51.jpg)
Design PICS filter: parallel integral curve system Execution:
Instantiate particles at seed locations Step particles to form integral curves
Analysis performed at each step Termination criteria evaluated for each step
When all integral curves have completed, create final output
![Page 52: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/52.jpg)
Design Five major types of extensibility:
How to parallelize? How do you evaluate velocity field? How do you advect particles? Initial particle locations? How do you analyze the particle paths?
![Page 53: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/53.jpg)
Inheritance hierarchy
avtPICSFilter
Streamline Filter
Your derived type of PICS
filter
avtIntegralCurve
avtStreamlineIC
Your derived type of integral
curve
We disliked the “matching inheritance” scheme, but this achieved all of our design goals cleanly.
![Page 54: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/54.jpg)
#1: How to parallelize?
avtICAlgorithm
avtParDomIC-Algorithm
(parallel over data)
avtSerialIC-Algorithm
(parallel over seeds)
avtMasterSlave-
ICAlgorithm
![Page 55: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/55.jpg)
#2: Evaluating velocity field
avtIVPField
avtIVPVTKField
avtIVPVTK- TimeVarying-
FieldavtIVPM3DC1
Field
avtIVP-<YOUR>HigherOrder-Field
IVP = initial value problem
![Page 56: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/56.jpg)
#3: How do you advect particles?
avtIVPSolver
avtIVPDopri5 avtIVPEuler avtIVPLeapfrog
avtIVP-M3DC1Integrato
r
IVP = initial value problem
avtIVPAdams-Bashforth
![Page 57: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/57.jpg)
#4: Initial particle locations avtPICSFilter::GetInitialLocations() = 0;
![Page 58: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/58.jpg)
#5: How do you analyze particle path? avtIntegralCurve::AnalyzeStep() = 0;
All AnalyzeStep will evaluate termination criteria avtPICSFilter::CreateIntegralCurveOutput(
std::vector<avtIntegralCurve*> &) = 0;
Examples: Streamline: store location and scalars for current
step in data members Poincare: store location for current step in data
members FTLE: only store location of final step, no-op for
preceding steps NOTE: these derived types create very
different types of outputs.
![Page 59: Particle Advection Defined & Usage](https://reader036.vdocuments.us/reader036/viewer/2022062302/5681668e550346895dda5d05/html5/thumbnails/59.jpg)
Putting it all togetherPICS Filter
avtICAlgorithmavtIVPSolv
er
avtIVPFieldVector<
avtIntegral-Curve>
Integral curves sent to other processors with some derived types of avtICAlgorithm.
::CreateInitialLocations() = 0;
::AnalyzeStep() = 0;