investigating scaling behaviour of monte carlo codes for ... · outlinemotivation linear algebra...
TRANSCRIPT
![Page 1: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/1.jpg)
Investigating Scaling Behaviour ofMonte Carlo Codes
for Dense Matrix Inversion
V. Alexandrov (BSC, Spain)J. Straßburg (BSC,Spain & UoR, UK)
![Page 2: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/2.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Outline
MotivationBackground on Linear Algebra Problems and Monte CarloScaling experimentsConclusions
![Page 3: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/3.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Motivation
Predict behaviour on different systemFind bottlenecks, sweet spot, scaling problemsEasier then running on several machinesReproducible
![Page 4: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/4.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Scientific and Engineering Problems
Many scientific problems revolve around:inverting a real n by n matrix (MI)
• Given A• Find A−1
solving a system of linear algebraic equations (SLAE)• Given A and b• Solve, for x , Ax = b• Or find A−1 and calculate x = A−1b
![Page 5: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/5.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Traditional Methods• Gaussian elimination• Gauss-Jordan• Both take O(n3) steps
Time prohibitive if:• a large problem• or a real time solution is required
![Page 6: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/6.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Idea behind Monte Carlo Methods
Wish to estimate the quantity α
Define a random variable ξ
Where ξ has the mathematical expectation α
Take N independent realisations ξi of ξ
• Then ξ̄ = 1N
N∑i=1
ξi
• And ξ̄ ≈ α
![Page 7: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/7.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Reason for Using Monte Carlo
O(NT ) steps to find an element of the:• matrix inverse A−1
• solution vector xwhere:
• N number of Markov Chains• T length of Markov Chains
Independent of n - size of matrix or problemAlgorithms can be efficiently parallelised
![Page 8: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/8.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Parallel Algorithm
Matrix Setup
Use parallel Monte Carloto find B−1
Use parallel iterativerefinement to improve
accuracy of B−1
MASTER SETUP
SLAVECALCS
SLAVECALCS
SLAVECALCS
MASTER COLLECTS
INVERSE
MASTER DISTRIBUTES
SLAVECALCS
SLAVECALCS
SLAVECALCS
MASTER COLLECTS
DECISION
INPUT
![Page 9: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/9.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Native vs. simulated scaling
1
10
100
1000
10000
5 10 15 20
tim
e
cores
500x500
1000x1000
2000x2000
3000x3000
0.1
1
10
100
1000
10000
5 10 15 20
tim
e
cores
500x500
1000x1000
2000x2000
3000x3000
24 core SMP development machinesimilar on ORNL cluster
![Page 10: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/10.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Core scaling
0
500
1000
1500
2000
2500
3000
3500
4000
4500
1 2 4 8 16 32 64 128 256 512 1024 2048
Exe
cuti
on
Tim
e in
Se
con
ds
Simulated MPI Processes
500 1000 2000 3000 4000
960 core system240 cores for simulation due to memory bandwidth restrictions
![Page 11: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/11.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Core scaling
1
10
100
1000
1 2 4 8 16 32 64 128 256 512 1024 2048
Exe
cuti
on
Tim
e in
Se
con
ds
Simulated MPI Processes
500 1000 2000 3000 4000
Recurring behaviour for increasing MPI process sizesScales well, then plateaus
![Page 12: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/12.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Problem scaling
1
10
100
1000
500 1000 2000 3000 4000
Exe
cuti
on
Tim
e in
Se
con
ds
Problem Size
1 2 4 8 16 32 64 128 256 512 1024 2048
Linear behaviour up to 2000x2000 matrix sizeSlight degradation for larger problem sizes
![Page 13: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/13.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
MPI message count scaling
0.E+00
5.E+03
1.E+04
2.E+04
2.E+04
3.E+04
3.E+04
1 2 4 8 16 32 64 128 256 512 1024 2048
MP
I Me
ssag
e C
ou
nt
Simulated MPI Processes
500 1000 2000 3000 4000
1.E+00
1.E+01
1.E+02
1.E+03
1.E+04
1.E+05
1 2 4 8 16 32 64 128 256 512 1024 2048
MP
I Me
ssag
e C
ou
nt
Simulated MPI Processes
500 1000 2000 3000 4000
Simulator also gathers MPI statisticsLinear increase of exchanged messages, as expected
![Page 14: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/14.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Outcome & Conclusions
Behaviour of code is predictableSimulation provides valuable informationForecast behaviour on varying systems possibleTime and resource concerns
![Page 15: Investigating Scaling Behaviour of Monte Carlo Codes for ... · OutlineMotivation Linear Algebra Monte Carlo Scaling experiments Motivation Predict behaviour on different system Find](https://reader034.vdocuments.us/reader034/viewer/2022042215/5ebb5603d157e6732c7df397/html5/thumbnails/15.jpg)
Outline Motivation Linear Algebra Monte Carlo Scaling experiments
Future work
Improve the code and retestProfiling + scaling simulationOptimize program to be able to handle simulation of largerproblem sizesRuns on larger parallel systems, different networks