staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · franc¸ois pellegrini –...
TRANSCRIPT
![Page 1: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/1.jpg)
Static and dynamic processing of discretedata structuresC2S@Exa
Francois Pellegrini
![Page 2: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/2.jpg)
Contents
1. Context
2. Works
3. Results to date
4. Perspectives
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 2
![Page 3: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/3.jpg)
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 3
1Context
![Page 4: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/4.jpg)
Purpose
• Means and ends: discrete data structures:- Graphs
- “Mesh-like”, not “social network-like”- Meshes
• Create locality (→ pole 3)- Compute efficient partitions / mappings of graphs / meshes- Improve cache locality by adequate local numbering
• Enable PDE solver writers to focus on their “core business” (→ pole 2)- Hide all the MPI “plumbing work”- Contribute to efficient parallelization
- At the node level
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 4
![Page 5: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/5.jpg)
People and support
• F. Pellegrini- Formerly Bacchus team, now TADaaM since 01/01/2015
(“Topology-Aware System-Scale Data Management for High-Perfor manceComputing Applications”)
• S. Fourestier- PhD defended on 20/06/2013- Left project on 11/2014
• C. Lachat- PhD defended on 13/12/2013- ADT “El Gaucho” from 01/10/2012 to 30/09/2014- PIA ELCI (“Bull”) ASAP
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 5
![Page 6: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/6.jpg)
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6
2Works
![Page 7: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/7.jpg)
The Scotch project
• Toolbox of graph partitioning methods, which can beused in numerous contexts
• Sequential Scotch library (v6.0)- Graph and mesh partitioning- Static mapping (edge dilation)- Graph and mesh reordering- Clustering- Graph repartitioning and remapping
• Parallel PT-Scotch library (v6.0)- Graph partitioning (edge)- Static mapping (edge dilation)- Graph reordering- Graph repartitioning, remapping (v6.1)
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 7
![Page 8: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/8.jpg)
Three challenges
• Scalability- How will the algorithms behave for large numbers of processing elements?
• Heterogeneity- How will the architecture of the target machine impact performance?
• Asynchronicity- Will our algorithms still be able to rely on fast collective communication?
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 8
![Page 9: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/9.jpg)
Design constraints
• Parallel algorithms have to be carefully designed- Algorithms for distributed memory machines- Preserve independence between the number of parts k and the number of
processing elements P on which algorithms are to be executed- Algorithms must be “quasi-linear” in |V | and/or |E |
- Constants should be kept small• Data structures must be scalable
- In |V | and/or |E |: graph data must not be duplicated- In P and k : arrays in k |V |, k2, kP, P|V | or P2 are forbidden
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 9
![Page 10: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/10.jpg)
Architectural considerations matter
• High-end machines comprise very large numbers of processing units, andwill possess NUMA / heterogeneous architectures
• Impacts on our research:- Target architecture has to be taken into account- Do static mapping and not only graph partitioning
- Reduces number of neighbors and improves communication locality, at the expenseof slight increase in message sizes
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 10
![Page 11: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/11.jpg)
Graph partitioning with fixed vertices
• Used to model repartitioning / remapping• Algorithms designed and implemented in Scotch 6.0
- Parallel version scheduled for Scotch 6.1
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 11
![Page 12: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/12.jpg)
PaMPA
• PaMPA: “Parallel Mesh Partitioning and Adaptation”• Library managing the parallel repartitioning and remeshing of unstructured
meshes modeled as interconnected valuated entities• The user can focus on his/her “core business”:
- Solver- Sequential remesher
- Coupling with MMG3D provided for tetrahedral remeshing
PT-Scotch
API
Seq. Qual.MeasurementRemeshing and redistribution
PaMPA
Physics, solver
Seq.remesherMMG3D
Seq.interpolator
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 12
![Page 13: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/13.jpg)
Data structures for representing distributed meshes
• Based on the notion of “enriched graph”- Labeled undirected loopless graph- Sub-labeling (for separating e.g. boundary and inner faces)- With adequate local and global vertex numbering
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 13
![Page 14: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/14.jpg)
Framework for parallel remeshing
• Iterative process until all tagged elements are remeshed
1b Tagging
2 Identification
3 Extraction 4 Remeshing
5 Reintegration
1a Tagging
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 14
![Page 15: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/15.jpg)
Extraction of zones to be remeshed
• Two criteria must be considered:- Load-balance according to the remesher workload- Minimize communication
• Requires partitioning with fixed vertices
P0
P1 P2
P3
9 7 5 5 3 7 3
P2 P3P0 P1
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 15
![Page 16: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/16.jpg)
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 16
3Results to date
![Page 17: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/17.jpg)
Features of PaMPA 1.0
• Description of distributed unstructured meshes- With values attached to enriched graph vertices
• Data exchange with overlap of any width- Point-to-point or collective communication
• Iterators to loop over entities and sub-entities• Parallel partitioning and redistribution
- Renumbering for cache miss reduction
• Parallel mesh I/O• Parallel remeshing based on a sequential remesher
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 17
![Page 18: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/18.jpg)
Example: Solve system (Jacobi) (1/2)
UaPrec = 0. ! Suppose A = L + D + U, system to solve : A x = bCALL PAMPAF dmeshItInit (dm, ENTITY NODE, ENTITY NODE, i t ngb , i e r r )DO i r e l a x = 1 , Nrelax
res = 0.CALL PAMPAF dmeshIt Ini tStart (dm, ENTITY NODE, PAMPAF VERT BOUNDARY, i t v r t , i e r r )DO WHILE ( PAMPAF itHasMore ( i t v r t ) )
i s = PAMPAF itCurEnttVertNum ( i t v r t )CALL PAMPAF dmeshMatLineData (dm, ENTITY NODE, is , I1 , I1Fin , i e r r )CALL PAMPAF itStart ( i t ngb , is , i e r r )res0 = RHS( i s ) ! res0 = b
i v = i 1DO WHILE ( PAMPAF itHasMore ( i t n g b ) )
j s = PAMPAF itCurEnttVertNum ( i t n g b )PAMPAF itNext ( i t n g b )res0 = res0 − MatCSR%Vals ( i v ) ∗ UaPrec ( j s ) ! res0 = b − ( L + U) x ˆ ni v = i v + 1
END DOUa( i s ) = res0 / MatCSR%Diag ( i s ) ! x ˆ n+1 = ( b − ( L + U) x ˆ n ) /DPAMPAF itNext ( i t v r t )
END DO
CALL PAMPAF dmeshHaloValueAsync (dm, ENTITY NODE, PAMPA TAG SOL, req , i e r r )
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 18
![Page 19: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/19.jpg)
Example: Solve system (Jacobi) (2/2)
CALL PAMPAF dmeshIt Ini tStart (dm, ENTITY NODE, PAMPAF VERT INTERNAL, i t v r t , i e r r )DO WHILE ( PAMPAF itHasMore ( i t v r t ) )
i s = PAMPAF itCurEnttVertNum ( i t v r t )CALL PAMPAF dmeshMatLineData (dm, ENTITY NODE, is , I1 , I1Fin , i e r r )CALL PAMPAF itStart ( i t ngb , is , i e r r )res0 = RHS( i s ) ! res0 = b
i v = i 1DO WHILE ( PAMPAF itHasMore ( i t n g b ) )
j s = PAMPAF itCurEnttVertNum ( i t n g b )PAMPAF itNext ( i t n g b )res0 = res0 − MatCSR%Vals ( i v ) ∗ UaPrec ( j s ) ! res0 = b − ( L + U) x ˆ ni v = i v + 1
END DOUa( i s ) = res0 / MatCSR%Diag ( i s ) ! x ˆ n+1 = ( b − ( L + U) x ˆ n ) /DPAMPAF itNext ( i t v r t )
END DO
CALL PAMPAF dmeshHaloWait ( req , i e r r )
UaPrec = UaEND DO ! end loop on i r e l a x
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 19
![Page 20: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/20.jpg)
Parallel remeshing test cases
Isotropic mesh Anisotropic mesh
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 20
![Page 21: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/21.jpg)
Parallel remeshing on an isotropic mesh
PaMPA-MMG3Don 240 on 480
processors processorsInitial number of elements 27 044 943
Used memory (kb) 651 185 792 542 832 960Elapsed time 00h34m59s 00h29m03s
Elapsed time × number procs 139h56m 232h24mFinal number of elements 609 671 387 612 426 645
Smallest edge length 0.2911 0.1852Largest edge length 8.3451 7.3611
Worst element quality 335.7041 190.4122Element quality between 1 and 2 98.92% 98.97%
Edge length between 0.71 and 1.41 97.20% 97.39%
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 21
![Page 22: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/22.jpg)
Industrialization of PaMPA
• Several Inria teams and groups already use PaMPA- Bacchus & Cagire: AeroSol solver framework for fluid dynamics- Castor: Plato prototype solver- Num3sis: Prototype interfacing work
• Industrial interest in parallel remeshing- CD Adapco, Dassault, Airbus, etc.
• Project in progress with involvement of DTI- Release as GPL’d free software- Creation of a community- Creation of a start-up?
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 22
![Page 23: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/23.jpg)
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 23
4Perspectives
![Page 24: Staticand dynamicprocessingofdiscrete datastructures · 2015. 1. 13. · Franc¸ois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 6 2 Works. The Scotch project](https://reader034.vdocuments.us/reader034/viewer/2022052006/601a956eb2d93c7dbb3a89e9/html5/thumbnails/24.jpg)
Perspectives
• PaMPA is now ready for solver software development- Testbed for the tuning of TADaaM works on mapping
• Development of (PT-)Scotch is going on- PhD on multi-constraint partitioning started on 08/12/2014 (CEA/DAM funding,
as “Projet Phare”)- Works with HiePACS on specific partitioning and ordering algorithms
• PIA ELCI (“Bull”)- Scalability studies of PT-Scotch and PaMPA- Man-powered by C. Lachat
• H2020 “AltExa” project submitted- With Inria teams HiePACS, ROMA and KIT, TU. Vienna, U. Utrecht, ETH Zurich
Francois Pellegrini – Pole 3: Discrete data structures – January 13, 2015 24