european centre for medium-range weather forecasts
TRANSCRIPT
1 Email: [email protected]
www.hpc-escape.euThe project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement No 800897
Andreas Müller1, Giovanni Tumolo, Willem Deconinck, Nils Wedi, Peter Bauer
European Centre for Medium-Range Weather Forecasts, Reading, UK
ESCAPE 2: Energy-efficient SCalable Algorithms for weather and climate Prediction at Exascale
Introduction• Numerical Weather Prediction (NWP) and Climate models contain decades
of algorithmic developments for conventional CPU hardware architectures • Paradigm shift towards more parallel and energy efficient many-core
hardware architectures due to breakdown of Dennard scaling • Large impact on programming models expected in the near future • Rethink of design choices for future software frameworks:
‣ Scalability ‣ Energy efficiency
‣ Flexibility in algorithmic choices ‣ Maintainability
Figure 1: ESCAPE 2 Partners
• Combine scientific and computer-science expertise
• Define and co-design necessary steps towards affordable exascale HPC simulations of weather and climate
ESCAPE 2 (EU Horizon 2020)
Weather and Climate benchmarks• HPWC: a hierarchy of benchmarking
components representing key elements in the workflow of weather and climate systems
Weather
Figure 6: Strong scaling study on the supercomputer Summit for the spectral transform dwarf ported to GPUs in ESCAPE1. Each node uses 6 GPUs resulting in a maximum total number of 11520 GPUs. TCO3999 (2.5km)
runt
ime
per t
imes
tep
in s
econ
ds
0.01
1
100
number of nodes
240 480 960 1920
perfect scalingIBM Power9 CPUsNVIDIA V100 GPUs
12.5x23.7x
This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE office of Science User Facility supported under contract DE-AC05-00OR22725.
Weather and Climate DwarfsWeather and Climate Dwarfs are self-contained algorithms representing key functional blocks of a NWP & Climate model. They must be verifiable and possible to integrate back in the model.
Figure 2: Overview of dwarfs used in ESCAPE 2 and models from which they originate. FVM and DG are two alternative dynamical cores for the IFS. ICON and NEMO represent ocean models.
Figure 3: Coverage of ESCAPE1 Weather and Climate Dwarfs in ECMWF’s operational Integrated Forecasting System (IFS). ESCAPE Dwarfs not covered by IFS: MPDATA advection; GCR(k) elliptic solver; BiFFT spectral transform.55%
5%4%13%
23% Dynamics — SH spectral transform Dynamics — semi-Lagrangian advectionPhysics: radiation Physics: cloud microphysicsNon-ESCAPE dwarf
now
tomorrow
time-step
spectral transform
semi-Lagrangian
discontinuous Galerkin
options:
ACRANEB2 radiation
time-stepping
advection,
gradient computation
physics
components:
elliptic solver, multigrid
HEVI
MPDATA
finite volume
CLOUDSC microphysics
IFS FVM
ALARO
DG ICON
NEMO
MUSCL finite difference
AI radiation
fault-tolerant GMRES
Combining advanced algorithms with High-Level IR
Weather Figure 4: Aiming at the combination of per formance, accuracy, resi l ience and portability for weather and climate prediction through novel mathematical and algorithmic methodologies, programming and uncertainty estimation.
Spatial resolution
Model complexity
Bene!t beyond the state-of-the-art
Uncertainty estimate
Time&energy-to-solution
Portability
Resilience
Mathematical methods and algorithmsSemi-implicit, semi-Lagrangian CG/DGHierarchical multigrid toolsFault resilient solverArti!cial neural networks
DSL toolchainEnsemble based URANIE
and:
State-of-the-art
Figure 5: Design of a modular domain specific language (DSL) toolchain for Weather & Climate dwarfs by using a High-Level intermediate representation (IR).
DSL Frontends
Dom
ain
Spec
i!c
Chec
kers
Opt
imiz
ers
Code
Gen
erat
or
Python DSLFrontend
Readbefore write
Missing UpdateBoundary
Data dependencyrace conditions
Naive C/FortranGenerator
OptimizedGridTools Generator
Out of BoundsStencil Access
Software Managed Caches
Full verticalparallelization
StageFusion
Data LocalityExploit
Data LocalityExploit
clang DSL(C++)
High-LevelIR
CLAW DSL(Fortran)
Energy-Efficient Scalable Algorithms for Weather and Climate Prediction at Exascale