the effect of in-network computing-capable interconnects
TRANSCRIPT
![Page 1: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/1.jpg)
NAFEMS
The Effect of In-Network Computing-Capable
Interconnects on the Scalability of CAE Simulations
Ophir Maor, Gerardo Cisneros, David Cho, Yong Qin, Gilad Shainer
HPC Advisory Council
![Page 2: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/2.jpg)
NAFEMS
The HPC-AI Advisory Council
• World-wide HPC non-profit organization
• More than 400 member companies / universities / organizations
• Bridges the gap between HPC-AI usage and its potential
• Provides best practices and a support/development center
• Explores future technologies and future developments
• Leading edge solutions and technology demonstrations
![Page 3: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/3.jpg)
NAFEMS
HPC Advisory Council Members
![Page 4: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/4.jpg)
NAFEMS
HPC-AI Advisory Council
Cluster Center (Examples)
• Supermicro / Foxconn 32-node cluster
• Dual Socket Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz
• Dell™ PowerEdge™ R730/R630 36-node cluster
• Dual Socket Intel® Xeon® 16-core CPUs E5-2697A V4 @ 2.60 GHz
• IBM S822LC POWER8 8-node cluster
• Dual Socket IBM POWER8 10-core CPUs @ 2.86 GHz
• GPU: NVIDIA Kepler K80 GPUs
![Page 5: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/5.jpg)
NAFEMS
• Lattice QCD
• LAMMPS
• LS-DYNA
• miniFE
• MILC
• MSC Nastran
• MR Bayes
• MM5
• MPQC
• NAMD
• Nekbone
• NEMO
• NWChem
• Octopus
• OpenAtom
• OpenFOAM
• OpenMX
• OptiStruct
• PARATEC
• PFA
• PFLOTRAN
• Quantum ESPRESSO
• RADIOSS
• SNAP
• SPECFEM3D
• STAR-CCM+
• STAR-CD
• VASP
• VSP
• WRF
Multiple Applications Best Practices Published
• Abaqus
• ABySS
• AcuSolve
• Amber
• AMG
• AMR
• ANSYS CFX
• ANSYS FLUENT
• ANSYS Mechanical
• BQCD
• BSMBench
• CAM-SE
• CCSM
• CESM
• COSMO
• CP2K
• CPMD
• Dacapo
• Desmond
• DL-POLY
• Eclipse
• FLOW-3D
• GADGET-2
• Graph500
• GROMACS
• Himeno
• HIT3D
• HOOMD-blue
• HPCC
• HPCG
• HYCOM
• ICON
App
App
App
App
![Page 6: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/6.jpg)
NAFEMS
Cloud andWeb 2.0
Big Data
Enterprise
Business Intelligence
HPC
Storage
Security
Machine Learning
Internet of Things
source: IDC
Exponential Data Growth Everywhere
![Page 7: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/7.jpg)
NAFEMS
Breaking the Application Latency Wall
10 years ago
~10
microsecond
~100
microsecond
NetworkCommunication
Framework
Today
~10
microsecond
Communication
Framework
~0.1
microsecond
Network
~1
microsecond
Communication
Framework
Future
~0.05
microsecond
Network
![Page 8: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/8.jpg)
NAFEMS
The Ever Growing Demand for
Higher Performance
2000 202020102005
“Roadrunner”
1st
2015
Terascale Petascale Exascale
Single-Core to Many-CoreSMP to Clusters Co-Design
HW SW
APP
Hardware
Software
Application
![Page 9: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/9.jpg)
NAFEMS
Data Centric Data Center
![Page 10: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/10.jpg)
NAFEMS
SHARP - Scalable Aggregation
and Reduction Technology
• Reliable Scalable General Purpose Primitive
– In-network Tree based aggregation mechanism
– Large number of groups
– Multiple simultaneous outstanding operations
• Applicable to Multiple Use-cases
– HPC Applications using MPI / SHMEM
– Distributed Machine Learning applications
• Scalable High Performance Collective Offload
– Barrier, Reduce, All-Reduce, Broadcast and more
– Sum, Min, Max, Min-loc, max-loc, OR, XOR, AND
– Integer and Floating-Point, 16/32/64 bits
Topology (Physical Tree)
![Page 11: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/11.jpg)
NAFEMS
SHARP Allreduce Performance
![Page 12: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/12.jpg)
NAFEMS
SHARP Allreduce Performance
![Page 13: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/13.jpg)
NAFEMS
The Niagara Supercomputer –
University of Toronto
![Page 14: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/14.jpg)
NAFEMS
SHARP AllReduce Performance
1500 Nodes, 60K MPI Ranks, Dragonfly+ Topology
![Page 15: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/15.jpg)
NAFEMS
OpenFOAM
• OpenFOAM® (Open Field Operation and Manipulation) CFD
• Toolbox in an open source CFD applications that can simulate
– Complex fluid flows involving
– Chemical reactions
– Turbulence
– Heat transfer
– Solid dynamics
– Electromagnetics
– The pricing of financial options
• OpenFOAM support can be obtained from OpenCFD Ltd
![Page 16: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/16.jpg)
NAFEMS
OpenFOAM Performance
(motorBike_160)
30%
![Page 17: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/17.jpg)
NAFEMS
OpenFOAM Scalability per
Interconnect Technology
![Page 18: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/18.jpg)
NAFEMS
OpenFOAM MPI Performance
![Page 19: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/19.jpg)
NAFEMS
Summary
• HPC cluster environments impose high demands on connectivity throughput and low latency with low CPU overhead, network flexibility, and high efficiency
• Fulfilling these demands enables the maintenance of a balanced system that can achieve high application performance and high scaling
• With the increase in number of CPU cores and application threads, there is a need to develop a new HPC cluster architecture - a data-focused architecture
• The Co-Design collaboration enables the development of In-Network Computing technology that breaks the performance and scalability barriers
• The OpenFoam application was benchmarked for this study to demonstrate the advantages of In-Network Computing technology
• We have witness nearly 30% performance advantage and linear scalability with InfiniBand In-Network Computing technology
![Page 20: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/20.jpg)
NAFEMS
2018 HPC-AI Advisory Council Activities
• HPC-AI Advisory Council– More then 400 members, http://www.hpcadvisorycouncil.com/
– Application best practices, case studies
– Benchmarking center with remote access for users
– World-wide conferences
• 2018 Conferences– USA (Stanford University) – February
– Switzerland (CSCS) – April
– Australia - August
– Spain (BSC) – Sep
– China (HPC China) – October
• 2018 Competitions– APAC HPC-AI Competition - March
– China - 6th Annual RDMA Competition - May
– ISC Germany - 7th Annual Student Cluster Competition - June
• For more information – www.hpcadvisorycouncil.com
![Page 21: The Effect of In-Network Computing-Capable Interconnects](https://reader031.vdocuments.us/reader031/viewer/2022021106/62059bbac6e60e70c4572439/html5/thumbnails/21.jpg)
All trademarks are property of their respective owners. All information is provided “As-Is” without any kind of warranty. The HPC Advisory Council makes no representation to the accuracy and completeness of the information
contained herein. HPC Advisory Council undertakes no duty and assumes no obligation to update or correct any information presented herein
Thank You