Download - April 2010
Mellanox Connectivity Solutions for Scalable HPC
Highest Performing, Most EfficientEnd-to-End Connectivity for Servers and Storage
April 2010
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL - 2
Connectivity Solutions for Efficient Computing
Leading Connectivity Solution Provider For Servers and Storage
Enterprise HPC
High-end HPC
Leading Connectivity Solution Provider For Servers and Storage
HPC Clouds
Mellanox Technologies is a leading supplier of high-performance connectivity solutions for servers & storage
• End-to-End Ifniniband solutions• 10 GbE and Low Latency Ethernet NIC solutions
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
Cluster Software Management
HPC Application Accelerations
Networking Efficiency/Scalability
Servers and Storage High Speed Connectivity
Complete End-to-End Highest Performance Solution
3
ICs Switches/GatewayAdapters Cables
CORE-Direct GPU-Direct
QoSCongestion Control
Adaptive Routing
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
Mellanox’s InfiniBand Leadership
HighestPerformance
Efficient, Scalable and
Flexible Networking
Converged Network
Efficient use ofCPUs and GPUs
CORE-Direct
GPU-Direct
Highest Throughput
Lowest Latency
CPU Availability
Virtual Protocol
Auto SensingHost
Hardware Bridging
Message Rate
Transport Offload
Adaptive Routing
CongestionControl
Multiple Topologies
Advanced QoS
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
ConnectX-2 Virtual Protocol Interconnect
StorageNFS, CIFS, iSCSI
NFS-RDMA, SRP, iSER,Fibre Channel, Clustered
NetworkingTCP/IP/UDP
Sockets
ClusteringMPI, DAPL, RDS, Sockets
ManagementSNMP, SMI-S
OpenView, Tivoli, BMC, Computer Associates
LLE
Consolidated Application Programming Interface
App1 App2 App3 App4 AppX…
Acceleration Engines
10GigE
Any Protocol over Any Convergence Fabric
Protocols
Applications
Networking VirtualizationClustering Storage RDMA
10/20/40 InfiniBand
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
Efficient Data Center Solutions
40Gb/s Network• InfiniBand • Eth over IB• FC over IB• FC over Eth*
Switches Bridge
Storage
ServersAdapters
Ethernet Storage
FC Storage
IB Storage
40G InfiniBand, FCoIB10G Ethernet, FCoE
IB to EthIB to FCEth to FC
* via ecosystem products
40G InfiniBand 10G Ethernet 8G Fibre Channel
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
Highest Performance
Highest throughput • 40Gb/s node to node and 120Gb/s switch to switch• Up to 50M MPI messages per second
Lowest latency• 1usec MPI end-to-end• 0.9us InfiniBand latency for RDMA operations• 100ns switch latency at 100% load• True zero scalable latency – flat latency up to 256 cores per node
7
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
Efficient use of CPUs and GPUs
GPU-direct• Works with existing NVIDIA Tesla and Fermi products• Enables fastest GPU-to-GPU communications• Eliminates CPU copy and write process in system memory• Reduces 30% of the GPU-to-GPU communication time
8
CPU
GPUChipset
GPUGPUMemorMemor
yy
MellanoxInfiniBand
System Memory1
2
CPU
GPUChipset
GPUGPUMemorMemor
yy
MellanoxInfiniBand
System
Memory
1
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
Efficient use of CPUs and GPUs
CORE-direct (Collectives Offload Resource Engine)• Collectives communication are communications used for system
synchronizations, data broadcast or data gathering• CORE-direct performs the collective on the HCA instead of the CPU• Eliminates system noise and jitter issue • Increases the CPU cycles available for applications
Transport Offload• Full transport offload maximizes CPU availability for user applications• Only solutions to achieve 40Gb/s throughput with ~5% CPU overhead
9
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
Efficient, Scalable and Flexible Networking
Congestion control• Eliminates network congestions (hot-spots) related to many senders and a single
receiver
10
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
Efficient, Scalable and Flexible Networking
Adaptive routing• Eliminated networks congestions related to point to point communications sharing
the same network path
11
Hot Spots Configuration
Average Bandwidth
Minimum Bandwidth
Average Bandwidth
Minimum Bandwidth
None 100.0% NA NA
2:41,3:4,4:1 79.4% 37.3% 99.7% 99.6%
2:35,3:5,4:1 80.3% 37.1% 99.7% 99.0%
2:30,3:10,4:1 75.9% 36.5% 99.7% 99.5%
2:41,3:5,4:1 82.4% 37.7% 99.7% 99.4%
No Adaptive Routing With Adaptive Routing
220 server node system, Mellanox InfiniBand HCAs and switches
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
Efficient, Scalable and Flexible Networking
Multiple Topologies• Fat-tree: non-blocking, oversubscription• Mesh, 3D-Torus• Hybrid solutions
Advanced Quality of Service• Fine grained QoS• Consolidation without performance degradation
12
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
Mellanox Performance/Scalability Advantage
Weather simulations at scale with Mellanox end-to-end
13
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL - 14
Top100 Interconnect Share Over Time
InfiniBand the natural choice for large scale computing• All based on Mellanox InfiniBand technology
© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -
The 20 Most Efficient Top500 Systems
15
InfiniBand solutions enables the most efficient system in the Top500 The only standard interconnect solution in the top 100 highest utilization systems
#55 US Army Research Laboratory
#22 The Earth Simulator Center
#93 National Institute for Materials Science #10 Jülich
Thank You