nec and cscs a first ivy bridge deployment · cscs in a nutshell • an autonomous unit of the...
TRANSCRIPT
![Page 1: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/1.jpg)
NEC and CSCS A First Ivy Bridge Deployment
Jason Temple, Luc Corbeil, HPC Solutions, CSCS
![Page 2: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/2.jpg)
What is CSCS
© CSCS 2013 2
Office building • 5 floors • 2’600 m2
• Standard Minergie • Offices for 55 people • Two conference rooms
![Page 3: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/3.jpg)
CSCS in a nutshell
• An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich)
• founded in 1991 in Manno • relocated to Lugano in 2012
• Develops and promotes technical and scientific services • for the Swiss research community in the field of high-performance
computing
• Enables world-class scientific research • by pioneering, operating and supporting leading-edge supercomputing
technologies
© CSCS 2013 3
![Page 4: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/4.jpg)
Management of Computing Resources
• Two groups of highly skilled HPC system administrators
• National Systems – Systems that are operated for the benefit of the Swiss scientific
community – Time allowed based on a call for proposal with peer-review
• HPC Solutions – Hosting and management of HPC resources for Swiss organizations – Current partners:
– MeteoSwiss (24/7 operational weather forecast) – EPFL/BlueBrain – CHiPP (data processing from LHC)
![Page 5: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/5.jpg)
New project for HPC Solutions
• Community of ETHZ professors needed HPC resources – Theoretical physics – Seismology – Polymer Physics – Astrophysics – Molecular dynamics
• CSCS acts as a broker – Pooling of resources – Reduced overhead for each professor – Provider of a turnkey solution
![Page 6: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/6.jpg)
Requirements
• Common analysis of the requirements – Scientific
– Subset of applications – Gromacs – VASP – AMR – SSE
– Technical – Facilities constraints – Cluster design – Reliability, Availability, Serviceability (RAS)
– Best solution within budget
![Page 7: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/7.jpg)
Solution
• NEC’s offer was selected – Full competitive WTO process
• First Intel Ivy Bridge cluster in Switzerland – Maximum power consumption was lower than expected – Throughput per node improved
![Page 8: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/8.jpg)
System Hardware Specifications 340 Standard Compute Nodes with two Sockets and 20 Cores
– Two Intel Ivy Bridge EP E5-2660v2 @ 2.2 GHz – 312 + 4 nodes with 32 GB (DDR3 1600 MHz), 24 with 64GB
8 of 21 © CSCS 2013
![Page 9: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/9.jpg)
System Hardware Specifications
• FDR fat tree, fully non-blocking, Mellanox SX6036
• >300TB NEC LXFS v3 (based on Lustre 2.3), >15 GB/s – Built with NEC SNA260-FS building blocks
• <100 kW under typical load
9
![Page 10: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/10.jpg)
System Management, Compilers and Tools
10 © CSCS 2013
• LX C3 (LX Cluster Command and Control) • Node provisioning • Cluster monitoring
• Slurm scheduler (installed and managed by CSCS)
• Intel Cluster Studio • Intel C, C++, Fortran Compiler • OpenMP Version 3 • Intel MPI (MPI 2.2 for C/C++ and Fortran) • Mathematical Libraries • Intel VTune XE for Linux
![Page 11: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/11.jpg)
• Video
![Page 12: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/12.jpg)
NEC: solid HPC partners
• Very professional approach – Detailed installation planning – Planned followed with very minor variations – No bad surprises
• Great benchmarking expertise – Prudent, conservative commitments – Performance commitments were all exceeded
• Efficient support – Personalized attention – Pro-active replacement of suspicious hardware
Conclusion: Success Story!
![Page 13: NEC and CSCS A First Ivy Bridge Deployment · CSCS in a nutshell • An autonomous unit of the Swiss Federal Institute of Technology in Zurich (ETH Zurich) • founded in 1991 in](https://reader034.vdocuments.us/reader034/viewer/2022050300/5f6918533d133d526673d3f9/html5/thumbnails/13.jpg)
Thank you for your attention. Questions?
13