Jaguar Super Computer

Topics Covered

Introduction Architecture Location & Cost Bench Mark Results Location & Manufacturer Machines

in top 500 Operating Systems & Supporting

Languages Typical Applications Interesting sibling interesting about the assigned

machine Questions

Introduction

Jaguar is ranked Number 2 in the world by the Top500 List of the world’s fastest computers.

Jaguar is Number 1 for open science, with major projects in climate science, astrophysics, fusion research, and many other critical areas

Jaguar achieved 101.7 teraflops on High-Performance Linpack benchmark, more than 85 percent of its theoretical peak of 119 teraflops

Jaguar also solved the largest Linpack problem ever (a matrix problem of order

2.2 million containing nearly 5 trillion elements)

Architecture

Processor :AMD X86_64 Opteron Quad Core 2300 MHz (9.2 GFlops) Has Two of dedicated Notes

– Compute nodes – Service nodes

– Compute nodes : – Designed to run MPI tasks efficiently and reliably to completion.– Each compute node contains a quad-core 2.1 GHz AMD Opteron processor and 8GB of

memory. Aggregate system performance is approximately 263 TF. – Approximately 600 TB are available in the scratch file system. – The compute nodes are running the Compute Node Linux (CNL) OS– Each node is connected to a Cray SeaStar router through HyperTransport, and the

SeaStars are all interconnected in a 3-D-torus topology. – The resulting interconnect has very high bandwidth, low latency, and extreme

scalability.

Architecture

Service nodes :– Designed to provide system and I/O connectivity and also serve as login

nodes from which jobs are compiled and launched. – Jaguar has a total of 7,832 XT4 compute nodes in addition to input/output (I/O)

and login service nodes. – The service nodes consist of a 2.6 GHz dual-core AMD Opteron processor

with 8 GB of memory. Service nodes run SuSE LINUX

– Service nodes run SuSE LINUX

Architecture

Scalable Interconnect: XT4 Internal Interconnect– The Cray XT5 system incorporates a high-bandwidth, low-latency interconnect based on the

Cray SeaStar2+™ chip. – The interconnect directly connects all the nodes in a Cray XT5 system in a 3D torus topology,

eliminating the cost and complexity of external switches and allowing for easy expandability and upgradability to future high performance Cray interconnect technologies.

– Communications processing and high-speed routing on a single device. – Each communications chip is composed of a HyperTransport™ link, a Direct Memory Access

(DMA) engine, a communications and management processor, a high speed interconnect router and a service port

– The peak bidirectional bandwidth of each link is 9.6 GB/s with sustained bandwidth in excess of 6Gb/s

Architecture

Architecture

Networking : InfiniBand network - The XT4 and XT5 parts of Jaguar are combined into a single system using an InfiniBand

network that links each piece to the Spider file system

- The InfiniBand network SION (or scalable I/O network) connects all major NCCS systems.

File System: - Spider, a 10-petabyte Luster-based shared file system, connects to every system in the

ORNL computing complex

- It will serve all NCCS platforms and connect to every internal network.

- Because all simulation data will reside on Spider, file transfers among computers and other systems will be un necessary.

Architecture

Scalable Software: – The Cray XT5 utilizes the Cray Linux Environment™ (CLE)– The Linux environment features a compute kernel which can be configured to

match different workloads– The Cray XT5 system maintains a single root file system across all nodes,

Location & Cost

Location: Oak Ridge National Laboratory (ORNL), United States– ORNL is a multi program science and technology laboratory managed for

the U.S. Department of Energy by UT-Battelle, LLC

Cost: Difficult to find as the computer has been evolved constantly over time.

Bench Mark -Results

Benchmark results: Jaguar, a Cray XT5 system at ORNL, is #2 in the top 500 supercomputers list .

It achieves a maximal Lin pack performance of 1.059 PFlop/s with a theoretical peak performance of 1.3814 PFlop/s

Cores Rmax (GFlops) Rpeak(GFlops) Power

150152 1059.00 1381.40 6950.60

Location & Manufacturer Machines in top 500

Location in Top 500 2

Machines in Top 10 4

Machines in Top 50 10

Manufacturer Including No of Machines in Top 500 22

Rank Computer

2 Jaguar –Cray XT5 Quadcore 2.3 Ghz Cray Inc

7 NERSC/LBNL Franklin-CrayXT4Quadcore 2.3Ghz Cray Inc

8 Jaguar –Cray XT4 Quadcore 2.1 Ghz Cray Inc

9 RedStorm- sandia/ Cray RedStorm,3/4,2.4/2.2 Ghz dual/ Quadcore Cray Inc

Operating Systems & Supporting Languages

The Compute Node Linux (CNL): is the next generation of lightweight kernels for compute nodes on the CRAY XT3 and Cray XT4 computer systems. The CNL operating system provides a runtime environment based on the SUSE SLES distribution

Supported languages:

Compilers C,C++,Fortran

Math Libraries BLAS, FFTs, LAPACK, ScaLAPACK, SuperLU

Scientific Libraries Cray scientific Library

Typical Applications

Energy Assurance :Petascale leadership systems will arm sci entists with better data to aggressively pursue renewable energy sources and more efficiently and safely exploit conven tional energy options

Climate :The potential of petascale simulations to clarify the evolution of the climate system is difficult to overstate. Nearly every aspect of climate simulation stands to benefit from the upcoming petascale

Materials: In materials science, innovations made possible by petascale computing promise to bolster American competitiveness in multiple technological sectors

Biology: Biologists will use petaflop computers for detailed studies showing how proteins carry out crucial tasks. Simulations of larger structures at longer timescales and finer resolution will allow exploration of protein structure and behavior

Interesting Sibling

Kraken at UT.

It’s a Cray XT4 but as of Feb 2 has been replaced by an XT5 and will probably be jumping up higher on the list of the top 500.

It is also located very near Jaguar as it is at the University of Tennessee and is part of a partnership with ORNL the owner of the Jaguar

Interesting About The Assigned Machine

Jaguar is actually the most powerful supercomputer in the world for open scientific use.

Its peak performance is more than 119 trillion calculations per second (119 teraflops).

To support its extraordinary concentration of computing power, the NCCS has put in place high-speed fiber-optic networks to expedite data movement

Jaquar At a Glance

Cray XT Top500 rank: 2 1.64-petaflops peak theoretical performance 1.059 + 0.205-petaflops actual performance on HPL benchmark program 182,000 processing cores AMD quad-core Opteron TM 2.3 gigahertz processors InfiniBand network Cray SeaStar network interface and router 362 terabytes of memory 578 terabytes per second of memory bandwidth 284 gigabytes per second of input/output bandwidth 10-gigabyte-per-second connections to ESnet and Internet 2 networks High-Performance Storage System scales to store increasing amounts of data Spider, a 10-petabyte Luster-based shared file system, connects to every system in the ORNL

computing complex disk subsystem transfers data at greater than 200 gigabytes per second

Questions??