Program Systems Institute Russian Academy of Sciences

11

Program Systems InstituteProgram Systems InstituteResearch Activities OverviewResearch Activities Overview

Extended Version Extended Version

Program Systems InstituteProgram Systems InstituteResearch Activities OverviewResearch Activities Overview

Extended Version Extended Version

Alexander Moskovsky,Alexander Moskovsky,Program Systems Institute RAS, Program Systems Institute RAS,

Russia, Pereslavl-ZalesskyRussia, Pereslavl-Zalessky

Program Systems Institute, Russian academy of ScienceProgram Systems Institute, Russian academy of Science

22

Institute Institute StructureStructure Institute Institute StructureStructure ~200 research staff~200 research staff Artificial Intelligence Research CentreArtificial Intelligence Research Centre Medical Informatics Research CentreMedical Informatics Research Centre Research Center for Research Center for

Multiprocessor SystemsMultiprocessor Systems System Analysis Research CentreSystem Analysis Research Centre Control Processes Research CentreControl Processes Research Centre Scientific and educational centreScientific and educational centre::

International Children’s Computer International Children’s Computer Centre named after A. AylamazyanCentre named after A. Aylamazyan

Kindergarten and Primary school Kindergarten and Primary school “Pochemuchka”“Pochemuchka”

Program Systems Institute, Russian academy of ScienceProgram Systems Institute, Russian academy of Science

““SKIF-GRID” PROJECT TIMELINESKIF-GRID” PROJECT TIMELINE““SKIF-GRID” PROJECT TIMELINESKIF-GRID” PROJECT TIMELINE

PSI RAS PSI RAS is a lead organizarion in Russian Federationis a lead organizarion in Russian Federation

1.1. 2000-2004 - 2000-2004 - SKIF project, SKIF K-1000 is #98 in SKIF project, SKIF K-1000 is #98 in Top500Top500

2.2. JuneJune 2004 2004 – – first proposal filedfirst proposal filed for “SKIF-GRID” for “SKIF-GRID” projectproject

3.3. MarchMarch 2007 2007 – – approved by Governmentapproved by Government

4.4. MarchMarch 200 20088 - - SKIF-MSU supercomputer deployed SKIF-MSU supercomputer deployed (#36 in June 08 Top 500)(#36 in June 08 Top 500)

5.5. May 2008May 2008 - “SKIF-Testbed” federation created. - “SKIF-Testbed” federation created.

6.6. March 2009March 2009 – alliance agreement signed for SKIF – alliance agreement signed for SKIF series 4 developmentseries 4 development

Program Systems Institute, Russian academy of ScienceProgram Systems Institute, Russian academy of Science

SKIF MSU SKIF MSU SKIF MSU SKIF MSU

Theoretical peak Theoretical peak performance performance 60 TFlops60 TFlops

47 TFlops 47 TFlops Linpack Linpack Advanced clustering Advanced clustering

solutionssolutions:: diskless diskless

computational computational nodesnodes

Original blade Original blade designdesign

Parameter Value

CPU architecture: x86-64

CPU model: Intel XEON E5472 3,0 GHz (4-cores)

Nodes (dual CPU) 625

CPU cores total 5 000

Interconnect Infiniband DDR,

Fat Tree

Program Systems Institute, Russian academy of ScienceProgram Systems Institute, Russian academy of Science

PROJECT ORGANIZATION: 2009-PROJECT ORGANIZATION: 2009-20102010

PROJECT ORGANIZATION: 2009-PROJECT ORGANIZATION: 2009-20102010

Project directionsProject directions1.1. Grid technologyGrid technology

2.2. SupercomputersSupercomputers SWSW HWHW

3.3. SecuritySecurity

4.4. Pilot projects – applications of Pilot projects – applications of HPC and grid technologyHPC and grid technology

TotallyTotally more than 20 more than 20 organizations in Russiaorganizations in Russia

Program Systems Institute, Russian academy of ScienceProgram Systems Institute, Russian academy of Science

SKIF-Aurora SKIF-Aurora SKIF-Aurora SKIF-Aurora Fulfillment of original SKIF-GRID project Fulfillment of original SKIF-GRID project

goals (back to 2004)goals (back to 2004) Highest density of performanceHighest density of performance

(increase number of FLOPS per 1U)(increase number of FLOPS per 1U) Interconnect:Interconnect: we need better scalability, we need better scalability,

bandwidth and latency that it’s provided by bandwidth and latency that it’s provided by best available solutions (eg. Infiniband QDR)best available solutions (eg. Infiniband QDR)

New approach to monitoring and New approach to monitoring and management management of the supercomputerof the supercomputer

CPUs and acceleratorsCPUs and accelerators in computational in computational nodes of the supercomputernodes of the supercomputer

Program Systems Institute, Russian academy of ScienceProgram Systems Institute, Russian academy of Science

SKIF Aurora OverviewSKIF Aurora OverviewSKIF Aurora OverviewSKIF Aurora Overview Nodes: Nodes:

dual-CPU boards (Nehalem)dual-CPU boards (Nehalem) FPGA for 3D torus interconnectFPGA for 3D torus interconnect Infiniband QDR integratedInfiniband QDR integrated liquid coolingliquid cooling

Chassis: Chassis: 32 nodes32 nodes Integrated management and Integrated management and

monitoring monitoring Integrated Integrated Infiniband QDRInfiniband QDR Cables integrated on backplaneCables integrated on backplane

Rack:Rack: Up to 8 chassis (512 CPU)Up to 8 chassis (512 CPU) No moving parts No moving parts

Designed by an alliance of Eurotech, PSI RAS Designed by an alliance of Eurotech, PSI RAS and RSC SKIF with support by Inteland RSC SKIF with support by Intel

chassis

RackDatacenter

Program Systems Institute, Russian academy of ScienceProgram Systems Institute, Russian academy of Science

SKIF-4 nodeSKIF-4 nodeSKIF-4 nodeSKIF-4 node

Program Systems Institute, Russian academy of ScienceProgram Systems Institute, Russian academy of Science

Subsidiary Interconnect, Infiniband

Our proposalOur proposalOur proposalOur proposal

Based on SKIF-Aurora architecture:Based on SKIF-Aurora architecture: Interconnect w. 3-D torus topologyInterconnect w. 3-D torus topology FPGA on network levelFPGA on network level

99

System Interconnect, 3D-torus

FPGA FPGA FPGA FPGA...

CPU CPU CPU CPUstandard part

non-standard part

Program Systems Institute, Russian academy of ScienceProgram Systems Institute, Russian academy of Science

Our proposal-2Our proposal-2Our proposal-2Our proposal-2 FPGA-basedFPGA-based Integral support of Integral support of

programming programming model by hardwaremodel by hardware

Customizable hw Customizable hw platformplatform

Project idea: Project idea: optimized runtime optimized runtime support for FP-support for FP-based languagesbased languages

1010