unc cause chris maher ibm high performance computing hpc

Technical Computing / High Performance ComputingUniversity Perspective

Chris Maher, IBM Vice President HPC [email protected]

Agenda

• Industry Expansion of HPC and Technical Computing

• Universities and HPC

• Taming the Growing Data Explosion

• Why HPC Cloud

• Technical Computing Systems

• University Resources and University Examples

• Putting it all together with Datatrend Technologies

Smarter traffic systems

Smarter water mgmt

Smarter energy grids

Smarter healthcare

Smarter food systems

Intelligent oil field technologies

Smarter regions

Smarter weather

Smarter countries

Smarter supply chains

Smarter cities

Smarter retail

...and this is driving a new economic climate.

The world is getting smarter – more instrumented, interconnected, intelligent

Technical computing is being applied to a broader set of industries enabling more areas for collaborative work at universities

Large Industrial sector applications

HPC

“en

try

cost

s”–

inve

stm

ent

and

skill

nee

ded

Timeline

High

Low

SupercomputersScience, Research &

Government


Government

Broad adoption across a variety of industries as technology becomes affordable & pervasiveUsage driven by modeling, simulation, predictive analysis workloadsDelivered via Clusters, Grids and Cloud


Government

Automotive Aerospace Engineering

PetroleumElectronic

Design Automation

Life SciencesFinancial Services Digital Media

ExascaleThe Next Grand

Challenge

1990’s 2010’s

HPC 1.0 HPC 1.5

HPC 2.0

HPC

Pro

blem

Dom

ains

Add

ress

ed

“PhysicsDriven”

“Data Driven”

“Applied”Technical

Computing

“Mainstream”Technical Computing

Research Engineering/Simulations Analysis/Big Data/Cloud deployments

+ +

• Computational analysis• Upstream/downstream processing• Next-generation genomics• Satellite ground stations• Video capture and surveillance• 3-D computer modeling• Social media analysis• Data mining/unstructured

information analysis (Watson-like)• Financial “tick” data analysis• Large-scale real-time CRM

Examples of growth for Technical Computing

6

Industry TrendsThe Data Deluge

• Big data, big data management consuming researchers now• Very large projects have data measured in the 100s of petabytes

Expanding the role of HPC and HPC Cloud on Campus• Myriad of campus needs for both high throughput computing and high

performance (capability) computing using a shared environment• Best practices show cost reduction with central condominium facility

where researchers can contribute their grant money and which serves the larger university community

• HPC makes a university more competitive for grantsExascale computing will be a reality in 2018/9

• Petascale has been delivered(2008)• Large scale is being tackled now• In 2018, will large university installations have a multi petaflop computer?

What will house it?What will be the power requirements?The Power Utilization Efficiency (PUE) of your datacenter is as important as the “green solution” you put in it.

Agenda




• Why HPC Cloud




8

What we are seeing as trends at the University Level

• HPC is growing at a robust CAGR (6.9% according to Tabor)

• HPC is required for a research university to attract faculty.

• VP of Research titles changing to VP of Research and Economic Development acknowledging that joint ventures with companies is a MUST for universities

• Greater partnerships with new industries

• Power, cooling and space are making universities think about central vs. decentralized computing (total cost of ownership)

• Next Generation Sequencing and in silico Biology, High Energy Physics, Search and Surveillance, Nanotechnology, Analytics are key workload areas Use of accelerators (for example nVidia)

• HPC in the CLOUD becoming more relevant

9

Sample Workload/ISVs• In silico Biology– Amber, NAMD, BLAST, FAST/A, HMMer, NGS.

• Computational Chemistry– Gaussian, Jaguar, VASP, MOE, Open Eye, Accelrys Material Studio.

• Matlab– used in most medical school settings

• Statistics– IBM SPSS or SAS

• High Energy Physics– workload from Cern LHC– Monte Carlo techniques

• Quantum Physics– Quantum Chromodynamics (QCD)

• Analytics– COGNOS, Big Insights, InfoSphere Streams (large data being generated by the Square Kilometer Array), CERN, and Smarter Planet initiatives.

Agenda




• Why HPC Cloud




All these and more are contributing to the Growing Data Explosion

1980 1990 2000 2010

Megabytes

Gigabytes

Terabytes

Petabytes

Kilobytes

Half a Zettabyte of Annual IP Traffic by 2013 (a trillion gigabytes; 1 followed by 21 zeroes)

MRIs will generate a Petabyteof data in 2010

Text messages generate 400TB of data per day (US)

“IDC’s annual Digital Universe… indicates that over the next 10 years, data generation is expected to increase a staggering 44x” *

“IDC’s annual Digital Universe… indicates that over the next 10 years, data generation is expected to increase a staggering 44x” *

* The Ripple Effect of Seagate's 3TB External HDDJuly 06, 2010 - IDC Link

Data Centric Thinking

inputoutput

Data lives on disk and tapeMove data to CPU as neededDeep Storage Hierarchy

Data becomes Center of AttentionWe are never certain exactly where it is

•Although we can ask Abstraction allows for specializationAbstraction allows for Storage Evolution

DataData

Today’s Compute-Focused Model Future Data-Focused Model

Top Storage/Data Issues

Source: 2010 InfoPro Survey

0% 10% 20% 30% 40% 50% 60% 70%

Storage Provisioning

Archiving / Archive Mgmt

Performance Problems

Managing Complexity

Backup Administration

Managing Costs

Forecasting / Reporting

Managing Storage Growth

At look at Next Generation Sequencing

• Growth is 10x YTY

Sequencers can generate 2TB+ of final data per week/sequencer. Processing the data is compute intensive; the data storage is PBs per medium sized institution. For

example, BGI in China currently has 10 PB.

Managing the data explosion from NGS

Average Storage Cost Trends

Source: Disk - Industry Analysts, Tape - IBM

2003 2004 2005 2006 2007 2008 2009 2010 2011$0.01

$50.00

$1.00

$10.00

$/G

B

Industry Disk HC LC Disk Average Tape

Projected Storage Prices

Use of Tape Technology

• Virtual Tape + deduplication growing technology for secondary data– Key value – time to restore

– Use compute to reduce hardware costs

– Add HA clustering and remote site replication

• Tape used as the “Store” in large HPC configurations– Files required for job staged from tape to disk ‘cache’ by a data mover (HPSS)

– Results written to disk, then destaged back to Tape

• Hybrid disk and tape use for archive applications – large capacity, long term retention– Metadata on Disk, Content on Tape

– Lowest cost storage

– Lowest power consumption

– Most space efficient

– Long life media• Specialty Niche – removable media interchange

Any statements or images regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

Agenda




• Why HPC Cloud




High Performance Computing continues to evolve

HPCCluster

HPCGrid

HPCCloud

SingleSystem

Why are Universities exploring Clouds?• Cost Efficiency

– Consolidation and sharing of infrastructure

– Leverage resource pooling for centralized policy administration• System/Configuration Management Policies

• Energy-related Policies

• Security-related Policies

• User-related Policies

• Flexibility– End-user self-service cloud portal enablement

– Exploit advanced automation to free technical resources for higher value work

– Enhanced access to specialized resources (e.g. GPUs)

– Dynamic on demand provisioning and scaling

20

IBM’s new HPC Cloud addresses the specific intersection of high performance computing and cloud computing

CloudBurst

ISDM, TPM, TSAM

Virtual Machine Provisioning

Intelligent Cluster

HPC Management Suite

Bare Metal & VM Provisioning

System x, BladeCenter

System p, System z

SAN, NAS

1Gigabit Ethernet

iDataPlex

BlueGene, System p 775

GPFS, SONAS

InfiBand, 10-40 GbE

General PurposeComputing

High PerformanceComputing

CloudComputing

Stand-aloneComputing

IBM’s HPC Cloud is being deployedat clients such as the phase 2 pilot at NTU

Needs include–Batch job scheduling – several unique schedulers and runtime libraries–Parallel application development and debugging, scaling and tuning–Parallel data access–Low latency, high bandwidth interconnects

Environment Characteristics

Full and direct access to system resources (bare metal pooling)

Efficient virtualization, where applicable (KVM and VMWare pooling)

Diverse technologies – Windows & Linux– Diverse cluster

managers

Agenda




• Why HPC Cloud




New Era of Technical Computing SystemsHardware + Software + Services = Systems and Solutions

Purpose built, optimized offerings for Supercomputing- iDataPlex, DCS3700 Storage, TS3500 Tape Library

Full array of standard hardware offerings for Technical Computing- Intel- based IBM blade servers, IBM Rack Servers, x3850X5 SMPs, Integrated Networking Solutions, Storage Products (DCS3700)

Hardware

+ Software- Parallel File Systems - Resource Management

= Systems & SolutionsHPC Cloud Offerings from IBM- IBM HPC Management Suite for Cloud- IBM Engineering Solutions for Cloud: HPC cloud offerings optimized for Electronics, Automotive & Aerospace clients

Intelligent Cluster- IBM Intelligent Cluster solutions: Integrated, optimized w/servers, storage and switches

+ Services

- HPC Cloud Quick Start Implementation Services - Technical Computing Services Offering Portfolio:Full range of customizable services to help clients design, develop, integrate, optimize, validate and deploy comprehensive solutions to address their Technical Computing challenges

- Parallel Application Development Tools- Systems Management

ISV Solutions - Partnering with leading ISVs to maximize the value of our joint solutions

IBMResearchInnovation

Agenda




• Why HPC Cloud




26

University Relations (UR) and STG University Alliances

• IBM University Relations: Resources for educators, researchers, staff and students

–https://www.ibm.com/developerworks/university/

• IBM Systems and Technolgy Group University Alliances

–Responsible for guiding STG research and collaboration with universities

–Enables new opportunities for deploying IBM systems and solutions at universities

–RTP Center for Advanced Studies headed by Dr. Andrew Rindos, [email protected]

University Relations Teaming Examples

IBM, Imperial College, government & industry partners to invest ~ $81M for Digital City Research project to develop & implement the next generation infrastructure, systems & services to modernize cities (i.e. make cities smarter) Goals include connecting citizens to real time intelligence,

bring value through smart decision making, generating commercial, creative and social opportunities to enhance quality of life

In addition catalyse the next generation of digital services inhealthcare, energy, transportation and creative industries.

Proposed Collaboration w/ Imperial College London : Digital City Lab (DCL)

SUR Project : Smarter Infrastructure Lab for Smarter Cities

MOU signed creating SI Lab collaboration taking a system of systems view of a university managed like a smart city using sensors, data, and analyticsGoals include development of fixed & mobile infrastructure

analytics technologies & solutions for a smarter city (e.g. smart water, waste, buildings, energy, transportation, healthcare, environment, etc.). Also to provide a showcase for client visits & demonstrations of IBM Smarter Cities technologies

Future proposal to have lab become part of larger Pennsylvania Smarter Infrastructure Incubator Initiative

SUR Project : Smarter City Solutions For China

Tongji University signed a Smarter City Initiative collaboration agreement aimed at building and providing integrated IBM Smarter City solutions for ChinaGoal of collaboration is to overcome the current silo decision

making by different government ministries and to provide a city mayor and other decision makers an integrated Smarter City framework, solution package, and a real life city modelToJU will partner with IBM on Smart City projects based on

ToJU's urban planning work in several cities (Shanghai Pudong, Hangzhou & Yiwu )

IBM & Swansea University (Wales UK)Partner for Economic Dev’t

The vision for the collaboration is economic dev’t & job creation ; build state of the art HPC capability across the universities in Wales to provide enabling technology that delivers research innovation, high level skills dev’t and transformational ICT for economic benefit. Wales infrastructure is linked to the larger UKQCD consortium (19 UK Particle Physicists and Computing Scientists from 19 UK universities) that share computing resourcesSeeded w/ SUR award which drove revenue of $2.4M in 2010

University of Victoria

Industries: Higher EducationURL: http://www.uvic.ca/

Upgrading old hardware while significantly boosting performance and research capabilities

The need:Requirement to replace original circa 1999 UNIX machines

Principal Investigator’s key requirement was research collaboration

Physics Department main requirement was only for FLOPs / $$ for performance was key

Solution:A research capability computing facility of 380 iDataplex Nodes (2x Intel x5650’s 1:1 InfiniBand)

A performance/capacity cluster of iDataplex nodes (2x Intel x5650’s 2x 1Gig)

High Performance Focused on Benchmark results (disk I/O and Jitter performance)

The benefits:Research time cut by 50%

Power and cooling was 40% less while gaining 30% throughput benefits

St. Jude’s Children’s Research HospitalSimplifies storage management to meet researchers needs

Business challenge:St. Jude’s Children’s Research Hospital , based in Memphis, TN, is a leading pediatric treatment and research facility focused on children's catastrophic diseases. The mission of St. Jude Children’s Research Hospital is to advance cures, and means of prevention, for pediatric catastrophic diseases through research and treatment. Their current NAS solution was not scalable to meet researchers needs and tiering of data was becoming an arduous process.

Solution:St. Jude’s viewed IBM as a thought leader is storage virtualization. IBM SONAS was deployed to provide a single, scalable namespace for all researchers. IBM Tivoli Storage Management and Hierarchical Storage Management automated tiering and backup of all data allow IT to focus on the needs of research. St Jude’s was able to simplify their storage management while providing the ability to meet researchers needs. Benefits:

A single, scalable, name space for all users that can be enhanced and upgraded with no down time

Avoided the expense, time and risk of manually moving data to improve reliability and access to the information

Able to adjust to dynamic business requirements, reduce maintenance, lower integration costs, and seamlessly bridge to new technologies

Solution components:IBM SONASTivoli TSM & HSMIBM ProtecTIERDS50003 years hardware & software maintenanceIBM Global Technology Services

East Carolina UniversityAdvancing Life Sciences Research with an IBM Intelligent Clustersolution based on IBM BladeCenter technologies

The need:Without a dedicated supercomputer capable of running massively parallel computational tasks, the Biology department at ECU could not run models as quickly as it needed. Researchers were frustrated by slow performance, and scientists were forced to spend time resolving IT issues.

The solution:ECU selected an IBM® Intelligent Cluster™ solution based on IBM BladeCenter® servers powered by Intel® Xeon® 5650 processors, working with Datatrend Technologies Inc. to deploy it. The solution was delivered as a preintegrated, pretested platform for high-performance computing, and includes remote management from Gridcore.

The benefit:ECU can now run up to ten typical computational tasks in parallelUsing all 100 Intel processor cores, models that might previously have taken a day are completed in a matter of minutesEfficient, easy-to-scale solution opens up new research possibilities for the future.

“There are some analyses that make use of all 96 cores… Previously, a task of this magnitude might have taken a full day of computation to complete. With the IBM Intelligent Cluster, it takes just minutes.”

—Professor Jason Bond, East Carolina University

Solution components:IBM® Intelligent Cluster™IBM BladeCenter® HS22

XSP03265-USEN-00

Agenda




• Why HPC Cloud

• Technical Computing Systems and University Resources

• University Examples


Putting it All Together… Literally with Datatrend Technolgies

Doug Beary, Technical Account ExecutiveDatatrend Technologies919-961-4777, [email protected]

High Performance Computing PlatformsDatatrend Technologies can help put it All Together –

Providing a Solution

• HPC Clusters– Compute, Interconnect & Storage

• Workload Fit– Distributed Memory (MPI)

• Scale Out: iDataplex, Blades, Rack

– Shared Memory (SMP)• Large Scale SMP: ScaleMP, NumaScale

– Hybrid Systems

• Management System– xCAT, MOAB, eGompute

33

HPC Clusters

34

Platform Optimization– Optimize Processor Selection

• Performance/$• Performance/W

– Optimize Form Factor– Optimize Delivery & Installation

Typical 84 Node Cluster• 100 to 1000 boxes• Optimize Form Factor• Optimize Delivery &

Installation

Datatrend Solution• One Item

Top Components• Fastest CPUs• Flexible Interconnect Choices

Fabric, Card, Switch, Cabling• Unmatched Storage to Meet

Any CapacityAny Performance

Workload Fit• Distributed Memory

– Most Common Cluster– Under Desk to PetaFlops– 100s to 100,000+ Cores– Many OS Images

• Shared Memory– Growing Demand– Dozens to 1000’s of Cores– 64+TB Memory– One OS

• Hybrid– Do Both on One platform!!

35

Hyper-Scale ClusterUp to: 126 Nodes, 1512 cores, 23.6TB • Simple Scaling

• 126 Nodes in 2 Racks• Full Blade Chassis: 9

• Bandwidth:• *Bi-sectional bandwidth:

64%• Largest non-blocking

Island: 14 nodes

• Low Latency• Max. 200ns

Distributed Memory, Shared Memory or BOTH!!

36