hpc trends for 2013 · 2020. 1. 14. · class 2011 2012 change growth servers 9,915 10,381 466 4...

HPC Trends for 2013

Addison Snell

addison@intersect360.com

New at Intersect360 Research

• HPC500 user organization, www.hpc500.com

– Goal: 500 users worldwide, demographically

representative of industry (verticals, geos, budgets)

– Over 100 end-user organizations signed up so far

– Free access to research

– First member call scheduled for April 4

• Hired Michael Feldman to analyst team

• Report highlight overviews in slide format

• Newsletter announcing research, articles,

podcasts, etc.: Sign up at intersect360.com

Topics for Discussion

• Forecast highlights

• Shifts in budget allocations

• Processor architectures and their implications

– Memory

– Programming models

– Efficiency

• Big Data and its potential for HPC

• What’s cloud got to do with it

Total WW HPC Revenue

Table 1: 2012 vs. 2011 Comparison – Total HPC Market Revenue

($, millions) by Product Class

Product Class 2011 2012 Change Growth

Servers 9,915 10,381 466 4.7%

Storage 3,832 4,837 1,005 26.2%

Services 3,180 3,328 147 4.6%

Software 6,456 6,651 195 3.0%

Networks 1,987 2,121 134 6.7%

Other 1,786 1,842 56 3.1%

Total 27,156 29,159 2,003 7.4% Source: Intersect360 Research, 2013

Storage revenue was

significantly affected

by flooding in

Southeast Asia in the

second half of 2011,

causing a significant

amount of storage-

related revenue to be

deferred from late

2011 and recognized

in early 2012.

WW HPC Forecast

HPC Budget Distribution by Year

• ~$29B total market implies

~$44B total budget

• Hardware declined every

year until sudden rebound

in 2012

• Facilities increased every

year until reversing in 2012

• Public cloud is a small part

of the market

From HPC User Site Census

The primary challenges for users are:

• How to plan the balance between processors per node,

cores per processor, memory per node, I/O and

interconnect on node, total nodes, etc.

• How to adapt applications for node parallelism and on-

chip (i.e., multi-core) parallelism.

• How to organize the overall job mix. Smaller nodes may

be a better fit for processing large numbers of small jobs

or large set of jobs with a broad range of requirements.

Larger nodes may work best with a job mix skewed to

larger problems.

Memory Configuration

Accelerators (Mostly NVIDIA GPUs)

Challenges of Architecture Trends

• Power consumption

• Cost of memory

• New models of parallelization

• Languages and programming models, especially

for accelerated solutions

• System efficiency

• Personnel for administration, optimization,

programming services, etc.

Technical vs. Enterprise Computing

Technical Computing

• Top-line missions:

– Find the oil

– Design the minivan

– Cure the disease

• Driven by

price/performance

• Fast adoption of new

technologies, algorithms,

and approaches

Enterprise Computing

• Keeps business running

– Communicate/collaborate

– Market and sell the product

– Accounting, HR, finance, …

• Driven by RAS: reliability,

availability, serviceability

• Slow adoption of new

technologies, algorithms,

and approaches

Where Big Data Comes From

• “Big Data” is not a specific application type, but

rather a trend – or even a collection of trends –

spanning multiple application types

• Data growing in multiple ways:

– More data (volume of data)

– More types of data (variety of data)

– Faster ingest of data (velocity of data)

– Accessibility of data (internet, instrumentation, …)

• Exceeds organizational ability to manage data or

make competitive decisions based on it

Different Types of Big Data

• “Big” in Big Data is a relative term, like “High” in

High Performance Computing, not absolute TB

or IOPS

• Different types of challenges:

– Large files

– Large numbers of files

– Many users of files (concurrent access, copies)

– Fast rate of ingest

– Long or short lifespan of data

• Implication: If data, then value.

Scaling an Enterprise Data

Infrastructure

Parallel file systems

I/O interconnects

Operating systems

Cloud

Important Insights on Big Data

1. It is much broader than Hadoop – many

different types of users and applications.

2. Money is being spent on it now – often 25% of

the annual IT budget.

3. Performance matters – even enterprise users

are buying based on performance.

Big Data trends will lead to the adoption

of HPC technologies in more areas.

Percent Rating Cloud Barrier

“Significant”

Data movement and

security are “top” barriers,

but there is a long list

• From special study on

HPC and cloud, 2011

• N = 139 – 144

Cloud + Big Data:

When Trends Collide

• Cloud is a major business computing trend. Big Data

is a major business computing trend. Therefore …

• But the barriers to Big Data in cloud are the same as

HPC in cloud (security, data movement, etc.)

• Not as simple as offloading everything to Amazon

• If cloud is a priority, invest in management software

to coordinate workloads across public and private

I like sushi. I like ice cream.

Therefore I like sushi-flavored ice cream.

Conclusions for HPC in 2013

• Users have options for breakout performance,

but they all require effort

– Users are evaluating multi-core, accelerators, cloud,

programming models, etc., but not committing yet.

– What solutions will be efficient?

– What are the software and human skills required?

• There is an opportunity to expand the use of

HPC technology

• Big Data is a bigger near-term opportunity than

“Missing Middle” for most technologies

HPC Trends for 2013

Addison Snell

addison@intersect360.com