hpc market update from idc
Post on 13-Aug-2015
130 Views
Preview:
TRANSCRIPT
IDC HPC Update at ISC’15
Earl Joseph – ejoseph@idc.comBob Sorensen– rsorensen@idc.com
Lloyd Cohen – lcohen@idc.comSteve Conway – sconway@idc.com
IDC’s HPC Team
Earl Joseph
IDC HPC research studies, HPC
User Forum
Steve Conway
Strategic consulting, HPC User
Forum, market trends, Big Data
Bob Sorensen
Strategic research projects,
government studies and
international analysis
Lloyd Cohen
HPC data and workstations
Mike Thorp and Kurt Gantrish
Government account support
and special projects
Mary Rolph
HPC User Forum conference
planning and logistics
About IDC: IDC HPC Activities
• Track all HPC servers sold each quarter
• 4 HPC User Forum meetings each year
• Publish 85 plus research reports each year
• Visit all major supercomputer sites & write reports
• Assist in collaborations between buyers/users and vendors
• Assist governments in HPC plans, strategies and direction
• Assist buyers/users in planning and procurements
• Maintain 5 year forecasts in many areas/topics
• New: Developing a worldwide ROI measurement system
• New: HPDA program
• New: Quarterly tracking of GPUs/accelerators
Agenda
1. HPC Market Update
2. The HPC Market in Europe
3. HPDA Update: Where Big Data Meets HPC
4. IDC ROI Research Update
5. HPC User Forum Update
6. IDC HPC Innovation Program Update
7. Conclusions
Important Dates For Your Calendar
2015 HPC USER FORUM MEETINGS:
• Broomfield, ColoradoLocation: Omni Interlocken
September 8-10, 2015
• Paris -- Institute of Earth Physics of Paris Host: GENCI
October 12-13, 2015
• Munich -- Leibniz RechenzentrumHost: LRZ
October 15-16, 2015
Top Trends in HPC
2014 ended soft, after a number of very strong years
• Top5 system purchases has slowed for ~2 years
• The IBM/Lenovo deal delayed many purchases
• 2014 was basically flat with 2013 ($10.2 billion in 2014
vs. $10.3 billion in 2013)
• 2015 to 2018 are predicted to be healthy growth years
Big data combined with HPC is creating new solutions
• Adding many new users/buyers to the HPC space
Software Issues continue to grow
GPUs & accelerators extend their impact
Top Trends in HPC:
Evolving Issues On Our Minds
Non-x86 processors could alter the landscape
• ARM, Power, others
• Coprocessors, GPUs, FPGAs
China looms large(r)
• Lenovo, growing domestic market, export intentions
• Other Chinese vendors are planning to extend to Europe
Growing influence of the data center in IT food chain
• Impact on HPC technology options
• Perhaps providing new approaches?
HPC in the Cloud Gaining Traction
• How much, how soon?
Top Trends in HPC:
HPC Storage
Storage is the fastest-growing part of the HPC market
HPC storage revenue will grow to record levels
The HPC storage market remains fragmented
The big players are turning their attention to this market
The HPC interconnect market is in transition
Data movement/management is a major pain point
• Multi-year shift away from today’s extreme compute-centrism
• Many buyers still not savvy enough about purchasing storage
HPDA will boost storage budgets
© IDC Visit us at IDC.com and follow us on Twitter: @IDC
IDC HPC Competitive Segments: 2014
Departmental ($250K - $100K)
$3.83B
Divisional ($250K - $500K)
$1.52B
Supercomputers(Over $500K)
$3.15B
Workgroup(under $100K)
$1.72B
HPC
Servers
$10.2B
2014 Revenue Share by Vendor:
Supercomputer Segment ($500K+)
© IDC Visit us at IDC.com and follow us on Twitter: @IDC
HPC Forecasts
• Forecasting a 8.2% yearly growth from 2014 to 2019
• 2019 should reach $15.2 billion
High Performance Data Analysis
Needs HPC resources• High complexity (algorithms)
• High time-criticality
• High variability
• (On premise or in cloud)
Data of all kinds• The 4 V’s: volume, variety, velocity, value
• Structured, unstructured
• Partitionable, non-partitionable
• Regular, irregular patterns
Simulation & analytics
• Search, pattern discovery
• Iterative methods
• Established HPC users + new
commercial users
IDC’s HPDA Program:
Example Research Documents • Supercomputer-Based Analytics Are Starting to Transform Healthcare
• United States Postal Service Turns to HPC for Big Data and Analytics
• High-Performance Data Analytics: Market Status and User Exemplars
• PayPal Says More Fortune 2000 Firms Could Benefit from HPC for Big Data Analytics
• Market Analysis Perspective: Worldwide High-Performance Data Analysis, 2014
• NEC's HPC Vision: Bringing Vector Supercomputing to a Broader Data-Intensive User Base
• Bull Launches an Exascale Program for 2020, But the Vision Extends to Big Data and Beyond
• Total S.A. Retakes the Lead in Energy Sector Supercomputer Race
• Los Alamos National Laboratory: Seven Decades of Computing Leadership
• Summary of HPC and HPDA Technology, Development, and Applications: HPC User Forum, September 15–17, 2014, Seattle, Washington
• HPC Interconnects: New Options Emerging from a Sector in Transition
IDC’s HPDA Program: Four New
Industry/Application Workload Segments Fraud and anomaly detection
• This "horizontal" workload segment centers around identifying harmful or potentially harmful patterns and causes using graph analysis, semantic analysis, or other high performance analytics techniques.
Marketing• This segment covers the use of HPDA to promote products or services, typically
using complex algorithms to discern potential customers' demographics, buying preferences and habits.
Business intelligence• The workload segment uses HPDA to identify opportunities to advance the market
position and competitiveness of businesses, by better understanding themselves, their competitors, and the evolving dynamics of the markets they participate in.
Other Commercial HPDA• This catchall segment includes all commercial HPDA workloads other than the three
just described. • Over time, IDC expects some of these workloads to become significant enough to
split out, i.e. the use of HPDA to manage large IT infrastructures, and Internet-of-Things (IoT) infrastructures.
Note Financial, classified buyers, etc. will continue to be listed under the existing IDC segments
Machine Learning/Deep Learning
Like a game: a goal and rules for reaching it.
Stage 1: Search
• Computer is given the goal and trained in the rules (e.g.,
Jeopardy Watson).
Stage 2: Unknown pattern discovery
• Computer is given the goal and has to discover the rules
(e.g., medical Watson, some x86 clusters)
Stage 3: Cognitive computing/AI
• Computer has to discover the goal and the rules (future).
Machine Learning: Typical Attributes
Mainstream IT Market Serial algorithms
Data scientists acquire
algorithms
Major data reduction before
analytics (sometimes >99%)
Not very time-critical
Public cloud often fine
HPC (HPDA) Market Parallel algorithms
Data scientists create
algorithms
No big data reduction: high-
resolution results needed
Very time-critical
Public cloud often inadequate
Summary: HPDA Market Opportunity
HPDA: simulation + newer high-
performance analytics
• IDC predicts fast growth from a small
starting point
HPC and high-end commercial
analytics are converging
• Algorithmic complexity is the common
denominator
Economically important use cases are
emerging
No single HPC solution is best for all
problems
Three-Year Research Plan
2013 pilot study:
Tested 3 approaches and set the models
Populated the models with 208 cases of scientific innovation
and industrial ROI (it now exceeds 400)
Created the innovation index
Three-year study: sponsored by DOE Science/NNSA
Refine the models to move from association towards causation
Collect thousands of cases:
• Dense collection in the U.S.
• Sparser collection elsewhere*
* DOE has asked IDC to invite other countries to participate
The Financial ROI Models
ROI models that have been developed include:
1. ROI based on revenues generated (similar to GDP)
divided by HPC investment
2. ROI based on profits generated (or costs saved)
divided by HPC investment
3. ROI based on jobs created
Two Innovation Index Scales
How would you rate this innovation compared to all other innovations in this field over the last ten years?
Scale One:
5 = One of the top 2 to 3 innovations in the last decade
4 = One of the top 5 innovations in the last decade
3 = One of the top 10 innovations in the last decade
2 = One of the top 25 innovations in the last decade
1 = One of the top 50 innovations in the last decade
Scale Two:
5 = It had a major impact and is useful to many organizations
4 = A minor innovation that is useful to many organizations
3 = A minor innovation useful only to 2 -3 organizations
2 = A minor innovation useful only to 1 organization
1 = An innovation that is recognized ONLY by experts in the field
HPC User Forum Mission
To Improve the Health of the High
Performance Computing Industry through
Open Discussions, Information-sharing
and Initiatives Involving HPC Users In
Industry, Government and Academia Along
with HPC Vendors and Other Interested
Parties
Steering Committee Members Paul Muzio, City University of New York, Chair Rupak Biswas, NASA Ames, Vice Chair Earl Joseph, Executive Director Swamy Akasapu, General Motors Vijay Agarwala, Penn State University Alex Akkerman, Ford Motor Company Doug Ball, HPC expert (formerly Boeing) Jeff Broughton. NERSC/Lawrence Berkeley National Lab Paul Buerger, Avetec Chris Catherasoo, Caltech Jack Collins, National Cancer Institute Steve Conway, IDC Research Vice President Steve Finn, Cherokee Information Services Merle Giles, NSCA/University of Illinois Keith Gray, British Petroleum James Kasdorf, Pittsburgh Supercomputing Center, Chairman Doug Kothe, Oak Ridge National Laboratory Jysoo Lee, NICN Michael Resch , HLRS, University of Stuttgart Vince Scarafino, Industry Expert Suzy Tichenor, Oak Ridge National Laboratory
50+ Meetings Worldwide Since 2000
Amsterdam, Netherlands (SARA)
Annecy, France
Bangalore, India (Indian Institute
of Technology)
Beijing, China (Chinese Academy
of Sciences)
Bologna, Italy (CINECA)
Bristol, UK
Bruyères-le-Châtel, France
(Teratec)
Canberra, Australia
Geneva, Switzerland (CERN)
Kobe, Japan (RIKEN)
Lausanne, Switzerland (EPFL)
London, UK (Imperial College)
Manchester, UK (Manchester
University)
Melbourne, Australia
New Delhi, India (Indian Institute
of Science)
Seoul, Korea (National Institute of
Supercomputing & Networking)
Stuttgart, Germany (HLRS)
Warsaw, Poland (University of
Warsaw)
Yokohama, Japan (Earth
Simulator Center)
Zurich, Switzerland (ETH Zurich)
United States (many locations)
Recent HPC User Forum Meetings 2014:
• April: Santa Fe, New Mexico
• July: Kobe, Japan (RIKEN)
• September: Seattle, Washington
• October: Canberra & Melbourne, Australia
• October: Stuttgart, Germany (HLRS)
Future HPC User Forum Meetings
2015:• September 8 to 10, Denver, Colorado
• October 12-13. Paris (host: GENCI)
• October 15-16. Munich (host: LRZ)
HPC Innovation Awards for ISC 15
After careful deliberation, IDC’s HPC User Forum Steering Committee, which serves as the judging panel for the IDC HPC Innovation Excellence awards, has decided to make no awards for this round (July 2015).
• Some of the submissions described work that seemed both important and interesting
• However, none of the submission included information that was complete enough to permit fair judging.
• IDC has invited them to apply for the fall award.
• To avoid this issue going forward, IDC will be updating the application process and making the rules/objectives more explicit and easier to follow.
HPC Award Program Process
The Original Innovation Awards Goals and Objectives:
Help to expand the use of HPC by showing real ROI examples:
Expand the “Missing Middle” – SMBs, SMEs, SMSs -- by
providing examples of what can be done with HPC
Show mainstream and leading edge HPC success stories
Create a large database of success stories across many
industries/verticals/disciplines
To help justify investments and show non-users ideas on how
to adopt HPC in their environment
Creating many examples for funding bodies and politicians to
use and better understand the value of HPC to help grow
public interest in expanding HPC investments
For OEMs to demonstrate success stories using their products
HPC Award Program Process
Look for changes to the application process soon:
A new application form better designed to:
More clearly explain the rules and key data required
More clearly explain the type of accomplishments that are
desired
Better accomplish the goals:
• Recognize users and their vendors for major HPC-supported
achievements in industry, government and academia.
• Build a large portfolio of quantified ROI success stories to
strengthen the case for boosting investments in and funding for
HPC.
• Increase HPC adoption by educating the market on its benefits –
using clear and specific ROI examples.
HPC Award Program New Judging Process
A new more timely and interactive multi-step review process designed to extract the best insights from the base of HPC submitters:
• Stage 1: Before preliminary deadline, application is submitted using
new form;
• Stage 2: Application is reviewed by a Steering Committee member
and marked ‘Accept, Reject, or Return for Additional Information’;
• Stage 3: When needed, a Steering Committee member will work
with a submitter to assemble needed information;
• Stage 4: Re-worked application is submitted before final deadline.
• Stage 5: Full Steering Committee membership judges applications
Please standby as IDC will soon be issuing new guidance for future
submissions, starting with SC15 in November.
Why HPC Is Projected To Grow
1. The low half of the market is finally back to a recovery
mode
2. It has become a competitive weapon
3. Governments view HPC leadership as critical
For national pride, but more importantly for economic
prosperity
4. There are very critical HPC issues that need to be
solved
Global warming, alternative energy, safe NE, financial
disaster modeling, healthcare, homeland security, …
And 3D movies and large scale games are fun
But There are Still Major Customer
Pain Points
Software is the #1 roadblock• Better management software is needed
• Parallel software is lacking for most users
Many applications will need a major redesign
Clusters are still hard to use and manage • System management & growing cluster complexity
• Power, cooling and floor space are major issues
• Third party software costs
• Storage and data management are becoming new bottle necks
• Lack of support for heterogeneous environment and accelerators
ROI is becoming a requirement, especially as system costs escalate
… Some good news in that there are new technologies in Big data, accelerators, clouds, etc.
HPC is still expect to be a strong growth market
• Growing recognition of HPC’s strategic value is helping to drive
high-end sales
• Low-end buyers are back into a growth mode
Vendor share positions will shift greatly in 2015
Recognition of HPC’s strategic/economic value will
drive the exascale race
The formative HPDA market will expand
opportunities for vendors
The high ROI will drive major segments (finance,
health care)
China will be a wild card – with the US-China
supercomputer battles
Conclusions
Important Dates For Your Calendar
2015 HPC USER FORUM MEETINGS:
• Broomfield, ColoradoLocation: Omni Interlocken
September 8-10, 2015
• Paris -- Institute of Earth Physics of Paris Host: GENCI
October 12-13, 2015
• Munich -- Leibniz RechenzentrumHost: LRZ
October 15-16, 2015
top related