ga techsusthpc patterson
TRANSCRIPT
1
Taking HPC beyond energy efficiency;sustainability, the new measure of success
Michael K Patterson, PhD, PE, DCEP
Eco-Technology Program Office
2
Why HPC is different
• Procurement and refresh timing
• Workload and Utilization
• HPC and infrastructure link
• Best opportunity for Energy Reuse
• Minimal UPS
• Availability needs lower than Enterprise / Financial / IPDC
• Performance measurements higher than Enterprise / Financial / IPDC
3
PUE – simple and effective
PUE is defined in terms of total annual energy and total annual IT energy, allowing a more valid site-to-site
comparison
4
PUEs: Reported and Calculated
PUE
EPA Energy Star Average 1.91
Intel Jones Farm, Hillsboro 1.41
T-Systems & Intel DC2020 Test Lab, Munich 1.24
Google 1.16
Leibniz Supercomputing Centre (LRZ) 1.15
National Center for Atmospheric Research (NCAR) 1.10
Yahoo, Lockport 1.08
Facebook, Prineville 1.07
National Renewable Energy Laboratory (NREL) 1.06
5
PUEs: Reported and Calculated
PUE
EPA Energy Star Average 1.91
Intel Jones Farm, Hillsboro 1.41
T-Systems & Intel DC2020 Test Lab, Munich 1.24
Google 1.16
Leibniz Supercomputing Centre (LRZ) 1.15
National Center for Atmospheric Research (NCAR) 1.10
Yahoo, Lockport 1.08
Facebook, Prineville 1.07
National Renewable Energy Laboratory (NREL) 1.06
It’s all about the “1”!
1.06Focus on driving the 0.06 down? Or work with the 1.0? How?
� Energy reuse� Continued improvement in compute
performance
6
Tick Tock of Energy EfficiencyMoore’s Law and IA Innovation
Source: SPECpower_ssj2008* 2 socket results from SPEC.org as of August 2010Performance and power consumption results are based on certain tests measured on specific computer systems. Any difference in system hardware, software or configuration will affect actual performance. Configurations: Two-socket Systems, Test Results for SPECpower_ssj2008, Testing by Hewlett-Packard. For more information go to http://www.intel.com/performance
Source: SPECpower_ssj2008* 2 socket results from SPEC.org as of August 2010
Double Efficiency every 16 months (67% CAGR)
0
50
100
150
200
250
300
350
400
450
0 200,000 400,000 600,000 800,000 1,000,000
Po
we
r (W
)
SSJ Ops
2 Socket Volume-Server Power vs. Performance
2010Intel® Xeon®
X5670
2009Intel® Xeon®
X55702008
Intel® Xeon®
E5450
2006Intel®
Xeon®
5160
2004Intel®
Xeon®
3.8GHz
Pow
er: L
ower
is b
ette
r
Performance: Higher is better
Higher Efficiency
65�45nm 45�32nmNehalemCore
TICK TOCK TICK TOCKSandy Bridge
TICK
7
LRZ-MunichIBM/Intel supplying the new LRZ-Munich system. 40C water to the servers. Free cooling 100% of the year. PUE ~ 1.15 Supplier must pay the power bill for the first several years of the system operations.
Power Bill
K- Euros/TF
€2308
€113
€7
Energy cost / Tflop plummeting, butPower bill hitting “political” ceilings
8
Free cooling in Atlanta?
Recall that LRZ is using 40 °°°°C cooling
water
9
“I am re-using waste heat from my
data center on another part of my
site and my PUE is 0.8!”
Ever hear someone say something like:
10
“I am reusing waste heat from my data center
on another part of my site and my PUE is 0.8!”
• While re-using excess energy from the data center can
be a good thing to do, it should not be rolled into PUE.
The definition of PUE does not allow this. PUE is ALWAYS
greater than or equal to 1.0
•But there is a new metric to do this; ERE
11
Energy Reuse Effectiveness
A new energy efficiency metric
Similar to PUE but accounts for reuse energy
PUE and ERE can both provide insight
• Different perspectives on efficiency vs reuse
12
Utility
Cooling
UPS PDU
IT
RejectedEnergy
(a)
(b)(c) (d)
(f)
(e)
ReusedEnergy
(g)
ERE – adds energy reuse to the PUE concept
13
ERE Definition
IT
Reused-IT+Lighting+Power+Cooling =ERE
IT
IT+Lighting+Power+Cooling =PUE
Energy IT
Energy Total =PUE
Energy IT
Energy Reused -Energy Total =ERE
14
ERE Alternate Development
PUE ERF)1(IT
Reused-IT+Light+Pwr+Cool =ERE ×−=
Energy Total
Energy Reuse =ERF
Define energy reuse factor (ERF) as:
Then:
PUE ERF)-(1 =ERE ×
And finally:
ERF and PUE are mathematically related, but differ and need to defined and reported clearly.
15
Comparison with PUEOne view of PUE is that is the “tax” or burden in energy costs you must pay above the IT load to run the Data Center; ERE allows the same vision
PUE = 1.0 means 100% of the energy you bring in to the data center goes to the IT
ERE = 1.0 means you only need to bring into the site an amount equal to 100% of the IT energy to support the Data Center
We need both!
Case 1 Case 2
PUE = 2.0 PUE = 1.2
ERF= 0.55 ERF=0.25
ERE = 0.9 ERE=0.9
Case 1 focus on PUE, Case 2 focus on ERF
16
Towards the Net-Zero Data
Center: Development and
Application of an Energy
Reuse Metric
Technical Paper presented last
June: ASHRAE Summer Meeting,
Montreal
17
PUE & ERE resorted….
PUE Energy Reuse
EPA Energy Star Average 1.91
Intel Jones Farm, Hillsboro 1.41
T-Systems & Intel DC2020 Test Lab, Munich 1.24
Google 1.16
National Center for Atmospheric Research (NCAR) 1.10
Yahoo, Lockport 1.08
Facebook, Prineville 1.07
Leibniz Supercomputing Centre (LRZ) 1.15 � ERE <1.0
National Renewable Energy Laboratory (NREL) 1.06 � ERE <1.0
18
Two new metrics for Data Center sustainability
Published by The Green Grid
Development of the Metrics will give better focus on Data Center sustainability
Water and Carbon– increasing focus on sustainability
19
New Metrics… all in the PUE family
��� =���� ����������
��������
��� =�������������� ��������������������������
��������
��� =���� �����������
��������
���!"#$%& =���� ��� ��������������� + ���� �����������
��������
WUE ~ Liters/kWh CUE ~ kgCO2eq/kWh
20
Why two WUE’s?
��� =���� �����������
��������
���!"#$%& =���� ��� ��������������� + ���� �����������
��������
21
The Sustainability Triangle – an example
Water
Technology choices are typically a trade-off
22
Exascale by 2019-2020Business-as-usual (2X perf/watt
every 16 months) ����140 MWTarget ����20 MW
23
Exascale Challenges
� 140 MW just won’t work (~$140M / year to operate)
� Target is 20 MW!
� Even with Moore’s Law like improvements we can’t get there (20 MW) by 2019-2020
� Government / Industry / Academia partnerships must happen to get us there
� If we don’t, HPC growth will stall!
� If we do, amazing things can happen…..
24
TECHNOLOGY AND THE ENVIRONMENT
Use Computing to Improve Energy Savings Outside
Information and Communications Technology
98%The Big Opportunity
Drive Computing to Be More Energy Efficient
~2%Opportunity
Does HPC have a Carbon ROI?Should we explore it?
~2%
25
Carbon Accounting
We will talk about a lot of metrics but here is one that we might want to develop…
���()$*"+ =��������,��� ���������
����������������
26
()$*"+
27ORNL ReView “Supercomputers Help Model Cars in Collisions”
()$*"+
28
Opportunities for the Sustainable HPC Center� HPC is NOT like your typical data center
� Harvest the differences
� Focus on the 1.0!
� Workload
� Architecture
� Carbon ROI
� Exascale R&D
� Beyond Energy Efficiency to Sustainability
� Sustainability Triangles
� Metrics – ERE, CUE, & WUE
29
Thank You! Questions?