interpack '09 asme/pacific rim technical conference and exhibition
TRANSCRIPT
![Page 1: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/1.jpg)
© 2009 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice
Unified Thermal and Power Management in Server Enclosures
Niraj Tolia, Zhikui Wang, Manish Marwah, Cullen Bash, Parthasarathy Ranganathan, Xiaoyun Zhu*
*Now at VMware
![Page 2: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/2.jpg)
2005: $26.5 Billion
![Page 3: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/3.jpg)
“The cost of power and cooling is likely to exceed that of hardware…”
- Luiz Barosso, Google
![Page 4: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/4.jpg)
“In the data center, power and cooling costs more than the IT equipment it supports.”
- Christian L. Belady, Microsoft (former HP)
![Page 5: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/5.jpg)
Zephyr• Unified Power and Cooling Management
Contributions• Demonstrate how to do per-blade cooling• Save both cooling (21--30%) + system power (23--29%)
• Without impacting performance
![Page 6: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/6.jpg)
Compact DesignDense Compute and Storage
Blade Servers:
Workloads are Virtualized
![Page 7: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/7.jpg)
Compact DesignDense Compute and Storage
Blade Servers:
Workloads are Virtualized
![Page 8: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/8.jpg)
Web ServerOS
Hardware
Virtualization:
![Page 9: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/9.jpg)
VM MonitorHardware
Virtual Machine
WebServerOS
Virtualization:
![Page 10: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/10.jpg)
Virtual Machine
WebServerOS
OSVM MonitorHardware
VM MonitorHardware
Virtual Machine
App ServerOS
Virtualization:
![Page 11: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/11.jpg)
Virtual Machine
WebServerOS
OSVM MonitorHardware
VM MonitorHardware
Virtual Machine
App ServerOS
Virtualization:
![Page 12: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/12.jpg)
Virtual Machine
WebServerOS
OSVM MonitorHardware
VM MonitorHardware
Virtual Machine
App ServerOS
Virtualization:
![Page 13: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/13.jpg)
CPU P-states BladeEnclosure
PowerUtil
Temp.Zephyr
Min.Power
Tref
Utilref
Models
Fan Speed
VM Migr.Blade On/Off
Power ModelsTemperature Models
![Page 14: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/14.jpg)
0
50
100
150
200
250
0 20 40 60 80 100
Pow
er (
W)
Utilization
Power - P0Power - P4Power Model - P0Power Model - P4
Power Model: Single Blade
![Page 15: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/15.jpg)
Power Model: Single Fan
0 6000 12000 180000
50
100
150
200
Speed (RPM)
Pow
er (W
)
Measured Fan PowerFan Power Model
![Page 16: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/16.jpg)
Unfortunately, temperature models are significantly more complex.
Models don’t existsMany things affect temperatureInsufficient sensors in servers
Zonal Variations
![Page 17: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/17.jpg)
Fan No.
Blad
e N
o.
1 2 3 4 5 6 7 8 9 10
123456789
10111213141516
Zonal Variation: Fan Influence
![Page 18: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/18.jpg)
Temperature Models• Steady State Model−Assumes workloads, ambient environment is constant
• Transient Model−Finer-grained control in real-world environments
![Page 19: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/19.jpg)
Temperature Models• Steady State Model−Assumes workloads, ambient environment is constant
• Transient Model−Finer-grained control in real-world environments
![Page 20: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/20.jpg)
ZephyrBlade
Enclosure
PowerUtil
Temp.
Min.Power
Tref
Utilref
Models
Fan SpeedCPU P-statesVM Migr.
Blade On/Off
![Page 21: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/21.jpg)
ZephyrBlade
Enclosure
PowerUtil
Temp.
Min.Power
Tref
Utilref
Models
Fan SpeedCPU P-statesVM Migr.
Blade On/Off
FC
ECGC
![Page 22: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/22.jpg)
Group Controller (GC)• Goal: Minimize Power(Enclosure)
• Changes state every 10 minutes• Monitors Utilization, Inlet Temperature• Uses two-step optimization−Simulated Annealing for space search−Convex Optimization for Fan Cost + Blade Power Model
![Page 23: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/23.jpg)
Experimental Setup
Storage Array
Controller
Temp and Util. DataVM Migration
Fan Speed Control
16 Blades (4 cores)5 P-States
10 Fans, 64 Xen VMs
![Page 24: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/24.jpg)
0
50
100
150
200
250
300
IT
Pow
er (W
)Static Integral Predictive Zephyr Zephyr + On/Off
Cooling Power Savings
626
![Page 25: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/25.jpg)
System Power Savings
0
500
1000
1500
2000
2500
3000
3500
4000
IT
Pow
er (W
)Static Integral Predictive Zephyr Zephyr + On/Off
![Page 26: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/26.jpg)
Simulated Servers
239
34
68
102
136
170
204
120
154
190
222
256
290
0
50
100
150
200
250
300
0 20 40 60 80 100
Power Usage (W
)
Utilization
Real
EnergyProportional
PracticalNext-Gen
![Page 27: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/27.jpg)
Simulation: Tamb+15C &13 servers types
![Page 28: InterPack '09 ASME/Pacific Rim Technical Conference and Exhibition](https://reader034.vdocuments.us/reader034/viewer/2022051405/58a2d7071a28ab90228c1c08/html5/thumbnails/28.jpg)
Conclusions• Zephyr combines:−Concepts from Heat Transfer Theory with Systems
• Unified power and cooling management can save:−30% Cooling Power−29% Enclosure Power
Full Length Papers:www.hpl.hp.com/personal/Niraj_Tolia/