![Page 1: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/1.jpg)
© 2006 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice
JouleSort:A Balanced Energy-Efficiency Benchmark
Suzanne Rivoire (Stanford), Mehul Shah (HP Labs),
Partha Ranganathan (HP Labs), Christos Kozyrakis (Stanford)
![Page 2: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/2.jpg)
2
Energy Use is Important (1 of 2)
• From data centers to mobile devices
• Data center: power and cooling
World wide cost of purchasing and operating servers
50%
$29 billion
"If performance per watt remains constant … power costs could easily overtake hardware costs…"
[Barroso,12/05] (Google)
![Page 3: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/3.jpg)
3
Energy Use is Important (2 of 2)
• Data center: power and cooling
− Implications on reliability, density, and scalability
− Pollution – 4M tons CO2 [C. Patel et al., 2006]
− Load on utilities
• Desktops: electricity costs
• Mobile devices: battery life affects usability
![Page 4: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/4.jpg)
4
• Inspire energy-efficiency improvements
• Current efforts− E.g., MIPS/Watt, SPECint/Watt, SWaP, …
− E.g., Ongoing activity in Green Grid, EPA, SPEC Power, …
• But often …− Focused on specific component
− Under-specified or “under construction”
− Application specific: realistic but complex
No simple holistic benchmark
Benchmarks
![Page 5: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/5.jpg)
5
JouleSort: Simple and Holistic
• Primarily meant for system designers
− Simple: easy to setup and experiment
− Evaluate disruptive technology, gain insights
− Technology bellwether: anticipate trends
• Measure whole-system energy-efficiency
• Workload, metric, and guidelines
• Based on external sort
![Page 6: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/6.jpg)
6
Our Contributions
I: JouleSort: Holistic energy-efficiency benchmark
− Design: workload, metrics, guidelines
− Rationale and pitfalls
II: Energy-efficient system design: 2007 “winner”
− 3.5X better than previous estimated best
− Insights on future designs
![Page 7: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/7.jpg)
7
Why External Sort? (1 of 2)
• Simple, balanced workload
−Exercises all core components
−CPU, memory, disk, I/O, OS, filesystem
• Applies to systems small and large
−PDAs, Laptops, Desktop, Supercomputers
• Representative of sequential I/O tasks
−Data warehousing, Business analytics, etc.
![Page 8: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/8.jpg)
8
Why External Sort? (2 of 2)
• Hard to cheat
−Measure system while doing useful work
• Technology trend bellwether
−E.g. supercomputers to clusters, GPU?
• Holistic measure of improvement
![Page 9: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/9.jpg)
9
Existing Sort Benchmarks
• Pure performance
− MinuteSort: How much can you sort in 1 min ?
− TeraByte: How fast can you sort 1 TB ?
• Cost efficient
− PennySort: How much can you sort for 1 penny ?
− Performance-Price: Maximum SRecs/$ in 1 min ?
![Page 10: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/10.jpg)
10
Our ^ JouleSort Proposal
• Workload
− Sort 100-byte records with 10-byte keys
− From file on non-volatile store to file on non-volatile store
• Metric?
− Energy (Joules) = Power (Watts)* Time (secs)
− Fixed time budget (like MinuteSort, Price-Perf Sort)• 1 minute budget
• Measure records sorted and Joules
• Winner: max SortedRecs/Joule?
Initial
![Page 11: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/11.jpg)
11
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
1.0E+05 1.0E+06 1.0E+07 1.0E+08 1.0E+09 1.0E+10
Records Sorted
SRecs/J .
Records Sorted
Problem with Fixed Time Budget
− Biased toward systems that sort fewer records
− Better efficiency with 1-pass sort and sleep
• System not doing useful work
1-pass sort
< 10 sec Energy not linear
function of N
Sort
edR
ecs/J
oule
![Page 12: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/12.jpg)
12
Our ^ JouleSort Proposal
• Fixed input size (like TeraByte)
− Three classes: 10GB, 100GB, 1TB
− Winner: minimum energy
− Report SortedRecs/Joule (like MPG for cars)
− Inter-class comparisons imperfect
− Adjust classes as technology improves
• Categories
− Daytona “street-car”: sold and supported
− Indy “no-holds-barred”
Revised
![Page 13: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/13.jpg)
13
Energy Measurement
Power
Power Meter
System Measured
Monitoring System
Wall AC-Power
Power readings
(serial cable)Sort timing (network)
![Page 14: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/14.jpg)
14
Energy Measurement
Power
Power Meter
System Measured
Monitoring System
Wall AC-Power
Power readings
(serial cable)Sort timing (network)
•Measure energy of all components
– No unaccounted potential energy
– Cooling devices attached to system
• 20-25 C at inlet or within 1 foot of device
![Page 15: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/15.jpg)
15
Energy Measurement
Power
Power Meter
System Measured
Monitoring System
Wall AC-Power
Power readings
(serial cable)Sort timing (network)
•Measure true power from wall
– Applies to AC and DC
– Report power-factor
•Leverage SPEC-Power/EPA specifications
![Page 16: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/16.jpg)
16
Energy Measurement
Power
Power Meter
System Measured
Monitoring System
Wall AC-Power
Power readings
(serial cable)Sort timing (network)
•Take at least 3 consecutive readings
•Report average
![Page 17: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/17.jpg)
17
Road Map
I: JouleSort: Holistic energy-efficiency benchmark
− Design: workload, metrics, guidelines
− Rationale and pitfalls
II: Energy-efficient system design: 2007 “winner”
− 3.5X better than previous estimated best
− Insights on future designs
![Page 18: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/18.jpg)
18
SortedRecs/Joule vs. Time
0
500
1000
1500
2000
2500
3000
3500
1996 1998 2000 2002 2004 2006 2008
Year
Pennysort Daytona
Pennysort Indy
MinuteSort Daytona
MinuteSort Indy
Terabyte Daytona
Terabyte Indy
Datamation
Historical Analysis (Estimate)SortedRecs/Joule
![Page 19: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/19.jpg)
19
SortedRecs/Joule vs. Time
0
500
1000
1500
2000
2500
3000
3500
1996 1998 2000 2002 2004 2006 2008
Year
Pennysort Daytona
Pennysort Indy
MinuteSort Daytona
MinuteSort Indy
Terabyte Daytona
Terabyte Indy
Datamation
Historical Analysis (Estimate)
Cost-Efficient Sorts: 24% / yr
Perf-Oriented Sorts: 12% / yr
SortedRecs/Joule
Target: GPUTeraSort
~3200 SortedRecs/Joule
![Page 20: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/20.jpg)
20
A Look at Existing Systems
406
22
140
90
290
Power (Watt)
~320059GBn/a9GPUTeraSort(estimated)
~340010GB1%1Modern Laptop
~380010GB90%+12Sort-balanced Fileserver
~120010GB26%2Low-end server DL360G3
11%
CPU %
~3005GB1Low-power Blade
SortedRecsper Joule
Input Size
# Disks
![Page 21: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/21.jpg)
21
A Look at Existing Systems
406
22
140
90
290
Power (Watt)
~320059GBn/a9GPUTeraSort(estimated)
~340010GB1%1Modern Laptop
~380010GB90%+12Sort-balanced Fileserver
~120010GB26%2Low-end server DL360G3
11%
CPU %
~3005GB1Low-power Blade
SortedRecsper Joule
Input Size
# Disks
DL360G5 server: 180W
Disk trays + disks: 226W
Active Idle: 370W
![Page 22: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/22.jpg)
22
Optimizing for Energy-Efficiency: Step 1
Lower power components w/o equal perf. loss
52% power
75% perf
FileserverFileserver Our winnerOur winner
Sort BW: 313 MB/s
65W (peak)
Sort BW: 236 MB/s
34W (peak)
![Page 23: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/23.jpg)
23
Optimizing for Energy-Efficiency: Step 1
Lower power components w/o equal perf. loss
15% power
50% perf
FileserverFileserver Our winnerOur winner
Seagate Barracuda
Seq. BW: 80MB/s
13W
Hitachi Travelstar
Seq. BW: 40MB/s
2W
![Page 24: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/24.jpg)
24
0
2000
4000
6000
8000
10000
12000
2 3 4 5 6 7 8 9 10 11 12 13
Disks Used
SortedRecs/Joule
0
20
40
60
80
100
120
140
SortedRecs/sec
(x 10E4)
SRecs/J Perf
Optimizing for Energy Efficiency: Step 2• Maximize performance
− Balanced sort: enough disks to fully utilize CPU
− Disks running near peak BW
GPUTeraSort
![Page 25: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/25.jpg)
25
Winner 100GB Category
• 11300 SortedRecs/Joule
−3.5x better than GPUTeraSort
−Average Power: 100W
−Ordinal Technology’s NSort (thanks Chris Nyberg)
![Page 26: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/26.jpg)
26
Winner 100GB Category
Asus motherboard:
Mobile CPU + 2 PCI-e slots
Asus motherboard:
Mobile CPU + 2 PCI-e slots
RocketRAID Disk ControllersRocketRAID Disk Controllers
Detailed SW/HW sensitivity
experiments in paper
13 Hitachi TravelStar 160GB13 Hitachi TravelStar 160GB
![Page 27: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/27.jpg)
27
Insights for Future Designs
• All components matter− CPU, Disks, Memory, …
− Low hanging fruit: use low-power HW
• Current technology − Limited dynamic range
− For fixed HW: peak efficiency = peak performance
• Want “scale-down efficiency”− 1TB � 100GB and give best of both
![Page 28: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/28.jpg)
28
Other Issues
• Benchmark design
− Data-center cooling and control
− Display power, GPUs, etc.
− Total cost of ownership
• System design
− Flash is becoming practical
− Cheaper, faster, and lower power
![Page 29: JouleSort: A Balanced Energy-Efficiency Benchmarkcsl.stanford.edu/~christos/publications/2007.jsort.sigmod.slides.pdf · 3 Energy Use is Important (2 of 2) • Data center: power](https://reader034.vdocuments.us/reader034/viewer/2022050507/5f989cd6f1bba20ff35dfc61/html5/thumbnails/29.jpg)
29
Conclusion
• Energy-use is important− From data centers to handhelds
• JouleSort− Simple, holistic energy-efficiency benchmark
• Built energy-efficient sorting system− 3.5x better than 2006 estimated winner (GPUTeraSort)
− Insights: low-power HW, limited dynamic range
• Part of Sort Benchmark suite− Entries welcome for 2008
− http://joulesort.stanford.edu