analyzing the energy (dis-)proportionality of scalable ... · scalable interconnection networks...
TRANSCRIPT
![Page 1: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/1.jpg)
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks
Felix Zahn, Pedro Yebenes, Steffen Lammel, Pedro J. Garcia, Holger Fröning
2nd IEEE International Workshop on High-Performance
Interconnection Networks in the Exascale and Big-Data Era
Barcelona, 12th March, 2016
![Page 2: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/2.jpg)
Does network power ma.er at all?
§ Pitfall: don’t make assumptions based on maximum power ratings • At TDP, processors
outshine anything • But are processors always
operating at 100% load? • Energy-proportional: at
x% load, a component consumes only x% power
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 1
![Page 3: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/3.jpg)
It does!
§ System power • Scalable energy-efficient network • Direct network, integrated switches
§ Dynamic range of components § Many memory-bound applications
• E.g., emerging integer applications (R. Murphy, Sandia) & graph computations • DFS & BFS • Connected Components • Isomorphism • Shortest Path • Graph Partitioning • BLAST (alignment search) • zChaff (satisfiability)
§ Exception: compute-bound applications with perfect overlap
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 2
![Page 4: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/4.jpg)
Mo5va5on
§ Serialization technology dominates power consumption • Clock recovery, high frequency, equalization, pre-emphasis, …
§ It is link width that matters, not frequency • CML = Current Mode Logic • Linear scaling for 10GHz case • Frequency dependent part is CMOS only
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 3
71%$
14%$
15%$
Power&share&for&NIC&with&integrated&switch&
Links$(6)$
PCIe$
Core$
0.0#
0.5#
1.0#
1.5#
2.0#
2.5#
3.0#
3.5#
4#Lanes# 8#Lanes# 12#Lanes#
Power&con
sump-
on&[n
ormalized
]&
Link&power&scaling&
2.5#GHz#
5#GHz#
10#GHz#
![Page 5: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/5.jpg)
Energy-‐propor5onality?
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 4
![Page 6: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/6.jpg)
Energy-‐propor5onality vs. today’s interconnects
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 5
![Page 7: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/7.jpg)
Energy-‐propor5onality – today’s possibili5es
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 6
§ Energy/data: 34.4 pJ/bit (65nm TSMC-produced Serializers only)
![Page 8: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/8.jpg)
Concept
§ Application-dependent potential for energy saving • Best case: no transition time, links switched on/off depending on whether they are busy or idling
• Worst case (common case today): all links run with full power
§ Setup: OMNeT++-based simulator
• 3D Torus topology • 64 nodes • XYZ-dimension-order routing
7 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
![Page 9: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/9.jpg)
Workloads -‐ NAMD
§ Satellite Tobacco Mosaic Virus (STMV), 64 MPI tasks
8 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
![Page 10: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/10.jpg)
Workloads – Graph500
§ scale factor = 12, edge factor = 16, 64 MPI tasks
9 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
![Page 11: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/11.jpg)
Results – Link inac5vity
10 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
§ Execution time: NAMD 3449 ms, Graph500 95.2 ms
![Page 12: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/12.jpg)
Results – U5liza5on
§ Link utilization is highly volatile due to: • Application • Dimension • Routing algorithm
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 11
![Page 13: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/13.jpg)
Results – Idle 5me
12 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
6% 2%
![Page 14: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/14.jpg)
Results – Power saving poten5als
13 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
20% 24%
![Page 15: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/15.jpg)
Discussion
§ Network power matters • Today’s interconnection networks contribute up to 20% to the total power consumption of large computing systems
§ Interconnects are highly energy dis-proportional • But they provide the opportunity to implement power saving strategies
§ Huge potential power savings • An optimal and fully energy-proportional NIC would save about 75% of the network power
14 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
![Page 16: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/16.jpg)
Outlook
§ Understanding energy consumption • Power saving possibilities depend highly on workloads • Communication for HPC applications is complex to model => trace based network simulation
§ Analysis of different hardware parameter • Transition time, granularity of link width, etc. => Useful input on design decisions for future hardware § Network energy model
• Predicting energy consumed by the network based on communication characteristics
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 15
![Page 17: Analyzing the Energy (Dis-)Proportionality of Scalable ... · Scalable Interconnection Networks Felix Zahn, Pedro Yebenes , Steffen Lammel, Pedro J. Garcia, Holger Fröning 2nd IEEE](https://reader035.vdocuments.us/reader035/viewer/2022071117/6000c2d5c0a2cc3f697a9d21/html5/thumbnails/17.jpg)
Thank you! Questions?
16 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
Credits Discussions: Maximilian Thürmer, Markus Müller, Benjamin Klenk, Alexander Matz, Daniel Schlegel
(Heidelberg University), Sébas5en Rumley (Columbia University), Francisco Andujar, Jesus Escudero, Juan
Villar (Universidad de Cas5lla-‐La Mancha)
Current main interac4ons