ultra-low power electronics and design
Post on 13-Jan-2015
13.240 Views
Preview:
DESCRIPTION
TRANSCRIPT
ULTRA LOW-POWER ELECTRONICS AND DESIGN
This page intentionally left blank
Ultra Low-PowerElectronics and Design
Edited by
Enrico MaciiPolitecnico di Torino,
Italy
KLUWER ACADEMIC PUBLISHERSNEW YORK, BOSTON, DORDRECHT, LONDON, MOSCOW
eBook ISBN: 1-4020-8076-XPrint ISBN: 1-4020-8075-1
©2004 Springer Science + Business Media, Inc.
Print ©2004 Kluwer Academic Publishers
All rights reserved
No part of this eBook may be reproduced or transmitted in any form or by any means, electronic,mechanical, recording, or otherwise, without written consent from the Publisher
Created in the United States of America
Visit Springer's eBookstore at: http://www.ebooks.kluweronline.comand the Springer Global Website Online at: http://www.springeronline.com
Dordrecht
Contents
CONTRIBUTORS…………………………………………………………………….VII
PREFACE…………………………………………………………….………………...IX
INTRODUCTION……………………………………………………………………XIII
1. ULTRA-LOW-POWER DESIGN: DEVICE AND LOGIC DESIGN
APPROACHES……………………………………….………………………………….1
2. ON-CHIP OPTICAL INTERCONNECT FOR LOW-POWER……………………21
3. NANOTECHNOLOGIES FOR LOW POWER……………….…………………….40
4. STATIC LEAKAGE REDUCTION THROUGH SIMULTANEOUS
Vt/Tox AND STATE ASSIGNMENT………………………………………………….56
5. ENERGY-EFFICENT SHARED MEMORY ARCHITECTURES FOR
MULTI-PROCESSOR SYSTEMS-ON-CHIP…………………………………...…..84
6. TUNING CACHES TO APPLICATIONS FOR LOW-ENERGY EMBEDDED
SYSTEMS……………………………………………………………………………..103
7. REDUCING ENERGY CONSUMPTION IN CHIP MULTIPROCESSORS
USING WORKLOAD VARIATIONS……………………………………………....123
8. ARCHITECTURES AND DESIGN TECHNIQUES FOR ENERGY
EFFICIENT EMBEDDED DSP AND MULTIMEDIA PROCESSING……….….141
9. SOURCE-LEVEL MODELS FOR SOFTWARE POWER OPTIMIZATION…..156
10. TRANSMITTANCE SCALING FOR REDUCING POWER DISSIPATION
OF A BACKLIT TFT-LCD…………………………………………………………..172
11. POWER-AWARE NETWORK SWAPPING FOR WIRELESS PALMTOP
PCS…………………………………………………………………………………… 198
12. ENERGY EFFICIENT NETWORK-ON-CHIP DESIGN…………………………214
13. SYSTEM LEVEL POWER MODELING AND SIMULATION OF
HIGH-END INDUSTRIAL NETWORK-ON-CHIP……………………………….233
14. ENERGY AWARE ADAPTATIONS FOR END-TO-END VIDEO
STREAMING TO MOBILE HANDHELD DEVICES…………………………….255
vi
Contributors
A. Acquaviva Università di Urbino L. Benini Università di Bologna D. Bertozzi Università di Bologna D. Blaauw University of Michigan, Ann Arbor A. Bogliolo Università di Urbino A. Bona STMicroelectronics C. Brandolese Politecnico di Milano W.C. Cheng University of Southern California G. De Micheli Stanford University N. Dutt University of California, Irvine W. Fornaciari Politecnico di Milano F. Gaffiot Ecole Centrale de Lyon J. Gautier CEA-DRT–LETI/D2NT–CEA/GRE A. Gordon-Ross University of California, Riverside R. Gupta University of California, San Diego C. Heer Infineon Technologies AG M. J. Irwin Pennsylvania State University I. Kadayif Canakkale Onsekiz Mart University M. Kandemir Pennsylvania State University B. Kienhuis Leiden I. Kolcu UMIST E. Lattanzi Università di Urbino D. Lee University of Michigan, Ann Arbor A. Macii Politecnico di Torino S. Mohapatra University of California, Irvine I. O’Connor Ecole Centrale de Lyon K. Patel Politecnico di Torino M. Pedram University of Southern California C. Pereira University of California, San Diego C. Piguet CSEMM. Poncino Università di Verona F. Salice Politecnico di Milano P. Schaumont University of California, Los Angeles U. Schlichtmann Technische Universität München D. Sylvester University of Michigan, Ann Arbor
vii
F. Vahid University of California, Riverside and University of California, Irvine
N. Venkatasubramanian University of California, Irvine I. Verbauwhede University of California, Los Angeles
and K.U.Leuven N. Vijaykrishnan Pennsylvania State University V. Zaccaria STMicroelectronics R. Zafalon STMicroelectronics B. Zhai University of Michigan, Ann Arbor C. Zhang University of California, Riverside
viii
Preface
Today we are beginning to have to face up to the consequences of the stunning success of Moore’s Law, that astute observation by Intel’s Gordon Moore which predicts that integrated circuit transistor densities will double every 12 to 18 months. This observation has now held true for the last 25 years or more, and there are many indications that it will continue to hold true for many years to come. This book appears at a time when the first examples of complex circuits in 65nm CMOS technology are beginning to appear, and these products already must take advantage of many of the techniques to be discussed and developed in this book. So why then should our increasing success at miniaturization, as evidenced by the success of Moore’s Law, be creating so many new difficulties in power management in circuit designs?
The principal source and the physical origin of the problem lies in the differential scaling rates of the many factors that contribute to power dissipation in an IC – transistor speed/density product goes up faster than the energy per transition comes down, so the power dissipation per unit area increases in a general sense as the technology evolves.
Secondly, the “natural” transistor switching speed increase from one generation to the next is becoming downgraded due to the greater parasitic losses in the wiring of the devices. The technologists are offsetting this problem to some extent by introducing lower permittivity dielectrics (“low-k”) and lower resistivity conductors (copper) – but nonetheless to get the needed circuit performance, higher speed devices using techniques such as silicon-on-insulator (SOI) substrates, enhanced carrier mobility (“strained silicon”) and higher field (“overdrive”) operation are driving power densities ever upwards. In many cases, these new device architectures are increasingly leaky, so static power dissipation becomes a major headache in power management, especially for portable applications.
ix
A third factor is system or application driven – having all this integration capability available encourages us to combine many different functional blocks into one system IC. This means that in many cases, a large part of the chip’s required functionality will come from software executing on and between multiple on-chip execution units; how the optimum partitioning between hardware architecture and software implementation is obtained is a vast subject, but clearly some implementations will be more energy efficient than others. Given that, in many of today’s designs, more than 50% of the total development effort is on the software that runs on the chip, getting this partitioning right in terms of power dissipation can be critical to the success of (or instrumental in the failure of!) the product.
A final motivation comes from the practical and environmental consequences of how we design our chips – state-of-the-art high performance circuits are dissipating up to 100W per square centimeter – we only need 500 square meters of such silicon to soak up the output of a small nuclear power station. A related argument, based on battery lifetime, shows that the “converged” mobile phone application combining telephony, data transmission, multimedia and PDA functions that will appear shortly is demanding power at the limit of lithium-ion or even methanol-water fuel cell battery technology. We have to solve the power issue by a combination of design and process technology innovations; examples of current approaches to power management include multiple transistor thresholds, triple gate oxide, dynamic supply voltage adjustment and memory architectures.
Multiple transistor thresholds is a technique, practiced for several years now, that allows the designer to use high performance (low Vt) devices where he needs the speed, and low leakage (high Vt) devices elsewhere. This benefits both static power consumption (through less sub-threshold leakage) and dynamic power consumption (through lower overall switching currents). High threshold devices can also be used to gate the supplies to different parts of the circuit, allowing blocks to be put to sleep until needed.
Similar to the previous technique, triple gate oxide (TGO) allows circuit partitioning between those parts that need performance and other areas of the circuit that don’t. It has the additional benefit of acting on both sub-threshold leakage and gate leakage. The third oxide is used for I/O and possibly mixed-signal. It is expected over the next few years that the process technologists will eventually replace the traditional silicon dioxide gate dielectric of the CMOS devices by new materials such as rare earth oxides with much higher dielectric constants that will allow the gate leakage problem to be completely suppressed.
x
Dynamic supply voltage adjustment allows the supply voltage to different blocks of the circuit to be adjusted dynamically in response to the immediate performance needs for the block – this very sophisticated technique will take some time to mature.
Finally, many, if not most, advanced devices use very large amounts of memory for which the contents may have to be maintained during standby; this consumes a substantial amount of power, either through refreshing dynamic RAM or through the array leakage for static RAM. Traditional non-volatile memories have writing times that are orders of magnitude too slow to allow them to substitute these on-chip memories. New developments, such as MRAM, offer the possibility of SRAM-like performance coupled with unlimited endurance and data retention, making them potential candidates to replace the traditional on-chip memories and remove this component of standby power consumption.
Most of the approaches to power management described briefly above will be employed in 65nm circuits, but there are a lot more good ideas waiting to be applied to the problem, many of which you will find clearly and concisely explained in this book.
Mike Thompson, Philippe Magarshack
STMicroelectronics, Central R&D Crolles, France
xi
This page intentionally left blank
Introduction
ULTRA LOW-POWER ELECTRONICS AND DESIGN
Enrico Macii Politecnico di Torino
Power consumption is a key limitation in many electronic systems today, ranging from mobile telecom to portable and desktop computing systems, especially when moving to nanometer technologies. Power is also a showstopper for many emerging applications like ambient intelligence and sensor networks. Consequently, new design techniques and methodologies are needed to control and limit power consumption.
The 2004 edition of the DATE (Design Automation and Test in Europe) conference has devoted an entire Special Focus Day to the power problem and its implications on the design of future electronic systems. In particular, keynote presentations and invited talks by outstanding researchers in the field of low-power design, as well as several technical papers from the regular conference sessions have addressed the difficulties ahead and advanced strategies and principles for achieving ultra low-power design solutions. Purpose of this book is to integrate into a single volume a selection of these contributions, duly extended and transformed by the authors into chapters proposing a mix of tutorial material and advanced research results.
The manuscript consists of a total of 14 chapters, addressing different aspects of ultra low-power electronics and design. Chapter 1 opens the volume by providing an insight to innovative transistor devices that are capable of operating with a very low threshold voltage, thus contributing to a significant reduction of the dynamic component of power consumption. Solutions for limiting leakage power during stand-by mode are also discussed. The chapter closes with a quick overview of low-power design techniques applicable at the logic level, including multi-Vdd, multi-Vth and hybrid approaches.
Chapter 2 focuses on the problem of reducing power in the interconnect network by investigating alternatives to traditional metal wires. In fact, according to the 2003 ITRS roadmap, metallic interconnections may not be able to provide enough transmission speed and to keep power under control for the upcoming technology nodes (65nm and below). A possible solution, explored in the chapter, consists of the adoption of optical interconnect networks. Two applications are presented: Clock distribution and data communication using wavelength division multiplexing.
xiii
In Chapter 3, the power consumption problem is faced from the technology point of view by looking at innovative nano-devices, such as single-electron or few-electron transistors. The low-power characteristics and potential of these devices are reviewed in details. Other devices, including carbon nano-tube transistors, resonant tunnelling diodes and quantum cellular automata are also treated.
Chapter 4 is entirely dedicated to advanced design methodologies for reducing sub-threshold and gate leakage currents in deep-submicron CMOS circuits by properly choosing the states to which gates have to be driven when in stand-by mode, as well as the values of the threshold voltage and of the gate oxide thickness. The authors formulate the optimization problem for simultaneous state/Vth and state/Vth/Tox assignments under delay constraints and propose both an exact method for its optimal solution and two practical heuristics with reasonable run-time. Experimental results obtained on a number of benchmark circuits demonstrate the viability of the proposed methodology.
Chapter 5 is concerned with the issue of minimizing power consumption of the memory subsystem in complex, multi-processor systems-on-chip (MPSoCs), such as those employed in multi-media applications. The focus is on design solutions and methods for synthesizing memory architectures containing both single-ported and multi-ported memory banks. Power efficiency is achieved by casting the memory partitioning design paradigm to the case of heterogeneous memory structures, in which data need to be accessed in a shared manner by different processing units.
Chapter 6 addresses the relevant problem of minimizing the power consumed by the cache hierarchy of a microprocessor. Several design techniques are discussed, including application-driven automatic and dynamic cache parameter tuning, adoption of configurable victim buffers and frequent-value data encoding and compression.
Power optimization for parallel, variable-voltage/frequency processors is the subject of Chapter 7. Given a processor with such an architecture, this chapter investigates the energy/performance tradeoffs that can be spanned in parallelizing array-intensive applications, taking into account the possibility that individual processing units can operate at different voltage/frequency levels. In assigning voltage levels to processing units, compiler analysis is used to reveal hetherogeneity between the loads of the different units in parallel execution.
xiv
Chapter 8 provides guidelines for the design and implementation of DSP and multi-media applications onto programmable embedded platforms. The RINGS architecture is first introduced, followed by a detailed discussion on power-efficient design of some of the platform components, namely, the DSPs. Next, design exploration, co-design and co-simulation challenges are addressed, with the goal of offering to the designers the capability of including into the final architecture the right level of programmability (or reconfigurability) to guarantee the required balance between system performance and power consumption. Chapter 9 targets software power minimization through source code optimization. Different classes of code transformations are first reviewed; next, the chapter outlines a flow for the estimation of the effects that the application of such transformations may have on the power consumed by a software application. At the core of the estimation methodology there is the development of power models that allow the decoupling of processor-independent analysis from all the aspects that are tightly related to processor architecture and implementation. The proposed approach to software power minimization is validated through several experiments conducted on a number of embedded processors for different types of benchmark applications.
Reduction of the power consumed by TFT liquid crystal displays, such as those commonly used in consumer electronic products is the subject of Chapter 10. More specifically, techniques for reducing power consumption of transmissive TFT-LCDs using a cold cathode fluorescent lamp backlight are proposed. The rationale behind such techniques is that the transmittance function of the TFT-LCD panel can be adjusted (i.e., scaled) while meeting an upper bound on a contrast distortion metric. Experimental results show that significant power savings can be achieved for still images with very little penalty in image contrast.
Chapter 11 addresses the issue of efficiently accessing remote memories from wireless systems. This problem is particularly important for devices such as palmtops and PDAs, for which local memory space is at a premium and networked memory access is required to support virtual memory swapping. The chapter explores performance and energy of network swapping in comparison with swapping on local microdrives and FLASH memories. Results show that remote swapping over power-manageable wireless network interface cards can be more efficient than local swapping and that both energy and performance can be optimized by means of power-aware reshaping of data requests. In other words, dummy data accesses can be preemptively inserted in the source code to reshape page requests in order to significantly improve the effectiveness of dynamic power management.
xv
Chapter 12 focuses on communication architectures for multi-processor SoCs. The network-on-chip (NoC) paradigm is reviewed, touching upon several issues related to power optimization of such kinds of communication architectures. The analysis goes on a layer-by-layer basis, and particular emphasis is given to customized, domain-specific networks, which represent the most promising scenario for communication-energy minimization in multi-processor platforms.
Chapter 13 provides a natural follow up to the theory of NoCs covered in the previous chapter by describing an industrial application of this type of communication architecture. In particular, the authors introduce an innovative methodology for automatically generating the power models of a versatile and parametric on-chip communication IP, namely the STBus by STMicroelectronics. The methodology is validated on a multi-processor hardware platform including four ARM cores accessing a number of peripheral targets, such as SRAM banks, interrupt slaves and ROM memories.
The last contribution, offered in Chapter 14, proposes an integrated end-to-end power management approach for mobile video streaming applications that unifies low-level architectural optimizations (e.g., CPU, memory, registers), OS power-saving mechanisms (e.g., dynamic voltage scaling) and adaptive middleware techniques (e.g., admission control, trans-coding, network traffic regulation). Specifically, interaction parameters between the different levels are identified and optimized to achieve a reduction in the power consumption.
Closing this introductory chapter, the editor would like to thank all the authors for their effort in producing their outstanding contributions in a very short time. A special thank goes to Mike Thompson and Philippe Magarshack of STMicroelectronics for their keynote presentation at DATE 2004 and for writing the foreword to this book. The editor would also like to acknowledge the support offered by Mark De Jongh and the Kluwer staff during the preparation of the final version of the manuscript. Last, but not least, the editor is grateful to Agnieszka Furman for taking care of most of the “dirty work” related to book editing, paging and preparation of the camera-ready material.
xvi
Chapter 1
ULTRA-LOW-POWER DESIGN: DEVICE AND
LOGIC DESIGN APPROACHES
Christoph Heer1 and Ulf Schlichtmann
2
1Infineon Technologies AG; 2Technische Universität München
Abstract Power consumption increasingly is becoming the bottleneck in the design of
ICs in advanced process technologies. We give a brief introduction into the
major causes of power consumption. Then we report on experiments in an
advanced process technology with ultra-low threshold voltage (Vth) devices. It
turns out that in contrast to older process technologies, this approach
increasingly is becoming less suitable for industrial usage in advanced process
technologies. Following, we describe methodologies to reduce power
consumption by optimizations in logic design, specifically by utilizing
multiple levels of supply voltage Vdd and threshold voltage Vth. We evaluate
them from an industrial product development perspective. We also give a brief
outlook to proposals on other levels in the design flow and to future work.
Keywords: Low-power design, dynamic power reduction, leakage power reduction, ultra-
low-Vth devices, multi-Vdd, multi-Vth, CVS
1.1 INTRODUCTION
The progress of silicon process technology marches on relentlessly. As
predicted by Gordon Moore decades ago, silicon process technology
continues to achieve improvements at an astonishing pace [1]. The number
of transistors that can be integrated on a single IC approximately doubles
every 2 years [2,3]. This engineering success has created innovative new
industries (e.g. personal computers and peripherals, consumer electronics)
and revolutionized other industries (e.g. communications).
Today, however, it is becoming increasingly difficult to achieve
improvements at the pace that the industry has become accustomed to. More
and more technical challenges appear that require increasing resources to be
1
solved [4]. One such problem is the increasing power consumption of
integrated circuits. It becomes even more critical as an increasing number of
today’s high-volume consumer products are battery-powered.
In the following, we will consider the sources of power consumption and
their development over time. We will show why reduction of power
consumption increasingly is becoming critical to product success and will
review traditional approaches in Sections 1.1 and 1.2. In Section 1.3 we will
then analyze a potential solution based on introduction of an optimized
transistor with a very low threshold voltage Vth. Thereafter, we will present
and discuss logic-level design optimizations for power reduction in Section
1.4. Also, we will briefly point out potential optimizations on higher levels.
Our observations are made from the perspective of industrial IC product
development where technical optimizations must be carefully evaluated
against the cost associated with achieving and implementing them. Mostly,
the presented methodologies are already being utilized in leading-edge
industrial ICs.
1.2 POWER CONSUMPTION BECOMES CRITICAL
Depending on the type of end-product and its application, different
aspects of power consumption are the primary concern: dynamic power or
leakage power.
Reduction of dynamic power consumption is a concern for almost all
IC products today. For battery-powered products, reduced power
consumption directly results in longer operating time for the product, which
is a very desirable characteristic. Even for non-battery-powered products,
reduced power consumption brings many advantages, such as reduced cost
because of cheaper packaging or higher performance because of lower
temperatures. Finally, reduced power consumption often leads to lower
system cost (no fans required; no or cheaper air conditioning for data /
telecom center etc.).
Dynamic power consumption is caused by the charging and discharging
of capacitances when a circuit switches. In addition, during switching a
short-circuit current flows, but this current is typically much smaller, and
will therefore be neglected in the following. The dynamic current due to
capacitance charging and discharging is determined by the following well-
known relationship:
2~ ddLdyn VCfP ••
2
Based on constant electrical field scaling, Vdd and CL each are reduced by
30% in each successive process generation. Also, delay decreases by 30%,
resulting in 43% increase in frequency. Therefore, the dynamic power
consumption per device is reduced by 50% from one process generation to
the next. As scaling also doubles the number of devices that can be
implemented in a given die area, dynamic power consumption per area
should stay roughly identical. However, historically frequency has increased
by significantly more than 43% from one process generation to the next (e.g.
in microprocessors, it has roughly doubled, due to architectural
optimizations, such as deeper pipeline stages), and in addition, die sizes have
increased with each new process technology, further increasing the power
consumption, due to an increased number of active devices [5]. For these
reasons, dynamic power consumption has increased exponentially, as is
shown in Figure 1-1 for the example of microprocessors.
Reduction of leakage power consumption today is primarily a concern
for products that are powered by battery and spend most of their operating
hours in some type of standby mode, such as cell phones.
For many process generations, however, leakage has increased roughly
by a factor of 10 for every two process nodes [6]. Due to this dramatic
increase with newer process generations, leakage is becoming a significant
contribution to overall IC power consumption even in normal operating
mode, as can be seen in Figure 1-1 as well. Leakage was estimated to
increase from 0.01% of overall power consumption in a 1.0µm technology,
to 10% in a 0.1µm technology [6]. For a microprocessor, Intel estimated
leakage power consumption at more than 50W for a 100nm technology
node[3]. This figure probably is extreme, and leakage depends strongly on a
number of factors, such as threshold voltage (Vth) of the transistor, gate
oxide thickness and environmental operating conditions (supply voltage Vdd,
temperature T). Nevertheless, for an increasing number of products leakage
power consumption is turning into a problem, even when they are not
battery-powered.
3
Figure 1-1. Development of dynamic and leakage power consumption over time [3,7]
1.3 TRADITIONAL APPROACHES TO POWER
REDUCTION
As outlined above, dynamic power consumption is governed by:
2~ ddLdyn VCfP ••
with f denoting the switching frequency, CL the capacitance being
switched, and Vdd the supply voltage . This formula immediately identifies
the key levers to reduce dynamic power:
• Reduce operating frequency
• Reduce driven capacity
• Reduce supply voltage
Traditionally, reduction in supply voltage Vdd has been the most often
followed strategy to reduce power consumption. Unfortunately, lowering Vdd
has the side effect of reducing performance as well, primarily because gate
4
overdrive (the difference between Vdd and Vth) diminishes if the threshold
voltage Vth is kept constant. Based on the alpha power law model [8], the
delay td of an inverter is given by
( )αthdd
ddLd
VV
VCt
−•=
with α denoting a fitting constant. As supply voltages are driven below
1.0V, the reductions in gate overdrive are more pronounced than previously.
In addition, newer process technologies give significantly less of a
performance boost compared to the previous process generation than has
traditionally been the case, therefore a further reduction in performance is
highly undesirable. Finally, the power reduction achieved by moving to a
new process generation has trended down over time, since supply voltages
have been scaled by increasingly less than the 30% prescribed by the
constant electrical field scaling paradigm.
Consequently, more advanced approaches are required.
In the following, our main focus will be on dynamic power consumption,
but we will also consider leakage power consumption.
1.4 ZERO-VTH DEVICES
The concept of zero-Vth devices was developed in the mid 90-ies. It
overcomes the diminishing gate overdrive by radically setting the threshold
voltage of the active devices to zero. It has been shown [9], that the optimum
power dissipation is obtained, if Pleak (leakage contribution) is in the same
order of magnitude as Pdyn (dynamic switching contribution). This can be
achieved for transistors with Vth close to 0V (‘zero-Vth transistor‘). Therefore
the devices will never completely switch off. But from an overall power
perspective the gain in active power consumption is tremendous.
Using these transistors the supply voltage of 130nm circuits can be
reduced to values below 0.3V to achieve a Pdyn reduction by 90% without
performance degradation. Alternatively, the circuit can be operated at twice
the clock frequency when keeping the supply voltage at 1.2V, as shown in
Figure 1-2. The corresponding Ion/Ioff-ratio for the zero-Vth transistor is about
10-100 instead of >105 for the standard transistor options. During standby,
the complete circuits are switched-off or are set into a low leakage mode to
cope with the very high leakage contribution. The low leakage mode is
achieved by ‘active well’ control, which denotes the use of the body effect.
The well potentials of the PFETs and NFETs are altered to change Vth. To
achieve a lower leakage current, the absolute value of Vth is increased by
5
reverse back biasing: a negative well-to-source voltage Usb is used.
Therefore voltages below Vss for NFETs and above Vdd for PFETs have to
be generated. Furthermore, active well is required to compensate the lot-to-
lot or wafer-to-wafer variations of Vth.
The initial ‘zero-Vth’ concept assumed constant junction temperatures Tj
below 40°C. For some high-end computer equipment the costs for active
chip cooling are affordable to achieve this junction temperature. But this is
definitely not the case for cost-driven consumer products. For this
application domain Tj in active mode ranges between 85°C and 125°C, and
in some applications the specified worst-case ambient temperature is even
80°C. The proposed zero-Vth concept is therefore not applicable without
changes and adaptations.
Figure 1-2. Simulated performance curves of transistors with ultra-low Vth. Compared to low-
Vth, either a performance gain or a Vdd reduction can be achieved. Curves for reg-Vth and
high-Vth transistors of a 130 nm technology are included
A more conservative approach with respect to zero-Vth, but still
aggressive compared to current devices, had to be chosen. An ultra-low Vth
device with about 150mV threshold voltage proved to be the best
6
compromise between zero-Vth and current low-Vth of about 300mV within a
130 nm CMOS technology.
To identify the optimal choice of Vth and Vdd in combination with the
higher junction temperature Tj, simulations with modified parameters of the
130nm low-Vth transistor are performed. In Figure 1-3 the power dissipation
is shown for a high activity circuit ( = 20%) with various options for the
transistor threshold voltages: reg-Vth, low-Vth, and transistors whose Vth are
reduced to 200mV, 150mV, 100mV and 50mV. The reg-Vth circuit
performance was used as the reference (Vdd = 1.5V), and the supply voltages
for the other transistor options were reduced to meet that reference
performance.
Device Option / Vth (mV)
Po
we
r [W
]
Figure 1-3. Power dissipation at T=125°C in active mode for several transistor options with
reduced Vth. A minimum power consumption is achieved at 150mV Vth. (At T=55°C the
minimum is achieved for the same option but process variations show less impact).
The reduced supply voltage leads to lower overall active power
consumption Pactive. A minimum power consumption is reached at Vth =
150mV. With even lower threshold voltages Pactive starts to increase again
because of the increase of the leakage current. The steep rise of Pactive
originates from the exponential relation between Vth and leakage current. As
a rule of thumb a 100mV reduction of the threshold voltage allows for a Vdd
0,0E+00
5,0E-06
1,0E-05
1,5E-05
2,0E-05
2,5E-05
3,0E-05
3,5E-05
reg-Vt low-Vt 200mV 150mV 100mV 50mV
T= 125°C
1.0V
1.2V
Vdd= 1.5V
0.8V 0.7V
0.6V
fast
nom
= target
slow
7
reduction by 0.15V but on the other hand results in a tenfold increase of
the leakage current. From Figure 1-3 also the impact of technology
variations is visible. Due to the high leakage contribution a power reduction
of only 25% is achieved under fast process conditions. Using back biasing in
reverse mode, the high performance of fast transistors can be reduced
through increasing Vth. The corresponding leakage current therefore
decreases and allows a power reduction by 50% (stippled arrow).
A process modification has been developed to manufacture devices with
the threshold voltage of 150 mV, which proves to be the most efficient for
the target application domain of mobile consumer products [10]. In Table1-1
the key transistor parameters of our ultra-low-Vth FETs (ulv) and of the
standard low-Vth transistor are listed. The Vth values are 165mV and 161mV
for the ulv-NFET and ulv-PFET respectively, Ion increases by 35% and 22%,
which translates into an average decrease of the CV/I-metric delay by 29%.
Circuit simulations showed a performance increase of 25%. Concerning Vth,
performance, and Ioff the target values have been nearly met.
Table 1-1. Extracted key parameters of the ulv-FETSs in comparison with the target values
and the low- Vth FETs
130nm low-Vt
NFET / PFET
130nm ulv-FET
NFET / PFET
Target
Ion
[µA/µm]
560 / 240 755 / 295
Ioff
[nA/µm]
1.2 / 1.2 48 / 17 35
Vth
[mV]
295 / 260 165/160 150
body effect
[mV/V]
150 / 135 60/65 90
Vth@ L=10nm
[mV]
35 / 30 65/30
Vth@ L=15nm
[mV]
65 / 70 100/90
Simulated gate delay
[relative units]
1 0.8 0.75
The sensitivity of Vth to gate length variation (roll-off) is expressed in
Vth-shift per 10nm or 15nm gate length decrease. A comparison with low-
Vth-FETs shows a pronounced increase. Therefore in addition to temperature
compensation, back biasing has also to be used to compensate for this strong
technology variation.
8
The values of the body effect are also included in Table 1-1. The body
effect is expressed in Vth-shift per 1V well bias. The ulv-FETs yield values,
which are lower by more than 50% compared to the low-Vth transistors. The
decrease of body effect in combination with the increased roll-off reduces
the leverage of back biasing for ulv-FETs very significantly. The leverage is
not even sufficient to compensate the technology variation, since the value
of the roll-off is higher than that of the body effect. As an example, the ulv-
NFET shows roll-off values of 65mV/10nm and 100mV/15nm and a body
effect of only 60mV/V.
To investigate the migration potential of the ulv-FETs for future
technology generations Ioff measurement results, obtained from a recent
90nm hardware, were used. Based on this measurement data the leverage of
active well with the standard reg-Vth and low leakage transistor options has
been analyzed. For supply voltages of 1.2V and 0.75V a reverse back biasing
voltage of 0.5V has been applied. For the NFET, the back biasing results in a
leakage reduction by 50% to 70% for all transistor widths and for both
values of Vdd. In the case of the PFET, the leakage reduction values are
similar (60% to 80%) for transistors with W> 0.5µm. For very narrow
PFETs with Vdd = 1.2V, the reduction is only 20% or even less. Since
narrow FETs are used within SRAMs, which contribute a major part of the
circuit’s standby current, this small reduction for narrow transistors in
addition reduces significantly the leverage of active well. The root cause is
an additional leakage mechanism based on tunnelling currents across the
drain-well junction, which limits the reverse back biasing to 0.5V. This
tunnelling current depends exponentially on the drain-well voltage and is
working against any reduction of the sub-threshold current via active well.
At Vdd = 0.75V the drain-well voltage is reduced and the tunnelling current
is therefore lower. In this case the effect of back biasing is not compensated
by a rising tunnelling current and a leakage current reduction by 70% is still
achieved.
For a 90nm technology the limit of 0.5V for the well potential swing
limits the reduction of the leakage currents to a factor between 2 and 4. This
is still a major contribution of all feasible measures to reduce standby power
consumption, but the leverage becomes quite small compared to the
reduction ratios of several orders of magnitude obtained in previous
technologies [11,12]. In future technologies, Ileak will become more strongly
affected by the emerging tunnelling current Igate through the gate of the FET.
This is due to the ever decreasing gate oxide thickness and also due to the
fact, that even the on-state transistors shows gate leakage. Igate is not affected
by well biasing reducing the leverage of active well even further.
9
In summary the zero-Vth-devices have become very susceptible to
process and temperature variations. Significant yield is only achievable with
back biasing via active well control and with active cooling. The latter
approach is not feasible for mobile applications. Therefore a more
conservative approach with respect to zero-Vth, but still aggressive compared
to current devices, had to be chosen. An ultra-low-Vth device with about
150mV threshold voltage proved to be the best compromise between zero-
Vth and current low-Vth of about 300mV within a 130 nm CMOS technology.
But even though fabrication of this ultra-low-Vth device is possible, it
affects some standard methods to overcome short-channel effects. The so
called halo- or pocket-implantation had to be removed to bring the threshold
voltage down. Unfortunately short-channel effects are now heavily
increased, leading as shown to a very strong Vth roll-off at slight variations
of the channel length. Finally this effect was prohibitive for the overall
approach and led to cancellation of many zero-Vth projects in the
industry[13].
1.5 DESIGN APPROACHES TO POWER
REDUCTION
As outlined above, solutions from process technology by itself will not
suffice to provide sufficient power reduction. Therefore, solutions must be
found in algorithms, product architecture and logic design. Increasingly,
differentiated device options provided by process technology are utilized on
these levels in the search for optimization of power consumption.
For leading-edge products which need to optimize both power
consumption and system performance, optimization techniques on
architecture and design level have been proposed and partly already been
implemented. While academic research often focuses on the tradeoff
between power consumption and performance, industrial product
development must also take other variables into consideration.
• Product cost: often, power optimization design techniques increase die
area, directly affecting manufacturing cost. Also, utilization of additional
devices (e.g. different Vth devices) increases mask count and
consequently manufacturing cost, and additionally requires up-front
expenditures for the development of such devices. Finally, increased
manufacturing complexity poses the risk of lowered manufacturing yield.
• Product robustness: it must be ensured that optimized products still work
across the specified range of operating conditions, also taking
manufacturing variations into account.
10
1.5.1 Multi-Vdd Design
As outlined in the introduction, the supply voltage Vdd quadratically
impacts dynamic switching power consumption. Thus, lowering Vdd is the
preferred option to reduce dynamic power consumption. However, as
discussed in Section 1.2, lowering Vdd reduces the system performance.
Thus, the incentive to lower Vdd to reduce power consumption is kept in
check by the need to maintain performance.
Reduction of Vdd can be applied on different abstraction levels of a
design. Most effective regarding power reduction, and also easiest to
implement is to lower Vdd for an entire IC. As this will directly impact the
performance of the IC design, this often is not an option. On a lower
abstraction level, it is possible to lower Vdd for an entire module. This is still
rather simple to implement, but if only modules are chosen such that overall
IC performance is not impacted, the achieved gains in power reduction will
often be very moderate.
Finally, a reduction in supply voltage can be applied specifically to
individual gates, such that the overall system performance is not reduced.
This approach, as shown in Figure 1-4, recognizes that in a typical design,
most logic paths are not critical. They can be slowed down, often
significantly, without reducing the overall system performance. This slowing
down is achieved by lowering the supply voltage Vdd for gates on the non-
critical paths, which results in lowered power consumption.
11
Figure 1-4. Multi-Vdd design
This technique will modify the distribution of path delays in a design to a
distribution skewed towards paths with higher delay, as indicated Figure 1-5
[14].
Single Supply Voltage SSV Multiple Supply Voltages MSV
td td
SSV
crit. paths
1/f 1/f
MSV
Single Supply Voltage SSV Multiple Supply Voltages MSV
td td
SSV
crit. paths
1/f 1/f
MSV
Figure 1-5. Distribution of path delays under single and multiple supply voltages
Non-critical path runs with reduced supply voltage
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Vdd_low
Vdd_low Vdd_low
10ns
5ns
Non-critical path may be delayed
10ns
8ns
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Non-critical path runs with reduced supply voltage
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Vdd_low
Vdd_low Vdd_low
10ns
5ns
Non-critical path may be delayed
10ns
8ns
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Vdd_low
Vdd_low Vdd_low
10ns10ns
5ns5ns
Non-critical path may be delayed
10ns10ns
8ns8ns
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
12
A number of studies have shown significant variation in dynamic power
reduction results from implementing a multi-Vdd design strategy, ranging
from less than 10% up to almost 50%, with 40% being the average [15,16].
Rules of thumb for selecting appropriate supply voltage levels have been
developed. When using two supply voltages, the lower Vdd was proposed to
be 0.6x-0.7x of the higher Vdd [17]. The optimal supply voltage level also
depends on Vth [18].
The benefit of using multiple supply voltages quickly saturates. The
major gain is obtained by moving from a single Vdd to dual-Vdd. Extending
this to ever more supply voltage levels yields only small incremental benefits
[18,19], even when the overhead introduced by multiple supply voltages (see
below) is not taken into consideration.
The power reduction achieved by this technique roughly depends on two
parameters: the difference between the regular supply voltage Vdd and the
lowered supply voltage Vdd_low, and the percentage of gates to which Vdd_low
is applied.
Regarding the first parameter, it has been pointed out some years ago that
the leverage of this concept decreases as process technologies are scaled
down further [18].
Recent work has analyzed this in more detail [14]. At least for high-Vth
devices, which are essential for low standby power design due to their lower
leakage current, Vth has scaled much slower than Vdd recently. Therefore,
gate overdrive (Vdd - Vth) is diminished, negatively impacting performance.
Thus, even a little reduction in Vdd will have a very significant impact on
performance. Therefore, the potential to lower Vdd while maintaining overall
system performance is greatly reduced. It is shown that from 0.25µm down
to 0.09µm, the effectiveness of dual-Vdd decreases by a factor of 2 (from
60% dynamic power reduction to 30%) for high-Vth designs, whereas it stays
about constant for low-Vth designs. This can however be countered by
introduction of variable threshold voltages, as will be seen later.
Regarding the second parameter, experience has shown that especially in
designs using the multi-Vth technique outlined below, path delays tend to be
skewed to higher delays already, thus reducing the number of gates that can
be slowed down further [14].
For the selection of those gates which will receive the lower supply
voltage Vdd_low, a number of techniques have been proposed. Most prevalent
is the concept of clustered voltage scaling (CVS). It recognizes that it is
desirable to have clusters of gates assigned to the same voltage, since
between the output of a gate supplied by Vdd_low and the input of a gate
supplied by Vdd a level shifter is required to avoid static current flow [20].
This concept has been enhanced by extended clustered voltage scaling
(ECVS)[17] which essentially allows an arbitrary assignment of supply
13
voltage levels to gates. This strategy implies more frequent insertion of level
shifters into the design. However, usually only power consumption and
delay are considered in the literature. The additional area cost is neglected.
In industry, this certainly is not feasible.
While conceptually simple, the implementation of a multi-Vdd concept
poses a number of challenges.
• The additional supply voltage Vdd_low needs to be created on-chip by a dc-
to-dc converter, unless the voltage already exists externally. This results
in area overhead, and in power consumption for the converter.
• The additional supply voltage Vdd_low must be distributed across the chip.
• Level-shifters are required between different supply domains. It is
feasible to integrate level shifters into flip-flops [21].
The penalties in area, power consumption and delay resulting from these
effects are not always taken into account by work published in the literature.
Studies indicate that a 10% area overhead will result from implementing a
dual-Vdd design [22].
An additional consideration for industrial IC product development is that
EDA tool support for implementing a dual-Vdd design is still only
rudimentary. It is not sufficient to have a single point tool which can perform
power-performance tradeoffs. Instead, this methodology needs to encompass
the entire design flow (e.g. power distribution in layout; automated insertion
of level shifters etc.).
1.5.2 Multi-Vth Design
Another essential technique is the use of different transistor threshold
voltages (multi-Vth design). Primarily this technique reduces leakage power
consumption, thus increasing standby time of battery-powered ICs. As
leakage power consumption becomes an increasingly important component
of overall power consumption in modern process technologies, this
technique increasingly also helps to reduce overall power consumption
significantly, as design moves to more advanced process technologies. The
idea is similar to multi-Vdd design: paths that do not need highest
performance are implemented with special leakage-reduced transistors
(typically higher Vth transistors, but also thicker gate-oxide Tox), as shown
in Figure 1-6.
14
Figure 1-6. Multi-Vth design
A typical industrial approach today is to first create a design using lower
Vth transistors to achieve the required performance and then to selectively
replace gates off the critical path with higher Vth (or thicker Tox) transistors
to reduce leakage.
Studies in the literature have reported reductions in leakage of around
50% up to 80%. Some approaches assume that different Vth levels are
provided by the process technology (through doping variations) and propose
algorithms to optimally assign Vth levels to transistors, ensuring that
performance is not compromised [23, 24]. Recently, it has also been
proposed to achieve modifications in Vth by modifying transistor length or
gate oxide thickness Tox [25].
Design-tool support for this technique is also rudimentary at best. While
it is becoming established to design different modules of an IC with different
Vth transistors, it is very challenging to do this on the level of individual
transistors within a module. The primary reason is that the entire design flow
must be able to handle cells with identical functionality and size, which
differ in their electrical properties. This poses no principal algorithmic
problems, but must be consistently implemented in all EDA tools within a
design flow.
10ns
5ns
Non-critical path may be delayed
10ns
8ns
Non-critical path runs with increased threshold voltage
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
high Vt
high Vt high Vt
10ns
5ns5ns
Non-critical path may be delayed
10ns
8ns8ns
Non-critical path runs with increased threshold voltage
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
Q
QSET
CLR
D
high Vt
high Vt high Vt
15
1.5.3 Hybrid Approaches
Recently approaches have been suggested in the literature which combine
implementation of multiple supply voltages and multiple threshold voltages
for further power reduction. Especially for designs where minimization of
total power consumption is key (as compared to e.g. minimization of standby
power for mobile products), it is possible to trade off leakage and dynamic
power, as originally proposed in the zero-Vth concept. Studies in the
literature indicate a total power optimum when leakage power contributes
10% to 30% [26,12]. This ratio depends significantly on the process
technology, operating environment, and clock frequency of a design.
For applications where leakage power minimization is critical (e.g.
mobile products), this approach usually is not feasible, as it requires a
relatively low Vth which causes high leakage currents [14].
With the increasing significance of gate leakage currents, variations of
gate oxide thickness Tox have also been proposed.
An overall framework for using two supply voltages and two threshold
voltages as well has been presented [19]. Theoretically, it is shown that
more than 60% of total power consumption can be saved this way (not
considering required overhead such as level shifters, routing etc.). Rules of
thumb are proposed and it is shown that the optimal second Vdd is about 50%
of the original Vdd in this case. It is also argued that the usefulness of multi-
Vdd strategies is not diminished, but actually increased in more advanced
technologies, if also a multi-Vth strategy is followed, since this strategy
allows to trade off leakage vs. dynamic power consumption by changing Vth
and Vdd to optimize power consumption, while maintaining a required timing
performance.
This approach has been applied to the practical example of an ARM
processor in [27]. Due to specific layout considerations it was not possible to
implement all four intended combinations of Vdd and Vth. Instead, three
different libraries were implemented. Using a CVS algorithm, a reduction in
dynamic power by 15% was achieved for a 0.18µm process technology.
Leakage power was reduced by 40%. As leakage power was more than
1000x smaller than dynamic power, overall active power reduction was
15%. To achieve this, a 14% increase in area was required.
A very recent approach considers also transistor width sizing in addition
to Vdd and Vth assignment [28]. Using a two stage, sensitivity-based
approach, total power savings of 37% on average over a suite of benchmark
circuits are reported. In this study, the threshold voltage is chosen rather low,
so that leakage represents 20-50% of total power consumption. Therefore,
optimization of both leakage and dynamic power consumption is essential,
which is achieved with the presented approach.
16
An enhanced approach for leakage power consumption considers
multiple gate oxide thicknesses Tox in addition to multi-Vth [29]. It is
motivated by the fact that gate leakage increases very dramatically with
newer process technologies. Gate leakage is of the same order of magnitude
as subthreshold leakage at the 90nm process node. Their relationship also
depends significantly on the operating temperature T. The key observation
that an OFF transistor suffers from subthreshold leakage, an ON transistor
from gate leakage, motivates the approach to analyze transistor states in
standby mode and assign Vth and Tox such that leakage power consumption
is minimized. Leakage reductions of 5-6x are obtained on benchmark
circuits, compared to designs using a single Vth and Tox.
Previous approaches that included Tox into the optimization varied Tox
only for different design modules, not on critical paths within modules.
These newer approaches promise further reductions in power
consumption. This will come, however, at a price (as seen e.g. in the ARM
example). Design complexity increases significantly when variations in
many parameters are made available at the same time. In some studies, the
resulting overhead is not considered.
1.5.4 Cost Tradeoffs
This overhead must be considered, however, since it is quite significant:
• Multi-Vdd: level-shifter (area, power consumption, delay), routing of
additional supply voltages (area).
• Multi-Vth: additional masks (manufacturing costs); potentially special
design rules at the boundary between different Vth devices (area).
• Multi-Tox: additional masks (manufacturing costs).
• In addition, IC development costs increase due to more complex design
flows. Also, special process options (Vth, Tox) must be developed,
qualified and continuously monitored. For each such option, the design
library must be electrically characterized, modelled for all EDA tools,
and potentially optimized regarding circuit design and layout. It must be
maintained and regularly updated (changes in electrical parameters,
changes in tools in the design flow) over a long period of time as well. If
a very specialized manufacturing flow is developed to fully optimize a
given product, it will be very difficult to shift manufacturing of this
product to a different fab (e.g. a foundry in case additional capacity is
required).
For these and potentially other reasons, we are not yet aware of industrial
products that have implemented such proposals in a fine-grained manner (i.e.
different Vth, Vdd and Tox combined within one design module).
17
Some approaches in the literature also determine optimum levels of
threshold voltages depending on a given design. In industry, this is rarely
feasible. Typically, a manufacturing process has to be taken as given, with
only predefined values of Vth (and Tox) being available.
1.6 APPROACHES ON HIGHER ABSTRACTION
LEVELS
The approaches outlined above on gate level and device level can be (and
often must be) supported by measures on higher levels of abstraction.
Some of the most promising concepts are as follows:
• partitioning the system such that large areas can be powered off for
significant periods of time (block turnoff)
• especially partitioning memory systems such that large parts can be
turned off in standby mode
• clock gating is an essential method which reduces dynamic power
consumption by local off-switching of non-active gates
• coding strategies (e.g. for buses) can reduce switching and thus dynamic
power consumption
1.7 CONCLUSION AND FUTURE CHALLENGES
There is no single “silver bullet” to solve the challenge of power
reduction. While ultra-low voltage logic based on special ultra-low-Vth
devices is a conceptually very convincing concept, its widespread
implementation is hindered by manufacturing concerns. An extrapolation of
current technology trends indicates that such a concept will become even
more difficult in the future.
Today, design techniques are the most promising approach to reduce
power – both dynamic and leakage.
The concepts outlined here can be further extended. It is feasible to
dynamically adjust supply and threshold voltages. These are theoretically
promising concepts which however still require more investigation
especially with regard to feasibility under industrial boundary conditions.
Quite likely, in the future even more emphasis than today will have to be
placed on power reduction schemes on algorithmic and system level. On
these levels, the levers to reduce power consumption are largest.
Acknowledgement
The authors wish to acknowledge and thank Jörg Berthold and Tim
Schönauer for their contributions and fruitful discussions.
18
References
[1] G. Moore, Cramming More Components onto integrated circuits, Electronics Magazine,
Vol. 38, No. 8, 1965, pp. 114-117.
[2] ITRS, International Technology Roadmap for Semiconductors, 2003, http://public.itrs.net.
[3] F. Pollack, New Microarchitecture Challenges in the Coming Generations of CMOS
Process Technologies, Micro32 Keynote, 1999.
[4] U. Schlichtmann, Systems are Made from Transistors: UDSM Technology Creates New
Challenges for Library and IC Development, IEEE Euromicro Symposium on Digital
System Design, 2002, pp. 1-2.
[5] S. Borkar, Design Challenges of Technology Scaling, IEEE Micro, July/August 1999, pp.
23-29.
[6] S. Thompson, P. Packan, and M. Bohr, MOS Scaling: Transistor Challenges for the 21st
Century, Intel Technology Journal, Q3 1998.
[7] N. Kim et al., Leakage Current: Moore's Law Meets Static Power, IEEE Computer, Vol.
36, No. 12, December 2003, pp. 68-75.
[8] S. Sakurai, A. R. Newton, Alpha-Power Law MOSFET Model and its Application to
CMOS Inverter Delay and Other Formulas, IEEE Journal of Solid-State Circuits, Vol.
25, No. 2, 1990, pp. 584-594.
[9] J.B. Burr, J. Schott, A 200 mV self-testing encoder/decoder using Stanford ultra-low-
power CMOS, 1994 IEEE International Solid-State Circuits Conference
[10] J. Berthold, R. Nadal, C. Heer, Optionen für Low-Power-Konzepte in den sub-180-nm-
CMOS-Technologien (In German), U.R.S.I. Kleinheubacher Tagung 2002.
[11] V. Svilan, M. Matsui, J. B. Burr, Energy-Efficient 32 x 32-bit Multiplier in Tunable
Near-Zero Threshold CMOS, ISLPED 2000, pp. 268-272.
[12] V. Svilan, J. B. Burr, L. Tyler, Effects of Elevated Temperature on Tunable Near-Zero
Threshold CMOS, ISLPED 2001, pp. 255-258.
[13] C. Heer, Designing low-power circuits: an industrial point of view, PATMOS 2001
[14] T. Schoenauer, J. Berthold, C. Heer, Reduced Leverage of Dual Supply Voltages in Ultra
Deep Submicron Technologies, International Workshop on Power And Timing
Modeling, Optimization and Simulation PATMOS 2003, pp. 41-50.
[15] K. Usami, M. Igarashi, Low-Power Design Methodology and Applications utilizing Dual
Supply Voltages, Proceedings of the Asia and South Pacific Design Automation
Conference 2000, pp. 123-128.
[16] M. Donno, L. Macchiarulo, A. Macii, E. Macii, M. Poncino, Enhanced Clustered
Voltage Scaling for Low Power, Proceedings of the 12th ACM Great Lakes Symposium
on VLSI, 2002, pp. 18-23.
[17] K. Usami et al., Automated Low-Power Technique Exploiting Multiple Supply Voltages
Applied to a Media Processor, IEEE Journal of Solid-State Circuits, Vol. 33, No. 3,
March 1998, pp. 463-472.
[18] M. Hamada, Y. Ootaguro, T. Kuroda, Utilizing Surplus Timing for Power Reduction,
Proceedings IEEE Custom Integrated Circuits Conference CICC, 2001, pp. 89-92.
[19] A. Srivastava, D. Sylvester, Minimizing Total Power by Simultaneous Vdd/Vth
Assignment, Proceedings of the Asia and South Pacific Design Automation Conference
2003, pp. 400-403.
[20] K. Usami, M. Horowitz, Clustered Voltage Scaling Technique for Low-Power Design,
Proceedings of the International Symposium on Low Power Design ISLPD, 1995, pp. 3-
8.
19
[21] K. Usami et al., Design Methodology of Ultra Low-power MPEG4 Codec Core
Exploiting Voltage Scaling Techniques, Proceedings of the 35th Design Automation
Conference 1998, pp. 483-488.
[22] C. Yeh, Y.-S. Kang, Layout Techniques Supporting the Use of Dual Supply Voltages for
Cell-Based Designs, Proceedings of the 36th Design Automation Conference 1999, pp.
62-67.
[23] Q. Wang, S. Vrudhula, Algorithms for Minimizing Standby Power in Deep
Submicrometer, Dual-Vt CMOS Circuits, IEEE Transactions on CAD, Vol. 21, No. 3,
March 2002, pp. 306/318.
[24] L. Wei, Z. Chen, K. Roy, M. Johnson, Y. Ye, V. De, Design and Optimization of Dual-
Threshold Circuits for Low-Voltage Low-Power Applications, IEEE Transactions on
Very Large Scale Integration (VLSI), Vol. 7, No. 1, March 1999, pp. 16-24.
[25] N. Sirisantana, K. Roy, Low-Power Design Using Multiple Channel Lengths and Oxide
Thicknesses, IEEE Design & Test of Computers, January-February 2004, pp. 56-63.
[26] K. Nose, T. Sakurai, Optimization of VDD and VTH for Low-Power and High-Speed
Applications, Proceedings of the Asia and South Pacific Design Automation Conference
2000, pp. 469-474.
[27] R. Bai, S. Kulkarni, W. Kwong, A. Srivastava, D. Sylvester, D. Blaauw, An
Implementation of a 32-bit ARM Processor Using Dual Power Supplies and Dual
Threshold Voltages, IEEE International Symposium on VLSI, 2003, pp. 149-154.
[28] A. Srivastava, D. Sylvester, D. Blaauw, Concurrent Sizing, Vdd and Vth Assignment for
Low-Power Design, Proceedings of the Design, Automation and Test in Europe
Conference DATE, 2003, pp. 718-719.
[29] D. Lee, H. Deogun, D. Blaauw, D. Sylvester, Simultaneous State, Vt and Tox
Assignment for Total Standby Power Minimization, Proceedings of the Design,
Automation and Test in Europe Conference DATE, 2003, pp. 494-499.
20
Chapter 2
ON-CHIP OPTICAL INTERCONNECT FORLOW-POWER
Ian O’Connor and Frederic GaffiotEcole Centrale de Lyon
Abstract It is an accepted fact that process scaling and operating frequency both contributeto increasing integrated circuit power dissipation due to interconnect. Extrapolat-ing this trend leads to a red brick wall which only radically different interconnectarchitectures and/or technologies will be able to overcome. The aim of this chap-ter is to explain how, by exploiting recent advances in integrated optical devices,optical interconnect within systems on chip can be realised. We describe ourvision for heterogeneous integration of a photonic “above-IC" communicationlayer. Two applications are detailed: clock distribution and data communicationusing wavelength division multiplexing. For the first application, a design methodwill be described, enabling quantitative comparisons with electrical clock trees.For the second, more long-term, application, our views will be given on the useof various photonic devices to realize a network on chip that is reconfigurable interms of the wavelength used.
Keywords: Interconnect technology, optical interconnect, optical network on chip
2.1 INTRODUCTION
In the 2003 edition of the ITRS roadmap [17], the interconnect problem wassummarised thus: “For the long term, material innovation with traditional scal-ing will no longer satisfy performance requirements. Interconnect innovationwith optical, RF, or vertical integration ... will deliver the solution”. Continu-ally shrinking feature sizes, higher clock frequencies, and growth in complexityare all negative factors as far as switching charges on metallic interconnect isconcerned. Even with low resistance metals such as copper and low dielectricconstant materials, bandwidths for long interconnect will be insufficient for fu-ture operating frequencies. Already the use of metal tracks to transport a signalover a chip has a high cost in terms of power: clock distribution for instance
21
requires a significant part (30-50%) of total chip power in high-performancemicroprocessors.
A promising approach to the interconnect problem is the use of an opticalinterconnect layer, which could empower an increase in the ratio between datarate and power dissipation. At the same time it would enable synchronous op-eration within the circuit and with other circuits, relax constraints on thermaldissipation and sensitivity, signal interference and distortion, and also free uprouting resources for complex systems. However, this comes at a price. Firstly,high-speed and low-power interface circuits are required, design of which isnot easy and has a direct influence on the overall performance of optical inter-connect. Another important constraint is the fact that all fabrication steps haveto be compatible with future IC technology and also that the additional costincurred remains affordable. Additionally, predictive design technology is re-quired to quantify the performance gain of optical interconnect solutions, whereinformation is scant and disparate concerning not only the optical technology,but also the CMOS technologies for which optics could be used (post-45nmnode).
In section 2.2, we will describe the “above-IC” optical technology. Sections2.3 and 2.4 describe an optical clock distribution network and a quantitativeelectrical-optical power comparison respectively. A proposal for a novel opticalnetwork on chip in discussed in section 2.5.
2.2 OPTICAL INTERCONNECT TECHNOLOGY
Various technological solutions may be proposed for integrating an opticaltransport layer in a standard CMOS system. In our opinion, the most promisingapproach makes use of hybrid (3D) integration of the optical layer above acomplete CMOS IC, as shown in fig. 2.1. The basic CMOS process remainsthe same, since the optical layer can be fabricated independently. The weaknessof this approach is in the complex electrical link between the CMOS interfacecircuits and the optical sources (via stack and advanced bonding).
In the system shown in fig. 2.1, a CMOS source driver circuit modulatesthe current flowing through a biased III-V microsource through a via stackmaking the electrical connection between the CMOS devices and the opticallayer. III-V active devices are chosen in preference to Si-based optical devicesfor high-speed and high-wavelength operation. The microsource is coupled tothe passive waveguide structure, where silicon is used as the core and SiO2
as the cladding material. Si/SiO2 structures are compatible with conventionalsilicon technology and silicon is an excellent material for transmitting wave-lengths above 1.2µm (mono-mode waveguiding with attenuation as low as 0.8dB/cm has been demonstrated [10]). The waveguide structure transports theoptical signal to a III-V photodetector (or possibly to several, as in the case of
22
drivercircuit
receivercircuit
electricalcontact
III−Vphotodetector
III−Vlaser source
Si photonicwaveguide (n=3.5)
SiO2 waveguidecladding (n=1.5)
CMOS IC
met
allic
inte
rcon
nect
str
uctu
re
Figure 2.1. Cross-section of hybridised interconnection structure
a broadcast function) where it is converted to an electrical photocurrent, whichflows through another via stack to a CMOS receiver circuit which regeneratesthe digital output signal. This signal can then if necessary be distributed over asmall zone by a local electrical interconnect network.
2.3 AN OPTICAL CLOCK DISTRIBUTIONNETWORK
In this section we present the structure of the optical clock distribution net-work, and detail the characteristics of each component part in the system: ac-tive optoelectronic devices (external VCSEL source and PIN detector), passivewaveguides, interface (driver and receiver) circuits. The latter represent ex-tremely critical parts to the operation of the overall link and require particularlycareful design.
An optical clock distribution network, shown in fig. 2.2, requires a singlephotonic source coupled to a symmetrical waveguide structure routing to anumber of optical receivers. At the receivers the high-speed optical signal isconverted to an electrical one and provided to local electrical networks. Hencethe primary tree is optical, while the secondary tree is electrical. It is not feasibleto route the optical signal all the way down to the individual gate level sinceeach drop point requires a receiver circuit which consumes area and power.The clock signal is thus routed optically to a number of drop points which willcover a zone over which the last part of the clock distribution will be carried out
23
by the electrical secondary clock tree. The size of the zones is determined bycalculating the power required to continue in the optical domain and comparingit to the power required to distribute over the zone in the electrical domain. Thenumber of clock distribution points (64 in the figure) is a particularly crucialparameter in the overall system.
The global optical H-tree was optimised to achieve minimal optical lossesby designing the bend radii to be as large as possible. For 20mm die width and64 output nodes in the H-tree at the 70nm technology mode, the smallest radiusof curvature (r3 in fig. 2.2) is 625µm, which leads to negligible pure bendingloss.
r2
r3
opticalreceivers
opticalwaveguides
electricalclock trees
CVL : source−waveguidecoupling loss
WL : waveguidetransmission loss
YL : Y−couplerloss
BL : bendingloss
CRL : waveguide−receivercoupling loss
opticalsource
die width, D
r1=D/8, r2=D/16, r3=D/32
r1 LB
LY
LCR
LW
LCV
Figure 2.2. Optical H-tree clock distribution network (OCDN) with 64 output nodes. r1−3 arethe bend radii linked to the chip width D
2.3.1 VCSEL sources
VCSELs (Vertical Cavity Surface Emitting Lasers) are certainly the mostmature emitters for on-chip or chip-to-chip interconnections. Commercial VC-SELs, when forward biased at a voltage well above 1.5V, can emit opticalpower of the order of a few mW around 850nm, with an efficiency of some40%. Threshold currents are typically in the mA range. However, fundamentalrequirements for integrated semiconductor lasers in optical interconnect appli-cations are small size, low threshold lasing operation and single-mode operation(i.e. only one mode is allowed in the gain spectrum). Additionally, the factthat VCSELs emit light vertically makes coupling less easy. It is clear that
24
significant effort is required from the research community if VCSELs are tocompete seriously in the on-chip optical interconnect arena, to increase wave-length, efficiency and threshold current in the same device. Long wavelength,and low-threshold VCSELs are only just beginning to emerge (for example, a1.5µm, 2.5Gb/s tuneable VCSEL [5], and an 850nm, 70µA threshold current,2.6µm diameter CMOS compatible VCSEL [11] have been reported). Ulti-mately however, optical interconnect is more likely to make use of integratedmicrosources as described in section 2.5, as these devices are intrinsically bettersuited to this type of application.
2.3.2 PIN photodetectors
In order to optimise the frequency and power dissipation performance of theoverall link, photodetectors must exhibit high quantum efficiency, large intrinsicbandwidth and small parasitic capacitance. The photodetector performance ismeasured by the bandwidth efficiency product.
Conventional III-V PIN devices suffer from two main limitations. On onehand, their relatively high capacitance per unit area leads to limitations in thedesign of the transconductance amplifier interface circuit. On the other hand,due to its vertical structure, there is a tradeoff between its frequency performanceand its efficiency (the quantum efficiency increases and the bandwidth decreaseswith the absorption intrinsic layer thickness) [9].
Metal-semiconductor-metal (MSM) photodetectors offer an alternative overconventional PIN photodetectors. An MSM photodetector consists of interdig-itated metal contacts on top of an absorption layer. Because of their lateralstructure, MSM photodetectors have very high bandwidths due to their lowcapacitance and the possibility to reduce the carrier transit time. However,the responsivity is usually low compared to PIN photodetectors [4]. MSMphotodiodes with bandwidth greater than 100GHz have been reported.
2.3.3 Waveguides
Optical waveguides are at the heart of the optical interconnect concept.In the Si/SiO2 approach, the high relative refractive index difference ∆ =(n2
1 − n22)/2n
21 between the core (n1 ≈ 3.5 for Si) and cladding (n2 ≈ 1.5 for
SiO2) allows the realisation of a compact optical circuit with dimensions com-patible with DSM technologies. For example, it is possible to realise monomodewaveguides less than 1µm wide (waveguide width of 0.3µm for wavelengthsof 1.55µm), with bend radii of the order of a few µm [15].
However, the performance of the complete optical system depends on theminimum optical power required by the receiver and on the efficiency of passiveoptical devices used in the system. The total loss in any optical link is the sum
25
of losses (in decibels) of all optical components:
Ltotal = LCV + LW + LB + LY + LCR (2.1)
where
LCV is the coupling coefficient between the photonic source and opticalwaveguide. There are currently several methods to couple the beamemitted from the laser into the optical waveguide. In this analysis weassumed 50% coupling efficiency LCV from the source to a single modewaveguide.LW is the rectangular waveguide transmission loss per unit distance ofthe optical power. Due to small waveguide dimensions and large in-dex change at the core/cladding interface in the Si/SiO2 waveguide theside-wall scattering is the dominant source of loss (fig. 2.3a). For thewaveguide fabricated by Lee [10] with roughness of 2nm the calculatedtransmission loss is 1.3dB/cm.LB is the bending loss, highly dependent on the refractive index difference∆ between the core and cladding medium. In Si/SiO2 waveguides, ∆ isrelatively high and so due to this strong optical confinement, bend radiias small as a few µm may be realised. As can be seen from fig. 2.3b,the bending losses associated with a single mode strip waveguide arenegligible if the radius of curvature is larger then 3µm.LY is the Y-coupler loss, and depends on the reflection and scatteringattenuation into the propagation path and surrounding medium. For highindex difference waveguides the losses for the Y-branch are significantlysmaller than for low ∆ structures and the simulated losses are less then0.2dB per split [14].LCR is the coupling loss from the waveguide to the optical receiver.Using currently available materials and methods it is possible to achievean almost 100% coupling efficiency from waveguide to optical receiver.In this analysis the coupling efficiency is assumed to be 87% (LCR =0.6dB) [16].
2.3.4 Interface circuits
High-speed CMOS optoelectronic interface circuits are crucial building blocksto the optical interconnect approach. The electrical power dissipation of thelink is defined by these circuits, but it is the receiver circuit that poses the mostserious design challenges. The power dissipated by the source driver is mainlydetermined by the source bias current and is therefore device-dependent. Onthe receiver side however, most of the receiver power is due to the circuit, whileonly a small fraction is required for the photodetector device.
26
0
10
20
30
40
50
60
1 2 3 4 5 6 7 8 9 10 11 12
Tra
nsm
issi
on lo
ss (
dB/c
m)
Sidewall roughness (nm)
Figure 2.3a. Simulated transmission lossfor varying sidewall roughness in a0.5µm× 0.2µm Si/SiO2 strip waveguide
1e-14
1e-12
1e-10
1e-08
1e-06
0.0001
0.01
1
100
2 3 4 5 6 7 8 9
Pur
e be
ndin
g lo
ss (
dB)
Bend radius (um)
Figure 2.3b. Simulated pure bending lossfor various bend radii in a 0.5µm× 0.2µmSi/SiO2 strip waveguide
2.3.4.1 Driver circuits. Source driver circuits generally use a currentmodulation scheme for high-speed operation. The source always has to bebiased above its threshold current by a MOS current sink to eliminate turn-ondelays, which is why low-threshold sources are so important (figures of theorder of 40µA [7] have been reported). A switched current sink modulatesthe current flowing through the source, and consequently the output opticalpower injected into the waveguide. As with most current-mode circuits, highbandwidth can be achieved since the voltage over the source is held relativelyconstant and parasitic capacitances at this node have reduced influence on thespeed.
2.3.4.2 Receiver circuits. A typical structure for a high-speed pho-toreceiver circuit consists of: a transimpedance amplifier (TIA) to convert thephotocurrent of a few µA into a voltage of a few mV; a comparator to gener-ate a rail-to-rail signal; and a data recovery circuit to eliminate jitter from therestored signal. Of these, the TIA is arguably the most critical component forhigh-speed performance, since it has to cope with a generally large photodiodecapacitance situated at its input.
The basic transimpedance amplifier structure in a typical configuration isshown in fig. 2.4 [8]. The bandwidth/power ratio of this structure can be max-imised by using small-signal analysis and mapping of the individual componentvalues to a filter approximation of Butterworth type.
It is then possible to develop a synthesis procedure which, from desiredtransimpedance performance criteria (gain Zg0, bandwidth and pole qualityfactor Q) and operating conditions (photodiode and load capacitances, Cd andCl respectively) generates component values for the feedback resistance Rf andthe voltage amplifer (voltage gain Av and output resistance Ro). Circuits withhigh Ro/Av ratio (≈ 1/
∑gm) require the least quiescent current and area and
this quantity constitutes therefore an important figure of merit in design space
27
I i
Rf
−Av
ClVo
CdVdd
Vi Vo
Ci
Cm
Co
M1
M2 M3
C = C + Cd i C = C + Colx y
ω 0=R Co y
1 1 + Av
f x mM (M + M (1 + M ))x
Z = −g0fR − R /Ao v
1 + 1/Av( ) fM = R / Rf o
iM = C / Cx y
M = C / Cmm y
Q = 1 + Avf x m xM (M + M (1 + M ))( )1 + Av1 + M (1 + M ) + M M ( )x f m f
Figure 2.4. CMOS transimpedance amplifier structure
exploration (fig. 2.5a). To reach a sized transistor-level circuit, approximateequations for the small-signal characteristics and bias conditions of the circuitare sufficient to allow a first-cut sizing of the amplifier, which can then be fine-tuned by numerical or manual optimisation, using simulation for exact results.The complete process is described in [13].
Amplifier Ro/Av requirementCi=500fF Cl=100fF
Ro/Av 300 250 200 150 100 50
1
3
10
Bandwidthrequirement
/GHz 1000
3000
10000
Transimpedancegain
requirement/ohms
050
100150200250300350400
Figure 2.5a. TIA Ro/Av designspace with varying bandwidth andtransimpedance gain requirements
0.1
1
10
100
350 180 130 100 70 45
Technology node (nm)
1THzohm Transimpedance amplifier characteristics against technology nodeCd = 400fF, Cl = 150fF
Area / um2Quiescent power / 100uW
Figure 2.5b. Evolution of TIA character-istics (power, area, noise) with technologynode
Using this methodology with industrial transistor models for technologynodes from 350nm to 180nm and predictive BSIM3v3/BSIM4 models for tech-nology nodes from 130nm down to 45nm [3], we generated design parametersfor 1THzΩ transimpedance amplifiers to evaluate the evolution in critical char-acteristics with technology node. Fig. 2.5b shows the results of transistor levelsimulation of fully generated photoreceiver circuits at each technology node.
28
2.4 QUANTITATIVE POWER COMPARISONBETWEEN ELECTRICAL AND OPTICAL CLOCKDISTRIBUTION NETWORKS
2.4.1 Design methodology
In an optical link there are two main sources of electrical power dissipation:(i) power dissipated by the optical receiver(s) and (ii) energy needed by theoptical source(s) to provide the required optical output power. To estimate theelectrical power dissipated in the system we developed the methodology shownin fig. 2.6.
losses in passivewaveguide network
minimum opticalpower at receiver
minimum opticalpower at source
photodiode characteristics
dark(R,C ,I )
d
transimpedanceamplifier
BER specification(SNR requirement)
electrical power dissipatedin optical system
emitterpower
receiverpower
sourceefficiency
Figure 2.6. Methodology used to estimate the electrical power dissipation in an optical clockdistribution network
The first criterion for defining the performance of the optoelectronic link isthe required signal transmission quality, represented by the bit error rate (BER)and directly linked to the photoreceiver signal to noise ratio. For an on-chipinterconnect network, a BER of 10−15 is acceptable. To calculate the requiredsignal power at the receiver, the characteristics of the receiver circuit have tobe extracted from the transistor-level schematic, which is generated from thephotodetector characteristics (responsivity R, Cd, dark current Idark) and fromthe required operating frequency using the method described in section 2.3.For the given BER and for the noise signal associated with the photodiode andtransimpedance circuit the minimum optical power required by the receiverto operate at the given error probability can be calculated using the Morikuniformula [12].
With this figure, and knowing the layout and therefore the optical losses thatwill be incurred in the waveguides, the minimum required optical power at thesource can be estimated. The total electrical power dissipated in the optical
29
link is the sum of the power dissipated by the number of optical receivers andthe energy needed by the source to provide the required optical power. Theelectrical power dissipated by the receivers can be extracted from transistor-level simulations. To estimate the energy needed by the optical source, laserlight-current characteristics given by Amann [1] were used.
2.4.2 Design performance
Our aim in this work was to quantitatively compare the power dissipationin electrical and optical clock distribution networks for a number of cases, in-cluding technology node prediction. For both electrical and optical cases weused technology parameters from the ITRS roadmap (wire geometry, materialparameters). For transistor models we used predictive model parameters fromBerkeley (BSIM3V3 down to 70nm and BSIM4 down to 45nm). The powerdissipated in the electrical system can be attributed to the charging and discharg-ing of the wiring and load capacitance and to the static power dissipated by thebuffers. In order to calculate the power we used an internally developed simu-lator, which allows us to model and calculate the electrical parameters of clocknetworks for future technology nodes [18]. For optical performance predictionswe used existing technology characteristics while for the optoelectronic deviceswe took datasheets from two real devices and used these figures.
The power dissipated in clock distribution networks was analysed in bothsystems at the 70nm technology node. Power dissipation figures for electricaland optical CDNs were calculated based on the system performance summarisedin tables 2.1a and 2.1b.
Table 2.1a. Electrical CDN characteris-tics
Electrical system parameter
Technology (nm) 70Vdd (V) 0.9Tox (nm) 1.6Chip size (mm2) 400Global wire width (µm) 1Metal resistivity (Ω-cm) 2.2Dielectric constant 3Optimal segment length (mm) 1.7Optimal buffer size (µm) 90
Table 2.1b. Optical CDN characteristics
Optical system parameter
Wavelength λ (nm) 1550Waveguide core index (Si) 3.47Waveguide cladding index (SiO2) 1.44Waveguide thickness (µm) 0.2Waveguide width (µm) 0.5Transmission loss (dB/cm) 1.3Loss per Y-junction (dB) 0.2Input coupling coefficient (%) 50Photodiode capacitance (fF) 100Photodiode responsivity (A/W) 0.95
What follows is the results of comparisons of the power dissipation in elec-trical and optical clock distribution networks. This was quantitatively carriedout for varying chip size, operating frequency, number of clock distributionpoints, technology node, and finally sidewall roughness. This latter perfor-
30
mance characteristic is the only non system-driven characteristic, but it givessome important design information to technology groups working on opticalinterconnect.
Fig. 2.7a shows a power comparison where we vary square die size from 10 to37 mm width. This analysis was carried out for the 70nm node at a distributionfrequency of 5.6GHz (which is the clock frequency associated with this node)and 256 drop points. Electrical CDN power rises almost linearly with die size,which is understandable since the line lengths increase and therefore requiremore buffers to drive them. Optical CDN power rises much more slowly sinceall that is really changing is transmission loss and this has a quite minor effecton the overall power dissipation.
When we vary clock frequency for constant chip width, fig. 2.7b we observea similar effect for the electrical CDN. Again, the number of buffers has toincrease since the segment lengths have to be reduced in order to attain the lowerRC time constants. For the optical CDN, what is changing is the receiver powerdissipation. The transimpedance amplifier requires a lower output resistance inorder to operate at higher frequencies and this translates to a higher bias current.
In fig. 2.7c, we vary the number of drop points and see that both electricaland optical CDN power dissipation rises, but optical rises much faster thanelectrical. There are two reasons for this: firstly, every time the number of droppoints is doubled, so is the number of receivers and this accounts for a large partof the power dissipation; secondly, the number of splitters is doubled, whichin turn means that the power at emission also has to be doubled. These twofactors cause the optical power to catch up with the electrical power at around4000 drop points.
Fig. 2.7e shows a comparison for varying technology node. Not only thetechnology is changing here, we are also changing the clock frequency asso-ciated with the node. We can see that at the 70nm node there is a five-folddifference between electrical and optical clock distribution. As the technologynode advances, this difference becomes even more marked.
A final analysis, fig. 2.7f, shows how technological advances are requiredto improve system performance, concerning in this case waveguide sidewallroughness. 5nm roughness translates to a transmission loss of around 8dB/cm,which in turn corresponds to a power dissipation figure of around 500mWfor the 70nm node at 5.6GHz and 20mm chip width. Looking at the 2nmroughness point, achieved at MIT [10] and corresponding to a transmissionloss of 1.3dB/cm, we obtain a power dissipation figure of about 10mW, a fifty-fold decrease in the overall power dissipation by going from 5nm roughness to2nm roughness. This demonstrates the importance of optimising the passivewaveguide technology for the whole system.
31
200
400
600
800
1000
1200
1400
1600
100 300 500 700 900
Pow
er d
issi
patio
n (m
W)
Die size (mm2)
Electrical CDNOptical CDN
Figure 2.7a. Comparison of power dissi-pation in electrical and optical clock dis-tribution networks for varying chip size(70nm technology, 5.6GHz, 256 droppoints)
0
200
400
600
800
1000
1200
1400
1 3 5 7
Pow
er d
issi
patio
n (m
W)
Clock frequency (GHz)
Electrical CDN 256Optical CDN 256
Electrical CDN 128Optical CDN 128
Figure 2.7b. Comparison of power dissi-pation in electrical and optical clock dis-tribution networks for varying clock fre-quency (70nm technology, 400mm2, 256drop points)
1
10
100
1000
10000
4 32 256 2048 8172
Pow
er d
issi
patio
n (m
W)
Number of drop points (nodes)
Electrical CDNOptical CDN
Figure 2.7c. Comparison of power dissi-pation in electrical and optical clock dis-tribution networks for varying number ofdrop points (70nm technology, 5.6GHz,400mm2)
-50
-40
-30
-20
-10
0
10
20
30
4 32 256 2048 8172
Pow
er g
ain
optic
al/e
lect
rical
(%
)
Number of drop points (nodes)
Figure 2.7d. Comparison of power dissi-pation in electrical and optical clock dis-tribution networks for varying number ofdrop points (70nm technology, 5.6GHz,400mm2)
0
200
400
600
800
1000
1200
130 100 70 45
Pow
er d
issi
patio
n (m
W)
Technology node (nm)
Electrical CDNOptical CDN
Figure 2.7e. Comparison of power dissi-pation in electrical and optical clock dis-tribution networks for varying technologynodes
10
100
1000
1 3 5 7 9
Pow
er d
issi
patio
n (m
W)
Waveguide transmission loss (dB/cm)
Optical CDN 256Optical CDN 128
Figure 2.7f. Evaluation of power dissipa-tion in optical clock distribution networksfor varying waveguide sidewall roughness(70nm technology, 5.6GHz, 400mm2)
For a BER of 10−15 the minimal power required by the receiver is -22.3dBm(at 3GHz). Losses incurred by passive components for various nodes in theH-tree are summarised in table 2.2.
32
Table 2.2. Optical power budget for 20mm die width at 3GHz
Number of nodes in H-tree 16 32 64 128
Loss in straight lines (dB) 1.3 1.3 1.3 1.3Loss in curved lines (dB) 1.53 1.66 1.78 1.85Loss in Y-dividers (dB) 12 15 18 21Loss in Y-couplers (dB) 0.8 1 1.2 1.4Output coupling loss (dB) 0.6 0.6 0.6 0.6Input coupling loss (dB) 3 3 3 3Total optical loss (dB) 19.2 22.5 25.8 29.1Min. receiver power (dBm) -22.3 -22.3 -22.3 -22.3Laser optical power (mW) 0.5 1.1 2.30 4.85
We can conclude from this analysis that power dissipation in optical clockdistribution networks is lower than that of electrical clock distribution networks,by a factor of five for example at the 70nm technology node. This factor will inthe future become larger due to two reasons: firstly due to improvements in opti-cal fabrication technology; and secondly with the rise in operating frequencies.However, this figure is probably not sufficient to convince semiconductor manu-facturers to introduce such large technological and methodological changes forthis application. To improve the figure, weak points can be identified for eachmain part of an integrated optical link. For the source, the efficiency betweenelectrical and optical power conversion is relatively low. This needs to be im-proved and one area is possibly in integrated microsources. For the waveguidestructures, most of the losses need to be improved, especially transmission lossand coupling loss. Sidewall roughness especially has a direct and considerableimpact on the power dissipation in the global system. Finally at the receiverend, the transimpedance amplifier power dissipation is too high. Better circuitstructures must be devised, or the photodetector parasitic capacitance needs tobe reduced.
2.5 OPTICAL NETWORK ON CHIP
In current SoC architectures, global data throughput between functionalblocks can reach up to tens of gigabits per second, the load being shared byseveral communication buses. In the future the constraints acting on such dataexchange networks will continue to increase: the number of IP blocks in anintegrated system could be as high as several hundred and the global throughputcould reach the Tb/s scale. To provide this level of performance, the communi-cation system itself is designed as an IP block into which the various functionalunits will be connected. This type of standardised hardware communicationarchitecture is called a network on a chip (NoC).
33
Using wavelength division multiplexing (WDM) techniques, photonics andoptoelectronics may offer new solutions to realise reconfigurable optical net-works on chip (ONoC). An ONoC, as an electronic router with routing based onwavelength λ, is actually a circuit-switching based topology and can thus ensuredata exchanges between IP blocks with very low contention. The advantagesof using an optical network are many: independence of interconnect perfor-mance from distance and data rate, crosstalk reduction, connectivity increase,interconnect power dissipation reduction, increase in the size of isochronoustiles, use of communication protocols. Figure 2.8 shows a 4× 4 ONoC with allelectronic interfaces: photodetector and laser in III-V technology and opticalnetwork in SOI technology, using similar heterogeneous integration techniquesas described in section 2.2. Intellectual property (IP) blocks shown can be pro-cessor cores, memory blocks, functional units etc. with standard interfaces tothe communication network. This is a multi-domain device with high speedoptoelectronic circuits (modulation of the laser current and photodetectors) andpassive optics (waveguides and passive filters). In the figure, M are masters(processor, IP, ...) which can communicate with targets T (memory, ...). Thenetwork is comprised of 4 stages, each associated with a single resonant wave-length. The operation of the 4×4 network is summarised in the table of figure2.3. This system is a fully passive circuit-switching network based on wave-length routing and is a non-blocking network. From Mi to Tj , there exists onlyone physical path associated with one wavelength. At any one time, single-wavelength emitters can make 4 connections and multi-wavelength emitterscan make 12 connections. The network is in principle scalable to an infinitenumber of connections. In practice, this number is severely limited by lithog-raphy and etching precision. For a 5nm tolerance on the size of the microdisk,corresponding to state of the art CMOS process technology, the maximum sizeof the network is 8 × 8.
Table 2.3. Truth table for optical network on chip
T1 T2 T3 T4
M1 λ2 λ3 λ1 λ4
M2 λ3 λ4 λ2 λ1
M3 λ1 λ2 λ4 λ3
M4 λ4 λ1 λ3 λ2
The basic element of the network is an optical filter, described in the nextsection. The ports 1 − 4 correspond to inputs/outputs of the optical filter.Its operation is the same as an electronic cross-bar: the cross function (outputin 4) is activated when the injected wavelength in 1 does not correspondto a resonant ring wavelength and the bar function is activated (output in 3)when the injected wavelength in 1 corresponds to a resonant ring wavelength.
34
1
1
2
3
3
4
M1
M2
M3
M4
T1
T2
T3
T4
IPM1
IPM2
IPM3
IPM4
IPT1
IPT2
IPT3
IPT4
masterinterface(driver,laser)
targetinterface(detector,receiver)
passive opticalnetwork on chip
=
=
i
n
n
n
i
i
elementary opticalfilter operation
targetIP blocks
masterIP blocks
#1 #3
#4#2
Figure 2.8. Architecture of 4x4 optical network on chip
Operation is symmetrical: the same phenomena happens if the wavelengthinjection is placed in the port 4.
2.5.1 Microresonators
Microring resonators are ideal device candidates for integrated photonic cir-cuits. Because they render possible the addition or extraction of signals froma waveguide based on wavelength in a WDM flow, they can be considered asbasic building blocks to build complex communication networks. The use ofstandard SOI technology leads to high compactness (structures with radii assmall as 4µm have been reported) and the possibility of low-cost photonic in-tegration. Figure 2.9 shows the structure of an elementary add-drop filter basedon microring resonators. The size of the structure is typically a few hundredµm2. It consists of two identical disks evanescently side-coupled to two signalwaveguides which are crossed at near right angles to facilitate signal direc-tivity. The microdisks make up a selective structure: the electromagnetic fieldpropagates in the rings for discrete propagation modes corresponding to specificwavelengths. The resonant wavelengths depend on geometric and structural pa-rameters (indices of the substrate and of the microrings, thickness and diameterof the disks).
The basic function of a microresonator can be thought of as a wave-length-controlled switching function. If the wavelength of an optical signal passingthrough a waveguide in proximity to the resonator (for example injected at port1) is close enough to a resonant wavelength λ1 (tolerance is of the order of a fewnm, depending on the coupling strength between the disk and the waveguide),then the electromagnetic field is coupled into the microrings and then out alongthe second waveguide (in the example, the optical signal is transmitted to the
35
r=
r=
r i
ri#2
#3 #4
#1
#1
#2
#3
r #4
10um
30um
Figure 2.9. Micro-disk realisation of an add-drop filter
output port 3, as shown in fig. 2.10a). If the wavelength of the optical signaldoes not correspond to the resonant wavelength, then the electromagnetic fieldcontinues to propagate along the waveguide and not through the structure (inthe example, the optical signal would then be transmitted to the output port 4,as shown in fig. 2.10b). This device thus operates as an elementary router, thebehaviour of which is summarised in the table in fig. 2.9.
Figure 2.10a. FDTD simulation of add-drop filter in on-state
Figure 2.10b. FDTD simulation of add-drop filter in off-state
First structures have been realised and preliminary results are promising.Fig. 2.11a shows an IR photograph of the structure in the cross state (top) andin the bar state (bottom), while fig. 2.11b represents the transmission coefficienton the cross output: the transmitted power on the cross output reaches 100%for wavelengths corresponding to the resonant frequencies of the microdisk.
2.5.2 Microsource lasers
From the viewpoint of mode field confinement and mirror reflection, mi-crodisk lasers operate on the principle of total internal reflection, as opposed tomultiple reflection, as is the case in VCSELs for example. This fact gives thistype of source two distinct advantages over VCSELs for on-chip optical inter-connect. Firstly, light emission is in-plane (as opposed to vertical), meaning
36
Figure 2.11a. Infra-red photograph ofstructure in both cross (top) and bar (bot-tom) states
Figure 2.11b. Transmission coefficient oncross output for varying wavelength
that emitted light can be injected directly into a waveguide with minimum loss[6]. Secondly, for communication schemes requiring multiple wavelengths,it is easier from a technological point of view to control the radius of such adevice than it is to control the thickness of an air gap in a VCSEL. In any casesuch devices, to be compatible with dense photonic integration, must satisfy therequirements of small volume and high optical confinement, with low thresholdcurrent and emitting in the 1.3-1.6µm range. Although these devices are notas mature as VCSELs, they seem extremely promising for optical interconnectapplications. An overview of microcavity semiconductor lasers can be foundin [2].
2.5.3 Demonstration of principle
Behavioural models enable us to verify the operation of the 4 × 4 ONoC athigh level simulation. An injection of 4 wavelengths is realised (λ1, λ2, λ3,and λ4) at the port 1 at the same moment (shown in figure 2.12). The inputsignal format is a matrix. Figure 2.12 is a 3-dimensional representation withwavelength on the X-axis (representing the 4 channels), time on the Y-axis andpower (normalised) on the vertical axis. Each injected wavelength has twopulses (Gaussian) in time. The behavioural simulation analyses the 4 outputsT1, T2, T3 and T4 (T2 shown in fig. 2.12). As predicted in table 2.3, only λ3 isdetected at the output T2.
Figure 2.12. Simulation of 4x4 optical network on chip
37
2.6 CONCLUSION
Integrated optical interconnect is one potential technological solution to al-leviate some of the more pressing issues involved in moving volumes of databetween circuit blocks on integrated circuits. In this chapter, we have shownhow novel integrated photonic devices can be fabricated above standard CMOSICs, designed concurrently with EDA tools and used in clock distribution andNoC applications. The feasibility of on-chip optical interconnect is no longerreally in doubt. We have given some partial results to quantitatively demon-strate the advantages of optical clock distribution. Although lower power canbe achieved (of the order of a five-fold decrease), more work is required toexplore new solutions that benefit from advances both at the architectural andat the technological level. Also the existing basic building blocks need to beintegrated together to physically demonstrate on-chip optical links. Research iswell under way in several research groups around the world to do this. Lookingfurther ahead, the use of multiple wavelengths in on-chip communication net-works and in reconfigurable computing is an extremely promising and excitingfield of research.
References
[1] M. Amann, M. Ortsiefer, and R. Shau: 2002, ‘Surface-emitting LaserDiodes for Telecommunications’. In: Proc. Symp. Opto- and Microelec-tronic Devices and Circuits.
[2] T. Baba: 1997, ‘Photonic Crystals and Microdisk Cavities Based onGaInAsP-InP System’. IEEE J. Selected Topics in Quantum Electronics 3.
[3] Y. Cao, T. Sato, D. Sylvester, M. Orchansky, and C. Hu: 2000, ‘NewParadigm of Predictive MOSFET and Interconnect Modeling for Early Cir-cuit Design’. In: Proc. Custom Integrated Circuit Conference.
[4] S. Cho et al.: 2002, ‘Integrated detectors for embedded optical interconnec-tions on electrical boards, modules and integrated circuits’. IEEE J. Sel.Topics in Quantum Electronics 8.
[5] A. Filios et al.: 2003, ‘Transmission performance of a 1.5-µm 2.5-Gb/sdirectly modulated tunable VCSEL’. IEEE Phot. Tech. Lett. 15.
[6] M. Fujita, A. Sakai, and T. Baba: 1999, ‘Ultrasmall and ultralow thresholdGaInAsP-InP microdisk injection lasers: Design, fabrication, lasing charac-teristics and spontaneous emission factor’. IEEE J. Sel. Topics in QuantumElectronics 5.
[7] M. Fujita, R. Ushigome, and T. Baba: 2000, ‘Continuous wave lasing inGaInAsP microdisk injection laser with threshold current of 40µA’. IEEElectron. Lett. 36.
38
[8] M. Ingels and M. S. J. Steyaert: 1999, ‘A 1-Gb/s, 0.7µm CMOS OpticalReceiver with Full Rail-to-Rail Output Swing’. IEEE J. Solid-State Circuits34(7).
[9] I. Kimukin et al.: 2002, ‘InGaAs-Based High-Performance p-i-n Photodi-odes’. IEEE Phot. Tech. Lett. 26(3).
[10] K. Lee et al.: 2001, ‘Fabrication of ultralow-loss Si/SiO2 waveguides byroughness reduction’. Optics Letters 26.
[11] J. Liu et al.: 2002, ‘Ultralow-threshold sapphire substrate-bonded top-emitting 850-nm VCSEL array’. IEEE Phot. Lett. 14.
[12] J. Morikuni et al.: 1994, ‘Improvements to the standard theory for pho-toreceiver noise’. IEEE J. Lightwave Technology 12.
[13] I. O’Connor, F. Mieyeville, F. Tissafi-Drissi, G. Tosik, and F. Gaffiot:2003, ‘Predictive design space exploration of maximum bandwidth CMOSphotoreceiver preamplifiers’. In: Proc. IEEE International Conference onElectronics, Circuits and Systems.
[14] A. Sakai, T. Fukazawa, and T. Baba: 2002, ‘Low Loss Ultra-SmallBranches in a Silicon Photonic Wire Waveguide’. IEICE Tran. Electron.E85-C.
[15] A. Sakai, G. Hara, and T. Baba: 2001, ‘Propagation Characteristics ofUltrahigh-∆ Optical Waveguide on Silicon-on-Insulator Substrate’. Jpn.J. Appl. Phys. – Part 2 40.
[16] S. Schultz, E. Glytsis, and T. Gaylord: 2000, ‘Design, Fabrication, andPerformance of Preferential-Order Volume Grating Waveguide Couplers’.Applied Optics-IP 39.
[17] Semiconductor Industry Association: 2003, ‘International TechnologyRoadmap for Semiconductors’.
[18] G. Tosik, F. Gaffiot, Z. Lisik, I. O’Connor, and F. Tissafi-Drissi: 2004,‘Power dissipation in optical and metallic clock distribution networks innew VLSI technologies’. IEE Elec. Lett. 4(3).
39
Chapter 3
NANOTECHNOLOGIES FOR LOW POWER
Jacques Gautier
CEA-DRT – LETI/D2NT – CEA/GRE
Abstract The conventional approach to improve the performance of circuits is to scale
down the devices and technologies. This is also convenient to lower the power
consumption per function. In this chapter, we overview the potential of
nanotechnologies for this purpose, with emphasis on few-electron devices in
the case of room-temperature operation. Other devices, especially carbon
nanotube transistors, resonant tunnelling diodes and quantum cellular
automata, are briefly discussed.
Keywords: nanotechnologies; Single Electron Transistor; SET; molecular electronics;
RTD; QCA; low power; Coulomb blockade
3.1 INTRODUCTION
In addition to packing-density increase and speed improvement, the
downscaling of technologies comes with a reduction of the power
consumption per function. However this gain is offset by the tremendous
increase in the number of transistors per chip. A possible solution is to go
further towards nano-scale devices where a lower amount of charge is
needed to code a bit. This is the basis of what is known as single electronics.
The use of molecules could be a realistic way to fabricate these tiny devices
and other useful nanostructures.
In this chapter we overview the potential of nanodevices for low power
electronics with emphasis on few-electron electronics in the case of room-
temperature (RT) operation. Other devices, especially carbon nanotube
transistors, resonant tunnelling diodes (RTD) and quantum cellular automata
(QCA), are briefly discussed.
40
3.2 SINGLE ELECTRONICS
In CMOS circuits, the total power consumption is the sum of the
dynamic power and of the contribution of leakages. For advanced
technology generations the later is rapidly rising, but it is still less than the
former. So, we will focus on this dynamic power consumption which is
given by the usual expression
( ) cDDergategated fVCCNaP ⋅⋅+⋅⋅= 2
int (1)
where a is the activity factor, Ngate is the amount of gates, (Cgate + Cinter) is
the load capacitance, gate and interconnect contributions, and f is the clock
frequency. This equation shows that the power is proportional to the amount
of charge in transistors and interconnects for coding a bit of information. For
dense circuits with local interconnects, the dominant contribution is usually
the one related to the gate capacitance of transistors which can also be
expressed as Pd=a.Ngate.Q.VDD.fc, where Q is the channel charge. So there is a
strong motivation to reduce it for power saving. This is currently obtained by
the downscaling of technologies. From the extrapolation of the historical
trend and from the ITRS roadmap anticipation[1], we can expect a value of
only 10-20 electrons for sub-10nm MOSFETs. This is much less than the
hundreds to thousands of electrons present in current devices. Is it possible
to go still further, towards only one electron, using what is called a single
electron transistor or SET [2]? That would be advantageous for power
consumption, knowing that the reduction of power per function due to the
scaling is more or less balanced by the tremendous increase of the number of
transistors per chip. However this gain would be effective only if the
capacitances of interconnects are not too large. Another factor in expression
(1) is the electrostatic potential at which the charge Q is brought. At present,
there is a strong incentive for reducing it. Whereas the supply voltage of
current high performance circuits is in the range 1.2-1.8V, operation at only
0.3V on experimental circuits has already been demonstrated [3], which is
close to the bottom limit anticipated by the ITRS. For a lower value the
device is not in well defined On or Off states which results in either leakage
or poor performance. What can be expected from SET's ? Before giving an
answer to this question, their properties and modes of operation are briefly
recalled.
3.2.1 Background on single electron transistors
A SET is a device which comprises a Source and a Drain reservoir of
electrons and a control gate, like MOSFET's. In between, there is an island
41
where carriers should be confined [2] (see Fig. 3-1). A common solution to
obtain this effect is to insert tunnelling or potential barriers between the
reservoirs and the island. This is the main structural difference from
MOSFET's, but it is essential for the operation of SET's. Due to this
confinement, there is always an integer number of electron in the island.
However, the probability to have a given amount of charge is a continuous
function of the device bias, such that there is also a continuous variation of
the average charge versus the external bias.
RT
Cj
Cg
Vg
RT
Cj
Figure 3-1. Schematics of a SET
Provided that just one electron more or less has a significant effect on the
electrostatic energy of the device, it is shown that, for a given device bias,
there are limited possible states of charge in the island [2]. Especially, there
are bias domains for which only one state of charge is possible. In this case,
there is no exchange of charge with the electron reservoirs and the device is
in Off state. This is the Coulomb blockade effect. For the other cases, the
number of electron oscillates between the highest probable states of charge
leading to a flux of carriers between source and drain. For instance, when the
two states n and n+1 are possible, the current is due to the repetition of the
sequence: one electron coming from the source to the island then leaving the
island to the drain.
As shown in Fig. 3-2, the electrical characteristics of SET's are very
different from those of MOSFET's. The ID(VG) curves have periodic
oscillations of current and the output characteristics look like a resistance (or
staircase for non-symmetrical device) with a low drain voltage domain
where the device is periodically Off and On as a function of VG. The period
of Coulomb Blockade Oscillations, CBO, is given by e/Cg. Between two
successive oscillations, the only difference is that the average number of
G
RT>>RQ
DS
island
42
electron in the island is incremented or decremented by one. At a peak of
current, two dominant states of charge have equal probability and, on the
average, there is a half integer number of electron in the island.
0
2 10-8
4 10-8
6 10-8
8 10-8
1 10-7
1,2 10-7
0 0,5 1 1,5 2
ID
(A)
Vg (V)
VD=0.4V
200mV
100mV
20mV0
5 10-8
1 10-7
1,5 10-7
2 10-7
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7
ID (
A)
Vd (V)
Vg=0.45V
Vg=0.1V
Figure 3-2. Typical ID(VG) and ID(VD) characteristics of a SET. They have been obtained by
simulation with the following parameters: Cj=0.1aF, Cg=0.2aF, RT=1MΩ, T=300K
To observe the previous typical characteristics, there are two important
conditions to meet. Firstly, the charging energy, which is the electrostatic
energy increase due to the arrival of one electron in the island, should be
large in comparison to the thermal energy kT:
kTC
eEC >>=
Σ2
2
(2)
where e is the electron charge (absolute value), CΣ is the total capacitance
of the island, CΣ=2Cj+Cg, where Cj is the junction capacitance and Cg is the
gate to island capacitance. For room-temperature operation, CΣ should be
less than 0.3 aF (Ec=10 kT, T=300K), which requires an island smaller than
a few nm. The second condition is related to the confinement of the electron
wave function in the island, which is essential to quantize the charge in this
island. The resistance of the tunnel barriers should exceed the quantum
resistance RK=h/e2~25.8 kΩ.
For the fabrication of SET’s, there are many different possibilities since
any kind of conductive material can be used for the island, metallic as well
as semiconductor and even molecular. However silicon is advantageous for
CMOS compatibility and also for the stability of devices [4].
43
3.2.2 Designing a low VDD inverter
With regard to the power consumption of digital circuits, we consider in
this part the case of a simple inverter, since this is a convenient reference to
make comparisons with CMOS. The design of a SET inverter has been
discussed by many authors [5,6,7].They pointed out that, since there is only
one kind of SET, the complementary action of the pull-up and pull-down
devices is not as easy to obtain as in CMOS where two types of transistor
exist. A first solution is to choose the supply voltage in order that both of
these devices are On or Off in a complementary way in the switching part of
the transfer characteristic. An example of such situation is shown in Fig. 3-3.
The shaded area displays the Coulomb blockage domains of the pull-up and
pull-down transistors at zero temperature. Based on that, the transfer
characteristics has been schematically drawn. Contrary to CMOS, we can
observe that the voltage swing is less than rail-to-rail and that the DC current
is minimal at the transition point.
-0,6
-0,4
-0,2
0
0,2
0,4
0,6
0,8
1
0,00 0,20 0,40 0,60 0,80
1,00
Vin (V)
Vout (V)
VDD
Figure 3-3. Theoretical Coulomb blockade domains, also known as Coulomb diamonds,
(shaded areas) at 0K, for the pull-down and pull-up SET's of an inverter. At RT they are a
little narrower. Cj=0.1aF, Cg=0.2aF, VDD=0.53V. The bold line is a drawing of the transfer
characteristics.
44
Since a low VDD is advantageous for low power applications, we discuss
now the possibility to minimize it for this simple SET inverter, taking
account of the design constraints and aiming room-temperature operation:
• Cg + 2.Cj < 0.3aF for RT operation (for Ec~10kT)
• Cg / Cj > 1 for voltage gain
• VDD = e / (Cg + Cj) for complementary action of transistors
As a result, a very low VDD and RT operation would be difficult to
achieve simultaneously. In fact, with the previous equations and for a ratio
of gate to junction capacitances of 2, the minimum VDD would be equal to
0.7V ! However, for temperatures above 0K, the switching of the SET from
Off to On state is not abrupt since there is an exponential variation of the
current, equivalent to the subthreshold current of MOSFET’s. Consequently,
the real Coulomb blockade diamonds are narrower than those shown in Fig.
3-3 and it is possible to reduce VDD. This is demonstrated in Fig. 3-4, where
the DC voltage gain and the DC current at the transition point of an inverter
have been plotted versus VDD. Note also that the constraint on CΣ has been a
little relaxed. As thoroughly discussed by A. Korotkov [6], the acceptable
VDD window is quite narrow. A too low VDD value would be detrimental for
the noise margin and for the speed since the DC current at the transition
point is exponentially decreasing with VDD. On the contrary, a higher value
would increase the power consumption.
0,2
0,4
0,6
0,8
1
1,2
1,4
1,6
10-10
10-9
10-8
10-7
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8
Vo
ltag
e g
ain Im
in (A
)
VDD
(V)
VDD
=e/(Cj+Cg)
Cj=0.1aF
Cg=0.2aF
RT=1MΩ
T=300K
Figure 3-4. DC voltage gain (solid line) and DC current (dashed line) at the transition point of
a SET inverter at room temperature.
45
To go further in reducing VDD, a solution is to add control gates to each
SET (Fig. 3-5). Based on this approach, NTT has demonstrated a quasi-
CMOS operation inverter at a supply voltage as low as 20 mV8, which is
very advantageous for the power consumption, but in this case the
temperature was only 27K. The bias of the control gates shifts the CBO,
making possible to select the optimal part of the ID(VG) characteristics of
each SET for complementary action. In this way, the equivalent of two types
of transistors can be obtained, like for CMOS. In addition, their equivalent
threshold voltages can be tuned, balancing the influence of eventual parasitic
(background) charges in the neighbourhood of the SET:
gc
g
gc
g VC
CV ∆−=∆ (3)
To get a symmetrical transfer characteristic, from the Coulomb
diamonds, it can be easily demonstrated that the sum of the control gate
voltages should be equal to VDD:
DDdgcss VVV =+gcd
(4)
As a result there is one more degree of freedom to design an inverter, in
comparison to the case without control gates. That gives flexibility to fix the
value of VDD. In fact, there is now one optimal supply voltage, leading to
complementary states of pull-up and pull-down transistors, for each bias of
the control gates. Taking equation (4) into account, it is given by:
jg
gcssgc
DDoptCC
VCeV
+−
=2
(5)
There is a consequent reduction of VDDopt thanks to the control gates, but
it is important to note that the constraint on the total capacitance (equation 2)
should also take account of the contribution of Cgc : CΣ=2Cj+Cg+Cgc
A drawback of this approach is the requirement of extra lines to distribute
the control gate voltages. However, this can be avoided in the particular case
where Vgcss=VDD and Vgcdd=0V (VSS=0V). For this condition, the optimum
value of VDD is given by:
gcjg
DDoptCCC
eV
2++= (6)
46
As discussed previously for the case without control gates, at RT the
Coulomb blockade area of SET is narrower than at 0K which makes possible
a reduction of VDD or a change of bias of the control gates for a given VDD.
However, it also results in a change of the SET current which may affect the
speed of circuits. Consequently, there is a design trade-off. To illustrate it, in
Fig. 3-5 we have plotted the variations of the DC voltage gain and of the
propagation delay along a chain of SET inverters versus the DC current at
the transition point of the transfer characteristics. The shaded area shows the
most advantageous design window. In this example, the load capacitance is
equal to 0.5fF, but for another value the design window would be the same,
since the propagation delay directly scales with this capacitance. This is a
difference with CMOS where the dominant load capacitance of dense logic
is due to the gate capacitance of MOSFETs. Here, the gate capacitance of
SETs is extremely small and the dominant load capacitance comes from the
local interconnects. In fact the later should be much larger than e/2VDD to
avoid any detrimental effects of the shot noise.
Regarding the dynamic power consumption of SET logic, as long as a
CMOS output buffer is not implemented, the major contribution would also
come from the load capacitance due to the interconnects.
0,5
1
1,5
2
0
1 10-8
2 10-8
3 10-8
4 10-8
5 10-8
10-10
10-9
10-8
10-7
DC
vo
lta
ge
ga
in
pro
pa
gatio
n d
ela
y (s
)
DC current @ transition point (A)
VDD
=0.3V
T=300K
VGCss
+VGCdd
=VDD
τp
GV
Cj=0.05aF
CG=0.1aF
CGC
=0.1aF
Cload
=0.5fF
RT=1MΩ
Master Equation
CG
CG
Vout
CL
VDD
Vin
VGdd
VGss
CG
CG
Vout
CL
VDD
Vin
VGdd
VGss
Figure 3-5. Variations of the DC voltage gain (solid line) and of the propagation delay along
a chain of SET inverters (dashed line) versus the DC current at the transition point of the
transfer characteristics. VDD=0.3V. The control gate voltages are varied as follow:
0.7V<VGCss<0.1V and VGCss+VGCdd=VDD. The simulations are performed using a model of the
SET based on the solution of the Master Equation [2]. The shaded area is the design window
for sufficient voltage gain and speed.
47
An example of switching characteristics is shown in Fig. 3-5 for
VDD=0.3V, T=300K and a load capacitance of 0.5fF. The corresponding
switching energy is 5.2x10-2
fJ. In the same figure, transfer characteristics
have been plotted for a lower supply voltage, showing that a voltage gain
higher than 1 can be obtained down to VDD=0.2V at RT.
0
0,1
0,2
0,3
0,4
0,5
0
1 10-9
2 10-9
3 10-9
4 10-9
5 10-9
0 5 10-8
1 10-7
1,5 10-7
2 10-7
Vin
, V
ou
t (V
)
Cu
rren
t (A)
time (s)
Vout
Vin
Ipull-down
Ipull-up
0
0,05
0,1
0,15
0,2
0 0,05 0,1 0,15 0,2
Ou
tpu
t (V
)
input (V)
T=300K
T=150K
Figure 3-5. On the left, switching characteristics for a SET inverter with control gates.
VDD=0.3V, T=300K, CL=0.5fF, Cj=0.05aF, Cg= Cgc =0.1aF, RT=1MΩ, Vgcss=0.3V, Vgcdd=0.
On the right, transfer characteristics for the same capacitances and VDD=0.2V.
3.2.3 Designing gates with increased functionality
Another approach to lower the power consumption is to build logic gates
with increased functionality in order to reduce the count of transistor needed
to obtain a given function. This can be done by taking advantage of both the
existence of CBO and the possibility to design SET's with multiple inputs
[9]. The principle is to choose the logic levels such as the multiple inputs
SET's are biased at either the minima or at the peaks of current, depending
on the combination of input signals. This is illustrated in Fig. 3-7 in the case
of a double input X-OR function. The logic level "1" is equal to the CBO
period of the equivalent single input SET, e/(2.Cin), where Cin is the input
gate capacitance. From this equivalent SET, it is obvious that the device is
48
Off when both of the inputs are either "0" or "1" and that the device is On
when one and only one of the inputs is "1".
0
5 10-9
1 10-8
1,5 10-8
2 10-8
2,5 10-8
3 10-8
0 0,5 1 1,5 2
ID (
A)
Vgeff (V)
VD=100mV
A="0"
and
B="0"
A ="1"
X-OR
B ="1"
A ="1"
and
B ="1"
T=300K
CGeff
VGeff
2
VVV
2CC
BA
Geff
inGeff
+=
=
Cin
VA
VB
Cin VDD
Figure 3-7. Principle of design of a X-OR gate with a double input SET. The output current is
a X-OR function of VA and VB. The figure shows also the equivalent input circuit and
associated equations.
This can be used to design pass-gate logic functions, as demonstrated by
Y. Ono [9]. For instance, for an input current signal C in the previous X-OR
gate, the output pass current is given by C.(A ⊕ B). Furthermore, one of the
inputs of this gate, B for instance, can be viewed as a control input, leading
to the output pass current AC. if B is "1". Applying this technique to a gate
comprising such a control input in addition to the inputs A and B, we get the
function ( )BAC ⊕. for the output current, when the control input is "1" (see
Fig. 3-8). This way, cascading SET structures, NTT has designed a 4-b adder
with only 40 SET's for operation at 30K9. In comparison with CMOS there
are less transistors and no crossing of pass signal routes thanks to the high
functionality of the SET gates. Furthermore, there is a low level signal on the
pass route. Consequently, a lower dynamic power is expected. Nevertheless,
this power gain is not yet evaluated.
49
C.(A⊕B)
Control gate A
BC
Figure 3-8. Design of complex gates using multiple input SETs. These gates can be cascaded.
Moreover, there are important issues, especially about the control of the
phase of CBO. Since it will be probably impossible to avoid the existence of
any parasitic charges, charge-tolerant solutions are required. A first approach
consists in incorporating redundancy into the circuit design in order to
replace the defective gates by reconfiguration [9]. This is valuable only if a
reasonable amount of spares is needed and if the area-overhead is not too
large. Another solution would be to balance the influence of parasitic offset
charges by opposite charges stored near the island of the SET. This concept
has been demonstrated in the case of SET in which nanostructures have been
embedded [10-12]. The resulting device is a merge of a SET and of a Non
Volatile Memory function. Further, a feedback loop can be implemented to
automatically control the phase of CBO [13]. The loop is closed to adjust the
amount of charges in a memory node then it is opened for the use of the
device.
There are other potential applications about the possibility to tune and
memorize the phase of CBO. A first example has been the demonstration of
a hybrid SET-MOSFET gate which can be programmed to be inverting or
non-inverting [11]. This feature has been obtained thanks to a SET active
device which can operate either in a positive or in a negative
transconductance region, depending on the amount of charge stored in a
nearby nanostructure. In this case, the SET was fabricated in a very thin
undulated SOI film in which a narrow source-drain percolation channel and
an electron pocket working as a memory node can be naturally formed for a
range of bias. In the hybrid gate, the MOSFET was just used as a load. Since
the output voltage swing was only 10mV, an output buffer has been
implemented. The reproducibility of the structure is not obvious, but RT
operation and peak-to-valley current ration (PVCR) as high as 102 were
obtained. The most important is the concept of programmable logic which is
feasible with SET based devices, since it has a high potential for low-power
and high packing density. The design of SET programmable logic array
(PLA) has been also reported by K. Uchida [11].
50
It is important to note that many other functions can be designed with
few-electron devices, taking advantage of their specific features. Especially,
several memory structures that are promising for low power consumption
have been reported [14-17]. For spiking neuron circuits, it has been proposed
to combine NVM MOSFET devices and single-electron circuit based on
multinanodot floating-gate arrays [18]. Also, some analog applications and
devices have been studied, like CCD [8], ADC [19], metrology [20] and
NEMS [21]. However, although some have been demonstrated, most of them
are still at the proof of concept level.
3.3 MOLECULAR ELECTRONICS
For the fabrication of SET's, any kinds of conducting materials can be
used. Whereas the basic research was done on metallic SET's [2], circuit
demonstrations are performed mainly on silicon [8-13], for complementary
with MOSFET's and to benefit of the huge investment in silicon technology.
However, it could be advantageous to use molecules for real applications
due to the size requirement discussed previously and because the
reproducibility of nanoscale structures is very challenging. In addition, the
load capacitance of circuits should be very low, for power consumption and
speed considerations, which implies short and narrow interconnects. The
most promising way to achieve it is the bottom-up approach, using naturally
formed tiny structures or self-assembling methods. The best example is the
carbon nanotube (CNT) which can be used to fabricate FET's [22-23], SET's
[24], interconnects [25] and even non volatile memory arrays [26].
CNT's are long cylinder of carbon atoms consisting of rolled-up sheets of
graphite. For Single Wall CNT's the diameter is as small as 1-5 nm.
Depending on their chirality, they are semiconductor or metallic materials.
Their mobility is much higher than the one of silicon, and a ballistic
transport has been demonstrated for lengths less than a few hundreds of nm,
but the subthreshold characteristics of CNFET's are not better than those of
MOSFET's. Worldwide, several teams are conducting research on the
selective growth or deposition of CNT's that would have the right chirality
and on the evaluation of CNFET's as potential candidate to replace
MOSFET's in the future. For low power applications, thanks to their
excellent transport properties [22], it should be possible to reduce the gate
overdrive and VDD, while meeting the ITRS specifications [1].
Different kinds of molecules are also currently investigated to make
nanometer-scale electronic components and circuits, but a single molecule
transistor has not yet been obtained. To date, one of the most advanced
achievement is a 1µm2 64 cells crossbar matrix fabricated by HP Labs [27]
in which the switching units are bundle of rotaxane molecules. The operation
51
of such molecules is not yet clear and others mechanisms like the formation
of tiny filaments across the molecule gap between the electrodes could
explain the switching [28]. However, on the long term, this research subject
has a great potential for high density, low cost and probably ultra low power
electronics.
True single molecule device will require interconnects at a similar scale.
This is also essential to reduce parasitic capacitances and the power
consumption. Since the needed resolution is far beyond the possibility of
lithographic tools, including NGL, the solution will come from the bottom-
up approach. An example is the realization by Caltech of a Pt nanowire
lattice with width and pitch of 8 nm and 16 nm respectively [29]. Biology
can also come to the rescue for the self-assembly of nano-circuits [30]. A
very different approach, also mitigating the arduous task of nanoscale
patterning, is the concept of self-assembled nanocells proposed by J. Tour
[31]. These nanocells are disordered arrays of metallic islands that are
interlinked with molecules and that are accessed by metallic input/output
leads. Switching-type functions have been observed, but like for the work of
HP [27], the creation and dissolution of metal filaments is probably
responsible for the behaviour. In fact, the behaviour of electrically active
molecules is strongly influenced by surrounding electrodes and other
materials, which make a difference between molecular nanotechnology and
bulk or solution-phase chemistry.
3.4 DISCUSSION
For CNT devices, as well as for nano MOSFET's, the supply voltage
reduction is dependent upon the effects of subthreshold leakage on the static
power, leading to trade-off with the speed of circuits. For SET's, the
steepness of the On-Off switching is not better, but they offer an increased
functionality and low charge operation. However, there are important issues,
especially about the sensibility to offset charges and the fabrication of nano-
scale structures with sufficient level of reproducibility, which require a lot of
work.
Although it is not yet clear if they could achieve a lower VDD, there are
other candidates, like the resonant tunnelling diodes (RTD). Their operation
is based on electron transport via discrete energy levels in double barrier
quantum well structures, leading to the existence of a negative differential
resistance. This implies the fabrication in suitable materials and a perfect
control of the geometry, since the output characteristics are extremely
sensitive to the dimensions. A promising approach is the implementation of
RTD along semiconductor nanowires [31]. There are also prospective
52
studies for a molecular version and for structures mixing Coulomb blockade
and resonant effects [32].
One of the most important features of nanodevices, especially for
molecular ones, is their size. That offers the possibility to lower the power
consumption by parallel processing. For instance, consider two blocks of
low capacitance molecular devices doing the same task as one block of
conventional devices, but at half the clock frequency. The Ngate . fc product
being unchanged, equation 1 shows that the power consumption is directly
related to the C.V [2] product, the gate switching energy, which can be
strongly reduced thanks to lower capacitances and to the possibility to have
devices with lower On current, since fc is divided by 2 in this case.
Going further, quantum-dot cellular automata (QCA) is an attractive
approach, yet speculative, to reduce the power consumption, since there is
no flow of current but only Coulomb interactions [33]. The principle is to
encode binary information by charge configuration in electrostatically
coupled cells in which there are two extra electrons. It has been shown that a
clock field is needed to control the direction of propagation of information
along the cells and to enable power gain. This clock could also be used for a
quasi-adiabatic switching, leading to extremely low power consumption. To
date, experimental demonstrations are performed at low temperature on
metallic structures, but molecular implementations are being investigated in
view of RT operation [34].
References
[1] Semiconductor Industry Association, International Technology Roadmap for
Semiconductors 2001 Edition, http://public.itrs.net, 2003
[2] H. Grabert and M. H. Devoret, Single charge tunnelling Coulomb blockade phenomena
in nanostructures, volume 294 of NATO ASI Series B, Plenum Press New York and
London, 1992
[3] T. Douseki, T. Shimamura, N. Shibata, A 0.3V 3.6GHz 0.3mW frequency divider with
differential ED-CMOS/SOI circuit technology, in Proc. ISSCC, February 2003
[4] N. M. Zimmerman and W. H. Huber, Excellent charge offset stability in a Si-based
single-electron tunneling transistor, APL Vol. 79, N.19, pp. 3188-3190, 2001
[5] J. R. Tucker, Complementary digital logic based on the Coulomb blockade, JAP 72 (9),
1, pp. 4399-4413, 1992
[6] A. N. Korotkov, R. H. Chen and K. K. Likharev, Possible performance of capacitively
coupled single-electron transistors in digital circuits, JAP 78 (4), pp. 2520-2529, 1995
[7] M-Y. Jeong, B-H. Lee and Y-H. Jeong, Design considerations for low-power single-
electron transistor logic circuits, JJAP. Vol.40, pp. 2054-2057, 2001
[8] Y. Takahashi, Y. Ono, A. Fujiwara and H. Inokawa, Silicon Single-Electron Devices for
logic applications, in Proc. ESSDERC September 2002, Florence, pp. 61-68
[9] Y. Ono, H. Inokawa and T. Takahashi, Binary adders of multigate Single-Electron
Transistors: specific design using Pass-Transistor Logic, IEEE Trans. on Nanotech.
Vol.1 pp. 93-99, 2002
53
[10] N. Takahashi, H. Hishikuro and T. Hiramoto, A directional current switch using silicon
Single Electron Transistors controlled by charge injection into silicon nano-crystal
floating dots, in Proc. IEDM, pp.371-374, 1999
[11] K. Uchida, J. Koga, R. Ohba and A. Toriumi, Programmable Single-Electron Transistor
logic for future low-power intelligent LSI: proposal and room-temperature operation,
IEEE Trans. on Elec. Dev. Vol.50, pp.1623-1630, 2003
[12] G. Molas, X. Jehl, M. Sanquer, B. de Salvo, M. Gely, D. Lafond and S. Deleonibus,
Manipulation of periodic Coulomb Blockade Oscillations in ultra-scaled memories by
single electron charging of silicon nanocrystals floating gates, Silicon Nano Workshop,
Honolulu, June 2004
[13] K. Nishiguchi, H. Inokawa, Y. Ono, A. Fujiwara and Y. Takahashi, Automatic control
of the oscillation phase of a Single-Electron Transistor, IEEE EDL25 (1), pp. 31-33,
2004
[14] K. Yano, T. Ishii, T. Hashimoto, T. Kobayashi, F. Murai and K. Seki, Room-
Temperature Single-Electron Memory, IEEE Trans. on Elec. Dev. Vol.41,
NO.9,pp.1628-1638, 1994
[15] Z. A. K. Durrani, A. Irvine and H. Ahmed, Coulomb blockade memory using integrated
Single-Electron Transistor/Metal-Oxide-Semiconductor transistor gain cells, IEEE
Trans. on Elec. Dev. Vol.47, pp.2334-2339, 2000
[16] H. Sunamura, H. Kawaura, T. Sakamoto and T. Baba, Multiple-valued memory
operation using a Single-Electron Device: a proposal and an experimental
demonstration of a ten-valued operation, JJAP Vol. 41, pp. L93-L95, 2002
[17] G. Molas, B. de Salvo, D. Mariolle, G. Ghibaudo, A. Toffoli, N. Buffet and S.
Deleonibus, Single electron charging phenomena at room temperature in a silicon
nanocrystal memory, in Proc. WODIM 2002, Grenoble
[18] T. Morie, T. Matsuura, M. Nagata and A. Iwata, A multinanodot floating-gate MOSFET
circuit for spiking neuron models, IEEE Trans. On Nanotechnology, Vol. 2, NO. 3, pp.
158-164, 2003
[19] H. Inokawa, A. Fujiwara and Y. Takahashi, A multiple-valued logic and memory with
combined Single-Electron and Metal-Oxide-Semiconductor transistors, IEEE Trans. on
Elec. Dev. Vol.50, NO.2, pp. 462-470, 2003
[20] H. E. van den Brom et al., Counting electrons one by one – overview of a joint european
research project, IEEE Trans. on Inst. and Meas. Vol. 52, NO.2, pp. 584-588, 2003
[21] S. Mahapatra, V. Pott, S. Ecoffey, A. Schmid, C. Wasshuber, J. W. Tringe, Y. Leblebici,
M. Declercq, K. Banerjee and A. Ionescu, SETMOS: a novel true hybrid SET-CMOS
high current Coulomb Blockade Oscillation cell for future nano-scale analog ICs, in
Proc. IEDM 2003, pp. 703-706
[22] A. Javey, H. Kim, M. Brink, Q. Wang, A. Ural, J. Guo, P. McIntyre, P. McEuen, M.
Lundstrom and H. Dai, High-K dielectrics for advanced carbon-nanotube transistors
and logic gates, Nature Materials, Vol 1, pp. 241-246, December 2002
[23] P. Avouris, Carbon nanotube electronics, Chemical Physics, 281 (2002), pp. 429-445
[24] K. Matsumoto, S. Kinoshita, Y. Gotoh, K. Kurachi, T. Kamimura, M. Maeda, K.
Sakamoto, M. Kuwahara, N. Atoda and Y. Awano, Single-Electron Transistor with
ultra-high Coulomb energy of 5000K using position controlled grown carbon nanotube
as channel, JJAP Vol.42 Part 1 N°4B, pp. 2415-2418, 2003
[25] J. Li, Q. Ye, A. Cassell, H. T. Ng, R. Stevens, J. Han and M. Meyyappan, Bottom-up
approach for carbon nanotube interconnects, APL Vol. 82, N°15, pp. 2491-2493, 2003
[26] Rueckes, K. Kim, E. Joselevich, G. Y. Tseng, C-L. Cheung and C. Lieber, Carbon
nanotubes-based nonvolatile Random Access Memory for molecular computing,
Science, Vol. 289, pp. 94-97, 7 July 2000
54
[27] Y. Chen, G-Y. Jung, D. A. Ohlberg, X. Li, D. R. Stewart, J. O. Jeppesen, K. A. Nielsen,
J. F. Stoddart and R. S. Williams, Nanoscale molecular-switch crossbar circuits,
Nanotechnology 14 (2003) 462-468
[28] R. F. Service, Next-generation technology hits an early midlife crisis, Science Vol. 302,
pp. 556-559, 24 October 2003
[29] N. Melosh, A. Boukai, F. Diana, B. Gerardot, A. Badolato, P. M. Petroff and J. R.
Heath, Ultrahigh-density nanowire lattices and circuits, Science, Vol. 300, pp.112-115,
4 April 2003
[30] P. Fairley, Germs that build circuits, IEEE Spectrum, pp. 37-41, November 2003
[31] M. T. Björk, B. J. Ohlsson, C. Thelander, A. I. Persson, K. Deppert, L. R. Wallenberg
and L. Samuelson, Nanowire resonant tunneling diodes, APL, Vol. 81, N°23, pp. 4458-
4460, December 2002
[32] M. Saitoh and T. Hiramoto, Room-temperature operation of highly functional Single-
Electron Transistor logic based on quantum mechanical effect in ultra-small silicon dot,
in Proc. IEDM IEDM 2003, pp. 753-756
[33] G. Bernstein, Quantum-dot Cellular Automata: computing by field polarization, in Proc.
DAC 2003, June 2-6, Anaheim (CA), pp. 268-273
[34] C. Lent and B. Isaksen, Clocked molecular Quantum-dot Cellular Automata, IEEE
Trans. on Elec. Dev. Vol.50, NO.9, pp. 1890-1896, 2003
55
Chapter 4
STATIC LEAKAGE REDUCTION THROUGH
SIMULTENEOUS VT/TOX AND STATE ASSIGN-
MENT
Dongwoo Lee, Bo Zhai, David Blaauw and Dennis Sylvester
University of Michigan, Ann Arbor
Abstract: Standby leakage current minimization is a pressing concern for mobile applica-
tions that rely on standby modes to extend battery life. In this paper, we
propose new leakage current reduction methods in standby mode. First, we pro-
pose a combined approach of sleep-state assignment and threshold voltage (Vt)
assignment in a dual-Vt process for subthreshold leakage (Isub) reduction. Sec-
ond, for the minimization of gate oxide leakage current (Igate) which has
become comparable to Isub in 90nm technologies, we extend the above method
to a combined sleep-state, Vt and gate oxide thickness (Tox) assignments
approach in a dual-Vt and dual-Tox process to minimize both Isub and Igate. By
combining Vt or Vt / Tox assignment with sleep-state assignment, leakage cur-
rent can be dramatically reduced since the circuit is in a known state in standby
mode and only certain transistors are responsible for leakage current and need
to be considered for high-Vt or thick-Tox assignment. A significant improve-
ment in the leakage/performance trade-off is therefore achievable using such
combined methods. We formulate the optimization problem for simultaneous
state/Vt and state/Vt/Tox assignments under delay constraints and propose both
an exact method for its optimal solution as well as two practical heuristics with
reasonable run time. We implemented and tested the proposed methods on a set
of synthesized benchmark circuits and show substantial leakage current reduc-
tion compared to the previous approaches using only state assignment or Vt
assignment alone.
Keywords: Leakage current, reduction, performance, dual threshold voltage, oxide thick-
ness, algorithm.
4.1 INTRODUCTION
There is a growing need for high-performance and low-power system,
especially for portable and battery-powered applications. Since these applica-
tions often remain in stand-by mode significantly longer than in active mode,
56
their stand-by (or leakage) current has a dominant impact on battery life.
Standby mode leakage current reduction therefore has been a concern for
some time and a number of such methods have been proposed to address this
problem [1]-[7][9]-[18]. However, with continued process scaling, lower sup-
ply voltages necessitate reduction of threshold voltages to meet performance
goals and result in a dramatic increase in subthreshold leakage current. New
methods for reducing the leakage current in standby mode are therefore criti-
cally needed.
In dual-Vt technology, the MTCMOS approach [1] was proposed where a
high-Vt sleep transistor is inserted between the power supply and the circuit
logic. In standby mode, this sleep transistor is turned off which dramatically
reduces leakage due to its high-Vt. However, the method requires routing of
an additional set of power supply lines in the layout as well as substantially
sized sleep transistors to maintain good supply integrity and circuit perfor-
mance [2]. Also, special latches that maintain state in standby mode need to
be used [3]. In addition, the method does not scale well into sub-1V technolo-
gies due to the increased delay penalty for the high-Vt sleep device [4].
A different approach to standby mode leakage reduction has been pro-
posed that leverages the state dependence of a leakage current due to the so-
called stack effect [5][6]. In [7], the circuit input state that minimizes leakage
current is determined and special flip-flops are inserted in the design to pro-
duce this state in standby mode. The flip-flops in the design are modified to
produce a predetermined state in standby mode while also maintaining the
previously latched state. The required modification to a flip-flop is minor and
can be incorporated in the feedback path of the slave latch with minimal
impact on performance [8]. In general, determining the minimum sleep state
is a difficult problem due to the inherent logic correlations in the circuit. How-
ever, a number of efficient heuristics for this problem have been proposed
[9][10]. The limitation of this approach is that for larger circuits, the reduction
in leakage current is typically only in the range of only 10 to 30% [9].
The above techniques are aimed primarily at subthreshold leakage current
reduction which has been the dominant component of leakage in CMOS tech-
nologies to date. However, in 90nm technologies the magnitude of gate
tunneling leakage, Igate, in a device is comparable to the subthreshold leakage,
Isub, at room temperature. With difficulties in achieving manufacturable high-
k insulator solutions to address the gate leakage problem, the burden address
this problem is primarily on circuit designers and EDA tools. As a result,
there has been recent work in the area of gate leakage analysis and reduction
techniques including pin reordering, PMOS sleep transistors, and the use of
NAND implementations rather than NOR implementations [11]-[13]. Also,
the MTCMOS technique was extended to combat gate leakage by using a
57
thick-oxide I/O device with a larger gate drive than the logic transistors as the
inserted sleep transistor [14].
Another previous approach to leakage reduction that targets only sub-
threshold leakage is to use individual assignment of transistor threshold
voltages in a dual-Vt process [15]-[18]. In these approaches, the trade-off
between high-Vt transistors with low leakage/low performance and low-Vt
transistors with high leakage/high performance is exploited. Circuit paths that
are non-critical are assigned high-Vt while critical circuit portions are given
low-Vt assignments. The method therefore provides a trade-off between cir-
cuit performance and leakage reduction. It was demonstrated that with a
modest performance reduction of 5–10%, significant reduction of 3-4X in
leakage could be obtained over a circuit with all low-Vt transistors [17]. In
these approaches, high/low-Vt assignments are performed without knowledge
of the states of the circuit. Therefore, in order to obtain sufficient leakage
reduction under all possible circuit states, all or most of the transistors in a
particular gate must be set to high-Vt and hence the gate incurs a substantial
performance degradation.
While such dual-Vt processes have been commonplace for several genera-
tions, the availability of multiple oxide thicknesses in a single process has
only become relevant at the 90nm node due to the rise of Igate [19]. Given a
process technology with dual oxide thicknesses for logic devices, the dual-Vt
approach can be easily extended to also consider gate leakage by assigning
thick-oxide transistors to non-critical paths as well. However, similar to the
dual-Vt assignment approach, a simultaneous dual-Vt and dual oxide thickness
assignment with unknown states of the circuit will set all or most of the tran-
sistors in a particular gate to both high-Vt and thick-oxide, to ensure that under
all possible circuit states in standby mode leakage current is acceptable. How-
ever, transistors that are simultaneously assigned a high-Vt and a thick-oxide
have a dramatic delay penalty compared to low-Vt transistors with thin oxide.
Therefore, this approach carries with it a significant delay penalty for process
technologies where both Isub and Igate need to be addressed.
In this paper, we therefore propose new methods to reduce standby mode
leakage current. We can divide our new methods into two categories: 1)
simultaneous dual-Vt and sleep state assignment for Isub reduction for technol-
ogies in which Isub is dominant in standby mode and 2) simultaneous dual-Vt,
dual oxide thickness and sleep state assignment for both Isub and Igate minimi-
zation for technologies which have comparable amount of Igate to Isub. First,
we combine the concepts of Vt assignment and sleep state assignment. This
approach is based on the key observation that, given a known input state for a
gate, the leakage of that gate can be dramatically reduced by setting only a
single OFF-transistor on each path from Vdd to Gnd to high-Vt. Since all other
58
transistors in the gate are kept at low-Vt and continue to have high drive cur-
rent, the performance degradation is limited while significantly gain in
leakage current is obtained. This approach therefore provides a much better
trade-off between leakage and performance compared to Vt assignment with
unknown input state where most or all of the transistors must be set to high-Vt
before a significant improvement in the leakage current is observed. The link
between the effectiveness of Vt assignment and state assignment was previ-
ously observed for Domino logic [8], since these circuits are by their own
nature in a known state in standby mode. However, we extend this concept to
general CMOS circuits by actively controlling the circuit state in standby
mode, thereby dramatically increasing the effectiveness of leakage reduction.
The second proposed approach minimizes the total leakage current (Isub
and Igate) by simultaneous assignment of sleep state, high-Vt and thick-oxide
transistors. In this approach, a key observation is that given a known input
state, a transistor need not be assigned both a high-Vt and a thick oxide since
Isub only occurs in transistors that are OFF while significant Igate occurs only
in transistors that are ON. Furthermore, depending on the input state of a cir-
cuit, only a subset of transistors need to be considered for either high-Vt or
thick-oxide. Therefore, the impact on the delay of the gate is significantly
reduced while obtaining leakage reductions comparable to when all transis-
tors are assigned to both high-Vt and thick-oxides. The proposed method is
compatible with existing library-based design flows, and we explore different
trade-offs between the number of Vt and Tox variations for each library cell
and the obtained leakage reduction. In addition, we compare the obtained
leakage reduction when Vt (the first method) and Vt / Tox (the second method)
assignments can be made individually for transistors in a stack as opposed to
when an entire stack is restricted to a uniform assignment due to manufactur-
ing or area considerations.
Since the circuit state / Vt and the circuit state / Vt / Tox assignments inter-
act, it is necessary to consider their optimization simultaneously. The state / Vt
and state / Vt / Tox assignment task is to find a simultaneous assignment that
minimizes the leakage current in standby mode while meeting a user specified
delay constraint. We formulate this problem as an integer optimization prob-
lem under delay constraints. The search space consists of all input state / Vt
and input states / Vt / Tox assignments and hence is very large. Therefore, in
addition to an exact solution, we also propose a number of heuristics. The pro-
posed methods are implemented on benchmark circuits synthesized using an
industrial cell library in 0.18 m technology for Isub minimization and in a pre-
dictive 65nm technology for both Isub and Igate minimization. On average, the
proposed Isub minimization method by simultaneous state / Vt assignment
approach improves leakage current by a factor of 6X over the traditional
59
approach using Vt assignment only. The second proposed method that mini-
mizes both Isub and Igate by simultaneous state / Vt / Tox assignment has an
average leakage reduction of 5-6X over an all low-Vt and thin-oxide design
solution with a 5% delay range point and achieves more than a 2X improve-
ment over the first proposed approach using Vt and state assignment only (i.e.,
without dual-Tox).
The remainder of this paper is organized as follows. In Section 4.2, we dis-
cus the used leakage model and the characteristics of Isub and Igate leakage
current. In Section 4.3, we present the approach using simultaneous Vt and
state assignment for Isub leakage reduction. In Section 4.4, we present the sec-
ond approach that also addresses Igate by performing simultaneous Vt, Tox,
and state assignment. In Section 4.5, we present our results on benchmark cir-
cuits and in Section 4.6 we present our conclusions.
4.2 LEAKAGE MODEL AND CHARACTERISTICS
In this section, we discuss our leakage current model and briefly review
the general characteristics of gate leakage current in CMOS gates.
Since the proposed leakage optimization approach is library-based, we use
precharacterized leakage current tables for each library cell, with specific
leakage table entries for each possible input state of a library cell. The pre-
characterized tables were constructed using SPICE simulation with BSIM3
models from 0.18 m technology for Isub minimization approach. In order to
represent both Isub and Igate components for the state / Vt / Tox assignment
approach, BSIM4 models were used to generate the precharacterization of
tables. The device simulation parameters were obtained using leakage esti-
mates from a predicted 65nm processes [20], and had a gate leakage
component that was approximately 36% of the total leakage at room tempera-
ture (at which all analysis is performed).1 (Detailed numbers will be shown at
Section 4.5.2.) Different high- and low-Vt versions of a cell as well as Tox and
Vt versions of a cell will be explained further in Section 4.4.2. Also, the delay
and output slope as a function of cell input slope and output loading were
stored in precharacterized tables.
The total gate leakage for a library cell consists of several different com-
ponents, depending on the input state of the gate, as illustrated for the inverter
cell in Figure 4.1. The maximum gate tunneling current occurs when the input
is at Vdd and Vs = Vd = 0V for the NMOS device. In this case, Vgs = Vgd = Vdd
and the Igate is at its maximum for the NMOS device. At the same time, the
1.Since this work aims at standby mode leakage, we expect junction temperatures during these idle periods
to be lower than under normal operating conditions, making room temperature analysis more valid.
60
PMOS device exhibits substantial subthreshold leakage current. When the
input is at Gnd, the output rises to Vdd and Vgs = 0 while Vgd will become -Vdd
for the NMOS device, resulting in a reverse gate tunneling current from the
drain to the gate node. In this case, tunneling is restricted to the gate-to-drain
overlap region, due to the absence of a channel. Since this overlap region is
much smaller than the channel region, reverse tunneling current is signifi-
cantly reduced compared to the forward tunneling current [21]. Note that
BSIM4 intrinsically considers this reverse tunneling current so it is included
in the precharacterized tables described above.
When the input voltage is Gnd, the PMOS device also exhibits gate cur-
rent from the channel to the gate since its Vgs = Vgd = -Vdd. The relative
magnitude of the PMOS gate current in comparison to the NMOS gate current
differs for different process technologies. If standard SiO2 is used as the gate
oxide material, then the Igate for a PMOS device is typically one order of mag-
nitude smaller than that for an NMOS device with identical Tox and Vdd
[19][22]. This is due to the much higher energy required for hole tunneling in
SiO2 compared to electron tunneling. However, in alternate dielectric materi-
als, the energy required for electron and hole tunneling can be completely
different. In the case of nitrided gate oxides, in use today in a few processes,
PMOS Igate can actually exceed NMOS Igate for higher nitrogen concentra-
tions [23][24]. In this paper, we assume that standard SiO2 gate oxide material
is used and the PMOS gate current is negligible. However, the presented
methods can be easily extended to include appreciable PMOS gate leakage as
well.
4.3 SUBTHRESHOLD LEAKAGE REDUCTION
4.3.1 Simultaneous Vt and State Assignment
Consider the leakage and performance of the simple NAND2 circuit
shown in Figure 4.2 under different input states and Vt assignments. It is clear
that given a particular input state, only those transistors that are OFF need to
Figure 4.1. Inverter circuit with NMOS oxide leakage current.
Isub
0VVdd
Igate
Igate
Isub
VddGnd Igate
Igate
61
be considered for high-Vt assignment as the ON-transistors are not leaking.
For instance, in state AB = 01, only transistor tn1 needs to be considered for
high-Vt assignment. Assigning other transistors to high-Vt will only decrease
the performance of the gate with no reduction in leakage current. On the other
hand, in state 11 both tp1 and tp2 must be assigned high-Vt in order to reduce
leakage, since they are parallel devices.
We can partition the transistors into so-called Vt-groups, corresponding to
the minimum sets of transistors that need to be set to high-Vt to reduce leak-
age in a particular state assignment. For the 2-input NAND gate in Figure 4.2,
three Vt-groups exist as shown. The concept of Vt-groups can be easily
applied to more complex structures in which case it may be possible that a
transistor belongs more than one Vt-group. It is clear that we can restrict our-
selves to setting only entire Vt-groups to either high or low-Vt. By considering
only Vt-groups, instead of individual transistors, we therefore significantly
reduce the number of possible Vt assignment and the optimization complexity.
In Table 4.1, we show the leakage current for the NAND2 in Figure 4.2 for
different input states and Vt-group assignments. Column 3 shows the leakage
current when we use high-Vt for one or more Vt-groups that are OFF in a par-
ticular input state. In column 4 and 5, the leakage current with all transistors
assigned to, respectively, high-Vt and low-Vt is shown. We can see that in
states 01, 10, and 11 only a single Vt-group is a candidate for high-Vt assign-
B
A
Group 1
Group 2
Group 3
tp1
tp2
tn1
tn2
Figure 4.2. The concept of groups for a NAND2 gate
Table 4.1. Leakage current of NAND2 gate
Input
State
Assigned
Group
Leakage current [pA]
with Group Assign. with All High Vt with All Low Vt
00
2 24.9
7.2 286.73 9.8
2 and 3 7.2
01 2 26.6 26.6 1054.0
10 3 25.7 24.4 922.6
11 1 14.2 14.2 357.2
62
ment. Also, setting only this one Vt-group to high-Vt results in equal or nearly
equal leakage compared with the leakage when all transistors are assigned
high-Vt demonstrating the effectiveness of the approach. In state 00, three
high-Vt assignments are possible: group 2, group 3, and both group 2 and 3.
However, the leakage current with both groups assigned to high-Vt is only
slightly better than that with only one group set to high-Vt, and assigning
group 3 to high-Vt reduces leakage somewhat more than assigning group 2 to
high-Vt. Hence, it is clear that we need to only consider assignment of group 3
to high-Vt without significant loss in optimality.
Table 4.1 shows that the leakage current varies considerably as different
groups associated with different input states are set to high-Vt. At the same
time, the impact of different high-Vt group assignments on the performance of
the circuit must be considered. By setting only a single group to high-Vt, the
performance degradation is restricted to only a single signal transition direc-
tion and is also reduced compared to high-Vt assignments where most or all
transistors are set to high-Vt. Therefore, the performance/power trade-off of Vt
assignment with known input state is much improved compared with that with
unknown input state.
The input state of a gate effects which transition direction is degraded by a
high-Vt group assignment to a gate. Also, the position of the high-Vt group in
a stack of transistors changes the impact of a high-Vt group assignment on the
different input to output gate delays. Therefore, the input state of a gate must
be chosen such that its associated high-Vt group results in the least degrada-
tion of the critical paths in the circuit. However, only the input state of the
circuit as a whole can be controlled and the logic correlations of the circuit
restrict the possible assignments of gate input states. Therefore, selection of
the circuit input state and of which gate is assigned a high-Vt group must be
made simultaneously to obtain the maximum improvement in leakage current
with minimum loss in performance.
4.3.2 Exact Solution to Vt and State Assignment
The size of the input state space is 2n, where n is the number of circuit
inputs. For each input state assignment, there are two possible Vt assignments
for each gate (one high-Vt group which is pre-determined by its input state,
and all low-Vt). The total number of possible Vt assignment is therefore 2m,
where m is the number of gates in the circuit and the total size of the search
space is 2n+m.
In order to find an exact solution to the problem, we developed an efficient
branch-and-bound method that simultaneous explores the state and Vt assign-
ments and that exploits the characteristics of the problem to obtain efficient
pruning of the search space to improve the run time. Due to the exponential
63
nature of the problem, an exact solution is only possible for very small cir-
cuits. However, the exact approach is still useful as the proposed heuristics
are based on it.
We use two types of branch and bound trees. The first branch-and-bound
tree determines the input state of the circuit and is referred to as the state tree.
The nodes of the state tree correspond to the input variables of the circuit
inputs. Each node of the state tree is associated with a so-called gate tree
which is searched to determine the group Vt assignment. In other words, for a
state tree with k nodes, there exist k copies of the gate tree. Each node in a par-
ticular gate tree corresponds to a gate in the circuit, as shown in Figure 4.3.
Each node has two fanout edges, representing the assignment of that gate with
all low-Vt groups (left branch) or with one high-Vt group, as determined by
the input state of the gate (right branch).
At the root of the state tree, the state of all input variables is unknown. As
the algorithm proceeds down the tree, the state of one input variable becomes
defined with each level that is traversed. At each node in the state tree, a solu-
tion of leakage current can be obtained by traversing the gate tree. Note that
the gate tree may be traversed both with a completely known input state at the
bottom of the state tree as well as with a partially or completely unknown
input state, at higher levels of the state tree.
For each node in the state and gate tree, an upper and low bound on the
leakage current is computed incrementally as explained in Section 4.3.2.1.
Note that early in the state tree the bounds on leakage will be very loose since
the state of the circuit is only partly defined. As the algorithm traverses down
the state tree, the input state becomes more defined and the leakage bounds
become closer. Similarly, the leakage bounds are very wide at the top of each
gate tree, as the Vt assignment of all gates are unknown, and becomes progres-
sively tighter as the algorithm traverses down the tree. Only at the bottom of
both the state tree and its associated gate tree do the upper and low bounds on
s2 s
2
s1
0 1 0 1
0 1
Sol.
0 1 0 1
sn s
n
g2 g
2
g1
L H L H
L H
gm
gm
Sol.
L H L H
State tree
Gate tree
Figure 4.3. State tree with gate tree at each node
64
leakage coincide. The algorithm first traverses down to the bottom of the tree
and then returns back up, to traverse down unvisited branches in DFS manner.
During the search, a tree branch is pruned when if it has a lower bound on
leakage that is worse than the best upper bound on leakage that has been
observed so-far. In addition to pruning based on leakage bounds, we also
compute a lower bound on the circuit delay at each node in the gate tree tra-
versal and prune all branches whose lower bound exceeds the specified delay
constraint. Computation of the delay bounds is also performed incrementally
and is discussed in Section 4.3.2.2.
Also, early in the state tree, computation of the exact minimum Vt assign-
ment by traversing the gate tree is not meaningful since even at that bottom of
the gate tree there is considerable uncertainty in the leakage current due to the
unknown input state. Therefore, the gate tree is searched only partially at the
higher levels of the state-tree which results in slightly more conservative
bounds, but an overall improvement in the run time of the algorithm.
The gate tree is also searched in DFS manner and edges are pruned based
on the computed leakage bounds. During the downward traversal of the gate
tree, the high-Vt branch is always selected, provide it meets the delay con-
straint. This is due to the fact that the high-Vt branch always has less leakage
current than the low-Vt branch. Only if the lower bound on the delay of the
high-Vt branch exceeds the delay constraint, is the low-Vt branch selected and
is the high-Vt branch pruned.
Finally, the gates in the circuit are assigned to nodes in the gate tree in
topological order to enable incremental delay computation. Gates of equal
topological level are further sorted by decreasing leakage to improve the
pruning of the search space. The input signals of the circuit are also assigned
to nodes in the state tree in specific order. We want to place inputs whose state
assignment strongly influences the total leakage of the circuit near the top of
the state tree. We estimate the influence of each input signal on the circuit
leakage by taking the sum of the leakage current of all gates connected to the
input signal. This input variable ordering is similar to that used in [25].
4.3.2.1 Incremental leakage bound computation
During the traversal of the gate tree, some of the gates will have a known
Vt assignment and others, which have not been visited, will have an unknown
Vt assignment. As shown in Figure 4.4, a lower bound on the leakage is com-
puted by assuming all unknown gates have a high-Vt group assignment and an
upper bound is computed by assuming all unknown gates have a low-Vt group
assignment. As the high branch is taken in the downward traversal, only the
upper bound is update (decreased) while when a low branch is taken, only the
lower bound must be updated and is increased.
65
4.3.2.2 Incremental delay bound computation
Similar to the leakage current bounds, a lower bound on the delay is com-
puted assuming all unknown gates have low-Vt group assignments. Delay is
changed only when a high branch is taken in the traversal and is computed
incrementally. We first compute the slack of the circuit for all circuit nodes at
the start of the tree traversal with all Vt assignments assumed to be low-Vt.
When a group changes from a low to a high-Vt group assignment during the
traversal, the slack of that gate will be updated. However, the Vt change of the
gate will affect not only the gate itself but also the delays of fanout gates due
to the slope change at the output of the changed gate. Since the slope at the
output of the changed gate will become slower due to its high-Vt assignment,
the delay of all fanout gates will increase, resulting an overall increased cir-
cuit delay. Ignoring the effect of slope change on fanout gates will therefore
result in the computation of an optimistic lower bound which ensures that the
optimal solution is not accidentally pruned. It also enables incremental delay
computation, given that the gates are visited in topological ordering. As gates
are visited, the changed input slope, due to high-Vt assignments of a fanin
gate, is processed to ensure that an exact delay bound is computed at the bot-
tom of the gate tree.
4.3.3 Heuristic Solution to Vt and State Assignment
We propose two fast heuristics that can be applied to large circuits and that
produce high quality solutions. The proposed heuristic are based on the exact
method described in Section 4.3.2, and are discussed below.
Heuristic 1
In this heuristic, the state and gate tree search is limited to only one down-
ward traversal. Note that while only a single traversal of the state tree is
performed, at each node of the state tree the decision to follow the left or right
child node is based on the computed bounds of the leakage using the gate tree.
gi
gi+1
gi+1
UBi-1
= leak(g1~g
i-1=known gates) + leak(g
i~g
n=L)
HL
LBi-1
= leak(g1~g
i-1=known gates) + leak(g
i~g
n=H)
UBi = UB
i-1 - leak(g
i=L) + leak(g
i=H)
LBi = (unchanged)
UBi = (unchanged)
LBi = LB
i-1 - leak(g
i=H) + leak(g
i=L)
Delay = (unchanged) Delay = (increased)
Figure 4.4. Incremental leakage bound computation
66
Each downward traversal of the gate tree visits m nodes, where m is the num-
ber of gates in the circuit. We perform exactly two such traversals at each
state tree node, leading to a total run time complexity that is O(nm), where n is
the number of circuit inputs. Since the number of inputs is generally thought
to grow approximately as the sqrt(m), the total complexity of this heuristic is
O(msqrt(m)).
Heuristic 2
In the second heuristic, the state tree is searched more extensively, subject
to a fixed run time constraint, while the gate tree search is kept to a single
downward traversal for each state tree node. Experimentally, it was found that
the quality of the first bottom node reached in the gate tree search is near the
optimal Vt assignment. This is due to the fact that the gate tree always chooses
the high-Vt child in its downward traversal which tends to produce a high
quality result. This is in contrast to the state tree, where choosing the correct
child during the downward traversal was found to be much more difficult.
Therefore, the solution quality was found to improve most by searching the
state tree more extensively, subject to a run time constraint, while limiting the
gate tree search to a single downward traversal.
4.3.4 Vt assignment Control within Stacks
We assume the ability to assign Vt on an individual basis within stacks of
transistors. Although it is generally possible to assign the Vt of each transistor
in a stack individually, this may result in the need for increased spacing
between the transistors in order not to violate design rules and ensure manu-
facturability [26]. Hence, at times it may be desirable to restrict the
assignment of Vt such that all transistors in a stack are uniform. In this case,
less flexibility exists in the assignment of Vt, and hence the obtained trade-off
in delay and leakage will degrade to some extent. In Section 4.5.1, we present
results showing the impact on the leakage optimization when uniform stack
assignments are enforced in the library.
4.4 LEAKAGE REDUCTION METHOD FOR BOTH
SUBTHRESHOLD AND GATE LEAKAGE
CURRENT
4.4.1 Leakage Reduction Approach
The proposed leakage optimization method performs simultaneous assign-
ment of standby mode state and high-Vt and thick-oxide transistors. The
67
proposed method is based on the key observation that given a known input
state, a transistor need not be assigned both a high-Vt and a thick oxide. This
is due to the fact that if a transistor that is OFF, gate leakage is significantly
reduced and hence the transistor only needs to be considered for high-Vt
assignment. Conversely, a transistor that, given a particular input state, is ON
may exhibit significant Igate, but does not impact Isub. Hence, conducting tran-
sistors only need to be considered for thick oxide assignment. If the input state
is unknown in standby mode, it cannot be predicted at design time which tran-
sistors will be ON or OFF and therefore all or most transistors must be
assigned to both high-Vt and thick-oxide in order to significantly reduce the
total average leakage. However, given a known input state, we can avoid
assignment of transistors to both high-Vt and thick oxide, thereby significantly
improving the obtained leakage / delay trade-off.
Furthermore, depending on the input state of a circuit, only a subset of
transistors needs to be considered for high-Vt or thick-oxide, as discussed in
Section 4.3.1. For instance, in a stack of several transistors that are OFF, only
one transistor needs to be assigned to high-Vt to effectively reduce the total
Isub. Similarly, Igate for transistors in a stack also has strong dependence on
their position. If a conducting transistor is positioned above a non-conducting
transistor in a stack, its Vgs and Vgd will be small and gate leakage will be
reduced. Hence, depending on the input state, only a small subset of all ON
transistors needs to be assigned thick-oxide and only a subset of all OFF tran-
sistors need to be considered for high-Vt assignment.
We illustrate the advantage of high-Vt and thick-oxide assignment with a
known input state for a 2-input NAND and NOR gate in Figure 4.5. In Figure
4.5(a) a 2-input NOR gate is shown with input state 01. Since only PMOS
transistors p2 is OFF in the pull-up stack, it is the only transistor that needs to
Figure 4.5. High Vt and thick oxide assignments at different input states
i1
p1
p2
n1
n2
i2
0
(a)
p1
p2
n1
n2
(b)
p1
p2
n1
n2
(c)
p1
p2
n1
n2
(d)
p1
p2
n1
n2
(e)
H igh-V t
Transistor
Thick
O xide
Transistor
1
i1
i2
1
1
i1
i2
0
0
i1
0
i2
1i1
0
i2
1
68
be set to high-Vt to reduce the subthreshold leakage of the gate. Similarly,
only NMOS transistor n2 exhibits gate leakage and needs to be assigned thick
oxide to reduce Igate. Hence only two out of four transistors are affected while
the total leakage current is reduced by nearly the same amount as when all
transistors in the gate are set to high-Vt and thick oxide simultaneously. As a
result, the delay of the rising input transition at input i1 is unaffected by the
high-Vt and thick-oxide assignments, while the other transitions are affected
only moderately.
In Figure 4.5(b), the worst-case input state for a NOR2 gate is shown,
which is when both inputs are 1. In this case, both NMOS devices must be
assigned to thick-oxide to reduce Igate, while at least one PMOS device is set
to high-Vt. Depending on the delay requirements, the best input state is either
the state 01 shown in Figure 4.5(a), or the state 00, shown in Figure 4.5(c),
which requires only two transistors to be set to high-Vt. Hence, it is clear that
the input state significantly impacts the ability to effectively assign high-Vt
and thick-oxides without degrading the performance of the circuit. This leads
to the need for a simultaneous optimization approach where both the input
state and the high-Vt and thick-oxide assignments are considered simulta-
neously under delay constraints.
In addition to high-Vt and thick-oxide assignment, we also take advantage
of the Igate dependence on input pin ordering to reduce leakage current [11].
This is illustrated in Figure 4.5(d), for a 2-input NAND gate with input state
01. In order to effectively reduce the leakage under this input state, NMOS
transistor n1 must be assigned to high-Vt and NMOS transistor n2 must be
assigned to thick-oxide. However, if input pins i1 and i2 are reordered, with i1positioned at the bottom of the stack, as shown in Figure 4.5(e), the Vgs and
Vgd voltage of NMOS transistor n1 will be reduced from Vdd to approximately
one Vt drop. Hence, the gate leakage current of n1 will be substantially
reduced and can be ignored. After reordering the input pins, it is necessary to
only set NMOS transistor n2 to high-Vt without further assignments of thick-
oxide transistors. It should be noted that pin reordering will impact the delay
of the circuit and hence some performance penalty might be incurred. How-
ever, this penalty will be readily offset by the elimination of the thick-oxide
assignment in the pull-down stack. In this paper, we therefore consider com-
bined input state assignment with pin-reordering and Vt / Tox assignment.
4.4.2 Cell Library Construction
In order to perform simultaneous Vt, Tox and state assignment, it is neces-
sary to develop a library where for each cell the necessary Vt and Tox version
are available. After such a library has been constructed, the process of assign-
ing Vt and Tox assignments can be performed by simply swapping cells from
69
the library. Since different Vt and Tox variations do not alter the footprint of a
cell, the leakage optimization can be performed either before or after final
placement and routing.
For each gate and input state, a number of different Tox and Vt assignments
is possible, providing different delay / leakage trade-off points. For the fastest
and highest leakage trade-off point, all transistors are assigned to low-Vt and
thin oxides, such as the NAND2 gate shown in Figure 4.6(a). On the other
hand, for the slowest and lowest leakage version of the cell all transistors con-
tributing to leakage are assigned either high-Vt or thick oxide. For instance,
for the NAND2 gate with input state 11, shown in Figure 4.6(b), all transistors
affect the leakage current and both NMOS transistors are assigned thick Tox
while both PMOS transistors are assigned high-Vt to obtain the minimum
leakage / maximum delay trade-off point.
In addition to the fastest version and minimum leakage version of the cell,
a number of other intermediate trade-off points can be constructed for a cell
by assigning only some of the transistors that contribute to leakage to high-Vt
or thick-Tox. These cell versions would have lower leakage than the fastest
cell version but would be faster than the lowest leakage version. It is clear that
a large number of possible cell versions can be constructed if all possible
trade-off points are considered for each possible input state. While a larger set
of cell versions provides the optimization algorithm with more flexibility, and
hence a more optimal leakage result, it also increases the size of the library,
which is undesirable. Therefore, we initially restrict our library to at most 4
different trade-off points for each input state of a library cell, which are: 1) the
minimum delay, shown in Figure 4.6(a), 2) minimum leakage, shown in Fig-
Figure 4.6. Complete Vt-Tox versions of NAND2 gate
A
B
tp1
tp2
tn1
tn2
1
1 A
B
tp1
tp2
tn1
tn2
A
B
tp1
tp2
tn1
tn2
A
B
tp1
tp2
tn1
tn2
(a) (b) (c)
(d)
1
1
1
1
A
B
tp1
tp2
tn1
tn2
(e)
0
0 A
B
tp1
tp2
tn1
tn2
(f)
0
1
70
ure 4.6(b), 3) fast falling transition but slow rising transition, with
intermediate leakage, shown in Figure 4.6(c), and 4) fast rising transition but
slow falling transition with intermediate leakage, shown in Figure 4.6(d).
Although other possible trade-off points could be considered, we empirically
found that these four points yield good optimization results and provide a sys-
tematic approach for constructing all versions of a cell.
In principle, using four possible trade-off points for each input combina-
tion could result in as many as 16 (4x4) cell versions for a 2 input gate.
However, in practice, many of the cell versions are shared between different
input states. Also, in some cases not all 4 trade-off points are realizable and
hence the total number of cell versions is significantly less. We illustrate this
for the NAND2 gate for input state 00. The fastest cell version is again shown
in Figure 4.6(a) and is shared for all input combinations, and the minimum
leakage version is shown in Figure 4.6(e). Note that only one transistor needs
to be set to high-Vt to achieve minimum leakage for this input state. This
results from the fact that PMOS devices have negligible gate leakage in the
target technology and only one transistor in a stack needs to be set to high-Vt
to reduce the leakage through the entire stack. Hence, for the input state 00,
only two trade-off points are needed and only one additional cell version is
added to the library.
Input state 10 again requires the assignment of only a single transistor to
high-Vt for the minimum leakage version, as shown in Figure 4.6(f). This is
due to the fact that the gate leakage through the top NMOS transistor n1 is
negligible since its Vgs and Vgd is reduced to approximately one Vt drop. Only
two trade-off points are therefore required for this input state and both ver-
sions are shared with the 00 state. Finally, if the 01 state occurs in the circuit,
the optimization will automatically perform input pin swapping for all but the
fastest trade-off point, thereby resulting in no additional cell version. The
NAND2 gate therefore requires a total of 5 cell versions to provide up to 4
trade-off points for each input state. In Table 4.2, we show the delay / leakage
Table 4.2. Trade-offs for different Vt-Tox versions of NAND2 gate
State Cell
Total
leakage
current
[nA]
Normalized
rise delay
Normalized
fall delay
pin A pin B pin A pin B
11
Minimum delay (a) 270.4 1.00 1.00 1.00 1.00
Fast rise delay (d) 109.1 1.00 1.36 1.27 1.27
Fast fall delay (c) 91.4 1.36 1.36 1.00 1.00
Minimum leakage (b) 19.5 1.36 1.37 1.27 1.27
00
Minimum delay (a) 41.2 1.00 1.00 1.00 1.00
Minimum leakage (e) 14.0 1.00 1.00 1.12 1.16
10
Minimum delay (a) 91.8 1.00 1.00 1.00 1.00
Minimum leakage (f) 13.3 1.00 1.00 1.12 1.16
71
trade-offs obtained for each input state using the described approach for the
NAND2 gate.
The same process can be applied to each cell in the library to construct the
full set of cell versions for the leakage characterization method. Table 4.3,
shows the number of cell version required for several common gates. Note
that the number of cell version is higher for NOR gates than NAND gates.
Since for a library the total number of cells would increase significantly, we
also explored reducing the number of cells by allowing only two trade-off
points for each cell (minimum delay, and minimum leakage), instead of 4
trade-off points. In this case, the number of cells for the NAND2 gate reduces
to only 3 versions. The number of cell version required for two trade-off
points for different cell types is shown in Table 4.3, column 3. In column 4,
we add one more cell library version - two trade-off points with reduced num-
ber of cells. In order to minimize the number of needed library cells, one or
two cells of NOR2 or NOR3, respectively, are removed from library with
small degradation of leakage/delay trade-off. Therefore, all gates have only
three cells in this option. In Section 4.5.2 we compare the final leakage results
using the full library with 4 trade-off points, the reduced library with only two
trade-off points, and minimum number of cell library with two trade-off
points.
Finally, we consider Vt and Tox assignment control within stacks similar to
the discussion for Vt stack control in Section 4.3.4. However, Tox assignment
differs from Vt assignment in that the assignment of Tox to transistors in a
stack is already uniform due to the use of pin-swapping. This is evident from
the 5 added cell versions for the NAND2 in Figure 4.6, and can be easily
shown to be true for all cell versions generated under the proposed approach.
This is a significant advantage since spacing design rules for different Tox
assignments are expected to be more severe that those for spacing between
different Vt assignments [26]. However, the Vt assignment is not always uni-
form as shown in Figure 4.6(e), where only a single transistor in a stack is
assigned to high-Vt. In the event that a uniform stack is required, both transis-
tors in the stack need to be set to high-Vt, resulting in a slightly worsened
Table 4.3. The number of needed library cells
4 trade-off points 2 trade-off points2 trade-off points
with reduced number of cells
Inverter 5 3 3
NAND2 5 3 3
NAND3 5 3 3
NOR2 8 4 3
NOR3 9 5 3
72
delay / leakage trade-off. Leakage current comparison results between indi-
vidual vs. uniform stack assignment control will be shown in Section 4.5.2.
4.4.3 Optimization - Approach and Heuristics
In this section, we present an exact solution and two heuristics to the prob-
lem of finding a simultaneous input state, high-Vt and thick-Tox assignments
for a circuit under delay constraints. As mentioned, the leakage minimization
problem can be formulated as a integer optimization problem under delay
constraints. The size of the input state space is 2n, where n is the number of
circuit inputs. As discussed in Section 4.4.2, for each input state assignment,
there are up to four possible Vt-Tox assignments for each gate. Note that while
the total number of cell versions can be larger than 4, only 4 of them need to
be considered for each specific input state. For instance, for the NAND2 gate
in Figure 4.6, only versions (a)-(d) are considered for a 11 input state. There-
fore, the total number of possible Vt-Tox assignments is 4m, where m is the
number of gates in the circuit and the total size of the search space is 2n+2m.
In order to find an exact solution to the problem, we extend the branch-
and-bound method with Section 4.3.2. The branch and bound algorithm for
Vt-Tox and state assignment uses two interdependent search trees: state tree
and gate tree. The state tree is searched to determine the input state of the cir-
cuit and the gate tree is searched to determine the Vt-Tox assignment of the
circuit, as shown in Figure 4.7. The only difference from Section 4.3.2 is the
gate tree. Each node in a particular gate tree corresponds to a gate in the cir-
cuit. Since there are four possible Vt-Tox assignments for a gate, each node of
the gate tree has four edges: minimum delay, minimum leakage, fast fall delay
with intermediate leakage, and fast rise delay with intermediate leakage. The
exponential nature of the problem makes it impossible to obtain an exact solu-
Figure 4.7. State tree with gate tree at each node
s2 s
2
s1
0 1 0 1
0 1
Sol
0 1 0 1
sn s
n
g2 g
2
g1
gm
Sol
State tree
Gate tree
73
tion for substantial circuits, such as Isub minimization approach in Section
4.3.2. Therefore, we also use the two heuristics discussed in Section 4.3.3.
4.5 RESULTS
4.5.1 Subthreshold Leakage Reduction
The proposed methods for simultaneous state and Vt assignment were
tested on the ISCAS benchmark circuits [27] and a 64-bit ALU circuit, syn-
thesized using a 0.18 m industrial library with Synopsys. This technology has
a difference of 14X (10X) in Isub and 16% (15%) in delay between low-Vt and
high-Vt NMOS (PMOS) devices. The leakage current for each Vt version of a
cell was computed using SPICE simulation and stored in precharacterized
tables. Delay computation was performed based on the Synopsys table delay
model and was verified to match with Synopsys timing analysis delay reports.
In addition to the proposed methods, traditional methods using only state or Vt
assignment were also implemented for comparison. The state-only assign-
ment was implemented using the approach discussed in [25] while for Vt-only
assignment a method similar to the sensitivity-based approach of [17] was
used.
Table 4.4 compares the leakage results obtained by the three proposed
heuristics for three delay constraints to the average leakage computed using
10,000 random input vectors. The columns marked 0%, 5%, and 10% refer to
leakage minimization results when the delay constraints were set at 0%, 5%,
and 10% respectively, of the full delay range between all low-Vt and all high-
Vt circuit delay, as illustrated in Figure 4.8. The 0% column is therefore the
Table 4.4. Leakage current comparison between heuristics
Minimized leakage current [nA] (reduction factor: vs. average leakage current)
Avg. Ileak by
random
(10K)vectors
0% in low Vt/high Vt delay range 5% in low Vt/high Vt delay range 10% in low Vt/high Vt delay range
Heuristic 1 Heuristic 2 Heuristic 1 Heuristic 2 Heuristic 1 Heuristic 2
Ileak X Time Ileak X Ileak X Time Ileak X Ileak X Time Ileak X
C432 32.9 7.7 4.3 1 4.3 7.7 4.9 6.7 1 3.6 9.2 4.7 7.0 1 3.6 9.1
C499 94.0 13.2 7.1 3 11.3 8.3 13.1 7.2 2 11.6 8.1 9.7 9.6 2 9.7 9.6
C880 73.4 9.7 7.5 4 8.9 8.3 8.9 8.2 3 8.3 8.8 8.9 8.3 4 8.3 8.8
C1355 85.1 19.0 4.5 3 12.7 6.7 14.6 5.8 3 11.7 7.3 12.0 7.1 3 11.0 7.7
C1908 82.8 19.0 4.3 2 15.1 5.5 15.5 5.3 2 12.2 6.8 13.4 6.2 2 10.3 8.0
C2670 162.5 12.7 12.8 58 12.5 13.0 12.7 12.8 55 12.4 13.1 14.3 11.3 55 12.2 13.3
C3540 173.1 20.1 8.6 10 16.4 10.6 20.5 8.4 10 14.6 11.8 17.4 10.0 9 14.5 11.9
C5315 309.1 26.4 11.7 169 25.9 11.9 27.5 11.2 164 25.2 12.3 28.5 10.9 165 25.2 12.2
C6288 451.5 157.5 2.9 47 153.9 2.9 145.5 3.1 44 141.4 3.2 135.8 3.3 43 128.4 3.5
C7552 385.8 31.0 12.4 330 30.6 12.6 30.8 12.5 330 30.1 12.8 30.7 12.6 328 29.6 13.0
alu64 332.3 46.0 7.2 405 43.6 7.6 47.2 7.0 408 44.5 7.5 43.0 7.7 406 42.0 7.9
AVG 7.6 8.6 8.0 9.2 8.5 9.6
74
most stringently constrained optimization as it corresponds to the best obtain-
able delay for the circuit (no performance penalty). Note that a simple
replacement of all low-Vt devices with all high-Vt ones would yield a ~20%
circuit delay increase. Thus, when interpreting the results in this section, a
10% delay point indicates that the circuit after Vt assignment has a delay that
is approximately 2% larger than the original fastest implementation. Since the
average leakage current with 10,000 random input vectors is computed with
all low-Vt transistors, it also corresponds to a 0% delay criteria. Runtimes for
heuristic 1 are given in Table 4.4 in seconds. Heuristic 2 was limited to a runt-
ime of 1800 seconds (30 minutes). We report the reduction factor relative to
the average leakage current over the 10,000 random vectors. Heuristic 2 has
~10% lower leakage results than heuristic 1 at 5% delay point across the
benchmark circuits. However, heuristic 2 has a 4-5X runtime overhead for
large circuits (~1000X for small circuits) over heuristic 1.
In Table 4.5, we compare the proposed approach with traditional tech-
niques, including state-only and Vt-only assignment methods. The state-only
Figure 4.8. Delay point from all low-Vt to all high-Vt range
Delay with all low-Vt
0%
Delay with all high-Vt
100%
5% 50%10%
Table 4.5. Leakage current comparison with traditional techniques
Circuits Minimized leakage current [nA]
Number ofAvg.
Ileak by
random
(10K)
vectors
State-only
assignment
Vt only & proposed heuristic (reduction factor: vs. average leakage current)
0% in the delay range 5% in the delay range 10% in the delay range
Input GateVt-only Heuristic 1 Vt-only Heuristic 1 Vt-only Heuristic 1
Ileak X Ileak X Ileak X Ileak X Ileak X Ileak X Ileak X
C432 36 177 32.9 26.3 1.25 30.8 1.1 7.7 4.3 29.5 1.1 4.9 6.7 29.2 1.1 4.7 7.0
C499 41 519 94.0 86.1 1.09 85.0 1.1 13.2 7.1 57.2 1.6 13.1 7.2 40.4 2.3 9.7 9.6
C880 60 364 73.4 63.7 1.15 64.6 1.1 9.7 7.5 63.9 1.1 8.9 8.2 20.2 3.6 8.9 8.3
C1355 41 528 85.1 81.4 1.04 94.0 0.9 19.0 4.5 65.1 1.3 14.6 5.8 53.4 1.6 12.0 7.1
C1908 33 432 82.8 74.6 1.11 67.0 1.2 19.0 4.3 46.5 1.8 15.5 5.3 30.3 2.7 13.4 6.2
C2670 233 825 162.5 146.2 1.11 44.7 3.6 12.7 12.8 39.7 4.1 12.7 12.8 27.8 5.8 14.3 11.3
C3540 50 940 173.1 155.7 1.11 161.9 1.1 20.1 8.6 148.4 1.2 20.5 8.4 82.4 2.1 17.4 10.0
C5315 178 1627 309.1 283.1 1.09 290.6 1.1 26.4 11.7 289.7 1.1 27.5 11.2 108.3 2.9 28.5 10.9
C6288 32 2470 451.5 412.4 1.09 417.0 1.1 157.5 2.9 259.5 1.7 145.5 3.1 233.0 1.9 135.8 3.3
C7552 207 1994 385.8 352.3 1.10 360.2 1.1 31.0 12.4 353.5 1.1 30.8 12.5 350.9 1.1 30.7 12.6
alu64 131 1803 332.3 294.5 1.13 312.8 1.1 46.0 7.2 288.5 1.2 47.2 7.0 230.1 1.4 43.0 7.7
Avg. 1.12 1.3 7.6 1.6 8.0 2.4 8.5
75
assignment method was limited to a runtime of 1800 seconds (30 minutes).
The results demonstrate that substantial improvement in standby leakage cur-
rent can be obtained using the proposed methods, with an average
improvement of ~80% (5-6X) for the 0% and 5% delay constraints over Vt-
only assignment.
Table 4.6 compares leakage current results for both individual and uni-
form stack control. Since uniform stack control degrades the delay/leakage
trade-off as discussed in Section 4.3.4, the results for uniform stack assign-
ment exhibit less leakage reduction than those of individual stack control. It is
interesting to note, however, that the leakage current degradation by moving
to a less fine-grained threshold voltage assignment scheme is not overly large
implying that even with manufacturing constraints, the proposed technique
provides significant leakage savings.
Finally, Figure 4.9 plots the leakage results for the proposed method and
the two traditional methods as a function of the delay constraint for circuit
c6288. The optimization was performed for a range of delay constraints. The
proposed method provides its largest improvements at tight delay constraints.
This is due to the fact that, as the delay constraint becomes looser, more tran-
sistors can be set to high-Vt in both approaches, and the relative advantage of
the proposed approach reduces. However, leakage reduction is most challeng-
ing under tight performance constraints at which the proposed technique
holds promise.
Table 4.6. Leakage current comparison between individual and uniform stack control.
Minimized leakage current [nA]
5% in low Vt/high Vt delay range (reduction factor: vs. average leakage current)
Average
Ileak by
random
(10K)vector
Vt-only assignmentHeuristic 1
Individual control Uniform control
Ileak X Ileak X Ileak X
C432 32.9 29.5 1.1 4.9 6.7 6.8 4.8
C499 94.0 57.2 1.6 13.1 7.2 12.5 7.5
C880 73.4 63.9 1.1 8.9 8.2 9.1 8.1
C1355 85.1 65.1 1.3 14.6 5.8 23.7 3.6
C1908 82.8 46.5 1.8 15.5 5.3 15.7 5.3
C2670 162.5 39.7 4.1 12.7 12.8 12.9 12.6
C3540 173.1 148.4 1.2 20.5 8.4 24.1 7.2
C5315 309.1 289.7 1.1 27.5 11.2 28.5 10.9
C6288 451.5 259.5 1.7 145.5 3.1 163.1 2.8
C7552 385.8 353.5 1.1 30.8 12.5 31.3 12.3
alu64 332.3 288.5 1.2 47.2 7.0 44.6 7.5
Avg. 1.6 8.0 7.5
76
4.5.2 Leakage Reduction for both Subthreshold and Gate
Leakage
The proposed methods for simultaneous state, Vt and Tox assignment were
implemented on a number of benchmark circuits [27] synthesized using a
library based on a predictive 65nm process [20]. In this technology, the differ-
ence in Igate for the thick-oxide NMOS devices vs. the thin-oxide device is
11X, whereas Isub is reduced by 17.8X (16.7X) when replacing a low-Vt
NMOS (PMOS) device with a high-Vt version. Table 4.7 shows relative leak-
age and delay values at the four possible Vt and Tox assignments for NMOS
devices in this technology. A comparison of our first and second heuristics
along with average leakage computed using 10,000 random input vectors is
shown in Table 4.8. The total leakage current value is given in A and runt-
ime is given in seconds. In heuristic 2, we set the runtime limit as 1800
Figure 4.9. Leakage current comparison for c6288
0 10 20 30 40 50 60 70 80 90 100
0
50
100
150
200
250
300
350
400
450 Average Current with Low-V
t
State Assignment Only with Low-Vt
Dual-Vt Assignment only
Our proposed method - Heuristic 1
State Assignment Only with High-Vt
To
tal L
ea
kag
e C
urr
en
t [n
A]
Delay Point from All Low-Vt to All High-V
t Range [%]
Table 4.7. Comparison of leakage and delay between four possible Vt-Tox assignment
for NMOS
Assignment Normalized values
Vt Oxide thickness
Leakage
Delay
Isub Forward Igate Reverse Igate
Low Thin 1.00 0.41 0.22 1.00
High Thin 0.06 0.31 0.22 1.33
Low Thick 0.73 0.04 0.00 1.26
High Thick 0.05 0.03 0.02 1.69
77
seconds (30 minutes). The average leakage computed using the random vec-
tors can be used to approximate the standby mode leakage if state assignment
as well as dual-Vt and dual-Tox techniques were not employed. Again, the
delay range points used in all results are defined by a percentage of the maxi-
mum possible delay that is associated with moving from an all low-Vt and
thin-oxide design to an all high-Vt and thick-oxide implementation. Note that
a simple replacement of all fast devices with their slowest counterparts would
yield a ~70% circuit delay increase. Thus, when interpreting the results in this
section, a 5% delay point indicates that the circuit after Vt and Tox assignment
has a delay that is approximately 4% larger than the original fastest
implementation.
As shown in Table 4.8, heuristic 2 generally provides somewhat better
results but at much greater runtimes. On average, heuristic 2 provides ~10%
lower leakage current than heuristic 1 across these benchmarks at the 5%
delay point, similar to the results in Section 4.5.1. The improvement of the
two proposed heuristics compared to the average leakage without state, Vt or
Tox assignment is dramatic and approaches 7X at the 10% delay point in the
best-worst delay range. More aggressively, with just a 5% delay penalty the
reduction in total standby leakage is 5.3-6X with a maximum improvement of
8.6X for heuristic 2 in circuit c2670.
In Table 4.9 we compare our results to other standby mode techniques,
including state assignment alone and simultaneous state and Vt assignment (as
in the previous section). The total leakage current value is given in A. Again,
we report the reduction factor in relation to the average leakage current with
10,000 random vectors for consistency. We first point out that state assign-
Table 4.8. Leakage current comparison between heuristics with 4-option, individual stack
control library
Average
Ileak by
random
(10K)
vectors
0% in the best-worst delay range 5% in the best-worst delay range 10% in the best-worst delay range
Heu1 Heu2 Heu1 Heu2 Heu1 Heu2
Ileak X Time Ileak X Ileak X Time Ileak X Ileak X Time Ileak X
c432 24.5 8.2 3.0 3 5.4 4.6 7.7 3.2 2 3.2 7.6 5.5 4.5 2 3.0 8.2
c499 65.8 32.2 2.0 7 31.1 2.1 26.1 2.5 7 24.6 2.7 22.7 2.9 6 20.8 3.2
c880 50.1 10.3 4.9 8 9.2 5.5 8.5 5.9 7 8.3 6.1 8.5 5.9 7 7.0 7.1
c1355 70.8 20.4 3.5 8 20.4 3.5 15.8 4.5 6 13.1 5.4 9.9 7.1 6 9.9 7.1
c1908 56.7 17.4 3.3 5 16.9 3.4 14.8 3.8 4 13.6 4.2 13.2 4.3 5 10.5 5.4
c2670 104.7 14.9 7.0 82 14.7 7.1 12.3 8.5 78 12.2 8.6 13.5 7.8 78 11.3 9.3
c3540 128.5 27.7 4.6 20 23.7 5.4 22.1 5.8 18 19.9 6.4 18.6 6.9 17 17.4 7.4
c5315 221.2 36.6 6.0 219 35.9 6.2 30.0 7.4 213 30.0 7.4 28.4 7.8 202 27.6 8.0
c6288 346.8 153.6 2.3 75 146.0 2.4 112.2 3.1 64 101.4 3.4 84.1 4.1 59 75.6 4.6
c7552 270.0 34.9 7.7 410 33.4 8.1 32.2 8.4 404 31.8 8.5 30.3 8.9 399 30.2 8.9
alu64 260.0 48.7 5.3 468 46.8 5.6 43.4 6.0 464 41.6 6.3 34.3 7.6 458 33.1 7.9
AVG 4.5 4.9 5.4 6.0 6.2 7.0
78
ment alone, which we accomplish by searching the state tree only, achieves
very little improvement in standby mode leakage; about 6%. By adding Vt
assignment, the algorithm of the first proposed method shows an average
reduction of 58% beyond state assignment alone at a 5% delay point. The full
Vt, Tox, and state assignment approach provides an additional 53% reduction
in current beyond state and Vt assignment for the 5% delay point.
Table 4.10 provides a comparison of results using the various cell library
options; 4 and 2 trade-off points with individual stack control, and also with
uniform stacks. The main result in Table 4.10 is that there is very little leak-
Table 4.9. Leakage current comparison with 4-option, individual stack control library
Average
Ileak by
random
(10K)
vectors
State
Assignment
Only
0% in the delay range 5% in the delay range 10% in the delay range
Vt & State Heu1 Vt & State Heu1 Vt & State Heu1
Ileak X Ileak X Ileak X Ileak X Ileak X Ileak X Ileak X
c432 24.5 22.7 1.08 13.3 1.8 8.2 3.0 12.5 2.0 7.7 3.2 12.7 1.9 5.5 4.5
c499 65.8 63.9 1.03 41.9 1.6 32.2 2.0 35.7 1.8 26.1 2.5 32.2 2.0 22.7 2.9
c880 50.1 46.0 1.09 18.9 2.6 10.3 4.9 17.5 2.9 8.5 5.9 16.9 3.0 8.5 5.9
c1355 70.8 67.4 1.05 39.9 1.8 20.4 3.5 33.0 2.1 15.8 4.5 29.8 2.4 9.9 7.1
c1908 56.7 54.8 1.04 27.6 2.1 17.4 3.3 25.8 2.2 14.8 3.8 22.9 2.5 13.2 4.3
c2670 104.7 101.4 1.03 33.3 3.1 14.9 7.0 32.7 3.2 12.3 8.5 31.9 3.3 13.5 7.8
c3540 128.5 121.8 1.05 54.5 2.4 27.7 4.6 51.5 2.5 22.1 5.8 48.5 2.7 18.6 6.9
c5315 221.2 215.1 1.03 81.2 2.7 36.6 6.0 77.1 2.9 30.0 7.4 73.7 3.0 28.4 7.8
c6288 346.8 306.7 1.13 209.3 1.7 153.6 2.3 180.4 1.9 112.2 3.1 153.7 2.3 84.1 4.1
c7552 270.0 262.6 1.03 88.9 3.0 34.9 7.7 86.6 3.1 32.2 8.4 86.1 3.1 30.3 8.9
alu64 260.0 237.2 1.10 90.7 2.9 48.7 5.3 86.1 3.0 43.4 6.0 81.1 3.2 34.3 7.6
AVG 1.06 2.3 4.5 2.5 5.4 2.7 6.2
Table 4.10. Leakage current comparison between cell library options (current unit: A)
Average
Ileak by
random
(10K)
vectors
5% in the best-worst delay range
Individual stack control Uniform stack control
4-option 2-option
2-option
3 cell versions
only
4-option 2-option
2-option
3 cell versions
only
Ileak X Ileak X Ileak X Ileak X Ileak X Ileak X
c432 24.5 7.7 3.2 7.4 3.3 7.1 3.4 7.3 3.4 7.9 3.1 8.6 2.8
c499 65.8 26.1 2.5 26.7 2.5 27.8 2.4 26.0 2.5 28.0 2.3 28.9 2.3
c880 50.1 8.5 5.9 9.7 5.2 8.0 6.3 10.0 5.0 10.7 4.7 10.8 4.6
c1355 70.8 15.8 4.5 16.2 4.4 14.1 5.0 23.4 3.0 25.2 2.8 23.9 3.0
c1908 56.7 14.8 3.8 14.9 3.8 14.3 4.0 15.9 3.6 15.3 3.7 16.8 3.4
c2670 104.7 12.3 8.5 12.1 8.7 12.4 8.4 16.1 6.5 15.4 6.8 16.5 6.3
c3540 128.5 22.1 5.8 24.2 5.3 25.3 5.1 27.1 4.7 25.8 5.0 29.2 4.4
c5315 221.2 30.0 7.4 30.9 7.2 30.7 7.2 32.1 6.9 32.9 6.7 33.8 6.6
c6288 346.8 112.2 3.1 114.2 3.0 114.2 3.0 134.0 2.6 147.8 2.3 145.4 2.4
c7552 270.0 32.2 8.4 31.4 8.6 30.6 8.8 31.8 8.5 31.1 8.7 31.1 8.7
alu64 260.0 43.4 6.0 44.0 5.9 43.2 6.0 42.0 6.2 47.0 5.5 46.1 5.6
AVG 5.4 5.3 5.4 4.8 4.7 4.6
79
age current penalty when moving from a full 4-option library to a simpler 2-
option library. There are several cases where the smaller library outperforms
the larger library due to the heuristic nature of the algorithm used (heuristic 1
is used in this table). Since the library size required in the 2-option scenario is
roughly half that of 4-option, we conclude that the use of 2-option represents a
very good trade-off between library complexity and potential leakage reduc-
tion. Moreover we can see that the simplest cell library of 2-option with a
reduced number of cells provides good leakage reduction results. In general, a
reduced number of cells degrades the leakage/delay trade-off as discussed in
Section 4.4.2. However we find that only complex, and infrequently used
cells, such as 3-input NORs require appreciable reductions in cell variants
which limits the impact on total leakage reduction. Therefore, very good leak-
age current minimization can be obtained even with libraries with 3 cell
versions for each cell. Also, the restriction that each stack of transistors must
use the same Vt and Tox is shown in Table 4.10 to have only a minor impact on
leakage. For instance, the uniform stack 4-option case shows a 10.6% average
power increase compared to the individual stack 4-option case; this still repre-
sents a nearly 5X reduction in standby leakage compared to the average case.
Note that library complexity is not reduced in moving from individual to
stack-based control; such a change would be dictated by manufacturing issues
as well as the trade-off between standby power (lower for individual control)
and cell area (expected to be slightly lower for stack-based control).
Finally, Figure 4.10 plots the leakage current results for the proposed
method and traditional methods as a function of the delay constraint for cir-
Figure 4.10. Leakage current comparison for c6288
0 10 20 30 40 50 60 70 80 90 100
0
50
100
150
200
250
300
350
Average Current with Low-Vt/Thin-T
ox
State Assignment Only with Low-Vt/Thin-T
ox
Dual-Vt & State Assignment
Our proposed method - Heuristic 1
State Assignment Only with High-Vt/Thick-T
ox
To
tal L
ea
ka
ge
Cu
rre
nt [u
A]
Delay Point from the best to the worst range [%]
80
cuit c6288. Here, a 100% delay point implies a complete replacement of low-
Vt and thin-oxide devices with high-Vt and thick-oxide. This is clearly the
lowest leakage solution but is also very slow. The key point in Figure 4.10 is
that the proposed approaches (heuristic 2 results are not shown but are nearly
identical to heuristic 1) provide substantial improvement beyond the average
leakage or the use of state assignment alone and that these gains are achiev-
able with very small and even zero delay penalties. The rapid saturation of the
gains as the delay point increases beyond 10% implies that the new approach
is best suited for achieving low-leakage standby states with very little perfor-
mance overhead (e.g., 5% or even less). Note that the leakage current
achieved by our proposed method does not converge to that by state assign-
ment using all high-Vt and thick-oxide devices. The reason is that the selected
library cells include only a limited number of thick-oxide assignments in
order to simplify the library. Many additional library cells would be needed to
achieve convergence to the minimal leakage solution; instead the bulk of this
leakage savings can be achieved with very little performance penalty.
4.6 CONCLUSIONS
In this paper, we propose new approaches for standby leakage current
minimization under delay constraints. Our approaches use simultaneous state
assignment and Vt or Vt / Tox assignment. Efficient methods for computing the
simultaneous state and Vt or Vt / Tox assignments leading to the minimum
standby mode leakage current were presented. The proposed methods were
implemented and tested on a set of synthesized benchmark circuits. Using the
new state and Vt assignment technique demonstrates 6X lower leakage than
previous Vt-only assignment approaches and 5X lower than state assignment
alone (at 5% delay point). In cases where gate leakage is prominent, as in
90nm CMOS technologies, these improvements are increased by an addi-
tional factor of 2 using state and Vt / Tox assignment. We also investigate the
leakage/complexity trade-off for various cell library configurations and dem-
onstrate that results are still very good even when only 2 additional variants
are used for each cell type.
Acknowledgement
The authors would like to thank Harmander Deogun for his work in leak-
age current model. The work has been supported by NSF, SRC, GSRC/
DARPA, IBM, and Intel.
81
References
[1] S. Mutoh, T. Douseki, Y. Matsuya, T. Aoki, S. Shigematsu and J. Yamada, “1-V
power supply high-speed digital circuit technology with multithreshold voltage
CMOS,” IEEE Journal of Solid-State Circuits, vol. 30, pp. 847-854, Aug. 1995.
[2] J. Kao, A. Chandrakasan, and D. Antoniadis, “Transistor sizing issues and tool
for multi-threshold CMOS technology,” Proc. Design Automation Conference,
pp. 409-414, 1997.
[3] S. Shigematsu, S. Mutoh, Y. Matsuya, Y. Tanabe and J. Yamada, “A 1-V high-
speed MTCMOS circuit scheme for power-down application circuits,” IEEE
Journal of Solid-State Circuits, vol. 32, pp. 861-869, June 1997.
[4] H. Kawaguchi, K. Nose and T. Sakurai, “A super cut-off CMOS (SCCMOS)
scheme for 0.5V supply voltage with picoampere standby current,” IEEE Journal
of Solid-State Circuits, vol. 35, pp. 1498-1501, October 2000.
[5] R. X. Gu and M. I. Elmasry, “Power dissipation analysis and optimization of
deep submicron CMOS digital circuits,” IEEE Journal on Solid-State Circuits,
vol. 31, no. 5, pp. 707-713, May 1996.
[6] Z. Chen, M. C. Johnson, L. Wei and K. Roy, “Estimation of standby leakage
power in CMOS circuit considering accurate modeling of transistor stacks,”
Proc. International Symposium on Low Power Electronics Design, pp. 239-244,
1998.
[7] J. Halter and F. Najm, “A gate-level leakage power reduction method for ultra-
low-power CMOS circuits,” Proc. CICC, pp. 475-478, 1997.
[8] V. De, Y. Ye, A. Keshavarzi, S. Narendra, J. Kao, D. Somasekhar, R. Nair and S.
Borkar, “Techniques for leakage power reduction,” in Design of High-Perfor-
mance Microprocessor Circuits, New York: IEEE Press, 2001.
[9] M.C. Johnson, D. Somasekhar and K. Roy, “Models and algorithms for bounds
on leakage in CMOS circuits,” IEEE Transactions on Computer-Aided Design of
Integrated Circuits and Systems, vol. 18, pp. 714-725, June 1999.
[10] A. Fadi, S. Hassoun, K. A. Sakallaha and D. Blaauw, “Robust SAT-based search
algorithm for leakage power reduction,” Proc. International Workshop on Power
and Timing Modeling, Optimization and Simulation, 2002.
[11] D. Lee, W. Kwong, D. Blaauw and D. Sylvester, “Analysis and minimization
techniques for total leakage considering gate oxide leakage,” Proc. Design Auto-
mation Conference, pp. 175-180, 2003.
[12] R.S. Guindi and F.N. Najm, “Design techniques for gate-leakage reduction in
CMOS circuits,” Proc. ISQED, pp.61-65, 2003.
[13] F. Hamzaoglu and M.R. Stan, “Circuit-level techniques to control gate leakage
for sub-100nm CMOS,” Proc. International Symposium on Low Power Electron-
ics and Design, pp. 60-63, 2002.
82
[14] T. Inukai, M. Takamiya, K. Nose, H. Kawaguchi, T. Hiramoto and T. Sakurai,
“Boosted Gate MOS (BGMOS): Device/circuit cooperation scheme to achieve
leakage-free giga-scale integration,” Proc. Custom Integrated Circuit Confer-
ence, pp. 409-412, 2000.
[15] Q. Wang and S.B.K. Vrudhula, “Static power optimization of deep submicron
CMOS circuits for dual Vt technology,” International Conference on Computer-
Aided Design, pp. 490-496, 1998.
[16] L. Wei, Z. Chen, M. C. Johnson, K. Roy and V. De, “Design and optimization of
low voltage high performance dual threshold CMOS circuits,” Proc. Design
Automation Conference, pp. 489-494, 1998.
[17] S. Sirichotiyakul, T. Edwards, C. Oh, R. Panda and D. Blaauw, “Duet: an accu-
rate leakage estimation and optimization tool for dual Vt circuits,” IEEE Transac-
tions on Very Large Scale Integration (VLSI) Systems, vol. 10, pp. 79-90, April
2002.
[18] M. Ketkar and S. Sapatnekar, “Standby power optimization via transistor sizing
and dual threshold voltage assignment,” Proc. ICCAD, 2002, pp. 375-378.
[19] S. Stiffler, “Optimizing performance and power for 130nm and beyond,” IBM
Technology Group New England Forum, 2003.
[20] International Technology Roadmap for Semiconductors, 2002.
[21] N. Yang, W. K. Henson, and J. J. Wortman, “A comparative study of gate direct
tunneling and drain leakage currents in N-MOSFETs with sub-2nm gate oxides,”
IEEE Trans. Electron Devices, vol. 47, pp. 1636-1644, Aug. 2000.
[22] B. Yu, H. Wang, C. Riccobene, Q. Xiang and M.-R. Lin “Limits of gate oxide
scaling in nano-transistors,” Proc. Symposium on VLSI Tech., pp. 90-91, 2000.
[23] Y.-C. Yeo, Q. Lu, W.-C. Lee, T.-J. King, C. Hu, X. Wang, X. Guo and T. P. Ma,
“Direct tunneling gate leakage current in transistors with ultra thin silicon nitride
gate dielectric,” IEEE Electron Device Letters, vol. 21, pp. 540-542, Nov. 2000.
[24] Q. Xiang, J. Jeon, P. Sachdey, B. Yu, K. C. Saraswat and M.-R. Lin, “Very high
performance 40nm CMOS with ultra-thin nitride/oxynitride stack gate dielectric
and pre-doped dual poly-Si gate electrodes,” Proc. International Electron Devices
Meeting, pp. 860-862, 2000.
[25] H. Kriplani, F. N. Najm and I. N. Hajj, “Pattern independent maximum current
estimation in power and ground buses of CMOS VLSI circuits: algorithms, sig-
nal correlations, and their resolution,” IEEE Transactions on Computer-Aided
Design of Integrated Circuits and Systems, vol. 14, pp. 998-1012, Aug. 1995.
[26] Ruchir Puri, IBM T.J. Watson Research, personal communication.
[27] F. Brglez and H. Fujiwara, “A Neutral Netlist of 10 Combinatorial Benchmark
Circuits”, Proc. ISCAS, 1985, pp.695-698.
83
Chapter 5
ENERGY-EFFICIENT SHARED MEMORYARCHITECTURES FOR MULTI-PROCESSORSYSTEMS-ON-CHIP
Kimish Patel1, Alberto Macii1 and Massimo Poncino2
1 Politecnico di Torino; 2 Universita di Verona
Abstract Most current multi-processor systems-on-chip (MPSoC) platforms do rely ona shared-memory architectural paradigm. The shared memory, typically usedfor storage of shared data, is a significant performance bottleneck because it re-quires explicit synchronization of memory accesses which can potentially occurin parallel. Multi-port memories are a widely-used solution to this problem; theyallow these potentially parallel accesses to occur simultaneously. However, theyare not very energy-efficient, since their performance improvement comes at anincreased energy cost per access. We propose an energy-efficient architecturefor the shared memory that can be used as an alternative to multi-port mem-ories, and combines their performance advantage with a much smaller energycost. The proposed scheme is based on the application-driven partitioning of theshared address space into a multi-bank architecture. This optimization can beused to quickly explore different power-performance tradeoffs, thanks to simpleanalytical models of performance and energy. Experiments on a set of paral-lel benchmarks show energy-delay product (EDP) savings of 50% on average,measured on a set of standard parallel benchmarks.
Keywords: Multi-Processor Systems, Shared Memory, Systems-on-Chip.
5.1 INTRODUCTION
Modern design paradigms for MPSoCs are pushing towards architectureswhich are fully distributed and that work as general networks, based on a mod-ular layered architecture, and that are able to support non-deterministic com-munications. Such architectures, called Networks-on-Chips (NoCs) [1], havebeen devised as an answer to the scaling of SoC complexity, especially in termsof the increased number of hosted processing elements, and of the decreasedreliability of the communication medium.
84
In spite of these scalability challenges, most current SoCs are still based on ashared-medium architecture, and, consequently on a shared-memory paradigm.One reason for this slow migration to more complex architectures is cost. Sharedon-chip buses represent a convenient, low-overhead interconnection, and theydo not require special handling during the physical design flow. Another reasonis a consequence of the limited support provided by system software for sucharchitectures. Although current silicon technology allows to build SoCs witha large number of embedded cores, the capabilities currently offered by theembedded software (e.g., in terms of OS primitives) does not allow to fullyexploit all the potential computational power; therefore, most implementationsof SoC consist of few (seldom more than 16) processor cores, for which a sharedinterconnect is perfectly suitable.
The architecture of these MPSoC platforms is thus reminiscent of tradi-tional multi-processor systems, where inter-processor communication and/orsynchronization is provided through the exchange of data through shared mem-ories of different types. Generally speaking, accessing the shared memories aresignificantly slower than accesses to local ones. First, they are placed fartheraway from the processors than private memories; in fact, the latter are oftentightly coupled to the cores by means of dedicated local buses, while sharedmemories are forcedly connected to a shared bus. Moreover, accesses to theshared buses by the processors requires some form of arbitration, which mayrequire the insertion of wait cycles in case of simultaneous accesses. As aconsequence, the shared memories tend to become a major bottleneck for thebandwidth of the overall system, especially for applications in which parallelismis built around shared data.
Caching of shared data might be a solution, but it raises the well-know issue ofcache coherence, i.e., the possible inconsistence between data stored in cachesof different processors. Cache coherence can be solved in hardware, yet withan extra overhead that may not be affordable in small-scale, low-cost SoC asthose considered in this work. Software-based cache coherence is also a viablesolution, but it essentially consists of limiting the caching of shared data to safetimes [2]. For applications in which parallelism is built around shared data, thisbasically amounts to avoid caching of shared data. In this paper, this will beour assumption: all accesses to shared data will always imply an access to theshared memory.
Providing sufficient memory bandwidth to sustain fast program execution anddata communication/transfers is mandatory for most embedded applications.Increasing memory bandwidth can be achieved by making use of differenttypes of on-chip embedded memories, which provide shorter latencies andwider interfaces [3–5]. One typical solution used to match the computationalbandwidth with that of memory is to use multi-port memories. This solutionincreases the sustainable bandwidth by construction, since a P -port memory
85
allows in fact up to P accesses in parallel (i.e., in a single memory cycle).Therefore, by properly choosing the number of ports of the memory versus thenumber of processors, the issue of synchronization of simultaneous accessescan be easily solved.
The adoption of multi-port memories, however, comes at the price of a sig-nificant increase in area, wiring resources, and energy consumption. On theother hand, architectures based on multi-port memories seem to be the onlyviable option in the cases where bandwidth optimization has absolute priority.
In this work we propose an alternative architecture for the shared memorywhich combines the advantages, in terms of bandwidth, of the multi-port ap-proach, with the advantages, in terms of energy consumption and access time,of partitioned memories [5]. We propose the use of small, single-port mem-ory blocks as a way to achieve memory bandwidth increase together with lowenergy demand. In our scheme, the memory addressing space is mapped oversingle-port banks that can be simultaneously accessed by different processors,so as to mimic for a large fraction of the execution time the behavior of a dual-port memory. Energy efficiency is enforced by two facts: First, the single-portblocks have an energy access cost which is smaller than that of monolithic(either single or dual-port) memories; second, address mapping is application-driven, and cell access frequency data is thus used to determine the optimalsizes of the memory blocks.
Based on analytical expressions for performance and energy consumptionthat allow to explore the energy-performance tradeoff, we present experimentalresults showing that the new architecture guarantees energy savings as high as69% with respect to a dual-port memory configuration (54% with respect tothe baseline, single-ported architecture), with comparable improvement of thememory bandwidth.
The rest of the chapter is organized as follows. Section 5.2 provides somebackground material on memory energy modeling, multi-port memories, andapplication-driven memory partitioning. Section 5.3 describes how partitionedmemories can be used to achieve an energy-efficient shared memory architec-ture. Section 5.4 illustrates the analytical models used to drive the energy-performance exploration engine, which is discussed in Section 5.5. Section 5.6presents the optimization results for a set of standard parallel applications. Fi-nally, some concluding remarks are provided in Section 5.7.
5.2 BACKGROUND
5.2.1 Modeling Memory Energy
Unlike generic hardware modules, the energy consumption of memories isbasically independent of the input activity. What matters, in fact, is whether weare reading or writing a value from or to the memory, regardless of the value.
86
This property allows to model memory energy consumption in an very abstractway, by explicitly exposing two independent variables affecting it: the cost ofan access and the total number of accesses. This translates into the followingformula:
etot =ctot∑i=1
ei (1)
where ctot is the total number of memory accesses, and ei is the cost of eachaccess. For the sake of simplicity, we equally weigh all accesses (i.e., we donot distinguish the cost of a read from that of a write).
Equation 1 exposes the two quantities we can consider to reduce the energyconsumption of a memory system and will be used throughout the paper asa reference. Techniques for reducing memory energy can be thus classifiedaccording to which variable is optimized [6].
5.2.2 Multi-Port Memories
A multi-port memory is simply a memory that allows multiple simultaneousaccesses for reads and writes to any location in memory. Multi-port memo-ries are typically employed as shared memories in multiprocessor designs, andare especially popular as dual-ended FIFO buffers for bus interfacing, or forvideo/graphics buffering.
Multiple simultaneous accesses are made possible by duplicating some ofthe resources required to access a cell: the address and data pins, the word-lines,and the bit-lines. Figure 5.1 shows the structure of a typical dual-port SRAMcell, and in particular the extra word-line (with the corresponding transistors)and extra bit-line.
Figure 5.1. Structure of a Dual-Port SRAM Cell.
In some devices, additional overhead is also required to handle the synchro-nization of multiple writes to the same cell; this is managed through a sort ofhardware semaphore which serializes the concurrent accesses.
The increase in bandwidth provided by multi-port memories comes at theprice of increased area, wiring resources and power consumption. Becauseof this considerable overhead, multi-port memories are usually limited to a
87
few ports (often 2, and seldom more than 4). One noticeable exception isrepresented by register files (although they are not strictly SRAMs), that aretypically highly multi-ported (even 16 or more ports) to provide very highbandwidth in superscalar processors.
Multi-port memories can also be characterized by the flexibility of the ports.In some memory devices, some of the ports can be specialized, i.e., they allowonly some type of access (read or write). This fact can be expressed by writingthe number of ports P = pr + pw + prw, where the three terms denote thenumber of read, write, and read/write ports, respectively. In this work, withoutloss of generality, we will assume that pr = pw = 0, and prw = P , that is, allports can be used for any type of access at any time.
When analyzing multi-port memories from the energy point of view, wemust take into account the two following non-idealities, supported by datafrom several multi-port memory providers ([7],[8],[9]).
a) Energy consumption of multi-port memories does not scale linearly withthe number of ports. For instance, the energy cost for accessing a dual-port memory is more than twice the energy required for accessing asingle-port memory of the same size.
b) When a multi-port memory is used as a shared memory in a multiproces-sor system, there are cases in which not all the ports are used simulta-neously. It may in fact happen that the access pattern of the applicationdoes not allow to group a set of accesses (from the processors) into asingle, multi-port access. In these cases, we must consider the fact thatenergy consumption does not scale linearly with the number of ports thatare accessed simultaneously. For instance, the energy cost for accessinga single port in a dual-port memory is larger than the one for accessing asingle-port memory of the same size.
With reference to the model of Equation 1, the use of multi-port memoriesreduces ctot, but it implies a sizable increase of the access cost ei.
5.2.3 Application-Driven Memory Partitioning
Partitioning a memory block into multiple blocks, based on the memory ac-cess profile, was originally proposed by Benini et al. [10]. Their techniqueexploits the fact that, due to the high locality exhibited by embedded applica-tions, the distribution of memory references is not uniform. As a consequence,some memory locations will be accessed more frequently than others. Thepartitioning is realized by splitting the address space (stored onto a single,monolithic memory block) into non-overlapping contiguous sub-spaces (storedonto several, smaller memory blocks).
88
Reduction of energy consumption is achieved because of two facts. First,each block is smaller than the monolithic one, and thus it has a smaller accesscost (ei). Second, and more relevant, only one of the blocks is active at a time.By properly partitioning the address space, it should be possible to access thesmallest blocks most of the times, and access the largest ones only occasionally.
The original implementation of [10] employs a sophisticated recursive algo-rithm to determine the optimal partition with an arbitrary granularity. In thiswork, we will exploit their idea, yet without employing the same partitioningengine. As a matter of fact, in our case partitioning is driven by the accesspatterns of more than one processor.
Memory partitioning specifically targets the reduction of the access cost ei,and it does not change ctot, since it does not modify the access patterns.
5.3 PARTITIONED SHARED MEMORYARCHITECTURE
The target MPSoC architecture considered is this work is depicted in Fig-ure 5.2. Each processor core has a cache and a private memory (PM) containingprivate data and code, which is accessed through a local bus. Processors arealso connected to another memory (SM), through a common global bus con-taining the data that are shared between the various threads executing on theprocessors. We do not consider here other types of interconnections, such aspoint-to-point ones (i.e., crossbars).
gure 5.2. Generic Architectural Template.
In this work, starting from the assumption that the shared memory is imple-mented as a conventional on-chip, single-port memory, we aim at improvingthe performance of the accesses to the shared memory, yet in a more energy-efficient way than resorting to a multi-port memory.
The proposed shared memory architecture combines the bandwidth advan-tages of multi-port memories (and thus the reduction of ctot) with the advan-
89
tages, in terms of energy consumption and access time, of partitioned, single-port memories (and thus the reduction of ei).
In our scheme, the memory address space is mapped over single-port banksthat can be simultaneously accessed by the different processors, so as to mimicthe behavior of a multi-port memory for a large fraction of the execution time.Each bank covers a subset of the address space, with no replication of memorywords; therefore, the address sub-spaces are non-overlapping. The latter issueis essential to understand why the partitioned scheme can only approach theperformance of the multi-port architecture. Since the memory blocks are single-ported and contain non-overlapping subsets of addresses, simultaneous accessesfrom the processor can be parallelized only if they fit into different memoryblocks. Otherwise, the potentially parallel access must take place into twoconsecutive memory cycles.
Energy efficiency is enforced by two facts: First, the single-port blocks havean energy access cost which is by far smaller than that of monolithic (eithersingle or dual-port) memories; second, address mapping is application-driven,and it accounts thus for the cell access frequency to determine the size of thememory blocks which is most suitable for memory minimization.
In the following, we will restrict our analysis to systems with two proces-sors. Consequently, we will consider dual-port memories, and the partitionedarchitecture will also consists of two blocks at most. Although the concepts thatwill be discussed apply in principle to an arbitrary number of processors (withmulti-port memories and multi-bank architectures), the quantitative analysisof energy and performance strictly refers to the case of two processors (withdual-port memory and two memory blocks).
Figure 5.3. Dual-Port (a) and Partitioned Single-Port (b) Architectures.
Figure 5.3 show a conceptual architecture of the dual-port and the partitionedsingle-port schemes. Label Ai refers to addresses from processor i, while Di
refer to data to/from processor i. In the dual-port scheme (Figure 5.3-(a)), the
DPM
Port2
D2
A2
Port1
A1
D1
P1
P2
SPM1
SPM2
P1
P2
A2
A1
D2
D1
A
D
A
D
(a) (b)
90
existence of two read/write ports allows to bind each processor to one port,realizing in fact a point-to-point interconnection.
In the partitioned architecture (Figure 5.3-(b)), addresses and data must bemultiplexed (from processor to memory) or de-multiplexed (from memory toprocessor) properly, to connect the processor to the required memory block.This block diagram just shows the high-level flow of data and addresses; theactual implementation of the decoder is actually more complex, and will bediscussed in the experimental section.
5.3.1 Related Work
The literature on energy optimization of embedded memories is quite rich(see [6] for a comprehensive survey); however, most techniques deal with theoptimization of caches, scratch-pad memories, or off-chip memories, and multi-port memories are seldom addressed.
Most energy optimizations for multi-port memories are concerned with theissue of the mapping of data structures (typically, arrays) to multi-port mem-ories, based on the access profiles of the applications. From these profiles,these techniques evaluate simultaneous array accesses (e.g., whether two ormore arrays are accessed in the same cycle), and build a so-called compatibilitygraph, which expresses the potential parallelization of accesses. The variousapproaches differ then in how this graph is used to decide the optimal allocationof array accesses to memory ports [3, 11–13].
One technique closer to the one proposed in this work has been discussedby Lewis and Brackenbury [14]. Their approach is based on the typical accesspatterns of DSP applications, and splits highly-multiported register files intomultiple banks of predefined sizes.
5.4 PERFORMANCE AND ENERGYCHARACTERIZATION
In this section we will derive analytical expressions for the number of memoryaccesses and for the total energy consumption for the architectures of Figure 5.3,referred to the case of a system consisting of two processors (hereafter denotedwith P1 and P2).
5.4.1 Performance Characterization
Let c1 and c2 be the number of memory accesses required by the executionof the application on processors P1 and P2, respectively. In the following,we will use the term memory cycle instead of memory access; we adopt thisterminology in order to distinguish accesses to the shared memory that canoccur in parallel. In fact, the total number of memory accesses by a processor
91
is fixed (and determined by the memory access pattern of the application, whichwe do not modify); What actually changes is the time (in cycles) required toserve these accesses. Furthermore, we will denotes sets with bold symbols, andtheir cardinalities with lowercase ones.
Our reference performance figure is the total number of memory cycles forthe case where shared memory is implemented as a monolithic single-portmemory. This value is cspm = c1 + c2.
5.4.1.1 Dual-Port Memory. When the shared memory is implementedby a monolithic dual-port memory, the total number of memory accesses willbe smaller than cspm because of the possibility of simultaneous accesses. Onlya fraction of the accesses, however, will occur simultaneously.
As Figure 5.4 shows, this fraction can be represented in terms of set notation.We denote with Cpar the set of memory cycles that can access memory simul-taneously; Cpar consists of the union of two subsets Cpar = Cpar,1 ∪ Cpar,2,where Cpar,1 ⊆ C1 and Cpar,2 ⊆ C2. These two subsets have same cardinality(i.e., cpar,1 ≡ cpar,2) because each element of one set matches one of the otherset to make a parallel access.
Figure 5.4. Classification of Execution Cycles.
The number of cycles for the dual-port configuration is therefore:
cdpm = (c1 − cpar,1) + (c2 − cpar,2) + cpar/2 (2)
where cpar = cpar,1 + cpar,2, denotes the total number of the parallel cycles.The division by two in the last term denotes the fact that parallel cycles areactually grouped in pairs, with each pair corresponding to a single memoryaccess. Equation 2 simplifies to cdpm = c1 + c2 − cpar/2, exposing the factthat the magnitude of cpar directly translates into a performance improvement.
5.4.1.2 Partitioned Memory. In the case of partitioned memory, thetwo memory banks now host two non-overlapping subsets of the address space.This implies that only a subset of the cycles in Cpar can be parallelized; inparticular, accesses that fall in the same subset of addresses now need to beserialized, since the two memory blocks are single-ported.
92
This further sub-setting of the cycles is depicted in Figure 5.5, using the sameset notation as above. We can notice that C1 and C2 are now both split intotwo subsets, where Ci,j denotes the cycles of processor i that fall into block j.
Figure 5.5. Classification of Execution Cycles for the Partitioned Architecture.
This induces a partition onto Cpar, as follows. The shaded areas labeledA and D in Figure 5.5 denote parallel accesses that fall into different memoryblocks: In region A (D), P1 accesses Block 1 (Block 2), and P2 accesses Block2 (Block 1). Conversely, the regions labeled B and C denote accesses thatfall in the same memory block (Block 2 for region b, and Block 1 for regionc). Cycles belonging to region B and C cause a performance penalty, because,although they can potentially occur in parallel, they must be serialized (and thusrequire two memory accesses).
These subsets can be characterized by using a quantity λ, that denotes thepercentage of the cycles in Cpar that fall in distinct memory blocks (and canthus be made parallel). λ will be used in the following as a compact metric toevaluate the cost of the partition. In fact, λ depends on where how the partitionhas been made, that is, how many addresses fall in each block. Therefore, Cpar
consists of λcpar cycles that can be parallelized, and (1 − λ)cpar that requirestwo separate accesses.
The number of cycles of the partitioned-memory architecture cspm,part istherefore:
cspm,part = (c1 − cpar,1) + (c2 − cpar,2) + λcpar/2 + (1 − λ)cpar (3)
The formula simplifies to cspm,part = c1 + c2 − λcpar/2, exposing the factthat cspm,part ≥ cdpm, since λ ≤ 1. Analyzing the dependency of cspm,part
versus λ, We notice that cspm,part (and thus) the performance penalty of thepartitioned scheme is minimized when λ is maximized, as expected. In par-ticular, when λ = 1, all accesses in Cpar are parallelized, and the partitionedscheme is equivalent to the dual-port memory, performance-wise. When λ = 0,all accesses by Cpar overlap on the same memory block, and the partitionedscheme is equivalent to the single-port memory architecture.
93
5.4.2 Energy Characterization
To compute energy, we stick to the high-level model of Equation 1; energyis thus simply obtained by multiplying each access for its cost.
5.4.2.1 Dual-Port Memory. In this case we have to consider twotypes of access costs, depending on whether one or both ports are accessed.Total energy is obtained thus by properly weighing the terms of Equation 2: Informula:
edpm = (c1 − cpar,1) · edpm,1 + (c2 − cpar,2) · edpm,1 + cpar/2 · edpm,2 (4)
The term edpm,x denotes the energy per access to the memory, in which theterm x = 1, 2 in the subscript denotes the number of ports used in the access.
5.4.2.2 Partitioned Memory. In the case of the partitioned memory,total energy cannot be conveniently expressed by a closed formula, for tworeasons. First, the energy per access depends on the size of the memory blockthat is accessed; the sizes of the blocks, however, are precisely the variables ofthe partitioning problem we are trying to solve. Second, we have two single-port memories, and each memory access from either processor will fall into oneof the two memory blocks. This implies that the energy per access can onlybe approximated by a “average” cost (i.e., the number of accesses to Block 1weighted by its energy cost, plus number of accesses to Block 2 weighted byits energy cost).
The accurate evaluation of energy for the partitioned architecture requiresthus a simulation of the dynamic address trace of the two processors, and theapplication of Equation 1 on an access-by-access basis.
Nevertheless, we can derive an approximate expression of total energy thatcan be used for a rough comparison with Equation 4:
espm,part = (c1 − cpar,1) · e′spm + (c2 − cpar,2) · e′′spm+(1 − λ)cpar · e′′′spm + λcpar/2 · (espm1 + espm2)
(5)
The first two term (e′spm and e′′spm) are the above mentioned average accesscosts and represent the non-parallel memory accesses. e′′′spm is the cost ofaccessing either Block 1 or Block 2 (depending on the subset of addresses),when accesses are potentially parallel but must be serialized. The last termrepresents the subset of potentially parallel accesses that will access Block 1and Block 2 simultaneously (espm1 + espm2).
Although approximate, Equation 5 allows to do some rough comparisonwith the dual-port scheme. First, all energy costs in Equation 5 are smaller thanedpm,2, and, in most cases (when the sizes of the two blocks are of comparablesize), also smaller than edpm,1. This implies that all four terms of Equation 5
94
are smaller than the corresponding ones in Equation 4, and energy is potentiallysmaller than the dual-port memory case, regardless of the value of λ.
The actual dependency of espm,part on λ is not easily observable from Equa-tion 5. A large value of λ increases the probability of accessing both blocks inthe same cycle (this corresponds to the largest term (espm1 +espm2)). Therefore,energy should be in principle reduced by choosing partitions which minimizeλ. In this case, in fact, only one of the two blocks (each one smaller than themonolithic memory) will be accessed in each cycle, thus using less energy; asmall value of λ, however, tends to increase the number of cycles, as alreadyobserved.
5.5 EXPLORATION FRAMEWORK
The models described in Section 5.4 show that there exists a tradeoff be-tween energy and performance in partitioning the shared memory. Althoughwe are searching for energy-efficient memory architectures, we cannot ig-nore performance implications; therefore, in order to search for the best en-ergy/performance tradeoff, we use energy/delay product (EDP) as a metric, andchoose to minimize EDP during the space exploration.
Thanks to the simple models of Section 5.4, the optimization space is rela-tively small, since λ is the only parameter of the models. λ is a function of theaccess pattern of the application, but it also depends on how the address spaceis partitioned. Partitions can be characterized by the boundary address B thatsplits the address space [0, . . . , N − 1] into two sub-spaces [0, . . . , B − 1] and[B, . . . ,N−1]. Therefore, λ is also a function of B. As an example, Figure 5.6shows the behavior of λ versus B for a parallel FFT kernel; we can observe thatthe curve is not monotonic, showing the sensitivity of λ to the access pattern.
Figure 5.6. Behavior of λ(B) vs. B.
These observations leads us to the following exploration procedure, for ashared memory of N words:
1 Compute epm(λ) and cpm(λ) as in Section 5.4;
95
2 For all possible values of B = 0, . . . , N − 1, Compute EDPpm(λ) asepm · cpm. EDPpm(λ(B)) is not a function, since there may be morevalues of B (and thus of EDP ) for a given value of λ. An example ofsuch curve is shown in Figure 5.7, for the parallel FFT benchmark.
3 Compute the function EDP paretopm (λ), obtained by selecting, for each
value of λ, the smallest value of EDPpm(λ). EDP paretopm contains the
Pareto points of EDPpm(λ), and can possibly contain some discontinu-ities. Figure 5.8 shows the resulting curve for the FFT benchmark.
Figure 5.7. Behavior of EDP (λ(B)) vs. λ.
Figure 5.8. Pareto Points of EDP (λ(B)).
4 Compute the minimum EDPmin, of this function, and let λmin the cor-responding value of λ;
5 On the λ vs. B plot, identify the corresponding value Bmin of B. Incase of multiple values of B, choose the one that makes the partitions asequal (in size) as possible.
96
5.6 EXPERIMENTAL RESULTS
5.6.1 Experimental Setup
We have implemented our partitioned memory scheme in ABSS [15]. ABSSis an execution-driven architectural simulator for multiprocessor systems devel-oped at Stanford University, that extends the ideas implemented in the AUG-MINT simulator. ABSS is based on the idea of augmentation, that is, theinstrumentation of the assembly code with various hooks that allow to makecontext switches to the simulator; augmentation translates the program intoa functionally equivalent program that runs on the simulated version of theprocessor.
The memory architecture provided by ABSS includes both private and sharedmemory. All the memories are connected through a single shared bus. Yet,ABSS does not provide any specific predefined cache or shared bus model;rather, it a defines a specific interface to which user-defined cache and busmodels can be easily hooked.
We have integrated Dinero [16] into ABSS, in order to provide accuratecache simulation data, and we have derived performance and energy modelsfor the shared memory (both single- and dual-port) by interpolation of theresults obtained from an industrial memory generator by ST Microelectronics.The target technology for all the models is 0.18µ.
Concerning the benchmarks, we have used Stanford’s SPLASH suite [17]which includes a set of kernels and parallel applications widely used in theparallel computing community.
5.6.2 Energy/Performance Tradeoff Analysis
Table 5.1 shows energy-delay product (EDP) results for the above bench-marks, for the monolithic, single-port architecture (EDPmm) and the parti-tioned one (EDPpm), obtained using the exploration procedure of Section 5.5.The EDP reduction (Column ∆) ranges from 40.5% to 62.3% (50.2% on aver-age).The exploration procedure also allows to compute the best performance andenergy points; these are summarized in Table 5.2, where performance improve-ments (number of cycles) and energy saving with respect to the monolithic,single port architecture are reported (Columns Best Performance and Best En-ergy).
The comparison of Tables 5.1 and 5.2, shows that savings in the EDP ismostly due to energy savings than to performance savings. Minimum EDPpoints are in fact very close to minimum energy points, for most of the bench-marks, while performance improvements are less significant. Notice also thatonly benchmarks that exhibit a sizable amount of parallel cycles (e.g., FFT,LU-Cont, Radix) results in a sizable performance improvement. Conversely,energy does not seem to be that sensitive to the amount of parallel cycles.
97
Table 5.1. Energy-Delay Product Results.
Application EDPmm EDPpm ∆ [%]
Barnes 24987.8 11357.5 54.6FFT 6.4 3.7 41.2FMM 853.4 389.6 54.4LU 3931.3 2339.2 40.5LU-CONT 3734.4 2073.1 44.5Radix 59512.5 23180.5 61.0Volrend 869794.2 453283.8 47.9Water-N2 150460.7 56710.0 62.3Water-S 10581.2 5770.5 45.5
Average 50.2
Table 5.2. Optimal Performance and Energy Points.
Application Best Performance [%] Best Energy [%]
Barnes 1.5 54.4FFT 34.0 37.2FMM 2.1 54.4LU 10.9 40.3LU-CONT 19.8 40.5Radix 25.4 60.9Volrend 0.3 50.3Water-N2 13.9 62.3Water-S 8.7 45.9
Average 13.0 49.6
Figure 5.9 shows the energy savings of the the partitioned architecture withrespect to the dual-port case. Numbers refer to best-performance points, sincewe want to reduce the performance penalty as much as possible. The savings donot include the cost of the decoding logic. The partitioned architecture resultsin an average energy saving of 56% (maximum 70%). This energy saving isachieved at an increase of the total number of memory cycles of 2.4% on average(10.1% maximum).
5.6.3 Decoder Implementation
The partitioned architecture requires an ad-hoc encoder which implementsthe conceptual scheme of Figure 5.3. The encoder must provide two mainfunctionalities. First, it must drive the selectors that decide to which block agiven memory access is directed; to do this, it must contain the information aboutthe boundary of the partition of the address space. Second, and more important,
98
Figure 5.9. Energy Savings of the Partitioned Architecture vs. the Multi-Port One.
it must handle the connection between processors and memory blocks; thisrequires a sort of arbitration mechanism that allows to serialize accesses thatare potentially parallel, but fall in the same subset of addresses (i.e., memoryblock).
Figure 5.10 shows a more detailed block diagram of the encoder. It takesas inputs the addresses A1 and A2 from the two processors, the correspondingrequest signals Reqi, and the value B of the address corresponding to the parti-tion. It then generates the addresses to be sent to each memory block AB1 andAB2 , and the signals used to allow the processors to access memory Granti.The latter are both active but in the cases where potentially parallel accessesmust be serialized.
The decoder contains two main blocks. The first block (RH, Request Han-dler) checks the two addresses A1 and A2, and generates the Busyi outputs aswell as a signal that determines whether the accesses can be parallelized or not(S/NS). The other block (SEL), uses three inputs to decide to what memoryblock to send what address: the S/NS input, and the outputs A1i and A2i of twocomparators (the boxes labeled with “=”) which determine in which block A1
and A2 are falling, respectively. By using the value of B as an external input, itis possible to make the decoder application-independent, and therefore to haveone single encoder for any application. We have implemented the decoder inVHDL, and synthesized it on a 0.18µm technology library by ST Microelec-tronics, using Synopsys Design Compiler. When applying the memory accesstrace of the FFT benchmark, the dissipation of the decoder is 0.35 µJ , about1.7% of total memory energy consumption (19.8 µJ).
99
Figure 5.10. Block Diagram of the Decoder.
Concerning delay, although the decoder is on the critical path (its delayadds up to the memory access time), this is not really an issue in the par-titioned architecture. In fact, the memory cycle time in this case is smallerthan that of the dual-port case, since we are accessing smaller memory blocks.Quantitatively, the partitioned architecture results in a slack equal to ddpm −max( dspm,1, dspm,2), where di denotes access time to the corresponding mem-ory block. The delay of the decoder obtained from synthesis is 310ps, wellwithin this slack.
5.7 CONCLUSIONS
We have proposed an energy-efficient alternative to multi-port memories suit-able for the implementation of the shared memory of multi-processor systems-on-chip. The architecture is based on application-driven partitioning of theaddress space into multiple banks.
The target of the architecture is to achieve little or no performance penaltywith respect to multi-port memories; therefore, we pursue maximum perfor-mance partitioning solutions, corresponding the case where the chance of par-allelizing the accesses is maximized. The architecture can be enhanced so thatzero performance penalty is achieved, thank to the use of an extra memorybuffer.
Experiments on a set of parallel benchmarks has shown average energy-delayproduct (EDP) reductions of 50% on average, with respect to the baseline caseof a single-port memory, and energy savings of 56%, with respect to the caseof a multi-port memory, with an average 2% performance penalty.
100
References
[1] L. Benini, G. De Micheli, “Networks on Chips: A New SoC Paradigm,”IEEE Computer, Vol. 35, No. 1, pp. 70–78, January 2002.
[2] P. Stenstrom, “A Survey of Cache Coherence Schemes for Multiprocessors,”IEEE Computer, Vol. 23, No. 6, June 1990, pp. 12–24.
[3] F. Catthoor, et al. Custom Memory Management Methodology Explo-ration for Memory Optimization for Embedded Multimedia System Design,Kluwer Academic Publishers, 1998.
[4] P. Panda, N. Dutt, Memory Issues in Embedded Systems-on-Chip Optimiza-tion and Exploration, Kluwer Academic Publishers, 1999.
[5] A. Macii, L. Benini, M. Poncino, Memory Design Techniques for Low-Energy Embedded Systems, Kluwer Academic Publishers, 2002.
[6] L. Benini, A. Macii, M. Poncino, “Energy-Aware Design of EmbeddedMemories: A Survey of Technologies, Architectures and OptimizationTechniques”, ACM Transactions on Embedded Computing Systems, Vol.2, No. 1, Feb. 2003, pp. 5–32.
[7] Cypress Semiconductor, http://www.cypress.com/products.
[8] Integrated Devices Technology, http://www.idt.com/products/multi port.html.
[9] Artisan Components, http://www.artisan.com/products/memory.html.
[10] L. Macchiarulo, A. Macii, L. Benini, M. Poncino, “Layout-Driven Mem-ory Synthesis for Embedded Systems-on-Chip," IEEE Transactions on VeryLarge Scale Integration (VLSI), Vol. 10, No. 2, pp. 96-105, April 2000
[11] P.R. Panda, N.D. Dutt, “Behavioral Array Mapping into Multiport Mem-ories Targeting Low-Power,” VLSI’97: International Conference on VLSIDesign, Jan. 1997, pp. 268–272.
[12] P.R. Panda, L. Chitturi, “An Energy-Conscious Algorithm for MemoryPort Allocation,” ICCAD’02: International Conference on Computer AidedDesign, Nov. 2002, pp. 572–576.
[13] W.-T. Shiue, C. Chakrabarti, “Low-Power Multi-Module, Multi-PortMemory Design for Embedded Systems,” Journal of VLSI Signal Process-ing, pp.167-178, Nov 2001.
[14] M. Lewis, L. Brackenbury, “Exploiting Typical DSP Data Access Patternsand Asynchrony for a Low-Power Multi-ported Register Bank,” ASYNC’01:International Symposium on Asynchronous Circuits and Systems, March2001, pp. 4–14.
[15] D. Sunada, D. Glasco, M. Flynn, ABSS v2.0: A SPARC Simulator, Tech-nical Report CSL-TR-98-755, CSL, Stanford University, April 1998.
101
[16] M. D. Hill, J. Elder, DineroIV Trace-Driven Uniprocessor Cache Simula-tor, www.cs.wisc.edu/markhill/DineroIV, 1998.
[17] J. P. Singh, W.-D. Weber, A. Gupta, “SPLASH: Stanford Parallel Appli-cations for Shared-Memory”, Computer Architecture News, Vol. 20, No. 1,pages 5-44, March 1992.
102
Chapter 6
TUNING CACHES TO APPLICATIONS FOR
LOW-ENERGY EMBEDDED SYSTEMS
Ann Gordon-Ross1, Chuanjun Zhang
1, Frank Vahid
1,2, and Nikil Dutt
2
1University of California, Riverside;2 University of California, Irvine
Abstract The power consumed by the memory hierarchy of a microprocessor can
contribute to as much as 50% of the total microprocessor system power, and is
thus a good candidate for power and energy optimizations. We discuss four
methods for tuning a microprocessors’ cache subsystem to the needs of any
executing application for low-energy embedded systems. We introduce on-
chip hardware implementing an efficient cache tuning heuristic that can
automatically, transparently, and dynamically tune a configurable level-one
cache’s total size, associativity and line size to an executing application. We
extend the single-level cache tuning heuristic for a two-level cache using a
methodology applicable to both a simulation-based exploration environment
and a hardware-based system prototyping environment. We show that a victim
buffer can be very effective as a configurable parameter in a memory
hierarchy. We reduce static energy dissipation of on-chip data cache by
compressing the frequent values that widely exist in a data cache memory.
Keywords: Cache; configurable; architecture tuning; low power; low energy; embedded
systems; on-chip CAD; dynamic optimization; cache hierarchy; cache
exploration; cache optimization; victim buffer; frequent value.
6.1 INTRODUCTION
The power consumed by the memory hierarchy of a microprocessor can
contribute to 50% or more of total microprocessor system power [1]. Such a
large contributor to power is a good candidate for power and energy
optimization. The design of the caches in a memory hierarchy plays a major
role in the memory hierarchy’s power and performance.
Tuning cache design parameters to the needs of a particular application
or program region can save energy. Cache design parameters include: cache
size, meaning the total number of data byte storage; cache associativity,
meaning the number of tag and data ways simultaneously read per cache
103
access; cache line size, meaning the number of bytes in a block when
moving data between cache and the next memory level; and victim buffer
use, meaning a small fully-associative buffer storing recently-evicted cache
data lines. Every application has different cache requirements that cannot be
efficiently satisfied with one predetermined cache configuration. For
instance, different applications have vastly different spatial and temporal
locality and thus have different requirements [2] with respect to cache size,
cache line size, cache associativity, victim buffer configuration, etc. In
addition to tunable cache parameters, widely existing frequent values in data
caches for some applications can enable data encoding within the cache for
reduced power consumption. We define cache tuning as the task of
choosing the best configuration of cache design parameters for a particular
application, or for a particular phase of an application, such that
performance, power and/or energy are optimized.
New technologies enable cache tuning. Core-based processors allow a
designer to choose a particular cache configuration [3-7]. Some processor
designs allow caches to be configured during system reset or even during
runtime [2,8,9].
Manual tuning of the cache is hard. A single-level cache may have many
tens of different cache configurations, and interdependent multi-level caches
may have thousands of cache configurations. The configuration space gets
even larger if other dependent configurable architecture parameters are
considered, such as bus and processor parameters. Exhaustively searching
the space may be too slow even if fully automated. With possible average
energy savings of over 40% through tuning [2,10], we sought to develop
automated cache tuning methods.
In this chapter, we discuss four methods of cache tuning for energy
savings. We discuss an in-system method for automatically, transparently,
and dynamically tuning a level-one cache; an automatic tuning methodology
for two-level caches applicable to both a simulation-based exploration
environment or a hardware-based prototyping environment; a configurable
victim buffer; and a data cache that encodes frequent data values.
6.2 BACKGROUND – TUNABLE CACHE
PARAMETERS
Many methods exist for configuring a single level of cache to a particular
application during design time and in-system during runtime. Cache
configuration can be specified during design time for many commercial soft
cores from MIPS [6], ARM [5], and Arc [4] and for environments such as
Tensilica’s Xtensa processor generator [7] and Altera’s Nios embedded
processor system [3].
104
Configurable cache hardware also exists to assist in cache configuration.
Motorola’s M*CORE [9] processors offer way configuration which allows
the ways of a unified data/instruction cache to individually be specified as
either data or instruction ways. Additionally, ways may be shut down
entirely. Way shut-down is further explored by Albonesi [8] to reduce
dynamic power by an average of 40%. An adaptive cache line size
methodology is proposed by Veidenbaum et al.[11] to reduce memory traffic
by more than 50%.
Exhaustive search methods may be used to find optimal cache
configurations, but the time required for an exhaustive search is often
prohibitive. Several tools do exist for assisting designers in tuning a single
level of cache. Platune [12] is a framework for tuning configurable system-
on-a-chip (SOC) platforms. Platune offers many configurable parameters
beyond just cache parameters, and prunes the search space by isolating
interdependent parameters from independent parameters. The level one
cache parameters, being dependent, are explored exhaustively.
Heuristic methods exist to prune the search space of the configurable
cache. Palesi et al. [13] improves upon the exhaustive search used in Platune
by using a genetic algorithm to produce comparable results in less time.
Zhang et al. [14] presents a cache configuration exploration methodology
wherein a cache exploration component searches configurations in order of
their impact on energy, and produces a list of Pareto-optimal points
representing reasonable tradeoffs in energy and performance. Ghosh et
al.[15] uses an analytical model to efficiently explore cache size and
associativity and directly computes a cache configuration to meet the
designers’ performance constraints.
Few methods exist for tuning multiple levels of a cache hierarchy.
Balasubramonian et al. [10] proposes a hardware-based cache configuration
management algorithm to improve memory hierarchy performance while
considering energy consumption. An average reduction in memory hierarchy
energy of 43% can be achieved with a configurable level two and level three
cache hierarchy coupled with a conventional level one cache.
6.3 A SELF-TUNING LEVEL ONE CACHE
ARCHITECTURE
Tuning a cache to a particular application can be a cumbersome task left
for designers even with the advent of recent computer-aided design (CAD)
tuning aids. Large configuration spaces may take a designer weeks or
months to explore and with a small time-to-market, lengthy tuning iterations
may not be feasible. We propose to move the CAD environment on-chip,
eliminating designer effort for cache tuning. We introduce on-chip hardware
105
implementing an efficient heuristic that automatically, transparently, and
dynamically tunes the cache to the executing program to reduce energy [16].
6.3.1 Configurable Cache Architecture
The on-chip hardware tunes four cache parameters in the level-one cache:
cache line size (64, 32, or 16 bytes), cache size (8, 4, or 2 Kbytes),
associativity (4, 2, or 1-way), and cache way prediction (on or off). Way
prediction is a method for reducing set-associative cache energy, in which
one way is initially accessed, and other ways accessed only upon a miss.
I$
Tuner
D$
Micro-
processor
Off chip
Memory
Figure 6-1. Self-tuning cache architecture
The exploration space is quite large, necessitating an efficient exploration
heuristic implemented with specialized tuning hardware, as illustrated in
Figure 6-1. The tuning phase may be activated during a special software-
selected tuning mode, during startup of a task, whenever a program phase
change is detected, or at fixed time intervals. The choice of approach is
orthogonal to the design of the self-tuning architecture itself.
The cache architecture supports a certain range of configurations [2]. The
base level-one cache of 8 Kbytes consists of four banks that can operate as
four ways. A special configuration register allows the ways to be
concatenated to form either a direct-mapped or 2-way set associative 8
Kbyte cache. The configuration register may also be configured to shut
down ways, resulting in a 4 Kbyte direct-mapped or 2-way set associative
cache or a 2 Kbyte direct-mapped cache. Specifically, due to the bank layout
for way shut down, 2 Kbyte 2- or 4-way set associative and 4 Kbyte 4-way
set associative caches are not possible using the configurable cache
hardware.
6.3.2 Heuristic Development Through Analysis
A naïve tuning approach would simply try all possible combinations of
configurable parameters in an arbitrary order. For each configuration, the
miss rate can be measured and used to estimate the energy consumption of
the particular cache configuration. After all configurations are executed, the
approach would simply choose the configuration with the lowest energy
106
consumption. However, such an exhaustive method may involve the
inspection of too many configurations. Therefore, we wish to develop a
cache tuning heuristic that minimizes the number of configurations explored.
When developing a good heuristic, the parameter (cache size, line size,
associativity, or way prediction) with the largest impact in performance and
energy would likely be the best parameter to search first. We analyzed each
parameter to determine the parameter’s impact on miss rate and energy by
fixing three parameters and varying the third.
We observed that varying the cache size had the largest average impact
on energy and miss rate – changing the cache size can impact the energy by
a factor of two or more. From our analysis, we developed a search heuristic
that first determines the best cache size, determines the best line size, then
the best associativity, and finally, if the best associativity is greater than one,
our heuristic determines whether to use way prediction or not.
6.3.3 Search Heuristic
The heuristic developed based on the importance of parameters is
summarized below:
1. Begin with a 2 Kbyte, direct-mapped cache with a 16 byte line size.
Increase the cache size to 4 Kbytes. If the increase in cache size causes a
decrease in energy consumption, increase the cache size to 8 Kbytes.
Choose the cache size with the best energy consumption.
2. For the best cache size determined in step 1, increase the line size from
16 bytes to 32 bytes. If the increase in line size causes a decrease in
energy consumption, increase the line size to 64 bytes. Choose the line
size with the best energy consumption.
3. For the best cache size determined in step 1 and the best line size
determined in step 2, increase the associativity to 2 ways. If the increase
in associativity causes a decrease in energy consumption, increase the
associativity to 4 ways. Choose the associativity with the best energy
consumption.
4. If step (3) determined the best associativity to be greater than 1,
determine if enabling way prediction results in energy savings.
The cache tuning heuristic can be implemented in either software or
hardware. In a software-based approach, the system processor would execute
the search heuristic. Executing the heuristic on the system processor would
not only change the runtime behavior of the application but also affect the
cache behavior, possibly resulting in the search heuristic choosing a non-
107
optimal cache configuration. Therefore, we prefer a hardware-based
approach that does not significantly impact overall area or power.
6.3.4 Experiments and Results
We simulated numerous Powerstone [9] and MediaBench [18]
benchmarks using SimpleScalar [19], a cycle-accurate simulator that
includes a MIPS-like microprocessor model, to obtain the number of cache
accesses and cache misses for each benchmark and configuration explored.
For power dissipation, we considered both static power dissipation due to
leakage current and dynamic power dissipation due to logic switching
current and the charging and discharging of the load capacitance. We obtain
the energy of a cache hit from our own CMOS 0.18 µm layout of our
configurable cache (we found our energy values correspond closely with
CACTI values). We obtain the off-chip memory access energy from a
standard Samsung memory, and the stall energy from a 0.18 µm MIPS
microprocessor. Furthermore, we obtained the power consumed by our cache
tuner, through simulation of a synthesized version of our cache tuner written
in VHDL.
Table 6-1. Results of search heuristic. Ben. is the benchmark considered, cfg. is the cache
configuration selected, No. is the number of configurations examined by our heuristic, and
E% is the energy savings of both the I-cache and D-cache.
Ben. I-cache cfg No. D-cache cfg No. I-cache E% D-cache E%
padpcm 8K_1W_64B 7 8K_1W_32B 7 23% 77%
crc 2K_1W_32B 4 4K_1W_64B 6 70% 30%
auto 8K_2W_16B 7 4K_1W_32B 6 3% 97%
bcnt 2K_1W_32B 4 2K_1W_64B 4 70% 30%
bilv 4K_1W_64B 6 2K_1W_64B 4 64% 36%
binary 2K_1W_32B 4 2K_1W_64B 4 54% 46%
blit 2K_1W_32B 4 8K_2W_32B 8 60% 40%
brev 4K_1W_32B 6 2K_1W_64B 4 63% 37%
g3fax 4K_1W_32B 6 4K_1W_16B 5 60% 40%
fir 4K_1W_32B 6 2K_1W_64B 4 29% 71%
jpeg 8K_4W_32B 8 4K_2W_32B 7 6% 94%
pjpeg 4K_1W_32B 6 4K_1W_16B 5 51% 49%
optimal 4K_2W_64B
ucbqsort 4K_1W_16B 6 4K_1W_64B 6 63% 37%
tv 8K_1W_16B 7 8K_2W_16B 7 37% 63%
adpcm 2K_1W_16B 5 4K_1W_16B 5 64% 36%
epic 2K_1W_64B 5 8K_1W_16B 6 39% 61%
g721 8K_4W_16B 8 2K_1W_16B 3 15% 85%
pegwit 4K_1W_16B 5 4K_1W_16B 5 37% 63%
mpeg2 4K_1W_32B 6 4K_2W_16B 6 40% 60%
optimal 8K_2W_16B
Average 5.8 Average: 5.4 45% 55%
108
Table 6-1 shows the results of our search heuristic, for instruction and
data cache configurations. Our search heuristic is quite effective: it searches
on average only 5.8 configurations, compared to 27 configurations for an
exhaustive approach. Furthermore, our heuristic finds the optimal
configuration in nearly all cases. For the two data cache configurations
where the heuristic does not find the optimal, pjpeg and mpeg2, the
configuration found is only 5% and 12% worse than the optimal,
respectively. On average, the dynamic self-tuning cache can reduce memory-
access energy by 45% to 55%. Additionally, be observed that way prediction
is only beneficial for instruction caches and that only a 4-way set associative
instruction cache has lower energy consumption when way prediction is
used. However, for the benchmarks we examined, the cache configurations
with the lowest energy dissipation were mostly direct mapped caches where
way prediction is not applicable.
To determine the area and power overhead of our cache tuner, we
designed the cache tuner hardware using VHDL and synthesized the tuner
using Synopsys Design Compiler. The total tuner size was about 4,000 gates,
or 0.039 mm2 in 0.18 µm CMOS technology. Compared to the reported size
of the MIPS 4Kp with caches [20], this represents an increase in area of just
over 3%. The power consumption of the cache tuner is 2.69 mW at 200
MHz, which is only 0.5% of the power consumed by a MIPS processor.
Furthermore, we only use the tuning hardware during the tuning stage; the
tuner can be shutdown after the best configuration is determined, thereby
minimizing the effects of additional static power dissipation due to the tuner.
6.4 AUTOMATIC TUNING OF A TWO-LEVEL
CACHE ARCHITECTURE – THE TCAT
In the previous section, we described an automatic method for tuning a
single level of cache in system during run-time. We extend the single level
cache tuner to tune two-level caches to embedded applications for reduced
energy consumption [21]. This method is applicable to both a simulation-
based exploration environment and a hardware-based prototyping
environment. We present the two-level cache tuner, or TCaT – a heuristic for
searching the huge solution space of possible configurations. The heuristic
interlaces the exploration of the two cache levels and searches the various
cache parameters in a specific order based on their impact on energy.
109
6.4.1 Configurable Cache Architecture
The configurable caches in each of the two cache levels explored here are
based on the configurable cache architecture described for a single level
configurable cache in Section 6.3.1. The target architecture for our two-level
cache tuning heuristic contains separate level one instruction and data caches
and separate level two instruction and data caches. For the first level cache,
we explore the same search space as the single level cache tuner: cache line
size (64, 32, or 16 bytes), cache size (8, 4, or 2 Kbytes), and associativity (4,
2, or 1-way). For the second level of cache, we expand the cache size to a
possible 64, 32, or 16 Kbytes while the line size and associativity parameters
are the same. We do not explore way prediction with the TCaT.
An exhaustive exploration of all cache configurations for a two level
cache hierarchy is too costly. For a single level separate instruction and data
cache design, an exhaustive exploration would explore a total of 28 different
cache configurations. However, the addition of a second level of hierarchy
raises the number of cache configurations to 432.
Nevertheless, for comparison purposes, we determined the optimal cache
configuration for each benchmark by generating exhaustive data. It took over
one month of continual simulation time on an UltraSparc compute server to
generate the data for our nine benchmarks.
In addition, we have chosen a base cache hierarchy configuration
consisting of an 8 Kbyte, 4-way set associative level-one cache with a 32
byte line size, and a 64 Kbyte 4-way set associative level two cache with a
64 byte line size – a reasonably common configuration.
6.4.2 Initial Two-Level Cache Tuning Heuristic – Search Each Level
Independently
Initially, we extended the heuristic described in Section 6.3.3 for a two-
level cache by tuning the level-one cache while holding the level-two cache
at the smallest size, then tuning the level-two cache using the same heuristic.
We applied the initial heuristic to the benchmarks and found that this
heuristic did not perform well for two levels (the original heuristic was
intended for only one level, where it works well). The cache configuration
determined by our initial heuristic consumed, on average over all
benchmarks, 1.41 times more energy than the optimal configuration. In the
worst case, our initial heuristic found a cache configuration using 2.7 times
more energy than the optimal configuration. In one benchmark, the initial
heuristic found a cache configuration that was worse than the base cache.
The naïve assumption that the two levels of cache could be configured
independently was the reason that our initial heuristic did not perform well
110
for a two level system. In a two-level cache hierarchy, the behavior of each
cache level directly affects the behavior of the other level. For example, the
miss rate of the level one cache does not solely determine the performance of
the level two cache. The performance of the level two cache is also
determined by what values are missing in the level one cache. To fully
explore the dependencies between the two levels, we decided to explore both
levels simultaneously.
6.4.3 The Two-Level Cache Tuner - TCaT
To more fully explore the dependencies between the two cache levels, we
expanded our initial heuristic to interlace the exploration of the level one and
level two caches. Instead of entirely configuring the level one cache before
configuring the level two cache, the interlaced heuristic explores one
parameter for both levels of cache before exploring the next parameter,
while adhering to the parameter ordering of the initial heuristic. The basic
intuition behind our heuristic is that interlacing the exploration allows for
better modeling and tuning of the interdependencies between the different
levels of cache hierarchy. We applied the interlaced heuristic to the
benchmarks and found that the interlaced heuristic performed much better
than the initial heuristic, but there was still much room for improvement.
We examined the cases where the interlaced heuristic did not yield the
optimal solution. We discovered that in these cases, the optimal was not
being reached for two reasons. First, the initial heuristic did not fully explore
each parameter. For instance, if an increase from a 2 Kbyte to 4 Kbyte cache
size did not yield an improvement in energy, an 8 Kbyte cache size was not
examined. The second reason the optimal configuration was not being found
was not due to a failure in the heuristic, but rather due to the limitations set
on certain cache configurations by the configurable cache itself. For
example, in the level two cache, if a 16 Kbyte cache is chosen as the best
size, the only associativity available is a direct-mapped cache. With no
energy improvement by increasing the cache from a 16 Kbyte direct-mapped
to a 32 Kbyte direct-mapped cache, no other associativities are searched by
the previous heuristics. To allow for all associativities to be searched, we
added a final adjustment to the associativity search step of the interlaced
heuristic with full parameter exploration. The final adjustment allows the
cache size to be increased for both the level one and level two caches in
order to search larger associativities. We refer to this final heuristic as the
two-level cache tuner - the TCaT.
111
6.4.4 Experiments and Results
The experimental setup and energy calculations are the same as those
described in Section 6.3.4. We explored nine different benchmarks obtained
from MediaBench [18] and EEMBC [22] benchmarks suites.
Figure 6-2. Energy consumption for the initial heuristic cache configuration, the TCaT cache
configuration, and the optimal cache configuration, normalized to the base cache
configuration for each benchmark.
Figure 6-2 shows the results for the initial heuristic and the TCaT for
each benchmark. The energy consumptions have been normalized to the base
cache configuration for each benchmark’s cache hierarchy. The results show
that the TCaT finds the optimal cache configuration in most cases.
Compared to the base cache configuration and averaged over all
benchmarks, the initial heuristic achieves an average energy savings of 32%
while the TCaT achieves an average energy savings of 53%. Additionally,
we found that for every benchmark, there is no loss of performance due to
cache configuration for optimal energy consumption. In fact, the benchmarks
receive an average of a 28% speedup, which we found was due to the tuning
of the cache line size.
Furthermore, the TCaT reduces the configuration search space
significantly. The exhaustive approach for separate instruction and data
caches for a two level cache hierarchy explores 432 cache configurations.
The improved heuristic explores only 28 cache configurations, or only 6.5%
of the search space. This reduction in the search space speeds up both a
simulation approach and a hardware-based prototyping platform approach.
00.20.40.60.8
11.2
g721
raw
caudio
pegw
it
AIF
FT
R01
AIF
IRF
01
BIT
MN
P01
IDC
TR
N01
PN
TR
CH
01
TT
SP
RK
01
avera
ge
Base Cache
Initial Heuristic
TCaT
Optimal
112
6.5 USING A VICTIM BUFFER IN AN
APPLICATION SPECIFIC MEMORY
HEIRARCHY
In addition to tuning cache parameters such as cache size, line size, and
associativity, the cache subsystem can include a configurable victim buffer
which can be beneficial in systems with a direct-mapped cache. Direct-
mapped caches are popular in embedded microprocessor architecture due to
their simplicity and good hit rates for many applications. A victim buffer is a
small fully-associative cache, whose size is typically 4 to 16 cache lines,
residing between a direct-mapped L1 cache and the next level of memory.
The victim buffer holds lines discarded after an L1 cache miss. The victim
buffer is checked whenever there is an L1 cache miss, before going to the
next level memory. If the desired data is found in the victim buffer, the data
in the victim buffer is swapped back to the L1 cache. Jouppi [23] reported
that a four-entry victim buffer could reduce 20% to 95% of the conflict
misses in a 4 Kbyte direct-mapped data cache. Albera and Bahar [24]
evaluated the power and performance advantages of a victim buffer in a high
performance superscalar, speculative, out-of-order processor. They showed
that adding a victim buffer to an 8 Kbyte direct-mapped data cache results in
10% energy savings and 3.5% performance improvements on average for the
Spec95 benchmark suite.
A victim buffer improves the performance and energy of a direct-mapped
cache on average, but for some applications, a victim buffer actually
degrades performance without much or any energy savings, as we will show
later. Such degradation occurs when the victim buffer hit rate is low.
Checking a victim buffer requires an extra cycle after an L1 miss. If the
victim buffer hit rate is high, that extra cycle actually prevents dozens of
cycles for accessing the next level memory. But if the buffer hit rate is low,
that extra cycle does not save much and thus is wasteful. Whether a victim
buffer’s hit rate is high or low is dependent on what application is running.
Such performance overhead may be one reason that victim buffers are not
always included in embedded processor cache architectures.
In this section, we will show that treating the victim buffer as a
configurable memory parameter to a direct-mapped cache is superior to
either using a direct-mapped cache without a victim buffer or using a direct-
mapped cache with an always-on victim buffer [25]. Furthermore, we show
that a victim buffer parameter is even useful with a cache that itself is highly
parameterized.
113
6.5.1 Victim Buffer as a Cache Parameter
We consider adding a victim buffer to both core-based and pre-fabricated
platform based design situations.
A core-based approach involves incorporating a processor (core) into a
chip before the chip has been fabricated, either using a synthesizable core
(soft core) or a layout (hard core). In either case, most core vendors allow a
designer to configure the level 1 cache’s total size (typical sizes range from
no cache to 64 Kbyte), associativity (ranging from direct mapped to 4 or 8
ways), and sometimes line size (ranging from 16 bytes to 64 bytes). Other
parameters include use of write through, write back, and write allocate
policies for writing to a cache, as well as the size of a write buffer. Adding a
victim buffer to a core-based approach is straightforward, involving simply
including or not including a buffer into the design.
A pre-fabricated platform is a chip that has already been designed, but is
intended for use in a variety of possible applications. To perform efficiently
for the largest variety of applications, recent platforms come with
parameterized architectures that a designer can configure for his/her
particular set of applications. Recent architectures include cache parameters
[2,8,9] that can be configured by setting a few configuration register bits. We
therefore developed a configurable victim buffer that could be turned on or
off by setting bits in a configuration register.
6.5.2 Experiments and Results
The experimental setup and energy calculations are the same as those
described in Section 6.3.4. The benchmarks examined include programs
from the Powerstone [9], MediaBench [18], and Spec2000 [26] benchmark
suites.
6.5.2.1 Victim Buffer with a Direct-Mapped Cache
Figure 6-3 shows the performance and energy improvements when
adding an always-on victim buffer to a direct-mapped cache. Performance is
the program execution time. Energy is estimated as described in section
6.3.4. 0% represents the performance and energy consumption of an 8 Kbyte
direct-mapped cache. From Figure 6-3, we see that a victim buffer improves
both performance and energy for some benchmarks, like mpeg, epic, and
adpcm. For other benchmarks, energy is not improved but performance is
degraded, as for vpr, fir, and padpcm. A victim buffer should be excluded or
turned off for these benchmarks. Some benchmarks, like jpeg, parser, and
auto2, yield some energy savings at the expense of some performance
degradation using a victim buffer – a designer might choose whether to
114
include/exclude or turn on/off the buffer in these cases depending on
whether energy or performance is more important.
Figure 6-3. Performance and energy improvements when adding a victim buffer to an 8 Kbyte
direct-mapped cache. Positive values mean the victim buffer improved performance or
energy, with 0% representing an 8 Kbyte direct-mapped cache without a victim buffer.
Benchmarks with both bars positive should turn on the victim buffer, while those with
negative performance improvement and little or no energy improvement should turn off the
victim buffer.
6.5.2.2 Victim Buffer with a Parameterized Cache
Figure 6-4 shows the performance and energy improvement of adding a
victim buffer to a parameterized cache having the same configurability
described by Zhang et. Al. [2] 0% represents the performance and energy of
the original configurable cache when tuned optimally to a particular
application. The bars represent the performance and energy of the
configurable cache when optimally tuned to an application assuming a
victim buffer exists and is always on. The optimal cache configurations for a
given benchmark are usually different for each of the two cases (no victim
buffer versus always-on victim buffer).
We see that, even though the configurable cache already represents
significant energy savings compared to either a 4-way or direct-mapped
cache [2], a victim buffer extends the savings of a configurable cache by a
large amount for many examples. For example, a victim buffer yields an
additional 32%, 43%, and 23% energy savings for benchmarks adpcm, epic,
and mpeg2. The savings of adpcm and epic come primarily from the victim
buffer that reduces the visits to off-chip memory. The saving of epic comes
primarily from the victim buffer enabling us to configure the configurable
cache to use less associativity without increasing accesses to the next
memory level. Yet, for other benchmarks, like adpcm, auto2 and vpr, the
21% 24%38% 43% 60%
-4%
0%
4%
8%
12%
16%
pad
pcm
crc
au
to2
bcn
t
bil
v
bin
ary
bli
t
bre
v
g3
fax
fir
pje
g
cb
qso
rt
v4
2
ad
pcm
ep
ic
g7
21
peg
wit
mp
eg
jpeg
art
mcf
pars
er
vp
r
performance
energy
115
victim buffer yields performance overhead with no energy savings and thus
should be turned off.
Figure 6-4. Performance and energy improvements when adding a victim buffer to an 8 Kbyte
configurable cache. 0% represents a configurable cache without a victim buffer, tuned
optimally to the particular benchmark.
6.6 LOW STATIC-POWER FREQUENT-VALUE
DATA CACHES
Recently, a frequent value (FV) low power data cache design was
proposed based on the observation that a major portion of data cache
accesses involves frequent values, which can be dynamically captured [27].
Frequent values are encoded in the cache, occupying only a few bits.
We improve upon previous FV data caches by reducing static power by
shutting off the unused bits in the larger sub-array for encoded frequent
values [28]. Since frequent values are stored in encoded form using only the
few bits in the smaller sub-array, the remaining bits in the larger sub-array
serve no purpose as long as the value stays frequent. Such shutoff may be
beneficial since FVs occupy many words in data caches [27].
Furthermore, the original FV low power cache design suffers from an
extra cycle when reading non-FVs [27], which account for 68% of all data
cache accesses, resulting in a 5% increase in execution time. We used circuit
design to remove the extra cycle.
6.6.1 Overview of Original FV Cache Design
In this section, we give a brief overview of the original FV data cache
designed by Yang and Gupta [27].
The FV cache was proposed based on the observation that a small
number of distinct frequently occurring data values often occupy a large
portion of program memory data spaces and therefore account for a large
portion of memory accesses [27]. This frequent value phenomenon was
32% 43% 23%
-4%
0%
4%
8%
12%p
ad
pcm
au
to2
bil
v
bli
t
g3
fax
pje
g
v4
2
epic
peg
wit
jpeg
mcf
vp
r
performance energy
116
exploited in designing a data cache that trades off performance with energy
efficiency.
From the perspective of the frequent value cache, data values are divided
into two categories: a small number of frequent values, in our case 32 FVs,
and all remaining values that are referred to as non-frequent values. The
frequent values are stored in encoded form and therefore can be represented
in 5 bits; the non-frequent values are stored in unencoded form in 32 bit
words. Additionally, a flag bit is needed for each word in the cache to
determine if the value stored in that location is encoded or not. The set of
frequent values remains fixed for a given program run.
When reading a word from the cache, initially we simply read from the
low-bit array. Since every word read out contains a flag bit, the flag is
examined to determine what comes next. The flag being 1 means the desired
word is in un-encoded form, so the remaining bits should be read out from
the high-bit array to form the original value. On the other hand, the flag
being 0 means that the desired word is a frequent value and stored in
encoded form. In this case, the access proceeds to decode the value. Since
the access to the high-bit array is avoided, cache activity is reduced.
A write to the FV cache is performed as follows. Before a value is
written, it is first encoded through an encoder. If encoding is successful, it
means that the value is a frequent value and thus a 5-bit code is stored in the
low-bit array and the flag bit is cleared. In this case, accessing the high-bit
array is avoided. If the encoding fails, the value to be written is a non-
frequent value and thus both low-bit and high-bit data arrays are accessed as
well as the flag bit being set. Note that writing non-FVs does not need to
take two cycles as does reading non-FVs, because the value is encoded early
in the pipeline and thus the decision of driving one array or two is clear
before the access.
6.6.2 Improving the FV Cache Design
The FVs are not only accessed frequently, but also distributed widely in
caches [29]. This phenomenon provides a good opportunity for reducing
static power. Our approach is the following. Since the 32-bit FVs are
encoded in 5 bits, the remaining 27 bits do not store any useful information.
Therefore, they can be shut down to save static power and as long as a value
stays frequent, static power is saved. The overall savings depend on the
occupancy of FVs in the cache. Our studies show that on average nearly half
of the cache content contains FVs, which indicates the benefit of reducing
static power through finding FVs.
The flag bits are initially set to 1, which means initially all words are
non-FVs. Any data to the data cache is checked with the FV encoder. If the
117
word is an FV, the corresponding flag bit is set to 0 and this cache word is
encoded and stored in the 5-bit array. At the same time, the flag bit turns off
the 27-bit portion of the word. Similarly, on reading FVs, only the 5-bit
portion is read and the 27-bit portion is gated off using the flag bit. On a
non-FV read or write, the flag bit is set to 0 and the original 32 bits are
written into the cache as usual. Our new circuit design improves the original
FV cache design in that there is no extra delay in determining accesses of the
27-bit portion.
6.6.3 Designers’ Choices of Using the FV Cache
We have described a low static power FV cache. When utilized into a
processor system, the FV cache can be designed with different degrees of
complexity and flexibility. In this section, we provide three approaches that
are suitable for a variety of processors targeting different types of
applications. Essentially, the complexity comes from how FVs are identified
and if they are allowed to vary for different applications. As always, the
more flexibility the processor provides, the more complex the FV cache is.
The first approach is appropriate to application specific processors. Since
only a single type of application runs on the processor, its FVs tend to be
stable over time. In such cases, the FVs can be first obtained from a profiling
run through simulations, and then synthesized into the cache as part of the
cache data storage. The advantage of this approach is that once the FVs are
hard coded on-chip, the cache does not perform operations other than reads.
Thus, the logic of this component is simple and can be designed to consume
minimum power.
The second approach extends the first one with the ability of changing
the FVs according to different applications. This approach is suitable for a
multi-task environment in which the processor runs multiple programs
instead of single program. Each program’s FVs are still obtained off-line.
Instead of synthesizing the FVs on-chip, a register file may be used to store
FVs so that they can be rewritten on each activation of a different program.
The size of the register file depends on the number of FVs of interest to the
designer, which is heavily dependent on each program’s behavior.
The third approach provides the maximum flexibility in maintaining FVs.
According to a previous study [29], some programs’ FVs are sensitive to
different inputs. This suggests that another dimension of varying FVs might
be added into the design. Since it is infeasible to profile every program on all
possible inputs to catch FVs, detecting FVs on-line would be useful. Thus,
on top of the second approach, the register file could be extended to
dynamically capture FVs using extra logic. In the scheme proposed by Yang
and Gupta [27], an inexpensive hardware FV finder was developed that
118
monitored cache accesses. The FV finder was turned on for only the first 5%
of memory accesses assuming that the total memory access numbers are
known a priori. After that, the FVs were captured in the finder and
transmitted to the cache so that the cache starts operating as an FV cache.
The energy overhead of the finder was estimated to be 0.3%-6.1% of the L1
D-cache (8 Kbyte to 64 Kbyte caches were tested). The area overhead is
similar to our second approach, and thus modest. One potential issue is that
the FV finder described detects frequently accessed values, which may or
may not correspond to frequently distributed values in memory, though they
usually are the same. We leave an FV finder for frequently distributed values
for future work.
6.6.4 Experiments and Results
To determine the benefits of our FV cache architecture in reducing static
energy, we ran 11 SPEC2000 [26] benchmarks through the SimpleScalar
tool set [19]. We used a 4-issue out-of-order processor simulator with a 32
Kbyte L1 instruction and data cache. The benchmarks were fast-forwarded
for 1 billion instructions and executed for 500 million instructions
afterwards, using reference inputs.
6.6.4.1 Static Energy Savings
Our main goal is to reduce the static energy consumed by the data cache
without losing performance. As mentioned earlier, the overall static energy
saving depends on the average coverage of FVs inside data cache. Through
experiments, we found that there are abundant FVs in the L1 data cache at
any time for Spec 2000 benchmarks, as shown in Figure 6-5. The percentage
shown is the average for the 500 million instructions execution time. On
average, 49.2% of the total words are FVs, with the highest being 77.0% for
benchmark mcf and the lowest 9.4% for benchmark ammp. The static energy
savings are proportional to the number of FVs in the data cache. Thus, the
corresponding static energy savings on average are 35%
(49.2%×27/33×86%) considering that 27 bits out of 33 bits (we need a flag
bit per 32-bit word) are shut off and 86% of static power can be saved using
a pMOS Gated-Vdd. When compared with the conventional 32-bit per word
cache, the static energy savings can be calculated as 100%- (100%-
35%)*33/32 = 33%.
119
Figure 6-5. Percentage of data cache words that are FVs
6.6.4.2 Performance Improvement
Our second achievement is the performance improvement over the
original FV data cache design. Recall that the original FV cache
performance overhead was due to the prolonged non-FV accesses. The more
non-FV accesses, the slower the execution and the less the overall power
savings (less energy savings), since the system would consume more energy
when the program runs longer. We measured the average percentage of
cache hits that are FVs, as shown in Figure 6-6(a). On average, the hit rate
on data FVs is 32% with the highest being 62.7% for votex and the lowest
11.4% for mcf. Therefore, we can see that on average, 68% of cache accesses
are non-FVs.
Figure 6-6. (a) Hit rate of FVs in data cache; (b) Performance (IPC) degradation of two-cycle
FV cache
With our improved circuitry (1-cycle latency for non-FVs as well as for
FVs), we are able to maintain the same execution speed as the base case. To
see how much performance we have gained over the original FV cache, we
measured the IPCs for a normal cache and a 2-cycle FV cache and plot them
in Figure 6-6(b). The IPC for our improved design is the same as the normal
cache. Figure 6-6(b) shows the slowdowns of the original FV cache design,
which is the same value as our performance improvement. We can see that
there is a 5.2% difference in the averaged IPCs between the original FV
cache and our improved version. This also means that in addition to the
static energy we saved by shutting off partial FV words, we also saved more
dynamic energy than the original FV cache design.
0 %
2 0 %
4 0 %
6 0 %
8 0 %
art
mcf
pars
er
vpr
bzip
am
mp
mesa
vote
x
equake
gcc
gzip
Ave
0.0
0.5
1.0
1.5
2.0
art
mcf
pars
er
vpr
gzip
am
mp
mesa
vort
ex
equake
gcc
bzip
2
Ave
IPC
Normal Cache 2-cycle FVC
(b)
0%
25%
50%
75%
art
mcf
pars
er
vp
r
bzip
am
mp
mesa
vo
tex
eq
uak
e
gcc
gzip
Av
e
(a)
120
Another feature in our new design is that it is safe in the sense that it does
not increase power consumption significantly even when FVs are not
abundant. Thus, our improved FV cache design is an appealing approach in
reducing both static and dynamic energy of caches.
Acknowledgements
This work was supported by the National Science Foundation (CCR-
0203829, CCR-9876006) and by the Semiconductor Research Corporation
(2003-HJ-1046G).
References
[1] S. Segars. Low power design techniques for microprocessors, International Solid State
Circuit Conference, February 2001.
[2] C. Zhang, F. Vahid, and W. Najjar. A highly-configurable cache architecture for
embedded systems. 30th Annual International Symposium on Computer Architecture,
June 2003.
[3] Altera, Nios Embedded Processor System Development, http://www.altera.com/corporate/
news_room/releases/products/nr-nios_delivers_goods.html.
[4] Arc International, www.arccores.com.
[5] ARM, www.arm.com.
[6] MIPS Technologies, www.mips.com.
[7] Tensilica, Xtensa Processor Generator, http://www.tensilica.com/.
[8] D. H. Albonesi. Selective Cache Ways: On Demand Cache Resource Allocation. Journal
of Instruction Level Parallelism, May 2002.
[9] A. Malik, W. Moyer, and D. Cermak. A Low Power Unified Cache Architecture
Providing Power and Performance Flexibility. International Symposium on Low Power
Electronics and Design, 2000.
[10] R. Balasubramonian, D. Albonesi, A. Buyuktosunoglu, and S. Dwarkadas. Memory
Heirarchy Reconfiguration For Energy and Performance in General-Purpose Processor
Architecture. 33rd International Symposium on Microarchitecture, December 2000.
[11] A. Veidenbaum, W. Tang, R. Gupta, A. Nicolau, and X. Ji. Cache Access and Cache
Time Model. IEEE Journal of Solid-State Circuits, Vol 31, No 5, 1996.
[12] T. Givargis and F. Vahid. Platune: A Tuning Framework For System-On-a-Chip
Platforms. IEEE Transactions on Computer Aided Design, November 2002.
[13] M. Palesi and T. Givargis. Multi-Objective Design Space Exploration Using Genetic
Algorithms. International Workshop on Hardware/Software Codesign, May 2002.
[14] C. Zhang and F. Vahid. Cache Configuration Exploration on Prototyping Platforms. 14th
IEEE International Workshop on Rapid System Prototyping , June 2003.
[15] A. Ghosh and T. Givargis. Cache Optimization For Embedded Processor Cores: An
Analytical approach. International Conference on Computer Aided Design, November
2003.
[16] C. Zhang, F. Vahid, and R. Lysecky. A Self-Tuning Cache Architecture for Embedded
Systems. Design Automation and Test in Europe Conference (DATE), February 2004.
[17] M. Powell, A.Agarwal, T. Vijaykumar, B. Falsafi, and K. Roy. Reducing Set-Associative
Cache Energy via Way-Prediction and Selective Direct Mapping, 34th
International
Symposium on Microarchitecture, 2001.
121
[18] C. Lee, M. Potkonjak, and W.H. Mangione-Smith. MediaBench: A Tool For Evaluating
and Synthesizing Multimedia and Communication Systems. Proc 30th
Annual
International Symposium on Microarchitecture, December 1997.
[19] D. Burger, T. Austin, and S. Bennet. Evaluating Future Microprocessors: The
Simplescalar Toolset. University of Wisconsin-Madison. Computer Science
Department Tech. Report CS-TR-1308, July 2000.
[20] http://www.mips.com/products/s2p3.html, 2003.
[21] A. Gordon-Ross, F. Vahid, and N. Dutt. Automatic Tuning of Two-Level Caches to
Embedded Applications. Design Automation and Test in Europe Conference (DATE),
February 2004.
[22] EEMBC, the Embedded Microprocessor Benchmark Consortium, www.eembc.org.
[23] N. Jouppi. Improving Direct-Mapped Cache Performance by the Addition of a Small
Fully-Associative Cache and Prefetch Buffers, Proceedings of International Symposium
on Computer Architecture, 1990.
[24] G. Albera and R. Bahar. Power/performance Advantages of Victim Buffer in High-
Performance Processors, IEEE Alessandro Volta Memorial Workshop on Low-Power
Design, 1999.
[25] C. Zhang and F. Vahid. Using a Victim Buffer in an Application-Specific Memory
Hierarchy. Design Automation and Test in Europe Conference (DATE), February 2004.
[26] http://www.specbench.org/osg/cpu2000.
[27] J. Yang and R. Gupta. Energy Efficient Frequent Value Data Cache Design, Int. Symp.
on Microarchitecture, Nov. 2002.
[28] C. Zhang, J. Yang, and F. Vahid. Low Static-Power Frequent-Value Data Caches. Design
Automation and Test in Europe Conference (DATE), February 2004.
[29] J. Yang and R. Gupta. “Frequent Value Locality and its Applications,” ACM
Transactions on Embedded Computing Systems (inaugural issue), Vol. 1, No. 1, pages
79-105, November 2000.
122
Chapter 7
REDUCING ENERGY CONSUMPTIONIN CHIP MULTIPROCESSORS USINGWORKLOAD VARIATIONS
I. Kadayif1, M. Kandemir2, N. Vijaykrishnan2, M. J. Irwin2 and I. Kolcu3
1Canakkale Onsekiz Mart University;2Pennsylvania State University;3UMIST
Abstract Advances in semiconductor technology are enabling designs with several hundredmillion transistors. Since building sophisticated single processor based systemsis a complex process from design, verification, and software development per-spectives, the use of chip multiprocessing is inevitable in future microprocessors.In fact, the abundance of explicit loop-level parallelism in many embedded ap-plications helps us identify chip multiprocessing as one of the most promisingdirections in designing systems for embedded applications. Another architecturaltrend that we observe in embedded systems, namely, multi-voltage processors, isdriven by the need of reducing energy consumption during program execution.Practical implementations such as Transmeta’s Crusoe and Intel’s XScale tuneprocessor voltage/frequency depending on current execution load. Consideringthese two trends, chip multiprocessing and voltage/frequency scaling, this chapterpresents an optimization strategy for an architecture that makes use of both chipparallelism and voltage scaling. In our proposal, the compiler takes advantage ofheterogeneity in parallel execution between the loads of different processors andassigns different voltages/frequencies to different processors if doing so reducesenergy consumption without increasing overall execution cycles significantly.Our experiments with a set of applications show that this optimization can bringlarge energy benefits without much performance loss.
Keywords: Chip multiprocessing, voltage scaling, loop-level parallelism, embedded systems,optimizing compilers.
123
7.1 INTRODUCTION
Rising development costs motivate computer architecture companies to de-sign fewer systems-on-chip, but to make each one they do design more flex-ible and programmable. Doing so makes it possible to reuse designs to takeadvantage of economies of scale and shorten time-to-market. Moreover, pro-grammability allows companies to keep products in the market longer, boostingintegrated profits.
High-performance embedded processors have traditionally relied mainly onclock frequency and superscalar instruction issue to boost performance. Whilefrequency and superscalarity have served the industry well and will continueto be used, we believe that they have limitations that will diminish the gainsthey will deliver in the future. The gains in operating frequencies, which havehistorically come at a rate of about 35 percent per year, are attributable to twomajor factors: semiconductor feature scaling and deeper pipelining. But eachof these factors is approaching the point of diminishing returns. Similarly,superscalar processing is nearing its limits, mainly due to the exponential in-crease in complexity in dispatch logic with increasing issue width. In addition,superscalar processing is limited by the inherent instruction-level parallelismin the code. Although VLIW implementations are less complex than theirsuperscalar counterparts (since most of execution decisions are made by thecompiler), they still employ power-hungry components and are limited by theavailable instruction-level parallelism. It should also be noted that both super-scalar and VLIW architectures are not efficient from an energy consumptionviewpoint. Therefore, it is not clear whether current architectures will be suffi-cient for meeting continuously increasing power and performance demands ofapplications.
These observations motive system designers to investigate different archi-tectures. When one looks at computer architecture industry today, two differenttrends in system design can easily be observed: on-chip multi-processing andmulti-voltage processors. On-chip multi-processors take advantage of high-level, coarse-grain parallelism that exists due to the natural independence ofseparate program fragments (e.g., functions and loops). As compared to super-scalar and VLIW architectures, they are much more suitable for array-intensiveembedded applications. Another advantage of using an on-chip multiprocessor,instead of a more powerful and sophisticated uniprocessor, is that there is lessdifficulty in designing a smaller, less complex chip. This also speedups chipverification and validation. Thus, time required to put the chip in the marketbecomes shorter. One can see several examples of on-chip multi-processingtoday in both academia and industry. For example, the four-core Hydra fromStanford University [14] is built around Integrated Device Technology Inc.’sRC32364 processor, which uses a 0.25-micron process, and runs at 250 MHz.
124
As manufacturing processes keep getting refined, it becomes even easier toreplicate the core several times on a single die. The MAJC architecture fromSun Microsystems [11] allows one to four processors to share the same die, andfor each to run separate threads. Each processor is limited to four functionalunits (each of which are able execute both integer and floating point operations,making the MAJC architecture more flexible). Another example of an on-chipmulti-processor from industry is the Power4 processor from IBM [15], wheretwo processors are placed into the same die.
The second trend, multi-voltage processors, is mainly driven by the need toreduce energy consumption during program execution. Practical implementa-tions such as Transmeta’s Crusoe [10] and Intel’s XScale [8] scale processorvoltage/frequency depending on execution load. Observing that one rarelyneeds an application to exercise a processor’s maximum performance and theunused extra performance usually represents wasted energy, Crusoe designerstry to match the operating level of the processor (in terms of voltage and fre-quency) to the performance requirements of the application being executed.Depending on the voltage regulator, a Crusoe processor can change its voltagein steps of 25mV and its frequency in steps of 33MHz.
Considering the continuously pressing power and performance demands, wecan expect these two techniques to be co-exist in the future embedded archi-tectures. Specifically, we believe that future architectures will be based onon-chip multi-processors, where each on-chip processor can be individuallyvoltage/frequency scaled. Considering such an architecture, this paper investi-gates the energy/performance tradeoffs in parallelizing array-intensive applica-tions taking into account the possibility that individual processors can operatein different voltage/frequency levels. In assigning voltage levels to processors,we make use of compiler analysis that reveals heterogeneity between the loadsof different processors in parallel execution. Our experiments with a set of ap-plications show that the proposed optimization can bring large energy benefitswithout much performance penalty.
The rest of this chapter is organized as follows. The next sections describesour chip multiprocessor. Section 7.3 discusses why we may be experiencingload imbalance across on-chip processors at runtime. Section 7.4 discussesthe necessary compiler analysis for determining workloads (on a loop nest ba-sis) of individual processors participating in parallel computation. Section 7.5discusses additional optimizations to further enhance our power savings. Sec-tion 7.6 describes our implementation, experimental platforms, and presentsperformance and energy numbers. Section 7.7 presents our concluding re-marks.
125
2 Cache3
L2 Cache Optional
Off-Chip
Cache
CPU0 CPU1 CPU2 CPU3
Cache1Cache0
Figure 7.1. Chip multiprocessor under consideration.
7.2 CHIP MULTIPROCESSOR ARCHITECTURE ANDEXECUTION MODEL
The chip multiprocessor we consider here is a shared-memory architecture;that is, the entire address space is accessible by all processors. Each processorhas a private L1 cache, and shared memory is assumed to be off-chip. Option-ally, we may include a (shared) L2 cache as well. Note that several architecturesfrom academia and industry fit in this description [1, 14, 11, 12]. We keep thesubsequent discussion simple by using a shared bus as the interconnect (thoughone could use fancier/higher bandwidth interconnects as well). We also usethe MESI [19] protocol (the choice is orthogonal to the focus of this paper)to keep the caches coherent across the CPUs. We assume that voltage leveland frequency of each processor in this architecture can be set independentlyof the others, and this is the main mechanism through which we save power.This paper focuses on a single-issue, five-stage (instruction fetch (IF), instruc-tion decode/operand fetch (ID), execution (EXE), memory access (MEM), andwrite-back (WB) stages) pipelined datapath for each on-chip processor. Cur-rently, this is the only architectural model for which our compiler estimatesprocessor workload.
Note that progress in VLSI technology has allowed chip-makers to packmillions of transistors in a single die. Rather than throwing all these resourcesinto a single, powerful processing core and making this core very complex todesign and verify, chip-multiprocessors consisting of several simpler proces-sor cores can offer a more cost-effective and simpler way of exploiting thesehigher levels of integration. Chip multiprocessors also offer a higher granu-larity (thread/process level) at which parallelism in programs can be exploitedby compiler/runtime support, rather than leaving it to the hardware to extractthe parallelism at the instruction level on a single (larger) multiple-issue core.All these compelling reasons motivate the trends toward chip multiprocessor
126
architectures, and there is clear evidence of this trend in the several commercialofferings and research projects [1, 14, 11, 12].
Our application execution strategy can be summarized as follows. We focuson array-based applications that are constructed from loop nests. Typically,each loop nest in such an application is small but executes a large number ofiterations and accesses/manipulates large datasets (typically multidimensionalarrays). We employ a loop nest based application parallelization strategy. Morespecifically, each loop nest is parallelized independently of the others. In thiscontext, parallelizing a loop nest means distributing its iterations across proces-sors and allowing processors to execute their portions in parallel. For example,a loop with 1000 iterations can be parallelized across 10 processors by allo-cating 100 iterations to each processor. We also assume that after each loopnest execution, all processors get synchronized before they start executing thenext loop nest. Note that dropping this requirement would necessitate a so-phisticated compiler analysis to identify the cases under which a processor thatfinishes its portion of iterations from the previous loop nest can go ahead andstart executing its portion from the next loop nest without waiting for the others.Nevertheless, in our experiments to be presented later, we also evaluate suchan alternative strategy.
There are many proposals for power management of a dynamic voltagescaling-capable processor. Most of them are at operating system level andare either task-based [13, 17] or interval-based [21, 5]. While some proposalsaim at reducing energy without compromising performance, a recent study byGrunwald et al [6] observed noticeable performance loss for some interval-based algorithms using actual measurements. The existing compiler basedstudies such as [7, 16] target single processor architectures. In comparison, ourwork targets at a chip multiprocessor based environment.
7.3 LOAD IMBALANCE IN PARALLEL EXECUTION
We can broadly divide loop nest parallelization techniques into two cate-gories: static and dynamic. In the static case, the compiler (or the user) decidesa suitable parallelization strategy for each loop nest at compile time. The ideais to assign each loop iteration to a processor. There are at least two ways ofdoing this. In block assignment, a group of consecutive loop iterations are as-signed to the same processor. Since such iterations typically access data storedin consecutive memory locations, this type of assignment can also be expectedto be data locality friendly. In cyclic assignment, the iterations assigned toprocessors are interleaved using some stride. While this type of assignmentis known to be good from a load balance viewpoint, it generally exhibits poordata locality. Consider, as an example, the loop nest shown below and the arrayreference in it:
127
2V1V0V
3V2V1V0V
3P
2P
1P
V
(c)(b)(a)
(e)(d)
2P3P 1P 0P>>> 3
0
3P2P1P 0P3P2P1P 0PP
P
3P
2P
1P
0P
2P
3P
1P
0
Figure 7.2. Different array accesses imposed by different iteration assignments (the array isassumed to be row-major).
for i: 1..1024for j: 1..1024
..X[i,j]..
Assuming that only the i-loop is parallelized across four processors (P0
through P3), Figure 7.2(a) illustrates how array X is accessed by the processorswhen block iteration assignment is used. In this assignment, each processorexecutes 256 × 1024 iterations, and accesses a group of consecutive rows ofthe array as depicted in Figure 7.2(a). However, it is also possible to parallelizethis loop (i) by distributing its iterations cyclicly across processors using someregular stride. For example, we can give the first 128 × 1024 iterations to thefirst processor, the next 128 × 1024 to the second one and so on, and whenwe give its quota to the last processor, we can repeat the whole process (untilall loop iterations have been assigned) starting over with the first processor.Figure 7.2(b) shows how array X is accessed by the processors under this cycliciteration assignment scheme. Note that the cyclic iteration distribution is flexi-ble in the sense that it can work with any stride. For example, instead of using128 × 1024 iteration chunks, we could have easily used 16 × 1024 or even 1× 1024 iteration chunks.
In comparison, in a dynamic parallelization strategy, the assignment of iter-ations to processors is performed dynamically during the course of executionby a central controller. Typically, this controller gives a new set of loop it-erations to a processor when that processor is done with executing its currentset of assigned iterations. While the dynamic strategy is expected to balance
128
the workloads of processors better than static strategies (as it can take run-time constraints into account), it also incurs a much higher runtime cost — interms of both execution cycles and power consumption — (as compared to thestatic parallelization schemes) since decisions regarding iteration assignmentsare made at runtime. Therefore, our focus in this study is on static loop nestparallelization.
Consider now the following loop nest:
for i: 1..1024for j: i..1024
..X[i,j]..
While this loop nest is similar to the previous one considered above, there isone significant difference: the lower bound of the inner loop (j) is i (insteadof 1). Figure 7.2(c) shows how the four processors access the array in questionwhen block iteration assignment is employed. Clearly, there is a significantload imbalance across the processors. Assuming that each iteration of this loopnest has the same cost (in terms of execution cycles) and all processors shouldsynchronize following the execution of the nest, there is not any advantage forthe processors with the light load to finish their set of iterations as soon as pos-sible. Instead, they can delay their executions (by reducing their frequencies)and lower their voltages to save energy while making sure that their executiondoes not take more time than that of the processor with the largest load (op-erating with the highest voltage level). Figure 7.2(d) illustrates such a voltageassignment, assuming that V0 is the highest voltage level available. The workpresented in this paper performs such a voltage-to-processor assignment foreach loop nest of a given array-based application. In a sense, in our frame-work the job of the compiler is not just to decide which loop iterations shouldbe assigned to which processors but also which supply voltage/frequency eachprocessor needs to use. Our objective is to save as much power as possiblewithout incurring much performance penalty.
At this point, someone might claim that it would be better in this case (Fig-ure 7.2(c)) to use cyclic assignment instead of block assignment as this wouldeliminate the load imbalance problem introduced by the latter to a large extent.However, this may not be a viable option in general. Consider, for example,the scenario depicted in Figure 7.2(e), where the direction of parallelization isreversed (due to data dependences for example). In this case, cyclic assignmentwould be very costly in terms of data locality (cache behavior), assuming thatthe array in question is stored as row-major. Considering the fact that off-chipmemory accesses are getting more and more expensive in terms of processorcycle times, one may not want to degrade data locality.
129
7.4 COMPILER SUPPORT
As mentioned earlier, the compiler’s job in our setting is to assign not onlyiterations to processors but also come up with a suitable voltage level for eachprocessor. To do this, the compiler needs to estimate the workload of eachprocessor and match it with an appropriate voltage/frequency level. Withoutloss of generality, we assume that there are s voltage/frequency levels availableto the compiler. Our compiler-based approach proceeds as follows:
• Parallelization Step. In this step, the compiler parallelizes an applica-tion in a loop nest basis. That is, each loop nest is parallelized independentlyconsidering the intrinsic data dependences it has. Since we are targeting achip multiprocessor, our parallelization strategy tries to achieve (for each nest)outer-loop parallelism to the best extent possible. In other words, we parallelizethe outermost loop (in the nest) that carries no data dependence. Our baselineresults are obtained using this parallelization strategy. Later in our experiments,we change our parallelization strategy to conduct a sensitivity analysis.
• Processor Load Estimation. In this step, the compiler estimates theload of each processor in each nest. To do this, it performs two calculations:(a) iteration count estimation and (b) per-iteration cost estimation. Since inmost array-based embedded applications bounds of loops are known beforeexecution starts, estimating the iteration count for each loop nest is not verydifficult. The challenge is in determining the cost (in terms of execution cycles)of a single iteration (for a given loop nest). Since the processors employed inour chip multiprocessor are simple single-issue cores, our cost computation isclosely dependent on the number and types of the assembly instructions thatwill be generated for the loop body. Specifically, we associate a base executioncost with each type of assembly instruction. In addition, we also estimatethe number of cache misses. Since loop-based embedded applications exhibitvery good instruction locality (as they spend most of their execution cycleswithin loop nests and there are not too many conditional-if executions), wefocus on data cache and estimate data cache misses using the method proposedby Carr et al [2]. An important issue is to estimate (at the source level) whatassembly instructions will be generated for the loop body in question. Weattack this problem as follows. The constructs that are vital to the studied codesinclude a typical loop, a nested loop, assignment statements, array references,and scalar variable references within and outside loops. Our objective is toestimate the number of assembly instructions of each type associated with theactual execution of these constructs. To achieve this, the assembly equivalentsof several codes were obtained using our back-end compiler (a variant of gcc)with the O2-level optimization. Next, the portions of the assembly code werecorrelated with corresponding high-level constructs to extract the number andtype of each instruction associated with the construct. In order to simplify the
130
correlation process and to partially isolate the impact of instruction choice dueto low-level optimizations, the assembly instructions with similar functionalityand energy consumption are grouped together. For example, both branch-if-not-equal (bne) and branch-if-equal (beq) are grouped as a generic branchinstruction (denoted bra).
To illustrate our parameter extraction process in more detail, we focus onsome specifics of the following example constructs. First, let us focus on aloop construct. Each loop construct is modeled to have a one-time overhead toload the loop index variable into a register and initialize it. Each loop also hasan index comparison and an index increment (or decrement) overhead whosecosts are proportional to the number of loop iterations (called trip count or trip).From correlating the high-level loop construct to the corresponding assemblycode, each loop initialization code is estimated to execute one load (lw) andone add (add) instruction (in general). Similarly, an estimate of trip+1 load(lw), store-if-less-than (stl), and branch (bra) instructions is associated with theindex variable comparison. For index variable increment (resp. decrement),2×trip addition (resp. subtraction) and trip load, store, and jump instructionsare estimated to be performed.
Next, we consider extracting the number of instructions associated with ar-ray accesses. First, the number and types of instructions required to computethe address of the element are identified. This requires the evaluation of thebase address of the array and the offset provided by the subscript(s). Our cur-rent implementation considers the dimensionality of the array in question, andcomputes the necessary instructions for obtaining each subscript value. Com-putation of the subscript operations is modeled using multiple shift and addi-tion/subtraction instructions (instead of multiplications) as this is the way ourback-end compiler generates code when invoked with the O2 optimization flag.Finally, an additional load/store instruction was associated to read/write thecorresponding array element. Note that these correlations between high-levelconstructs and low-level assembly instructions are a first-level approximationfor our simple architecture and array-dominated codes with the O2-level op-timization and obtained through extensive analysis of a large number of codefragments.
Based on the process outlined above, the compiler estimates iteration countfor each processor and per-iteration cost. Then, by multiplying these two, itcalculates the estimated workload for each processor. While this workloadestimation may not be 100% accurate, it allows the compiler to rank processorsaccording to their workloads and assign suitable voltage levels and frequenciesto them as will be described in the next item. As an example consider thesecond loop nest shown above, parallelized using 4 processors. Assuming thatour estimator estimates the cost of loop body as L instructions, the loads of
131
processors P0, P1, P2, and P3 are 256 × 1024 × L, 256 × (1024-257+1) × L,256 × (1024-513+1) × L, and 256 × (1024-769+1) × L, respectively.
• Voltage Assignment. In this step, the compiler first orders the proces-sors according to non-increasing workloads. After that, the highest voltage isassigned to the processor with the largest workload (the objective being not toaffect the execution time to the greatest extent possible). Then, the processorwith the second highest workload gets assigned to the minimum voltage levelVk available (where 1 ≤ k ≤ s) that does not cause its execution time to exceedthat of the processors with the largest workload. In this way, each processorgets the minimum voltage level (to save maximum amount of power) withoutincreasing overall parallel execution time of the nest (which is determined bythe processor with the largest workload). Continuing with the example above,suppose that we have two voltage/frequency levels (that is, V1/f1 and V2/f2,assuming s = 2 and V1/f1 > V2/f2), we first determine the execution time takenby processor P0 (denoted T0). Then, for each other processor, we use V2/f2
if doing so does not cause their execution times to exceed T0. If any of theseexecution times exceeds T0 (when using V2/f2), we switch back to V1/f1 forthat processor.
The success of our strategy critically depends on two important factors. First,there should be some load imbalance to exploit between different processors.This is because if there is no such imbalance then it is reasonable to executeeach processor with the highest voltage/frequency. Second, the compiler-basedworkload estimation should be reasonably accurate. If this is not the case, thenwe may assign a wrong voltage level/frequency to a processor, which may inturn impact overall execution time. In fact, in this scheme, the only time wepay some penalty is when our compiler-based workload estimation is not veryaccurate. In our experiments, we quantify this penalty in detail.
7.5 ADDITIONAL OPTIMIZATIONS
In this section, we discuss how the effectiveness of our strategy can be furtherincreased using additional optimizations.
7.5.1 Inter-Nest Optimization
In the description of our strategy above, we assumed that the processors willsynchronize at the end of each loop nest (before they start executing the nextloop nest). As noted by Tseng [20], such a global synchronization presentstwo major problems. First, to implement such a synchronization, the compilerneeds to generate extra (synchronization) code and insert it in the applicationcode. Obviously, this code presents extra performance and power overhead atruntime. Second, since this synchronization requires all processors to wait forthe slowest one, it makes poor use of available resources (from the performance
132
angle). Consequently, allowing a processor to continue without waiting forthe slower ones can allow small perturbations in processor execution times toeven out, thereby improving overall performance (by taking advantage of theloosely-coupled nature of chip multiprocessors). However, determining whenit is safe to allow a processor to continue without synchronization requires extracompiler analysis. In this study, we implemented a strategy that takes a number(called b) as a parameter, and for each loop nest, allows a processor to continuefor at most b next nests if doing so does not violate any data dependences.
7.5.2 Voltage/Frequency Reuse
Another optimization can be performed by being more careful in voltageassignment. Up to this point in our discussion we assumed that the processorassignment for each loop nest is done independently of the other nests. As aresult of this, as we move from one loop nest to another the same processor canget assigned different voltage levels. Consequently, we pay a penalty (in termsof both performance and energy consumption) for changing voltage levels. Thispenalty can be minimized by reusing the same voltage as much as possible forthe same processor throughout the execution. This can be achieved as follows.Suppose that in loop nest i, we used voltage level Vk for processor j. Whenwe move to loop nest i + 1 if we need to assign voltage level Vk to a processor,we use processor j for that. This can be repeated for each neighboring loopnest pair, and in this way, the processors reuse their voltage levels as much aspossible.
7.5.3 Adaptive Parallelization
So far in our treatment of the subject, we have assumed that we use allavailable processors in execution of all nests in the application. However, itis known from prior research [9] that, in some cases using fewer processors(and shutting off the unused ones along with their L1 caches) can result in abetter energy consumption behavior. We also conducted experiments with anadaptive strategy, where each loop nest is first profiled using different numberof processors in conjunction with our optimization strategy. After the profiling,for each loop nest, we identified the ideal number of processors, and used itin the actual execution. It should be noted that in adaptive parallelization weuse fewer number of processors than available (this means some performanceloss); however, turning off unused processors along with their L1 caches canbring energy benefits.
7.5.4 Combining Cyclic and Block Iteration Allocations
As has been discussed earlier in the paper, one may also opt to use cyclicdistribution of loop iterations across processors. Since our framework is able to
133
Table 7.1. Base simulation parameters used in our experiments.
Parameter Default Value
Number of Voltage/Frequency Levels 8Lowest/Highest Voltage Levels 0.8V/1.4V
Frequency Step Size 30MHzVoltage/Frequency Transition Penalty 10 cycles/2.10nJ
L1 Size 8KBL1 Line Size 32 bytes
L1 Associativity 4-wayL1 Latency 1 cycle
L2 Size (Shared) 2MBL2 Associativity 4-way
L2 Line Size 64 bytesL2 Latency 10 cycles
Memory Access Latency 100 cyclesBus Arbitration Delay 5 cycles
Replacement Policy Strict LRUL1 Energy (per access) 1.14nJL2 Energy (per access) 2.56nJ
Main Memory Energy (per access) 23.10nJ
estimate the number of cache misses, we can potentially have a better strategyas follows. For each loop nest, we can calculate the number of misses for bothblock and cyclic wise allocations and select the strategy that generates the bestenergy savings under a performance (execution cycles) constraint. We can referto such a strategy as hybrid since it makes use of both block and cyclic wiseallocation.
7.6 EXPERIMENTS
We tested the effectiveness of our algorithm in reducing energy consump-tion of chip multiprocessor using six array-intensive programs: 3D, DFE, LU,SPLAT, MGRID, and WAVE5. 3D is an image-based modeling applicationthat simplifies the task of building 3D models and scenes. DFE is a digitalimage filtering and enhancement code. LU is an LU decomposition program.SPLAT is a volume rendering application which is used in multi-resolution vol-ume visualization through hierarchical wavelet splatting. Finally, MGRID andWAVE5 are C versions of two Spec95FP applications. These C programs arewritten in such a fashion that they can operate on inputs of different sizes. Thedefault configuration parameters used in our experiments are given in Table7.1, and these are the values that are used unless explicitly stated/varied in thesensitivity experiments.
To conduct our experiments, we modified Simics [18]. Simics is a full systemsimulation platform that can simulate both uniprocessor and multiprocessor
134
Figure 7.3. Normalized energy consumption with different number of processors (8 voltagelevels).
machines. All energy results reported in this section include the energy spentin CPUs, their caches, and main memory and have been normalized with respectto the energy consumption when no voltage scaling is used and each processoris operated with maximum supply voltage and frequency.
The graph in Figure 7.3 gives the normalized energy consumptions withdifferent number of processors. We can make two main observations fromthis graph. First, all our six applications get some energy benefit from ourapproach with all processor sizes experimented. Second, our energy savingsget better with increased number of processors. This is because a larger numberof processors means more load imbalance to optimize, and our approach takesadvantage of it. When considering individual applications, one can see thatMGRID and WAVE5 perform poorly as compared to the others, mainly becausethese applications have very few cases where our approach is applicable. Incomparison, LU benefits much from increasing the number of processors sincemost of its few loops exhibit significant amount of load imbalance. Overall,the average savings across all six applications are between 16.03% (for the twoprocessor case) and 41.80% (for the thirty-two processor case). To evaluate theimpact of the number of voltage levels on energy savings, we also performedexperiments with different number of voltage levels. The results are presentedin Figure 7.4 for the 8 processor case. One can easily see from this graphthat the number of voltage levels has a significant impact on energy behavior.In particular, the difference in going from 4 levels to 8 levels is dramatic; thecorresponding savings are 6.63% and 29.02%. Increasing the number of voltagelevels further (to 16) does not bring too much additional energy benefits sincethere is little scope left to be optimized (beyond what could be optimized using
135
Figure 7.4. Normalized energy consumption with different voltage levels (8 processors).
8 levels). It should also be mentioned that when we have only 2 levels, theaverage saving across all applications is only 2.40%. This poor results is dueto the fact that our strategy tries not to increase execution cycles as much aspossible. Consequently, in many cases (when we have only 2 voltage levels)the compiler cannot use the lower voltage for a processor (even though theprocessor has low workload) since doing so would increase execution cyclesdramatically.
Recall that in Section 7.5 we discussed four different optimization strategiesthat can further increase energy savings. The graph shown in Figure 7.5 givesnormalized energy consumptions with these optimizations. The first bar foreach application corresponds to our strategy when none of these four optimiza-tions have been activated. Our first observation is that each application benefitsfrom one or more of these optimizations. Second, not every optimization is ef-fective for each benchmark. For example, using the hybrid iteration allocationbrings energy benefits in only 3D and DFE (since the nests in other allocationsexhibit a uniform behavior and prefer only one type of iteration allocation forthe best energy behavior). Similarly, adaptive parallelization is useful only for3D and SPLAT. To further study the impact of inter-nest optimization (one of theoptimizations discussed in Section 7.5), we also performed experiments withdifferent values for b (nore that the default value that we used in Figure 7.5 is 4).We see from the graph in Figure 7.6 that for the applications that benefit fromthis optimization, a b value of 4 seems to be reasonable. This is because in manycases the data dependences in the application prevent a processor from goingbeyond the next four nests to execute without waiting for the slower processors.
While the energy savings reported in this section are significant, for a faircomparison one also needs to consider the impact of our approach on perfor-
136
Figure 7.5. Impact of different optimizations on energy consumption (8 processors; 8 voltagelevels; and b = 4).
Figure 7.6. Impact of b on energy consumption (8 processors and 8 voltage levels).
mance. As has been pointed out earlier, our approach can lead to an increase inexecution cycles only if the compiler analysis is largely inaccurate. The graphin Figure 7.7 shows that the performance overhead incurred by our approachis below 2% in all but one (SPLAT) application. The reason that we have arelatively large performance penalty in SPLAT is the fact that this applicationexhibits a large number of conflict misses (over 68%, and rest are cold and ca-pacity misses), which cannot be captured by the cache miss estimation schemecurrently employed by our implementation. Consequently, the compiler is notvery successful in attaching suitable voltage levels to processors, and this inturn causes performance degradation. It is conceivable that a more accurate
137
Figure 7.7. Percentage increase in execution cycles (8 processors; 8 voltage levels).
cache miss estimation strategy (e.g., [4]) can help improve the behavior of thisbenchmark. This will be part of our future research on this topic.
7.7 CONCLUDING REMARKS
A chip multiprocessor lowers the number of functional units per processor,and distributes separate tasks/threads to each processor. This paper has evalu-ated a compiler-directed strategy that allows different processors to use differentvoltage levels/frequencies to take advantage of the load imbalances stemmingfrom loop parallelization. Our results with six applications clearly demonstratethe effectiveness of our strategy and makes a case for voltage-sensitive loopparallelization. Our results also show that it is possible to increase energysavings further by employing voltage/frequency reuse, adaptive parallelization,and inter-nest optimization.
Acknowledgments
This work was supported in part by NSF Career Awards #0093082 and#0093085, and a grant from GSRC PAS.
References
[1] L. A. Barroso, K. Gharachorloo, R. McNamara, A. Nowatzyk, S. Qadeer, B.Sano, S. Smith, R. Stets, and B. Verghese. Piranha: A Scalable ArchitectureBased on Single-Chip Multiprocessing. Proceedings of International Sym-posium on Computer Architecture, Vancouver, Canada, June 12–14 2000.
[2] S. Carr, K. S. McKinley, and C. Tseng. Compiler Optimizations for Im-proving Data Locality. Proceedings of the Sixth International Conference
138
on Architectural Support for Programming Languages and Operating Sys-tems, San Jose, October 1994.
[3] DAC’02 Sessions: Design Methodologies Meet Network Applications andSystem on Chip Design, New Orleans, LA, June 2002.
[4] S. Ghosh, M. Martonosi, and S. Malik. Cache Miss Equations: An An-alytical Representation of Cache Misses. Proceedings of the 11th ACMInternational Conference on Supercomputing, July, 1997.
[5] K. Govil, E. Chan, and H. Wasserman. Comparing Algorithms for DynamicSpeed-Setting of a Low-Power CPU. Proceedings of the 1st ACM Interna-tional Conference on Mobile Computing and Networking, November 1995.
[6] D. Grunwald, P. Levis, K. Farkas, C. Morrey III, and M. Neufeld. Poli-cies for Dynamic Clock Scheduling. Proceedings of the 4th Symposium onOperating System Design and Implementation, October 2000.
[7] C.-H. Hsu and U. Kremer. Dynamic Voltage and Frequency Scaling forScientific Applications. Proceedings of the 14th Workshop on Languagesand Compilers for Parallel Computing, August 2001.
[8] Intel XScale Technology. http://www.intel.com/design/intelxscale/.
[9] I. Kadayif, M. Kandemir, and U. Sezer. An Integer Linear ProgrammingBased Approach for Parallelizing Applications in On-Chip Multiproces-sors. In Proc. Design Automation Conference, New Orleans, LA, June 2002.
[10] A. Klaiber. The Technology Behind Crusoe Pro-cessors. Transmeta White Paper, January 2000.http://www.transmeta.com/about/press/white papers.html.
[11] MAJC-5200. http://www.sun.com/microelectronics/MAJC/5200wp.html
[12] MP98: A Mobile Processor. http://www.labs.nec.co.jp/MP98/top-e.htm.
[13] T. Okuma, T. Ishihara, and H. Yasuura. Real-Time Task Scheduling for aVariable Voltage Processor. Proceedings of the 12th International Sympo-sium on System Synthesis, 1999.
[14] K. Olukotun, B. A. Nayfeh, L. Hammond, K. Wilson, and K. Chang. TheCase for a Single Chip Multiprocessor. Proceedings of the 7th Intl Confer-ence on Architectural Support for Programming Languages and OperatingSystems, ACM Press, New York, 1996, pp. 2–11.
[15] POWER4 System Microarchitecture, White Paper, http://www-1.ibm.com/servers/eserver/pseries/hardware/whitepapers/power4.html
[16] H. Saputra, M. Kandemir, N. Vijaykrishnan, M. J. Irwin, J. S. Hu, C-H. Hsu, and U. Kremer. Energy-Conscious Compilation Based on Voltage
139
Scaling. Proceedings of ACM SIGPLAN Joint Conference LCTES’02 andSCOPES’02, Berlin , Germany, June, 2002.
[17] Y. Shin, K. Choi, and T. Sakurai. Power Optimization of Real-Time Em-bedded Systems on Variable Speed Processors. Proceedings of the Inter-national Conference on Computer-Aided Design, November 2000.
[18] SIMICS. http://www.virtutech.com/simics/simics.html.
[19] J. P. Singh and D. Culler. Parallel Computer Architecture: A Hardware-Software Approach, Morgan-Kaufmann, 1998.
[20] C.-W. Tseng. Compiler Optimizations for Eliminating Barrier Synchro-nization. Proceedings of 5th ACM Symposium on Principles and Practiceof Parallel Programming, Santa Barbara, CA, July 1995.
[21] M. Weiser, B. Welch, A. Demers, and S. Shenker. Scheduling for ReducedCPU Energy. Proceedings of the 1st Symposium on Operating SystemsDesign and Implementation, November 1994.
140
Chapter 8
ARCHITECTURES AND DESIGN TECHNIQUES
FOR ENERGY EFFICIENT EMBEDDED DSP
AND MULTIMEDIA PROCESSING
Ingrid Verbauwhede1,2
, Patrick Schaumont1, Christian Piguet
3, Bart
Kienhuis4
1University of California, Los Angeles; 2K.U.Leuven; 3CSEM; 4Leiden
Abstract Energy efficient embedded systems consist of a heterogeneous collection of
very specific building blocks, connected together by a complex network of
many dedicated busses and interconnect options. The trend to merge multiple
functions into one device makes the design and integration of these “systems-
on-chip” (SOC’s) even more challenging. Yet, specifications and applications
are never fixed and require the embedded units to be programmable. The topic
of this chapter is to give the designer architectures and design techniques to
find the right balance between energy efficiency and flexibility. The key is to
include programmability (or reconfiguration) at the right level of abstraction
and tuned to the application domain. The challenge is to provide an
exploration and programming environment for this heterogeneous architecture
platform.
Keywords: Embedded systems, architectures, low power, design tools, design exploration
8.1 INTRODUCTION
Embedded systems (e.g. a cell phone, a GPS receiver, a portable DVD
player, a HDD camcorder) use an architecture that is a heterogeneous
collection of very specific building blocks, connected together by a complex
network of many dedicated busses and interconnect options. General-
purpose programmable processors are not used for energy efficiency
reasons. Typically, multiple small embedded processor cores with
accelerators, IP cores, etc. are used. The trend to merge multiple functions
141
into one device (e.g. a cell phone with video capabilities) makes the design
and integration of these “systems-on-chip” (SOC’s) even more challenging.
Yet, specifications and applications are never fixed and require the
embedded units to be programmable. A good balance between energy
efficiency and programmability can be obtained by using programmable
domain-specific processors. A well known example are the programmable
digital signal processors (DSPs). DSPs are developed for wireless
communication systems (mostly driven by cellular standards). In a first
generation this meant that DSPs were adapted to execute many types of
filters (e.g. FIR, IRR), later communication algorithms such as Viterbi
decoding and more recently Turbo decoding are added.
A first trend we notice is that more applications and multiple applications
run in parallel or on demand on the device, e.g. video decoding, data
processing, multiple standards, etc. A second trend we notice is that these
new applications tend to run either on a separate domain specific
programmable processor or on a hardware accelerator (the distinction
between the two being rather blurry) next to the embedded DSP or micro-
controller instead of being tightly coupled into the instruction set of the host
processor.
A third trend we notice is that general-purpose programming
environments are getting more heterogeneous and domain-specific. The
general-purpose solutions are for energy efficiency reasons augmented with
domain specific units, accelerators, IP cores, etc. This is clearly visible in
FPGA’s, as the new generations now include specialized blocks such as
embedded core’s, block RAM’s and large numbers of multipliers. One
successful example is the Virtex-Pro family of Xilinx [17]. These devices
contain up to four Power PC cores, multiple columns of SRAM, multiple
columns of multipliers, Gbits IO transceivers, etc.
The architecture design of this heterogeneous SOC is a search in a three
dimensional design space, which we call the reconfiguration hierarchy [12].
First in the Y direction: at what level of abstraction should the programming
be introduced? Secondly in the X direction: which component of the
architecture should be programmable? Thirdly in the Z direction: what is the
timing relation between processing and the configuration/programming?
Programming can be introduced at multiple levels of abstraction. When it is
introduced at the instruction set level, it is called a “programmable
processor”. When it is introduced at the CLB level of an FPGA, it is called a
reconfigurable device. Regarding components, a processor has four basic
components: data paths, control, memory and interconnect. One has a choice
of making some or all of them programmable. Then the third question is to
compare the processing activity to the binding time. It makes a system
configurable, reconfigurable, or dynamic reconfigurable.
142
The challenge is to develop a design environment to navigate in this three
dimensional design space.
Several SOC platforms have been presented in literature. Most of them
focus on general -purpose regular architectures, e.g. [2]. Very few focus on
the low power issue and the need to tune the architecture towards the
application. One example is the low power Maya platform [18]. Unique to
our design approach is that we combine the design and programming of the
architecture with an environment to explore the best options.
The chapter is organized as follows. Section 8.2 and 8.3 look at the
architecture design, while section 8.4 and 8.5 discuss the design exploration,
co-design and co-simulation challenges.
8.2 ENERGY EFFICIENT HETEROGENEOUS SOC’S
The system designer needs an architecture platform that gives him the
lowest energy consumption, but at the same time provides enough flexibility
to allow re-programming or re-configuration. The key to energy efficiency is
to tune the architecture to the application domain. This means freezing
flexibility in the X (components) and Y (level of abstraction) direction of the
reconfiguration hierarchy. A hierarchy of so-called “Y charts” allows us to
do this in a top-down fashion [5].
A complex SOC will consist of multiple domain specific processing
engines. Each processor is programmable to a more or less degree. It can be
highly programmable if the processor is a micro-controller or a DSP engine
or a blank box of CLB units. The efficiency goes up as domain specific
instructions are added. An example of this is the addition of a MAC
instruction to a DSP processor. Loosely coupled co-processors will be more
energy efficient but less flexible as they fit a narrower application domain.
An example is the Turbo coder acceleration unit. The ultimate energy
efficient block is the optimized hard IP unit. Yet, it does not provide any
flexibility. In SOC a range and collection of these blocks are used.
Similarly arguments can be made for the interconnect component of a
SOC. Currently, we see only two extreme options: either dedicated one-to-
one connections and specialized busses, which have the lowest power
consumption (to a first order) or general-purpose global busses or inter-
connect, as provided by FPGA’s [17] or networks on chip [2]. The latter two
are both general-purpose solutions at different levels of abstraction to give
the designer a maximum flexibility and programmability.
143
Networking Security
ProtocolAlgorithm
ArchitectureMicroArchitectre
Circuit
MEMORY
Reconfigurable Interconnect
CPU
RF
BasebandProcessing
CryptoEngine
Domain-Specific
Hardware
Software
SignalProcessing
Signal ProcNetworking Video
StandardAlgorithm
ArchitectureMicroArchitectre
Circuit
MEMORY
CPU
RF
BasebandProcessing
VideoEngine
Domain-Specific
Hardware
Software
SignalProcessing
Signal Proc
Figure 8-1. Example RINGS Architecture.
The proposed RINGS architecture [16] is an architecture platform that
gives the designer the option to explore the energy flexibility trade-offs. An
example is shown in Fig. 8-1. A RINGS architecture contains a
heterogeneous set of building blocks: programmable cores, both DSP’s and
micro-controllers, programmable and/or reconfigurable hardware accelerator
units, specialized IP building blocks, front-end blocks, and so on. When
designing a solution based on RINGS, it is important that the domain expert
has freedom to select the appropriate level of flexibility, ranging from fully
programmable approaches, such as embedded micro controllers or FPGA
blocks to highly optimized IP blocks. For different domains, the flexibility
will be supported in different ways as domains have different characteristics.
This domain specific flexibility can be expressed as a do-main specific
abstraction pyramid as shown for Networking, Video, and Signal Processing
on Fig. 8-1. In case of Video, the engine will consist of elements expressed
in the Video pyramid, for example dedicated co-processors.
The SOC is connected together at the top level by a supervising software
program, which typically runs on an embedded micro-controller. At the
bottom level, the reconfigurable interconnect glues it together. The
programming paradigm used in RINGS is a reconfigurable network-on-chip.
Also in this network, flexibility can be traded for energy efficiency at
different levels of abstraction. Designers can instantiate an arbitrary network
of 1D and 2 D router modules leading to an architecture illustrated in Fig. 8-2.
144
Proc A Proc A
1D
router
2D
router
2D
router
2D
router
2D
router
Proc B
Proc X Proc Y
Figure 8-2. Example of Network-on-chip.
This network illustrates the three binding time concepts. At the level of
configuration, the static network architecture with routers is instantiated.
Reconfiguration is done by means of reprogramming the routing tables and
programming by giving each packet a target address. A traditional
reconfiguration is obtained by reprogramming the routing tables in each
node. An alternative approach is to use an easy to reconfigure physical
channel. One example of this is a CDMA based reconfigurable interconnect
[6][16]. Fig. 8-3 shows a conceptual picture of a source-synchronous CDMA
implementation. Each sender and receiver gets a unique spreading code. By
changing the Walsh code, a different configuration is obtained. Traditional
busses, which are a TDMA channel, require hardware switches for
reconfiguration. CDMA interconnect has the advantage that reconfiguration
can occur “on-the-fly.”
MOD1 MOD3
X
. . .
. . .
X X
Figure 8-3. Reconfigurable Interconnect (a) TDMA (b) SS-CDMA Bus Interface [1].
145
8.3 ULTRA LOW POWER COMPONENTS
The focus of this section is on the architecture design options to design
ultra low power processor components, in many cases without losing
performance.
DSP processors have real-time constraints or need to maximize their
throughput for a given task while at the same time minimize the power or
energy consumption. Therefore, the design of DSP processors is very
challenging, as it has to take into account contradictory goals: an increased
throughput request at a reduced energy budget. On top there are new issues
due to very deep submicron technologies such as interconnect delays and
leakage. For instance, hearing aids used analog filters 15 years ago and were
designed as digital ASIC-like circuits 5 years ago. Today they are designed
with powerful DSP processors below 1 Volt and 1 mW of power
consumption [8]. Hearing aids companies require DSP processors just
because they require flexibility, i.e. to program the applications in-house.
The design of ultra-low power DSP cores has to be performed at all
design levels, i.e. system, architecture, circuit and technology levels. We will
focus in this section to DSP architectures, but VHDL implementations as
well as cell libraries are important too. Latch-based implementations
including gated clocks described in VHDL or Verilog, low-power standard
cell libraries and leakage reduction circuit techniques are necessary to reduce
power consumption at these low levels.
Various DSP architectures can be and have been proposed to reduce
significantly the power consumption while keeping the largest throughput.
Beyond the single MAC DSP core of 5-10 years ago, it is well known that
parallel architectures with several MAC working in parallel allow the
designers to reduce the supply voltage and the power consumption at the
same throughput. It is why many VLIW or multitask DSP architectures have
been proposed and used even for hearing aids. The key parameter to
benchmark these architectures is the number of simple operations executed
per clock cycle, up to 50 or more. However, there are some drawbacks. The
very large instruction words up to 256 bits increase significantly the energy
per memory access. Some instructions in the set are still missing for new
better algorithms. Finally the growing core complexity and transistor count
becomes a problem because leakage is roughly proportional to the transistor
count.
To be significantly more energy efficient, there are basically two ways,
however impacting either flexibility or the ease of programming:
1. To design specific very small DSP engines for each task, in such a way
that each DSP task is executed in the most energy efficient way on the
smallest piece of hardware [9]. For N DSP tasks within a given
146
application, the resulting architecture will be N co-processors or
hardware accelerators around a controller or a simple DSP core as
illustrated on Fig. 8-1.
Memory Memory
x +
Memory Memory
- x
Figure 8-4. Hardware Reconfiguration Example [3].
2. To design reconfigurable architectures such as the DART cluster [3], in
which configuration bits allow the user to modify the hardware in such a
way that it can much better fit to the executed algorithms. Fig. 8-4 shows
an example.
Option 1 is definitively the best one regarding power consumption. Each
DSP task uses the minimal number of transistors and transitions to perform
its work. The control code unavoidable in every application is also
efficiently executed on the controller or on the simple DSP, and some
unexpected DSP tasks can be executed on the simple DSP if no accelerator is
available. However, the main issue is the software mapping of a given
application onto so many heterogeneous processors and co-processors (see
Section 4). Transistor count could be high and some co-processors fully
useless for some applications. Regarding leakage, unused engines have to be
cut off from the supply voltages, resulting in complex procedures to
start/stop them.
Reconfigurable DSP architectures are much more power efficient than
FPGAs. The key point is to reconfigure only a limited number of units
within the DSP core, such as some execution units and addressing units [11].
The latter are interesting, as the operands fetch from memory is generally a
severe bottleneck in parallel machines for which 8-16 operands are required
each clock cycle. So, sophisticated addressing modes can be dynamically
reconfigured depending on the DSP task to be executed. Fig. 8-5 shows an
example in which several addressing modes can reconfigured depending on
the user’s algorithms. This AGU (Address Generation Unit) contains 4 index
registers (a0 to a3), 4 offset registers (o0 to o3) and 4 modulo registers (m0
147
to m3). All these registers could be used to generate a given addressing
mode and to compute AGU registers updates. The VLIW AGU operation
register (AGUOP) is controlled by an AGU reconfiguration register (i0 to i3)
that could be reconfigured at any time and allows the programmer to
generate new addressing modes. Fig. 8-5 shows two examples of AGU
computations. In the first example, register i0 contains configuration data
such as the multiplexers and the PREAD adder are configured to generate
address a0 + (02>>1), while at the same time registers a1, a3 and o3 are
updated with new values computed through POSAD1, POSAD2 and
PREADR ALUs. The POSAD1 ALU is used to generate WP1 = (a1+o3)
modulo m2, while the POSAD2 ALU is used to generate WP2= m3 +
02<<2, and the result of PREADR is used to update register a0. The second
example (i2) generates WP2 that uses both POSAD1 and POSAD2 ALUs
connected in series. The operation (ao-02)%m0 is performed in the POSAD1
ALU, while adding 03 is performed in the POSAD2 ALU. This flexibility
allows the programmer to generate very complex addressing modes that
cannot be available in conventional DSP cores with addressing modes only
defined in their instruction sets.
DM ADDR
POSAD1 PREAD
POSAD2
I
RP1 RP2 RP3
P2A P2B
RP4 RP5
A O Ma0
a1
a2
a3
o0
o1
o2
o3
m0
m1
m2
m3
WP1 WP2
RP
1R
P2
RP
3R
P4
RP
5R
P6
RP
7P
2A
P2
BW
P1
WP
2P
RE
AD
PO
SA
D1
PO
SA
D2
AG
UO
P
i0
i1
i2
i3
VLIW AGU Reconfigurable
instruction registers
SELAGUOP
in = AGUOP
(n=0..3)
Examples of in operations:
i0: DM ADDR = a0+(o1>>1),
WP1: a1 = (a1+o3)%m2,
WP2: o3 = m3 + o2<<2
WP3: a0 = a0+(o1>>1),
i2: DM ADDR = a2+o1,
WP1: none
WP2: a0 = (a0-o2)%m0+o3
WP3: a2 = a2+o1
WP3
RP6RES1 RP7 RES2
DM ADDR
POSAD1 PREAD
POSAD2
I
RP1 RP2 RP3
P2A P2B
RP4 RP5
A O Ma0
a1
a2
a3
a0
a1
a2
a3
o0
o1
o2
o3
o0
o1
o2
o3
m0
m1
m2
m3
m0
m1
m2
m3
WP1 WP2
RP
1R
P2
RP
3R
P4
RP
5R
P6
RP
7P
2A
P2
BW
P1
WP
2P
RE
AD
PO
SA
D1
PO
SA
D2
AG
UO
P
i0
i1
i2
i3
i0
i1
i2
i3
VLIW AGU Reconfigurable
instruction registers
SELAGUOP
in = AGUOP
(n=0..3)
Examples of in operations:
i0: DM ADDR = a0+(o1>>1),
WP1: a1 = (a1+o3)%m2,
WP2: o3 = m3 + o2<<2
WP3: a0 = a0+(o1>>1),
i2: DM ADDR = a2+o1,
WP1: none
WP2: a0 = (a0-o2)%m0+o3
WP3: a2 = a2+o1
WP3
RP6RES1 RP7 RES2
Figure 8-5. Addressing Modes Reconfiguration Example (MACGIC DSP).
However, the power consumption is necessarily increased due to the
relatively large number of reconfiguration bits that have to be loaded in the
148
configuration registers. Similarly, the reconfigurable units are necessarily
more complex that non-reconfigurable units in terms of transistor count and
therefore consume more. Software issues are also difficult, as users can
define new instructions or new addressing modes that are difficult to support
by the development tools.
8.4 DESIGN & ARCHITECTURE EXPLORATION
The way a system behaves depends on the architecture, the way the
applications are written, and how these applications are mapped onto the
architecture as compactly expressed by the Y-chart [5]. Examples of
architectures for low-power have already been given in other sections. On
such architecture, mapping is typically done in case of reconfigurable fabrics
by the behavioral synthesis tool and the place and route tools. In case of
DSPs and CPUs, the mapping is typically performed by C-compilers
dedicated to a particular type of DSP or CPU. An important question
remains: how to specify the applications that they can take advantage of the
architecture in an effective manner.
A low-power architecture will typically employ different levels of
parallelism like bit-level parallelism, instruction parallelism or task-level
parallelism to take advantage of voltage scaling as already explained in the
previous section. To successfully map a DSP application at a high level, the
applications need to express task-level parallelism. This parallelism is
typically not present, as the applications are written in sequential languages
like C or Matlab. Therefore, mapping them is often a manual process that is
very tedious and time consuming, leading to a sub optimal system.
A designer would like to have tool support that converts automatically
the sequential specification into a parallel format. Moreover, the tool should
allow him to ‘play’ with the amount of parallelism extracted from the
specification. In general, such tools are lacking in embedded system design.
Some companies, like Pico and Art (ARM/Adelante) try to provide limited
commercial solutions but this field is still very much subject to research. The
Compaan tool suite [13] aims at providing designers the option to play with
parallelism for applications that are so-called “Nested Loop Programs”, a
very natural fit for DSP applications. A DSP application is specified in a
subset of Matlab and is automatically converted by Compaan into a network
of parallel processes. These processes can be specified in “C’ and mapped,
using a conventional C compiler, onto a DSP or CPU. On the other hand,
they can also be specified in VHDL and mapped using the appropriate tools
onto some reconfigurable fabric or realized as a dedicated IP core [19].
Hence, “programming” the RINGS architecture is reduced to putting some
149
processes onto the CPUs and DSPs while others are mapped onto FPGAs or
use dedicated IP cores.
There are many ways we can find parallelism in the application and in
the way we partition the processes of the CPUs, DSPs and reconfigurable
resources. Being able to explore these options early on in the design phase is
crucial to get efficient embedded low-power systems. To allow designers to
do this exploration, Compaan is equipped with a suite of techniques [14] like
Unfolding, Skewing and Merging, to allow designers to play with the level
of parallelism exposed in the derived network of processes. Skewing and
Unfolding increase the amount of parallelism, while Merging reduces
parallelism. By performing these techniques, many different networks can be
created that can be mapped in different ways onto the architecture. When
applied in a systematic way, the design space can be explored and the best
performing network of processes can be picked.
The difference in utilization of the architecture for a particular network
can be huge. By rewriting a DSP application (like Beam-forming) using the
presented techniques, we are able to achieve performances on a QR
algorithm (7 Antenna’s, 21 updates) ranging from 12MFlops to 472MFlops.
We realized QR using commercial floating point IP cores from QinetiQ,
which include pipelined 55 (Rotate) and 42 (Vectorize) stages. We achieved
this performance increase without doing anything to the architecture or
mapping tools, but only by playing with the way the QR application is
written, effectively improving the way the pipelines of the IP cores are
utilized. Using a system like Compaan, an experienced designer should be
able to obtain very different performing networks in days, having the
opportunity to explore different systems and picking the one that uses the
least amount of power.
8.5 DOMAIN-SPECIFIC CO DESIGN
ENVIRONMENTS
As discussed in the previous section, parallelism and distributed
processing are key to energy efficient architectures. Because the ensemble of
architecture elements (processors, busses, memories) cooperate towards a
common application, the designer faces a considerable co-simulation and co-
design problem. A key requirement is to have a good design model. Such a
model allows building of simulation tools, compilers and code generators.
We will look at a highly successful design model for programmable systems:
the instruction-set architecture (ISA). Next we will consider the approach
taken by the RINGS architecture.
In a classic Von-Neumann architecture, the instruction-set-architecture
(ISA) model maintains a single, consistent and abstracted view to the
150
operation of the system. Such a view ties four independent architecture
concepts together: control, interconnect, storage, and data operations [15].
This way the ISA becomes a template for the underlying target architecture,
for which compiler algorithms (scheduling etc) can be developed. Often
however, the ISA is unable to offer the right target template – in terms of
parallelism, storage capabilities or other.
In the RINGS architecture, we do not use an ISA as an intermediate
design model, but approach each of the four components that make up an
ISA independently. We enumerate them below and look at the requirements
they impose on co-simulation and co-design.
• Data Operations: Energy efficient operation requires us to specialize each
operator as much as possible. A RINGS system contains multiple
processing cores. These can include hardwired or programmable (DSP or
RISC) processors. We thus need to be able to combine instruction-set
simulation with hardware simulation.
• Storage: Energy efficient operation requires us to distribute storage. In
addition to the high-level design transformations discussed in the
previous section, we target to minimize storage bandwidth and use
multiple distributed memories. Each processor in RINGS will work
inside of a private memory space. Many operations in multimedia can be
implemented with dedicated storage architectures that take only a
fraction of the energy cost of a full-blown ISA. Examples are matrix
transposition or scan-conversion. Such dedicated storage can be captured
as a hardwired processor.
• Interconnect: The energy efficient interconnect architecture discussed in
section 2 requires explicit expression of interconnect operations – in
contrast to an ISA where this is implicitly encoded in the instruction
format. A network-on-chip can be modeled as a dedicated hardware
architecture [1]. On top of the network-on-chip a suitable network
protocol must be implemented, for example message-passing with the
MPI standard [7]. However, also this protocol is subject to specialization
and/or hard-coding. For example, a hardwired DCT coding unit attached
to a DSP core through RINGS will have a fixed communication pattern.
This pattern can be hard-coded in a collapsed and optimized protocol
stack.
• Control: Energy efficient operation requires us to split the data-flow and
control-flow in a RINGS architecture and handle them independently.
Fig. 8-6 clarifies this point. It shows the effect of moving an AES
encryption operation gradually from high-level software (Java)
implementation to dedicated hardware implementation, while at the same
time maintaining the interface to the high level Java model. It can be seen
that the interface overhead goes from 0.8% for a C-accelerated AES to
151
8000% for a hardware-accelerated AES! This overhead obviously is
caused by all the interfaces moving data from Java to C to hardware and
back. With the MPI message passing scheme, we have the freedom to
route control flow and a data flow independently as messages. This way,
we can eliminate or minimize this interface overhead.
Java
cycles
C
cycles
Rijndael
301,034
Interface
367Interface
892Rijndael
44,063
Rijndael
11
Co-processor
cycles
301, 034 44,430 903Total Cycles
acceleration
Figure 8-6. Overhead of Tightly Coupled Data/Control Flow.
When we put the elements together, we conclude that the RINGS co-
design environment should accommodate multiple instruction-set simulators
with user-specified hardware models. All of these must be embedded in a
model of an on-chip network. The timing accuracy of the simulation should
be precise enough to simulate interactions such as network-on-chip
communication conflicts. On the other hand, the simulation must also be fast
enough to support reasonable design exploration capabilities.
We have built the ARMZILLA environment to evaluate one class of
RINGS architectures, namely those that can be built with one or more ARM
cores, a network-on-chip, and dedicated hardware processors. Fig. 8-7
illustrates the ARMZILLA setup. There are three components: a hardware
simulation kernel (GEZEL), one or more instruction-set simulators (ISS),
and a configuration unit. The GEZEL kernel [4] captures hardware models
with the FSMD (Finite-State-Machine with Datapath) model-of-
computation. It uses a specialized language and a scripted approach to
promote interactive design exploration. The cycle-true models of GEZEL
can also be automatically converted to synthesizable VHDL. For the ARM
ISS we use the cycle-true SimIT-ARM environment [10]. The ARM ISS
uses memory-mapped channels to connect to the GEZEL hardware models.
Finally, the configuration unit specifies a symbolic name for each ARM ISS,
and associates each ISS with an executable. This way the memory-mapped
communication channels can be set up, and the hardware GEZEL models
can address each ARM memory space uniquely.
152
ARMZILLA
ARM ISS GEZEL
Kernel
Configuration
Unit
CC
C
EXE
EXE
EXE
ARM ISSARM ISS
Memory-mapped
Channels
Hardware
Processors
Network
On Chip
FDL
Config
Cross
Compiler
VHDL
FDL
Figure 8-7. The ARMZILLA Design Environment for ARM-based RING Processors.
An example of what can be done with the ARMZILLA environment is
shown in Table 8-1. This table shows cycle counts that were obtained after
partitioning a JPEG encoding algorithm. The reference implementation runs
on a single-ARM ISS model. In the second implementation, we separate the
chrominance and luminance channels over two ARM processors. This seems
a logical partition that splits the data operations roughly in two parts. But, it
also creates a communication bottleneck in the on-chip network and the
resulting implementation becomes slower then the O3-level optimized
single-processor implementation. The third implementation shows a better
partitioning. In this case, the data streams are routed out of the ARM and
into dedicated hardware processors for JPEG encoder subtasks. These
processors can communicate directly amongst themselves.
All these simulations are cycle-accurate yet they can run efficiently. For
the H.264 decoding on a dual ARM with network-on-chip for example,
ARMZILLA offers a simulation speed of 176K cycles per second. The
simulation speed varies with the complexity of the hardware model used. A
single, stand-alone SimIT-ARM simulator runs at 1 MHz cycle-true on a
3GHz Pentium.
153
Table 8-1. Multiprocessor JPEG Encoding Performance
Partition Cycle count 64x64 block
One single ARM 1.223 M
Dual ARM using split chrominance/
luminance channels
1.336 M
Single ARM with color conversion,
transform coding, huffman coding as stand-
alone hardware processors
313K
8.6 CONCLUSIONS
In this chapter, we presented architecture design and design exploration
for low power systems-on-chip. Low power is obtained by tuning all
components of the architecture (datapaths, control, memory and
interconnect) to the application. This can occur at different levels of
abstraction. The design of this type of SOC requires support by design
models and methods. The design environments Compaan and Gezel
/Armzilla are illustration of supporting tools for this design space
exploration.
References
[1] D. Ching, P. Schaumont, I. Verbauwhede, “Integrated Modeling and Generation of a
Reconfigurable Network-On-Chip,” Proc. 11th
Reconfigurable Architectures
Workshop, RAW 2004, Santa Fe, NM, April 2004.
[2] W. Dally, B. Towles, “Route Packets, not wires: on-chip interconnection networks,”
Proc. DAC 2001.
[3] R. David et al., “Low-Power Reconfigurable Processors”, Chapter 20 in “Low Power E
Electronics Design,” edited by C. Piguet, CRC Press, 2004.
[4] GEZEL kernel, http://www.ee.ucla.edu/~schaum/gezel
[5] B. Kienhuis, et al.,``A Methodology to Design Programmable Embedded Systems'',
LNCS, Vol 2268, Nov. 2001.
[6] J. Kim, et al., “A 2-Gb/s/pin Source Synchronous CDMA Bus Interface with
simultaneous Multi-Chip Access and Reconfigurable I/O capability,” CICC, Sept 2003.
[7] MPICH – A portable implementation of MPI, http://www.unix.mcs.anl.gov/mpi/mpich/
[8] P. Mosch et al., “A 720 mW 50 MOPS 1V. DSP for a Hearing Aid Chip Set,” Proc.
ISSCC, pp. 238-239, Feb. 2000.
[9] Õzgün Paker et al., “A heterogeneous multi-core platform for low power signal
processing in systems-on-chip,” ESSCIRC 2002.
[10] W. Qin, S. Malik, “Flexible and Formal Modeling of Microprocessors with Application
to Retargetable Simulation,” Proceedings of DATE 2003, Mar, 2003, pp.556-561.
[11] F. Rampogna et al., “Magic, a Low-Power, re-configurable DSP”, Chapter 21 in “Low
Power Electronics Design”, ed. C. Piguet, CRC Press, 2004.
[12] P. Schaumont, I. Verbauwhede, M. Sarrafzadeh, K. Keutzer, “A quick safari through
the reconfiguration jungle,” Proceedings DAC 2001, pg. 172-177, June 2001.
154
[13] T. Stefanov, C. Zissulescu, A. Turjan, B. Kienhuis, E. Deprettere,``System Design
using Kahn Process Networks: The Compaan/Laura Approach'', DATE2004, Feb 2004,
Paris, France.
[14] T. Stefanov, B. Kienhuis, E. Deprettere, “Algorithmic Transformation Techniques for
Efficient Exploration of Alternative Application Instances'', Proc. CODES'2002,
Colorado, May 2002.
[15] I. Verbauwhede, J. M. Rabaey. “Synthesis of Real-Time Systems: Solutions and
challenges” Journal of VLSI Signal Processing, Vol. 9, No. 1/2, Jan. 1995, pp. 67-88.
[16] I. Verbauwhede, M.C. F. Chang, “Reconfigurable Interconnect for next generation
systems”, Proc. SLIP, pp. 71-74, April 2002.
[17] Xilinx: Virtex-II-Pro Platform FPGAs: Introduction and Overview and Functional
Description, Aug. 2003, Oct. 2003, www.xilinx.com/bvdocs/publications/ds083-1.pdf,
ds083-2.pdf.
[18] H. Zhang, et al., “A 1V Heterogeneous Reconfigurable Processor IC for Baseband
Wireless Applications,” IEEE Journal on Solid State Circuits, November 2000.
[19] C. Zissulescu, et al., ``Laura: Leiden Architecture Research and Exploration Tool'',
Proc. FPL 2003.
155
Chapter 9
SOURCE-LEVEL MODELS FOR SOFTWAREPOWER OPTIMIZATION
Carlo Brandolese, William Fornaciari and Fabio SalicePolitecnico di Milano
Abstract This chapter presents a methodology and a set of models supporting energy-drivensource-to-source transformations. The most promising code transformation tech-niques have been identified and studied leading to accurate analytical and/orstatistical models. Experimental results obtained for some common embedded-system processors over a set of typical benchmarks are discussed, showing thevalue of the proposed approach as a support tool for embedded software design.
Keywords: Software optimization, Power optimization, Source-level modeling
9.1 INTRODUCTION
In a growing number of complex heterogeneous embedded systems the rele-vance of the software component is rapidly increasing. Issues such as develop-ment time, flexibility and reusability are, in fact, better addressed by softwarebased solutions. Another trend that is significantly pushing designers to moveas much functionality as possible toward software is the increased interest inplatform-based designs. In such systems much of the architecture is fixed andcan only be configured to match the design constraints. The greatest part ofthe application-specific functionality is thus naturally shifted from hardwarededicated components to software programs. In such a scenario it is clear thatthe importance of software is steadily increasing and poses new problems todesigners. Though performance, in the sense of computational efficiency, isstill the foremost requirement for many embedded systems, power consump-tion is gaining more and more attention. Optimization of the code is thus one ofthe key points and is currently addressed almost only by means of compilationtechniques. It is still not uncommon for designers to manually code criticalsections of the application directly in assembly. The recent technical litera-
156
ture proposes a different approach, based on source-to-source transformationsaimed at improving code quality either directly or by enabling better compileroptimizations. Source code transformations are extremely complex to auto-mate since they require a thorough semantic analysis of the code fragmentsto be optimized. This chapter proposes a sound and flexible methodology forthe analysis of the effect of source-to-source transformations mostly aimed atallowing rapid and accurate design space exploration. The proposed approachis based on a wide set of models studied to decouple the processor-independentanalysis from all technology specific aspects.
9.2 TRANSFORMATIONS OVERVIEW
Source-to-source transformation presented in literature, can be grouped into four main areas according to the code structures they operate on: loops, datastructures, procedures, control structures and operators. It is worth noting thatnot all the transformations are interesting when operating at source-level sincesome of them can as well be performed at RT or assembly-level and are thusperformed by modern compilers. The most promising transformations, eitherfound in literature [1, 2] or studied in the present work, are summarized in thefollowing. Particular attention must be devoted to loop transformations [3–6]since most of the execution time of a program is spent in loops.
Loop unrolling replicates the body of a loop a given number of times U (theunrolling factor), and modifies the iteration step from 1 to U . The trans-formation impacts on energy in two ways: on one hand, it reduces loopoverhead by performing less compare and branch instructions; on theother hand, it allows the compiler for better optimization and registerusage in the larger loop body.
Loop distribution breaks a single loop into multiple loops with the same it-eration range but each enclosing only a subset of the statements in theoriginal loop. Distribution is used to create sub-loops with fewer depen-dencies, improve instruction cache and instruction TLB locality due toshorter loop bodies, reduce memory requirements by iterating over fewerarrays and improve register usage by decreasing register pressure.
Loop fusion performs the opposite action of distribution, i.e. merging, by re-ducing loop overhead, increasing instruction parallelism, improving reg-ister, data cache, TLB or page locality. It also improves the load balanceof parallel loops.
Loop interchange exchanges the position of two loops in a loop nest, gener-ally moving one of the outer loops to the innermost position. It is oneof the most valuable transformations and can improve performance in
157
many ways: it enables and improves vectorization, increases data accesslocality and increases the number of loop-invariant expressions in theinner loop.
Loop tiling improves memory locality, primarily the at cache level, by access-ing matrices in N×M sized tiles rather than completely. It also improvesprocessor, register, TLB, and page locality.
Software pipelining breaks the operations of a single loop iteration into Sstages, and arranges the code in such a way that stage 1 is executed onthe instructions originally belonging to iteration i, stage 2 on those ofiteration i − 1, etc. Startup code must be generated before the loop toinitialize the pipeline for the first S − 1 iterations and cleanup code mustbe generated after the loop to drain the pipeline for the last S−1 iterations.
Loop unswitch is applied when a loop contains a branch with a loop-invarianttest condition. The loop is then replicated inside each branch of the con-ditional, saving the overhead of conditional branching inside the loop,reducing the code size of the loop body, and possibly enabling the paral-lelization of one or both branches.
The second class collects a number of data-structure and memory accesstransformations [7, 6].
Local to global array promotion allows compilers to use simpler addressingmodes since global arrays address does not depend on the stack pointer.
Scratch-pad array introduction has the goal of storing the most frequentlyaccessed array elements in a smaller array (the scratch-pad) to improvespatial locality.
Multiple indirection elimination identifies common chains of indirectionsand stores the address into a temporary variable.
The third group gathers those transformations [7] impacting on proceduresand functions.
Function inlining replaces the most frequently invoked function with the func-tion body. Inline expansion increases the spatial locality and decreasesthe number of function calls. This transformation increases the numberof unique references, which may result in more misses. However, a de-crease in the miss rate may also occur, since, without inlining, the calleecode might replace the caller code in the instruction cache.
Soft inlining is an intermediate solution between function calling and inlining.The transformation replaces calls and returns with jumps. This reducesthe code size w.r.t. inlining and eliminates context switching overheads.
158
Code linking directives can be used to suitably reorder the objects of differentfunctions to match as more as possible the dynamic call graph. Thispotentially leads to a reduction in instruction misses.
Most of the transformation in the last group are usually performed by com-pilers. Nevertheless, some of them can still be conveniently considered whenoperating at source-level [7, 8].
Conditional sub-expression reordering exploits shortcut evaluation of con-ditions usually performed by compilers. The transformation operates byreordering the sub-expressions according to their probability of beingtrue (for OR conditions) or false (for AND conditions). This reduces thenumber of instructions executed.
Special cases pre-evaluation allows avoiding a function call (usually a math-ematic library function) when the argument has a special value for whichthe result is known. This is done by defining suitable macros testing forthe special cases and leads to a reduction of actual calls.
Special cases optimization replaces calls to generic library or user-definedfunctions with optimized versions, suitable for common special cases.As an example, power raising on integers can be coded more efficientlythan it can be for real numbers.
9.3 METHODOLOGY
Transformations applied to source code might lead to very different resultsdepending on a number of factors: the specific structure of the code, the targetarchitecture, the parameters of the transformations etc. Furthermore, it is notunusual that a transformation applied on the source code as it is, leads to pooror no energy reduction, while, when applied to a pre-transformed code itseffectiveness is greatly increased. Thus sequences of transformations shouldbe considered, rather than single transformations. For this reason it is crucialto explore different transformations and sequences of transformations in termsof their energy reduction efficiency. The exploration strategy should allow toeasily modify the parameters of the transformation and of the target technologyand thus leading to a quick estimate of the expected benefits.
9.3.1 Conceptual Flow
Figure 9.1 shows the conceptual scheme of the estimation flow. The sourcecode is processed and its relevant characteristics are extracted by means of alexical and syntactical analysis leading to the set of code parameters. Typicalparameters are code size, loop body size, number of paths, number of loopiterations, etc.
159
Source Code↓
Transformationparameters
→ Code Analysis
↓∆I , ∆Minst, ∆Mdata
↓Technologyparameters
→ Energy Estimation
↓∆E
Figure 9.1. Phases of the methodology flow
The designer then chooses the transformations parameters such as unrollfactor, tiling size etc. and, finally, selects the target technology from a set oflibraries. Such libraries are collections of technology parameters specifyingarchitectural figures such as cache sizes, bus width etc. and electrical figuressuch as power supply voltages, average core currents, bus and memory capaci-tances etc. Based on all this data, the estimation models first provide the threedimensionless figures ∆I , ∆Minst and ∆Mdata expressing the variations ofnumber of instructions executed, of number of instruction cache misses and ofnumber of data cache misses, respectively. These figures, though still ratherabstract, already provide the designer with an indication of the potential bene-fits of a given transformation. To account for the target technology as well, thevariations are fed to a set of models, depending on the technology parameters,leading to an estimate of the energy reduction ∆E deriving from the applicationof the considered transformation.
9.3.2 Technology Models
Experimental results have shown that the energy consumption of an embed-ded system based on a processor executing some programs can be approximatedby considering three major contributions: the processor core and its on-chipcaches, the system bus and the main memory. All these components can bemodeled at different levels of accuracy by means of equations that involve twosets of parameters: those strictly related to the specific technology and thosesummarizing the properties and the behavior of the code being executed. Inparticular, as outlined above in the description of the conceptual flow, the en-ergy estimates can be based on three execution parameters only: the number ofassembly instructions executed and the number of instruction and data cachemisses. Though simple, the adopted models provide satisfactory results, es-pecially when considering energy variations rather than absolute values. Thetechnology parameters considered and used in the models adopted for the CPU,the cache, the bus and the main memory are summarized in Table 9.1.
160
Table 9.1. Technology parameters
Symbol Meaning Symbol Meaning
Tck CPU clock period B Cache block sizeCPI Average CPI1 S Cache sizeP cpu Average CPU power Edec Memory decode energyCtot Total capacitance on the bus Erw Memory read/write energyVsw Bus switching voltage Eref Memory refresh energyAsw Average bus switching activity Vm Memory supply voltageW Bus width Iref Average memory refresh current
The form of the equations, referred to relative energy variations, are reportedin the following using the symbols introduced. The processor energy variationis modeled as:
∆Ecpu = TckP cpuCPI∆I (3.1)
The contribution of system bus to energy variation ∆Ebus is:
∆Ebus = 12CtotV
2sw(∆Nbus,addr + ∆Nbus,data + ∆Ninst) (3.2)
where:
∆Nbus,addr = Asw,addrWaddr(∆Mdata + ∆Minst) (3.3)
∆Nbus,data = Asw,dataWdataBdata∆Mdata (3.4)
∆Nbus,inst = Asw,instWdataBinst∆Minst (3.5)
Finally, the adopted memory model expresses the energy variation ∆Em as:
∆Em = ∆Em,data + ∆Em,inst + ∆Em,ref (3.6)
where:
∆Em,data = (Edec + ErwBdata)∆Mdata (3.7)
∆Em,inst = (Edec + ErwBinst)∆Minst (3.8)
∆Em,ref = TckVmIrefCPI∆I (3.9)
9.4 CASE STUDIES
In this section, two case studies are reported: Loop unrolling and Loopfusion. For each transformation, the source code parameters and the modelequations are reported and discussed.
9.4.1 Loop Unrolling
Loop unrolling is a parametric transformation whose results in terms ofenergy reduction are influenced by the unrolling factor U , i.e. the number of
161
times the loop body is replicated to build the modified loop. The parameterU , thus, completely defines the transformation. The effects of loop unrollingclearly depend also on the characteristics of the source code being transformed.Such properties are captured by the set of source code parameters reported inTable 9.2.
Table 9.2. Source code parameters for loop unrolling
Symbol Meaning
LI Number of loop instructionsLS Size of loop instructions (bytes)
LBI Number of loop-body instructionsLBS Size of loop-body instructions (bytes)N Loop iterations
The number of instructions of the original loop is:
Io = N · LI (4.1)
The transformed loop executes Nt = N/U times and:
LIt = LI + (U − 1)LBI (4.2)
instructions per iteration. Therefore, the total number of instructions executedby the transformed loop is:
It = Nt · LIt =⌊
N
U
⌋· [LI + (U − 1)LBI] (4.3)
The instructions gain obtained with unrolling is thus:
∆I =⌊
N
U
⌋· [LI + (U − 1)LBI] − Io (4.4)
The transformation has also effects on the number of instruction cache missesdue to the increased dimension of the loop body. A more accurate analysisleads to the results—summarized in the following—that show a non-lineardependence of the number of misses on the relative values of the loop sizeLS and the instruction cache size Sinst
2. Three significant cases have beenidentified:
LS ≤ Sinst
In this case there are no capacity misses since the entire loop code can
2The loop size and number of instructions are linearly related assuming a fixed instruction size.
162
be loaded into the cache. Hence, there are only cold misses, during thefirst iteration. The number of instruction cache misses is thus:
Minst =⌈
LS
Binst
⌉(4.5)
Sinst < LS < 2Sinst
In this case capacity misses also take place. The number of cold misses isthe same as in the previous case, but, in addition, for every additional iter-ations, there are 2(LS mod Sinst)/Binst capacity misses. Therefore,the total number of misses is:
Minst =⌈
LS
Binst
⌉+ 2(N − 1)
⌈LS mod Sinst
Binst
⌉(4.6)
LS ≥ 2Sinst
The number of misses in every iteration is equal to the number of coldmisses, i.e.:
Minst = N
⌈LS
Binst
⌉(4.7)
For all these cases, the relevant figure is the variation of the number ofinstruction cache misses ∆Minst = IMt − IMo. Such difference depends onthe variation of number of instructions due to the transformation:
∆LS = LSt − LSo = (U − 1)LBS (4.8)
and must be calculated for all the 32 = 9 cases. It is worth noting that sincethe transformed code will always be larger than the original one, only 6 out ofthe 9 cases are significant. For the sake of conciseness, only the two boundarycases are described in the following.
(LSo ≤ ICS) ∧ (LSt ≤ ICS)In this case both the original and the transformed code completely fit intothe cache and thus only cold misses take place. The variation, recallingEquation (4.5), is:
∆Minst =⌈
LSo
Binst
⌉−
⌈LSt
Binst
⌉≈
⌈(U − 1)LBS
Binst
⌉(4.9)
(LSo ≥ 2ICS) ∧ (LSt ≥ 2ICS)In this other limiting case, both codes are larger than the double of thecache size and thus each instruction fetch causes a miss. Recalling Equa-tion (4.7), the instruction miss variation is:
∆Minst = Nt
⌈LS + (U − 1)LBS
IBS
⌉− No
⌈LS
IBS
⌉(4.10)
163
In a similar manner and referring to Equations (4.5)–(4.7), the variationsfor the other four cases can be calculated. The last effect to be considered isthe variation of data cache misses. Since the transformation does not modifythe data access pattern of the code, the term ∆Mdata can be assumed to be0, at least at a first approximation. A first validation can be performed atthis level comparing the dimensionless estimated figures ∆I and ∆Minst withthose derived from simulation. Figure 9.2 shows the results for the variation ofnumber of instruction executed. It is worth noting that ∆I does not depend onthe cache size but only on the structure of the code and the effectiveness of theoptimizations that the compiler can exploit on the modified loop.
0 10 20 30 40 50 60 70 80 90 100Unroll factor (U)
-700
-600
-500
-400
-300
Inst
ruct
ion
gain
ActualEstimated
Loop Unrolling
Figure 9.2. Loop unrolling: ∆I
As far as the variation of instruction cache misses, different scenarios havebeen considered by varying the cache size from 256 to 4096 bytes. Table 9.3summarizes the results obtained by averaging the estimation error over theinterval U = [2; 100] and Figure 9.3 shows the two boundary cases.
Table 9.3. Loop unrolling: ∆Minst average error and standard deviation
Sinst(bytes) ε% σ%
256 -1.881 8.026512 -2.557 7.1011024 -2.531 6.9102048 -2.750 9.2524096 -1.691 5.065
The two contributions ∆I and ∆Minst (remembering that ∆Mdata = 0)can now be fed to the technology models to derive the overall energy saving.Table 9.4 reports the average error and the corresponding standard deviation interms of energy gain for the five cache-size scenarios just considered.
These results show that the model tends to underestimate the potential gainderiving from loop unrolling. A possible reason is that unrolling a loop leads to a
164
Loop Unrolling
0 10 20 30 40 50 60 70 80 90 1000
100
200
300
400
Inst
ruct
ion
mis
s ga
in
ActualEstimated
0 10 20 30 40 50 60 70 80 90 100Unrolling factor (U)
0
100
200
300
400
Inst
ruct
ion
mis
s ga
in
ActualEstimated
Cache size = 256 Bytes
Cache size = 4 KBytes
Figure 9.3. Loop unrolling: ∆Minst
Table 9.4. Loop unrolling: ∆E average error and standard deviation
Sinst(bytes) ε% σ%
256 -1.754 9.144512 -4.552 7.3221024 -7.663 6.9662048 -6.203 5.7774096 -4.409 3.011
longer loop body, i.e. a larger basic block where the compiler can better performoptimizations. Despite the light biasing of the model, the overall average erroris, in absolute value, approximately 4.9% and this can be considered more thansatisfactory when operating at source code level.
9.4.2 Loop Fusion
This transformation has the purpose of combining into a new single loopthe bodies of different subsequent loops. Some constraint must be satisfied,in particular the loops to be merged need to have the same iteration range andthe statements in their bodies must be independent. The only transformationparameter characterizing loop fusion is the number NF of loops to be merged.The source code parameters that influence the effect of this transformation are
165
all those considered for loop unrolling (see Table 9.2) plus the number and sizeof control instructions, defined as:
LCI = LI − LBI (4.11)
LCS = LS − LBS (4.12)
In the following the subscript k ∈ [1, NF ] is used to indicate a specific loopamong those to be fused. An additional useful parameter is the average numberof control instructions over all the considered loops:
LCI =1
NF
NF∑k=1
LCIk (4.13)
Using the symbols just introduced, the number of instructions in the originaland transformed codes are:
Io = N
NF∑k=1
(LBIk + LCIk) (4.14)
It = N(LCI +NF∑k=1
LBIk) (4.15)
The variation ∆I is thus given by:
∆I = N(LCI +NF∑k=1
LBIk −NF∑k=1
(LBIk + LCIk) =
= N(LCI −NF∑k=1
LCIk)
(4.16)
Assuming that LCI = LCIk ∀k yields:
NF∑k=1
LCIk =NF∑k=1
LCI = NF · LCI (4.17)
and thus Equation (4.16) can be rewritten as:
∆I = N(LCI −NF∑k=1
LCIk) = N(1 − NF )LCI (4.18)
166
To study the effect of loop fusion with respect to instruction misses, thesame cases considered for loop unrolling and expressed by Equations (4.5)–(4.7) turn out to be applicable. Nevertheless, when considering the original codecomposed of NF loops, the number of instruction misses must be estimated foreach single loop according to the three mentioned equations and then summedover all loops. On the other hand, the estimates for the transformed code canbe obtained by simply substituting LS with the overall transformed code sizeLSt, defined as:
LSt = LCS +NF∑k=1
LBSk (4.19)
According to Equations (4.5)–(4.7) and referring to the original code sizesLSo,k and the transformed code size LSt, the number of instruction misses ofthe original loops IMo,k and the transformed one IMt can be derived. Theresulting overall variation is thus:
∆Minst = IMt −NF∑k=1
IMo,k (4.20)
It is worth noting that the number of possible cases derived from the limitingconditions on the cache size is, in general, 3NF+1. Similar considerationsapply to the estimation of data cache misses. Since in most cases the differentloops operate on different arrays, data misses tend to be increased, the best-case condition being that all data fit into the cache in which case the number ofmisses will approximately be invariant. A validation procedure similar to thatused for loop unrolling has been applied for loop fusion also, considering thesimplest and most common case where NF = 2. To analyze the behavior ofthe transformation, loops with different body sizes have been considered andthe results for instruction misses are shown in Figure 9.5, where the x axis isan index related to the loop body size ratio. For the same combinations of loopbody sizes and for an instruction cache size varying from 256 to 4096 bytes,the gain in terms of instruction misses have also been estimated and comparedwith actual results, leading to the data collected in Table 9.5 and the graphs ofFigure 9.5 relative to the two limiting cases.
Again the accuracy obtained is more than satisfactory since the average abso-lute error is approximately 2.1% with very low standard deviation. Combiningdimensionless figures with the energy models of the different component ofthe considered system led to the energy estimates. Such estimates show a verylimited error, as reported in Table 9.6, and are not biased. It is though worthnoting that the reported results refer to loops manipulating very small arraysfor which the hypothesis of being fully contained in the data cache may beassumed to hold. This translates into the models by assuming ∆Minst = 0.
167
0 10 20 30 40 50 60Loop size index
-1200
-1100
-1000
-900
-800
-700
-600
Inst
ruct
ion
gain
ActualEstimated
Loop Fusion
Figure 9.4. Loop fusion: ∆I
Table 9.5. Loop fusion: ∆Minst
Sinst(bytes) ε% σ%
256 +2.423 2.701512 +3.004 2.804
1024 -3.150 4.2532048 +0.153 1.6724096 -0.258 1.419
0 10 20 30 40 50 60-500
0
500
1000
1500
Inst
ruct
ion
mis
s ga
in
ActualEstimated
Loop Fusion
0 10 20 30 40 50 60Loop size index
-5
-4
-3
-2
-1
0
Inst
ruct
ion
mis
s ga
in
ActualEstimated
Cache size = 64 Bytes
Cache size = 4 KBytes
Figure 9.5. Loop fusion: ∆Minst
More complex cases show higher errors but preliminary experimental resultssuggest that a 10–15% error is a reasonable and conservative upper bound.
168
Table 9.6. Loop fusion: ∆E
Sinst(bytes) ε% σ%
256 +1.945 3.882512 +0.177 3.469
1024 -0.194 3.9162048 +1.592 2.4254096 +0.168 0.017
9.5 EXPERIMENTAL RESULTS
The estimates of∆I , ∆Minst and∆Mdata, combined with the energy models(see Section 9.3.2) adopted to account for the technology-dependent parameters,lead to a new set of results showing the accuracy of the complete methodology interms of energy reduction (∆E) estimation. The models for 5 transformationshave been tested on a set of SPEC95 benchmarks in order to quantify theenergy gain estimation error. The actual energy gain has been obtained bysimulating both the original and the transformed code and then compared withthe estimated gain derived from the models. Experiments have been performedon four architectures based on different processors and operating systems usingthird-party timing and/or power profiling tools (see Table 9.7).
Table 9.7. Operating environments for validation
Processor Operating system Simulation engine
Intel strongARM Linux RedHat 9.0 SimpleScalar 3.0 / SimPAnalyzerIBM PowerPC 405 Linux RedHat 9.0 SimpleScalar 3.0Sun microSPARC II EP Solaris 8 SpixToolsMIPS Tech. MIPS-32 Linux RedHat 9.0 SimpleScalar 3.0
Each benchmark has been analyzed varying both the instruction cache size(Sinst) and the input data and all compatible transformations have been appliedin a proper sequence using the predicted optimal values for their parameters(unroll factor, tile size, etc.). Table 9.8 collects the relative error between theestimated gain ∆Eest and the actual value ∆Eact derived from simulation.
The results confirm that the models are reliable since they can correctlypredict both energy reductions and undesirable energy increases. In conclusion,the average estimation error has shown to be around than 3%.
169
Table 9.8. Energy gain estimation relative errors
FIB FIR WAVE-1 WAVE-2 IIRSinst ε% σ% ε% σ% ε% σ% ε% σ% ε% σ%
256 +4.16 3.90 n/a n/a -1.97 2.81 +4.29 3.63 -1.63 1.20512 +7.18 4.02 -3.67 4.48 -1.83 2.67 +4.63 3.52 -1.82 1.151024 +3.31 1.49 -2.11 4.95 -2.87 3.51 +4.81 0.79 -3.93 1.512048 -1.42 2.15 +1.03 7.68 -2.37 3.71 +4.20 0.57 -0.53 1.594096 -2.08 1.91 +11.25 7.57 -1.86 3.71 +3.74 0.20 +0.03 16.00
Average 3.63 2.69 4.51 6.17 2.18 3.28 4.33 1.74 1.58 4.29
9.6 CONCLUSIONS
The presented work has addressed the problem of the fast estimation of theeffects induced by a set of specific source code transformations by using astructured methodological approach based on technology-independent models.In particular, the presented analysis flow, by providing an appropriated set ofboth technological and transformation parameters, allows the designer to an apriori evaluation of the impact of a specific transformation and/or the effectof a sequence of interdependent transformations. Two specific transformationshave been accurately described: loop unrolling and loop fusion. As far as loopunrolling is concerned, it has been shown that the proposed model can be con-sidered more than satisfactory since the average error between the estimatedgain and the simulated gain is, approximately, 4,9% with a low standard devi-ation. Concerning loop fusion, the model has produced estimates—for a wideset of technological options—displaying an average absolute error of 2,1% withan high level of reliability. Both the methodology and the models has been val-idated on a set of benchmarks showing an overall average error of the estimatedenergy gain around 3%. This result is more than satisfactory and confirmsthat the models of the different transformation are sufficiently accurate and themethodology, though subject to further improvements, is promising.
170
References
[1] L. Benini and G. De Micheli. System-level power optimization: Tech-niques and tools. Transactions on Design Automation of Electronic Sys-tems, 5:115–192, 2000.
[2] F. Catthoor, H. De Man, and C. Hulkarni. Code transformations for lowpower caching in embedded multimedia processors. Proc. of IPPS/SPDP,pages 292–297, 1998.
[3] D.F. Bacon, S.L. Graham, and O.J. Sharp. Compiler transformations forhigh performance computing. Technical Report N. UCB/CSD-93-781, Uni-versity of California at Berkeley, 1993.
[4] M.S. Lam. Software pipelining: An effective scheduling technique for vliwmachines. SIGPLAN Conference on Programming Language Design andImplementation, pages 318–328, 1988.
[5] M.S. Lam, E.E. Rothberg, and M.E. Wolfe. The cache performance andoptimization of blocked algorithms. Conference on Architectural Supportfor Programming Languages an Operating Systems, pages 63–74, 1991.
[6] M.J. Wolfe. More iteration space tiling. ACM Proceedings of Supercom-puting, pages 655–664, 1989.
[7] C. Brandolese, W. Fornaciari, F. Salice, and D. Sciuto. The impact of sourcecode transformations on software power and energy consumption. Journalof Circuits, Systems and Computers, 11(5):477–502, 2002.
[8] C. Brandolese, W. Fornaciari, F. Salice, and D. Sciuto. Library functionstiming characterization for source-level analysis. Conference on DesignAutomation and Testing in Europe, pages 1132–1133, March 2003.
171
Chapter 10
TRANSMITTANCE SCALING FOR REDUCING
POWER DISSIPATION OF A BACKLIT TFT-LCD
Wei-Chung Cheng and Massoud Pedram
University of Southern California
Abstract This chapter presents transmittance scaling; a technique aimed at conserving
power in a transmissive TFT-LCD with a cold cathode fluorescent lamp
(CCFL) backlight by reducing the backlight illumination while compensating
for the luminance loss. This goal is accomplished by adjusting the
transmittance function of the TFT-LCD panel while meeting an upper bound
on a contrast distortion metric. Experimental results show that an average of
3.7X power saving can be achieved for still images with a mere 10% contrast
distortion.
Keywords: CCFL; transmissive LCD; TFT-LCD; backlight luminance dimming;
transmittance scaling; concurrent brightness and contrast scaling; power
efficiency; low power design.
10.1 INTRODUCTION
TFT-LCD is the most popular flat-panel display used in today's consumer
electronics and computer systems. TFT stands for "Thin Film Transistor"
and describes the control elements that actively control the individual pixels.
For this reason, one speaks of so-called "active matrix TFT's". LCD means
"Liquid Crystal Display" and stands for monitors that are based on liquid
crystals. To obtain a high image quality and low power dissipation in a TFT-
LCD, low off-current and high on-current are necessary.
Previous studies on battery-powered electronics point out that the display
subsystem dominates the energy consumption of the whole system. In the
SmartBadge system, for instance, the display consumes 29%, 29%, and 50%
of the total power in the active, idle, and standby modes, respectively [1].
Direct-view LCDs can largely be categorized into reflective and transmissive
displays which utilize ambient light and light from an artificial light source
172
(e.g., fluorescent backlight tube) respectively. In a transmissive TFT-LCD
monitor, the backlight contributes more than 50% of the display subsystem
when using a cold cathode fluorescent lamp (CCFL) [2]. To reduce the
backlight power consumption, Choi et al. proposed a technique called
backlight luminance dimming. This technique dims the backlight and
compensates for the luminance loss by adjusting the grayscale of the image
to increase its brightness or contrast. The grayscale of the image is adjusted
by multiplying the pixel values by a scaling factor. In this chapter, we
describe the transmittance scaling technique, which compensates for the
luminance loss by adjusting the transmittance function of the TFT-LCD
panel. More precisely, transmittance scaling means “scaling the
transmittance function of the TFT-LCD panel.” This is a general technique
that can achieve concurrent brightness and contrast scaling of the whole
image to compensate for the effects of the backlight dimming.
In the following sections, we explain how CCFL works and show how to
model the non-linearity between its backlight illumination and power
consumption. Next, we propose a contrast distortion metric to quantify the
image quality loss after transmittance scaling. Finally, we formulate and
optimally solve the optimal transmittance scaling problem subject to a
constraint on the contrast distortion.
10.2 PRELIMINARIES
A transmissive LCD uses a dedicated backlight. A reflective LCD uses the
ambient light or/and a dedicated frontlight. A transflective LCD uses both
the ambient light and backlight. The frontlight and backlight use the same
light source. The difference between the two lighting schemes is in the light
path from the light source through the LCD panel to the observer. A back-lit
or front-lit LCD offers superior contrast ratio compared to the one that is lit
by the ambient light. A backlight can be direct or indirect type. A direct
backlight is positioned directly beneath the LCD panel. An indirect (or side-
lit) backlight is positioned at the side of the LCD panel and requires a
carefully designed light-guide and a diffuser to illuminate the LCD panel
evenly.
Most TFT-LCD monitors use CCFL for backlighting due to its unrivaled
luminance density – emitting the most light within the minimum form factor.
The CCFL can be designed to generate an arbitrary color, which is critical
for reproducing pure white in the backlighting applications. Technology for
CCFL manufacturing is mature; therefore, its production cost is rather low.
However, compared to power consumption of the TFT-LCD panel, the
power consumption of the CCFL backlight is quite high.
173
10.2.1 Radiometry and Photometry Terminology
Radiometry refers to the science of measuring light in any portion of the
electromagnetic spectrum [3]. In practice, radiometry is usually limited to
the measurement of infrared, visible, and ultraviolet light using optical
instruments.
Light is radiant energy. Electromagnetic radiation transports energy
through space. Radiant energy (denoted as Q) is measured in joules. A
broadband source such as the Sun emits electromagnetic radiation
throughout most of the electromagnetic spectrum, from radio waves to
gamma rays. However, most of its radiant energy is concentrated within the
visible portion of the spectrum. A single-wavelength laser, on the other
hand, is a monochromatic source; all of its radiant energy is emitted at one
specific wavelength. Energy per unit time is power, which we measure in
joules per second, or watts. A laser beam, for example, has so many watts of
radiant power. Light “flows” through space, and so radiant power is more
commonly referred to as the “time rate of flow of radiant energy” or radiant
flux. It is defined as: =dQ/dt where Q is radiant energy and t is time.
Radiant flux is measured in watts. In terms of a photographic light meter
measuring visible light, the instantaneous magnitude of the electric current is
directly proportional to the radiant flux. The total amount of current
measured over a period of time is directly proportional to the radiant energy
absorbed by the light meter during that time.
Radiant flux density is the radiant flux per unit area at a point on a surface.
There are two possible conditions. The flux can be arriving at the surface, in
which case the radiant flux density is referred to as irradiance. The flux can
also be leaving the surface due to emission and/or reflection. The radiant
flux density is then referred to as radiant exitance. Radiant flux density is
measured in watts per square meter. The radiant flux density at a point on a
surface due to a single ray of light arriving (or leaving) at a solid angle to
the surface normal is d /(dA·cos ). The radiance at that point for the same
angle is then d2 /(dA·d ·cos ), or radiant flux density per unit solid angle.
Radiance is measured in watts per square meter per steradian. We can
imagine an infinitesimally small point source of light that emits radiant flux
in every direction. The amount of radiant flux emitted in a given direction
can be represented by a ray of light contained in an elemental cone. This
gives us the definition of radiant intensity: I =d /d . Radiant intensity is
measured in watts per steradian.
Photometry is the science of measuring visible light in units that are
weighted according to the sensitivity of the human eye [3]. It is a
quantitative science based on a statistical model of the human visual
response to light -- that is, our perception of light -- under carefully
174
controlled conditions. The human visual system is a complex and highly
nonlinear detector of electromagnetic radiation with wavelengths ranging
from 380 to 770 nanometers (nm). The sensitivity of the human eye to light
varies with wavelength. A light source with a radiance of one watt/m2-
steradian of green light (540nm wavelength), for example, appears much
brighter than the same source with a radiance of one watt/m2-steradian of red
(650nm wavelength) or blue light (450nm wavelength). In photometry, we
attempt to measure the subjective impression produced by stimulating the
human eye-brain visual system with radiant energy. This task is complicated
immensely by the eye’s nonlinear response to light. It varies not only with
wavelength but also with the amount of radiant flux, whether the light is
constant or flickering, the adaptation of the iris and retina, the spatial
complexity of the scene being perceived, the psychological and
physiological state of the observer, and a host of other variables [4].
According to studies done by the Commission Internationale d’Eclairage
(CIE), the photopic luminous efficiency of the human visual system as a
function of wavelength looks like a near-normal distribution as depicted in
Figure 10-1 (cf. [5].) The CIE photometric curve thus provides a weighting
function that can be used to convert radiometric measurements into
photometric measurements. Today the international standard for a light
source is a point source that has a luminous intensity of one candela (the
Latin word for “candle”). It emits monochromatic radiation with a frequency
of 540*1012
Hertz (or approximately 555nm, corresponding with the
wavelength of maximum photopic luminous efficiency) and has a radiant
intensity (in the direction of measurement) of 1/683 watts per steradian.
Figure 10-1: Photopic luminosity function.
Together with the CIE photometric curve, candela provides the weighting
factor needed to convert between radiometric and photometric
measurements. Consider, for example, a monochromatic point source with a
175
wavelength of 510nm and a radiant intensity of 1/683 watts per steradian.
The photopic luminous efficiency at 510nm is 0.503. The source therefore
has a luminous intensity of 0.503 candela. Luminous flux is the
photometrically-weighted radiant flux (power). Its unit of measurement is
the lumen, defined as 1/683 watts of radiant power at a frequency of
540*1012
Hertz. As with luminous intensity, the luminous flux of light with
other wavelengths can be calculated using the CIE photometric curve.
Luminous energy is photometrically-weighted radiant energy. It is measured
in lumen seconds. Luminous flux density is photometrically-weighted radiant
flux density. Luminous flux density is measured in lumens per square meter.
Illuminance is the photometric equivalent of irradiance, whereas luminous
exitance is the photometric equivalent of radiant existence. Illuminance can
be used to characterize the luminous flux emitted from a surface. Most
photographic light meters measure the illuminance. Luminance is
photometrically-weighted radiance. In terms of visual perception, we
perceive luminance. It is an approximate measure of how “bright” a surface
appears when we view it from a given direction. Luminance is measured in
lumens per square meter per steradian. The maximum brightness of a CRT
or LCD monitor is described by luminance in its specification. Luminous
intensity is photometrically-weighted radiant intensity. It is measured in
lumens per steradian (i.e., candelas). Luminous intensity can be used to
characterize the optical power emitted from a spot light source, such as a
light bulb.
There is much more that we have not covered here, such as reflectance,
transmittance, absorption, scattering, diffraction, and polarization. We have
also ignored the interaction of the human visual system with light, including
scoptic and mesopic luminous efficiency, temporal effects such as flicker,
and most important, color perception. The study of light and how we
perceive it fills volumes of research papers and textbooks.
10.2.2 Cold Cathode Fluorescent Lamp
A transmissive LCD uses a dedicated backlight. A reflective LCD uses the
ambient light or/and a dedicated frontlight. A transflective LCD uses both
the ambient light and backlight. The frontlight and backlight use the same
light source. The difference between the two lighting schemes is in the light
path from the light source through the LCD panel to the observer. A back-lit
or front-lit LCD offers superior contrast ratio compared to the one that is lit
by the ambient light. A backlight can be direct or indirect type. A direct
backlight is positioned directly beneath the LCD panel. An indirect (or side-
lit) backlight is positioned at the side of the LCD panel and requires a
carefully designed light-guide and a diffuser to illuminate the LCD panel
176
evenly.
Most TFT-LCD monitors use CCFL for backlighting due to its unrivaled
luminance density – emitting the most light within the minimum form factor.
The CCFL can be designed to generate an arbitrary color, which is critical
for reproducing pure white in the backlighting applications. Technology for
CCFL manufacturing is mature; therefore, its production cost is rather low.
However, compared to power consumption of the TFT-LCD panel, the
power consumption of the CCFL backlight is quite high.
A CCFL backlight module consists of the fluorescent lamp, the driving
DC-AC inverter, and the light reflector. A CCFL is a sealed glass tube with
electrodes on both ends. The tube is filled with an inert gas (argon) and
mercury. The inner glass surface of the tube is coated with phosphor, which
emits visible light when excited by photons. The wavelength of the visible
light (i.e., color) depends on the type of the gas and phosphor. In the LCD
backlighting application, a proper mix of red, green, and blue phosphors
produces the desired three-band white light. In other applications where the
pure white is not required, the emitted light is color-shifted (e.g., the bluish
or yellowish household hot cathode fluorescent lamps).
The CCFL converts electrical energy into visible light by a process known
as the gas discharge phenomenon. When a high voltage is applied to the
electrodes turning on the lamp, electrical arcs are generated that ionize the
gas, which allows the electrical current to flow. The collision among the
moving ions injects energy to the mercury atoms. The electrons of the
mercury atoms receive energy and jump to a higher energy level followed by
emitting ultraviolet photons when falling back to their original energy level.
The ionized gas conducts the electrical current as a gas conductor. The
impedance of the gas conductor, unlike that of the metal conductor having a
linear behavior, decreases as the current increases. Therefore, the CCFL has
to be driven by an alternative current (AC). Otherwise, the CCFL explodes
due to the heat caused by the runaway current. The CCFL is usually
manufactured as a thin long tube, which can be straight or in any different
shape.
A DC-AC inverter is usually used to drive a CCFL in battery-powered
applications. A DC-AC inverter is basically a switching oscillator circuit that
supplies high-voltage AC current from a low-voltage battery. The nominal
AC frequency of modern CCFL is in the range of 50-100 kHz to avoid
flickering. The nominal operate voltage has to be higher than 500 VRMS to
keep inert gas ionized.
To conserve energy in battery-powered applications, dimming control is a
desirable feature for DC-AC inverters. Different methods of dimming the
CCFL have been used, including linear current, pulse-width-modulation, and
current chopping [6]. In a DC-AC inverter with dimming control, an analog
177
or digital input signal is exposed for adjusting the CCFL illumination in
addition to the on/off input signal. Most well designed DC-AC inverters
have high electrical efficiency (>80%) and linear response of output
electrical power to input power. Most fluorescent lamps, however, have low
optical efficiency (<20%) and non-linear response of output optical power
versus input power [7].
10.2.3 Power Modeling for the CCFL
The CCFL illumination is a complex function of the driving current,
ambient temperature, warm-up time, lamp age, driving waveform, lamp
dimensions, and reflector design [7]. In the transmissive TFT-LCD
application, only the driving current is controllable. Therefore, we model the
CCFL illumination as a function of the driving current only and ignore the
other parameters.
Figure 10-2: Normalized CCFL illuminance (i.e., backlight factor, b)
versus driver’s power consumption for a CCFL light source.
Relationship between the CCFL illumination (i.e., luminous flux incident
on a surface per unit area) and the driver’s power dissipation for the CCFL
in LG Philips transmissive TFT-LCD LP064V1 [8] is shown in Figure 10-2.
The CCFL illumination increases monotonically as the driving power
increases from 0 to 80% of the full driving power. For values of driving
power higher than this threshold, the CCFL illumination starts to saturate.
The saturation phenomenon is due to the fact that the increased temperature
and pressure inside the tube adversely impact the efficiency of emitting
visible light [9].
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.5 1 1.5 2 2.5 3
Power Consumption (W)
CC
FL
illu
min
an
ce
178
Let backlight factor b∈[0,1] denote the normalized CCFL illumination
factor with b=1 representing the maximum backlight illumination and b=0
representing no backlight illumination. Accounting for the saturation
phenomenon in the CCFL light source, we use a two-piece linear function to
characterize the power consumption of CCFL as a function of the backlight
factor:
. , 0( )
. , 1
A b C b Blin lin sb
backlight A b C B bsat sat sP
+ ≤ ≤+ ≤ ≤
= (Watt). (1)
We conduct experiments to derive the regression coefficients, A’s and C’s.
A precision luminance meter such as [10] provides accurate absolute
illuminance readings. These meters are expensive and can be replaced with
the cheaper photographic light meters. Indeed the absolute illuminance
readings are not required to characterize the CCFL power consumption as a
function of the backlight factor. An accurate photographic light meter can
serve the purpose so far as it is capable of sensing minor illuminance
changes as described next.
We use a light meter in a way similar to how we compare the mass of two
objects on a scale (weighing machine). We simultaneously adjust the
backlight factor b and the TFT-LCD grayscale, x∈[0,255], while
maintaining the same CCFL illuminance. Let q(b) denote the required analog
or digital dimming control input of the DC-AC inverter for generating
backlight factor b. We start by setting the maximum CCFL illuminance
(b=1) and the minimum TFT-LCD grayscale (x=0). The grayscale x is
obtained by displaying a pure gray image in which Red=Green=Blue=x for
every pixel. The grayscale x is increased until the light meter senses a
variation and reports a different reading. We then reduce the backlight factor
b (by adjusting the dimming control q) until the meter reports the previous
reading. The change in the TFT-LCD grayscale (which determines the
transmittance of the TFT-LCD panel) is thus known. Now the change in the
backlight factor is assumed to be the equal to the TFT-LCD transmittance.
We record q as the dimming control value for the backlight factor b=(255-
x)/256. At the same time, the power consumption of the backlight Pbacklight is
measured and recorded. We repeat the above procedure for x=0,1...,255.
After interpolation, we obtain q(b) and Pbacklight(b). For the CCFL in LG
Philips transmissive TFT-LCD LP064V1, the following coefficient values
were obtained:
Alin=0.4991, Asat=0.1489, Clin=0.1113, Csat=0.6119,
Bs=0.8666. (2)
179
10.2.4 Transmissive TFT-LCD Monitor
The major components of a transmissive TFT-LCD monitor subsystem
include a video controller, a frame buffer, a video interface, a TFT-LCD
panel, and the backlight (cfr. Figure 10-3.) The frame buffer is a portion of
memory used by software applications to deliver video data to the video
controller. The video data from the application is stored in the frame buffer
by the CPU. The video controller fetches the video data and generates
appropriate analog (e.g., VGA) or digital (e.g., DVI) video signals to the
video interface. The video interface carries the video signals between the
video controller and the TFT-LCD monitor. The TFT-LCD monitor receives
the video data and generates a proper shade – i.e., transmittance – for each
pixel according to the corresponding pixel value. All of the pixels on a
transmissive LCD panel are illuminated from behind by the backlight. To the
observer, a displayed pixel looks bright if its transmittance is high (i.e., it is
in the 'on' state), meaning it passes the backlight. On the other hand, a
displayed pixel looks dark if its transmittance is low (i.e., it is in the 'off'
state), meaning that it blocks the backlight. If the transmittance can be
adjusted to more than two levels between the 'on' and 'off' states, then the
pixels can be displayed in grayscale. If the shade can be colored as red,
green, or blue by using different color filters, then pixels can be displayed in
color by mixing three sub-pixels each in a different color and with its own
grayscale. In other words, the perceived brightness of a pixel is determined
by its transmittance and the backlight illumination.
Figure 10-3: Block diagram of a TFT-LCD monitor.
Frame
CPU
Video
Line
Timing
TFT-LCD
panel
Video data
Scann
er
Grayscale
Ba
cklig
ht
DC-AC
Display Control
Power supply
180
A TFT-LCD panel consists of the following ordered layers: front
polarizer, color filter, glass, indium tin oxide (ITO), polymide film, liquid
crystals, polymide film, ITO, glass, and rear polarizer. The light
transmittance is determined by the front and rear polarizers and the
orientation of the liquid crystals. A polarizer is a light filter that blocks the
light wave in different directions. After passing thought the rear polarizer,
the backlight becomes in a single direction. If the front polarizer and rear
polarizer are perpendicular to one another and there is no liquid crystal in
between, then the backlight will be blocked and the LCD looks dark.
Otherwise, the backlight will pass through and the LCD looks bright. The
liquid crystals can be considered as tiny lenses between the two polarizers.
The direction of light wave can be twisted by changing the orientation of the
liquid crystals. Thus, the liquid crystals can be considered as voltage-
controlled light switches (cfr. Figure 10-4.)
Figure 10-4: LCD as voltage-controlled light switch.
The liquid crystals are in a phase of matter between liquid and solid states.
In the liquid state the molecules can move freely, whereas in solid state the
molecules are fixed in certain order. The liquid crystals are in the state that
the molecules can move until they form a certain order according to an
external force. Three types of liquid crystal phases are used in LCD:
nematic, cholesteric, and smectic. The twisted nematic (TN) liquid crystal is
the most widely used in today's LCD monitors.
181
The nematic liquid crystals can be considered as transparent rods (rod-like
lenses.) These rods are locally aligned with their long axes nearly parallel to
each other on a two-dimensional plane. Their orientation can be denoted by
an angle. When there are two stacked planes of nematic liquid crystals, the
molecules on each plane align with each other, but the angle of each plane
can be different, or twisted. When more twisted planes of nematic liquid
crystals are stacked, a series of twisted rod-like lenses forms a spiral light
path, which can twist the direction of the light wave from the backlight. If
the polarizers are perpendicular, then the backlight passes and a bright pixel
is seen. This configuration of crossed polarizers is called normally white.
When an external electrical field is applied, all nematic liquid crystals
change their orientation such that their long axes point to the electrodes
uniformly. In this case, the direction of the light wave is not affected and
forms a dark pixel in a normally white LCD. In other words, a normally
white LCD consumes power to generate a dark pixel. By controlling the
electrode voltage, the amount of light passing the LCD can be modulated to
generate grayscale pixels.
The nematic liquid crystals are sandwiched between two glasses. The
spacing in between decides the number of planes of nematic liquid crystals.
To twist the liquid crystals, parallel groves are produced on the glasses by
using polymide film. The groves on the front glass set the orientation of the
nearby nematic liquid crystals.
The groves on the rear glass set the orientation of the nearby nematic
liquid crystals to be perpendicular to the first one. The orientations of other
planes in between are twisted by the first and last plane from 0 to 90 degree
accordingly when no voltage applied. The color filter determines a sub-pixel
to be red, green, or blue.
Figure 10-5: The electrical waveforms of the gate-source Vg and
drain-source voltage Vd of a TFT-LCD.
Each sub-pixel has an individual liquid crystal cell, a thin-film-transistor
and a storage capacitor. The layout of the transparent ITO electrodes defines
the shape of a sub-pixel. The liquid crystals between the electrodes form a
Vcom
Vg
Vd
182
conceptual cell. The electrical field of the capacitor controls the
transmittance of the cell. The capacitor is charged and discharged by its own
TFT. The gate electrode of the TFT controls the timing of
charging/discharging when the pixel is scanned (or addressed) for refreshing
its content. The (drain-) source electrode of the TFT controls the amount of
charge (cfr. Figure 10-5).
Figure 10-6: The equivalent circuit of a TFT-LCD sub-pixel. CST
is the storage capacitor. CGS and Clc are parasitic capacitances.
The gate electrodes and source electrodes of all TFT’s are driven by a set of
gate drivers and source drivers, respectively. A single gate driver (called a
gate bus line) drives all gate electrodes of the pixels on the same row. The
gate electrodes are enabled at the same time the row is scanned. A single
source driver (called a source bus line) drives all source electrodes of the
pixels on the same column (cfr. Figure 10-6). The source driver supplies the
desired voltage level (called grayscale voltage) according to the pixel value.
In other words, ideally, the pixel value transmittance, t(x), is a linear
function of the grayscale voltage v(x), which is itself a linear function of the
pixel value x. If there are 256 grayscales, then the source driver must be able
to supply 256 different grayscale voltage levels. For the source driver to
provide a wide range of grayscales, a number of reference voltages are
required. The source driver mixes different reference voltages to obtain the
desired grayscale voltages. Typically, these different reference voltages are
fixed and designed as a voltage divider. For example in [8], an Analog
Devices input LCD reference driver [11] is used with a 10-way voltage
divider.
Source bus line
Vn
Cst Clc
G
Vg
Vd
SD
Cgs
Vs
t
Vcom
Gate bus line
Vcom
183
Figure 10-7: The voltage divider generating reference
voltages for grayscale controller.
Assume that the transmittance of the TFT-LCD is linear and the resistors
of the voltage divider are identical. If k+1 identical resistors r0…rk are
connected in series between ground and Vdd in that order, then the output
voltage seen from the top terminal of ri is (cfr. Figure 10-7):
1
1i dd
iV V
k
+=
+. (3)
Figure10- 8: Normalized pixel transmittance, t(x), versus
power consumption of a pixel in the LCD panel.
r0
r1
r2
rk-3
rk-2
rk-1
rk
Vdd
Vk-1
Vk-2
Vk-3
V2
V1
V0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.96 0.965 0.97 0.975 0.98 0.985 0.99 0.995 1
TFT-LCD Panel Power
Tra
nsm
itta
nc
et
184
10.2.5 Power Modeling for the TFT-LCD
The hydrogenated amorphous silicon (a-Si:H) is commonly used to
fabricate the TFT in display applications. For a TFT-LCD panel, the a-Si:H
TFT power consumption can be modeled by a quadratic function of pixel
value x∈[0,255] [13]:
PTFT (x)=c0+c1x+c2x2(Watt). (4)
We performed the current and power measurements on the LG Philips,
LP064V1 LCD. The measurement data are shown in Figure10- 8. The
regression coefficients are thus determined as:
c0=2.703E-3, c1=2.821E-4, c2=2.807E-5. (5)
The power consumption of a normally white TFT-LCD panel decreases as
its global transmittance increases. In other words, while maintaining the
same luminance, the power consumption of the TFT-LCD decreases when
dimming the backlight. The change in the TFT-LCD power consumption is,
however, quite small.
10.3 BACKLIGHT AND TRANSMITTANCE SCALING
The general framework for the backlight luminance dimming and the
transmittance scaling is depicted in Figure 10-9. The pixel illumination is
determined by the backlight illumination and the LCD transmittance. The
backlight illumination is controlled by adjusting the amplitude of the
dimming control signal to the DC-AC inverter. The LCD transmittance is
controlled by the pixel values and the grayscale controller. The backlight
scaling technique dims the backlight to save power and increases pixel
values to compensate for the brightness loss. The transmittance scaling
technique dims the backlight to save power and increases the reference
grayscale voltages to compensate for the brightness loss. Note that
transmittance scaling does not change the pixel values.
185
Figure 10-9: A framework for backlight luminance dimming and transmittance
scaling.
The luminance of a transmissive object is the product of the luminance of
the incident light and the transmittance of the object [5]. For a pixel on a
transmissive TFT-LCD monitor, its transmittance, t(x), is a function of its
pixel value x. More precisely, a pixel value of zero means zero transmittance
and hence the perceived shade will be black whereas a pixel value of 255
means a transmittance of one and hence the perceived shade will be white.
Other pixel values between 0 and 255, result in various shades of gray. Now,
luminance, L(x), of a pixel in the TFT-LCD panel is calculated as:
L(x) = b·t(x) (6)
The ambient light is not considered here because it has little effect for a
transmissive TFT-LCD when compared with a reflective or transflective
one. Figure 10-10 depicts the relation in (6) assuming that the TFT-LCD
transmissivity is a linear function of the pixel value.
b * =
Backlight
Factor
TFT-LCD Transmittance
Function Luminance Function
Figure 10-10: The luminance of a normalized pixel value (right) is the
product of the backlight factor b and the TFT-LCD transmissivity function.
In a non-backlight-scaled TFT-LCD monitor, the backlight luminance
(denoted by the normalized backlight factor b) is fixed at full CCFL driver
power (b=1).
Pixel valueDimming value
DC-AC inverter
CCFL Illumination LCD Transmittance
Control voltage
Source driver
Grayscale controller
* = Pixel Illumination
Reference
grayscale
voltages
10
Pixel valueDimming value
DC-AC inverter
CCFL Illumination
Control voltage
Source driver
Grayscale controller
* =
Reference
grayscale
voltages
10
186
10.3.1 Backlight Luminance Dimming
Reference [2] describes two backlight luminance dimming techniques,
which dim the backlight luminance to save power consumption. To
compensate for brightness loss, the authors reduce b while increasing the
pixel values from x to x' by two mechanisms. The “backlight luminance
dimming with brightness compensation” technique uses:
x’=x+b. (7)
The “backlight luminance dimming with contrast enhancement” technique
uses:
x’=x/b. (8)
The pixel values are adjusted by software before being written into the frame
buffer or by hardware after being fetched by the video controller. The
distortion after backlight luminance dimming is evaluated by the percentage
of saturated pixels that exceed the range of pixel values, i.e., [0,255]. The
optimal backlight factor is determined by the backlight luminance dimming
policy subject to the given distortion rate. To calculate the distortion rate, a
histogram estimator is required for calculating the statistics of the input
image.
10.3.2 Programmable LCD Reference Driver
Recall that the pixel value transmittance, t(x), is a linear function of the
grayscale voltage v(x). The transmittance scaling approach is to control the
mapping of v(x) in order to control the transmittance function t(x). We
propose using a programmable LCD reference driver (PLRD) described as
follows.
The PLRD is implemented by adding an extra logic to the original voltage
divider expressed by (6). The logic contains a number of p-channel and n-
channel switches and multiplexers. Recall that k+1 identical resistors r0…rk
are connected in series between ground and Vdd. The PLRD takes two input
arguments gl and gu, and then short circuits the top terminal of rgl to ground
and the top terminal of rgu to Vdd. In this way, the output voltage seen from
the top terminal of ri becomes:
,
',, ,
0, 0
V gu i kdd
i glV gl i guddi gl gu gu gl
i gl
V
≤ ≤
− ≤ <−
≤ ≤
= . (9)
Clearly, the PLRD performs a linear transformation (limited by 0 and Vk)
187
on the original reference voltages, and therefore, provides the transmittance
scaling policy a mechanism for adjusting the TFT-LCD transmittance
function as shown in Figure 10-11a. The luminance function is shown in
Figure 10-11b.
b *
(a)
=
(b)
Figure 10-11: (a) The LCD transmittance function (b) and the
luminance function when using a programmable LCD reference driver.
The similar concept of PLRD has been implemented in TFT-LCD
controllers such as [12] to control contrast or gamma-correction. The PLRD
represents a class of linear transformations on the backlight-scaled image. It
covers both brightness scaling (adjusting gu and gl simultaneously) and
contrast scaling (adjusting gu-gl). On the other hand, non-linear
transformations are not desired in transmittance scaling because they cannot
preserve the uniformity of contrast.
10.3.3 Contrast Fidelity
The term contrast describes the concept of the differences between the
dark and bright pixels. Brightness and contrast are the two most important
properties of any image. In the Human Visual System [5][14], which models
the perception of human vision as a three-stage processing, the brightness
and contrast are perceived in the first two stages. Virtually every single
display permits the users to adjust the brightness and contrast settings. For
transmissive LCD monitors, the brightness control changes the backlight
illumination and the contrast control changes the LCD transmittance
function.
0 gl
x1
1
t(x)
gu0 gl
x1
1
t(x)
gu 0
b
x
L(x)
1
1
gl gu0
b
x
L(x)
1
1
gl gu
188
(a) Original image (b) Dim backlight to
50% without
compensation
(c) Backlight
luminance dimming
(d) Transmittance
scaling
Figure 10-12: Luminance functions and visual effects of adjusting brightness (b), contrast (c),
and both (d) when the backlight is dimmed to 50%.
Figure 10-12 shows how the brightness and contrast control change the
luminance function and their visual effects when the maximum brightness is
limited to 50%. Figure 10-12a depicts the original image of the USC girl. In
Figure 10-12b, when the backlight is reduced to 50%, the image contrast is
noticeably reduced. If we compensate for the contrast loss as shown in
Figure 10-12c, then the darker (<50%) pixels will preserve their original
brightness while the brighter (>50%) pixels will overshoot completely (there
will be no contrast present among these pixels.) Figure 10-12d shows how
the transmittance scaling generates a better image by balancing the contrast
loss and number of overshot pixels. The luminance functions in Figure 10-
12b and Figure 10-12d represent the following class of linear
transformations that can be implemented by the PLRD as expressed in (9):
0, 0
, ,
, 1
( )
dx gl glccx d gl x gu where
b dgub gu x c
b t x
−≤ ≤ =+ ≤ ≤ −=≤ ≤
⋅ = . (10)
Here (gl,0) and (gu,b) are the points where y=cx+d intersects y=0 and
y=b, respectively. The luminance function consists of three regions: the
189
undershot region [0,gl], the linear region [gl,gu], and the overshot region
[gu,1]. In other words, the gl and gu are the darkest and the brightest pixel
values that can be displayed without contrast distortion (overshooting or
undershooting). Notice that the slope of the linear region is very close to that
of the original luminance function, which is unity. The image has very few
pixels in the undershot and overshot regions. Its histogram is shown in
Figure 10-13a.
The kernel of transmittance scaling is to find the dissimilarity between the
original and backlight-scaled images, which can be solely determined by
examining the luminance function b.t(x). We define the contrast fidelity
function as the derivative of b.t(x):
0, 0
, , 0 1
0, 1
( )
x gl
c gl x gu cc
gu x
f x
≤ <≤ ≤ ≤ ≤
< ≤=
.
(11)
c is limited between 0 and 1. If c>1, the contrast increases and deviates
from that of the original image while the dynamic range [gl,gu] shrinks. The
overall contrast fidelity decreases from this point, so we do not include c>1
in our solution space.
The contrast fidelity is defined without quantifying contrast itself, which
has no universal definition [15] and cannot help solve the optimal
transmittance scaling policy problem. However, the definition of contrast
fidelity does convey the concept of the classical definitions of contrast such
as Weber's or Michelson's that express contrast as the ratio of the luminance
difference to the maximum luminance [5][14][15]. If the normalized image
histogram providing the probability distribution of the occurrence of pixel
value x in the image is given as:
p(x)∈[0,1], x=0..255, (12)
Then the global contrast fidelity of the backlight-scaled image will be
defined as:
( ) ( ).
gu
C c
gl
F f x p x= ⋅ (13)
Fc is a function of p, gl and gu. Finding the optimal solution that
minimizes Fc is called the optimal transmittance scaling policy problem.
190
More precisely, given the image, we would like to find the optimal backlight
factor and the PLRD transformation function, which maximize the global
contrast fidelity. The variables of optimization are b, gu, and gl.
The global contrast fidelity captures the brightness distortion due to
backlight luminance dimming, also. When the backlight is dimmed, the
dynamic range [gl,gu] is shrunk accordingly, so that more pixels have
contrast fidelity of zero.
Figure 10-13: (a) Histogram of the example image, (b) Optimal values of gl (left
curve) and gl+dr (right curve) as functions of dynamic range dr, (c) Global contrast
fidelity Fc as a function of dynamic range dr for b=1 (upper curve ) and b=0.5
(lower curve), (d) Optimal solutions < Fc,Pbacklight>, (e) After transmittance scaling
with 10% contrast distortion (f) After backlight luminance dimming.
10.3.4 Transmittance Scaling
To simplify the optimal transmittance scaling policy problem, our
191
approach is first to find the optimal linear transmittance function for each
given backlight factor. This problem is called the contrast fidelity
optimization problem. In this simplified version, given the image and
backlight factor, we would like to find the optimal PLRD transformation
function that maximizes the global contrast fidelity. The variables of
optimization are gu and gl. Next, we sweep the backlight factor domain to
find the globally optimal solutions.
Our goal is to find the optimal gl and gu that maximize the overall contrast
fidelity Fc. After that, the optimal coefficients c and d can be calculated from
(10). The optimal transmittance function t(x) that should be applied to the
LCD can then be determined as:
0, 0
,
1, 1
( )
x gl
cx dgl x gu
b
gu x
t x
≤ <+ ≤ ≤
< ≤
= , (14)
Clearly the backlight should be concurrently dimmed to b.
The optimal solution to the contrast fidelity optimization problem for an
arbitrary histogram can be found by the following procedures. Let dr=gu-gl
denote the required dynamic range [gl,gu] and the backlight factor b denote
the available dynamic range [0,b]. For each dr, we can find the required
dynamic range [gl,gl+dr] that maximizes ( )
gl dr
gl
p x+
. The optimal gl is found
by scanning gl=0*256/k, 1*256/k,…(k-1)*256/k, where k represents the
resolution of the PLRD in (9). Based on the histogram shown in Figure 10-
13a, Figure 10-13b shows the optimal gl and gl+dr in the x axis as functions
of dr in the y axis. The left and right curves are the optimal gl and gl+dr,
respectively for different dr values. This means that when the backlight is
dimmed to dr, by using the available dynamic range [0,dr] from the
transmissive LCD to display the required dynamic range [gl,gl+dr] by the
image, we are able to generate a backlight-scaled image that minimizes the
number of undershot or overshot pixels.
Now consider the contrast fidelity c in (14). If the available dynamic range
is larger or equal to the required dynamic range (dr b), the optimal contrast
fidelity c=1 can be obtained with d 0 and the overall contrast fidelity Fc is
simply ( )
gu
gl
p x . Otherwise, if dr>b, the highest possible contrast fidelity is
c=b/dr with t=1 and d=0. Thus, Fc becomes:
192
( ).
gl dr
gl
bp x
dr
+
(15)
Figure 10-13c shows Fc as a function of dr for b=1 (upper) and b=0.5
(lower). The Fc increases as dr increases from dr=0 to dr=0.5. For the b=1
curve, the example image needs no more than 70% of available dynamic
range to represent the whole histogram with the best contrast fidelity c=1.
For the b=0.5 curve, the Fc decreases from dr=0.5 to dr=1 because in (15)
the ( )
gl dr
gl
p x+
increases slower than dr. The optimal Fc happens at dr=0.5 and
the contrast fidelity c=1 in the region [gl,gl+dr]. Notice that c=1 is not
always the optimal solution when dr>b. If the distribution in the histogram is
not normal (e.g. has two peaks) the optimal dr can be greater than b, such
that ( )
gl dr
gl
p x+
can be increased. For each backlight factor b, the complexity of
finding the optimal Fc, gl and gu is O(k2) with a small k (<12).
Given the solution to the contrast fidelity optimization problem for any
backlight factor b, the optimal transmittance scaling policy problem can be
solved by sweeping the backlight factor range between bmin and bmax, where
bmin and bmax are user-specified minimum and maximum backlight factors,
respectively. All of the optimal solutions are recorded along with their power
consumptions. The inferior solutions, i.e., those with higher or equal power
consumptions but lower fidelity, are discarded. The remaining solutions are
stored for the transmittance scaling policy to select the most suitable solution
according to the user preferences. Figure 10-13d shows the optimal solutions
for b=0.8, 0.7,…0.2 from top-right to bottom-left. The x and y coordinates of
each solution indicate the global contrast fidelity and backlight power,
respectively. The two inferior solutions for b=1.0 and 0.9 are discarded
because they have the same fidelity, Fc=1, as that of b=0.8. Results show
that more than 50% power savings can be achieved by the transmittance
scaling policy while maintaining almost 100% of contrast fidelity at a
backlight factor of 70%. The visual effect is shown in Figure 10-13e, in
comparison with Figure 10-13f generated from the brightness-invariant
policy from (8).
A pseudo-code transmittance scaling procedure is provided below.
193
transmittance_scaling(p[0..255],k) cdf[0]=p[0]; for (i=0; i<256; i++) cdf[i]+=p[i];
for (b=bmin; b<=bmax; b+=(1/k)) Pb=Pbacklight(b); for (dr=1; dr<=255; dr+=(256/k)) Rmax=-1; for (g=0; g<=255-dr; g+=(256/k)) R=cdf[g+dr]-cdf[g]; if (R>Rmax) gl=g; Rmax=R; if (b>=dr)
Fc=R; else
Fc=(b/dr)*R;
gu=gl+dr; Sol = <Fc,Pb,b,gl,gu>; Search solution database for <Fc,*,*,*> and <*,Pb,*,*>; if (Sol is not inferior) Insert Sol into solution database;
We use a set of benchmark images from the USC SIPI Image Database
(USID) [16]. The USID is considered the de facto benchmark suite in the
signal and image processing research field [5]. The results reported here are
from 8 color images from volume 3 of USID. All of these images are 256 by
256 pixel images. The color depth is 24 bits, i.e., 8 bits per color-channel in
the range of 0-255.
Tables 10-1 and 10-2 show the results of the optimal transmittance scaling
policies for the benchmarks. We use 90% as the global contrast fidelity
threshold to find the minimum backlight factor and the optimal transmittance
transformation. The results show an average of 3.7X savings within 10% of
contrast distortion.
194
Table 10-1. Optimal transmittance scaling solutions to the USID benchmark images.
Image
#
Backlight
factor
b
Contrast
fidelity
c
Brightness
shift
d
Overall
fidelity
Fc
CCFL
Power
(mW)
4.1.01 0.51 1 0.00 0.91 803.84
4.1.02 0.38 1 0.00 0.91 549.99
4.1.03 0.65 1 0.00 0.91 1077.21
4.1.04 0.75 1 0.00 0.91 1272.47
4.1.05 0.75 1 0.01 0.91 1272.47
4.1.06 0.84 1 0.04 0.90 1448.21
4.1.07 0.71 1 0.06 0.90 1194.36
4.1.08 0.72 1 0.04 0.92 1213.89
Table 10-2. Original images (upper images) vs. transmittance-scaled images (lower images).
195
10.4 SUMMARY
This chapter presented a technique for reducing the backlight illumination
while compensating for the luminance loss by adjusting the transmittance
function of the TFT-LCD panel. First, background information about CCFL
and TFT-LCD monitors was presented. Next, a contrast distortion metric to
quantify the image quality loss after dimming backlight was described.
Finally, the transmittance scaling problem was precisely formulated and
solved. Experimental results showed that an average of 3.7X power saving
can be achieved with a mere 10% contrast distortion. The proposed
transmittance scaling technique was applied to still images only. However,
the basic technique can be extended to video streams. For video, the decision
about the backlight scaling factor is made for each frame one at a time.
Consequently, the backlight factor may change significantly across
consecutive frames. The enormous change in the backlight factor may
introduce inter-frame brightness distortion to the observer. Therefore, when
the transmittance scaling technique is applied to video applications such as
an MPEG2 decoder, the inter-frame change in the backlight dimming factor
should be controlled carefully such that the change is hardly noticeable to
human eyes.
References
[1] T. Simunic et al., “Event-driven power management,” IEEE Tran. Computer-Aided
Design of Integrated Circuits and Systems, vol. 20, pp. 840-857, July 2001.
[2] I. Choi, H. Shim, and N. Chang, “Low-power color TFT LCD display for hand-held
embedded systems,” Proc. of Symp. on Low Power Electronics and Design, Aug. 2002,
pp. 112-117.
[3] ANSI/IES. 1986. Nomenclature and Definitions for Illuminating Engineering, ANSI/IES
RP-16-1986. New York, NY: Illuminating Engineering Society of North America.
[4] Radiosity: A Programmer’s Perspective by Ian Ashdown, © October 2002 by Heart
Consultants Limited. (Originally published by John Wiley & Sons in 1994.)
[5] W. K. Pratt, Digital Image Processing, Wiley Interscience, 1991.
[6] Maxim, MAX1610 Digitally Controlled CCFL Backlight Power Supply.
[7] J. Williams, “A fourth generation of LCD backlight technology,” Linear Technology
Application Note 65, Nov. 1995.
[8] LG Philips, LP064V1 Liquid Crystal Display.
[9] Stanley Electric Co., Ltd., [CFL] cold cathode fluorescent lamps, 2003.
[10] Minolta, Minolta Precision Luminance Meter LS-100.
196
[11] Analog Devices, AD8511 11-Channel, Muxed Input LCD Reference Drivers.
[12] Hitachi, HD66753 168x132-dot Graphics LCD Controller/Driver with Bit-operation
Functions, 2003.
[13] H. Aoki, “Dynamic characterization of a-Si TFT-LCD pixels,” HP Labs 1996 Technical
Reports (HPL-96-19), February 21, 1996.
[14] S. Daly, “The visible differences predictor: an algorithm for the assessment of image
fidelity,” Digital Images and Human Vision, pp. 179-206, Cambridge: MIT Press, 1993.
[15] E. Peli, “Contrast in complex images,” J. Opt. Soc. Amer. A, vol. 10, no. 10, pp. 2032-
2040, Oct. 1990.
[16] A. G. Weber, “The USC-SIPI image database version 5,” USC-SIPI Report #315, Oct.
1997. Also http://sipi.usc.edu/services/database/Database.html.
[17] Toshihisa Tsukada, TFT/LCD, Liquid-Crystal Displays Addressed by Tin-Film
Transistors, Amsterdam: Gordon and Breach Publishers, 1996.
[18] W. C. O'Mara, Liquid crystal flat panel displays: manufacturing science & technology,
New York: Van Nostrand Reinhold, 1993.
197
Chapter 11
POWER-AWARE NETWORK SWAPPING FORWIRELESS PALMTOP PCS
Andrea Acquaviva, Emanuele Lattanzi and Alessandro BoglioloUniversita di Urbino
Abstract Virtual memory is considered to be an unlimited resource in desktop or notebookcomputers with high storage memory capabilities. However, in wireless mobiledevices like palmtops and personal digital assistants (PDA), storage memory islimited or absent due to weight, size and power constraints. As a consequence,swapping over remote memory devices can be considered as a viable alternative.Nevertheless, power hungry wireless network interface cards (WNIC) may limitthe battery lifetime and application performance if not efficiently exploited. Inthis chapter we explore performance and energy of network swapping in compar-ison with swapping on local microdrives and flash memories. Our study pointsout that remote swapping over power-manageable WNICs can be more efficientthan local swapping and that both energy and performance can be optimizedthrough power-aware reshaping of data requests. Experimental results show thatapplication-level prefetching can be applied to save up to 60% of swapping energywhile also improving performance.
Keywords: Memory management, power management, remote memory swapping.
11.1 INTRODUCTION
Mass storage devices provide to desktop and laptop computers the supportto implement a virtual memory that can be viewed as an unlimited resource tobe used to extend the main memory whenever needed. However, in wirelessmobile devices like palmtops and personal digital assistants (PDAs), storagememory is limited or absent due to weight, size and power constraints, thuslimiting the application of virtual memory. On the other hand, if a wirelessnetwork interface card (WNIC) is available, unlimited swapping space couldbe found on remote devices made available by a server and managed by theoperating system as either network file systems (NFS) or network block devices
198
(NBD). However, swapping over a power hungry WNIC may limit the batterylifetime and application performance if not efficiently exploited.
In this chapter we report the results of extensive experiments conducted toevaluate and optimize the performance and power efficiency of different localand remote swap devices for wireless PDAs (namely, a compact flash (CF), amicro drive (HD) and two different WNICs). The contribution of the chapter isthree-fold. First we characterize all swap devices in terms of time and energyinherently required to swap a single page. Second, we test the effectivenessof the dynamic power management (DPM) support made available by eachdevice. Third, we show that dummy data accesses can be preemptively insertedin the source code to reshape page requests in order to significantly improvethe effectiveness of DPM.
Experimental results show that WNICs are less efficient than local devicesboth in terms of energy and time per page. However, the DPM support pro-vided by WNICs is much more efficient than that of local micro drives, makingnetwork swapping less expensive than local swapping for real-world applica-tions with non-uniform page requests. Finally, we show that application-levelreshaping of page requests can be used in conjunction with DPM to save up to60% of energy while improving performance.
The rest of this chapter is organized as follows. In Section 11.2 we providesome background on remote storage space. In Section 11.3 we discuss the keyfeatures of local and remote storage devices that can be used for swapping, andwe briefly outline the software support for remote swapping provided by Linux.In Section 11.4 we describe the experimental setup used for our experiments. InSection 11.5 we outline the benchmarks used to characterize each swap devicein terms of power and performance and we present characterization results. InSection 11.6 we discuss the support for dynamic power management providedby each swap device and we propose an application-level prefetching strategyfor energy-aware swapping. In Section 11.7 we report and discuss the resultsof extensive experiments conducted on a simple case study. In Section 11.8 wedraw conclusions.
11.2 REMOTE STORAGE SPACE
The concept of remote storage has been exploited by deeply networked sys-tems for mainly three reasons: to provide extra storage space, to enable filesharing and to enhance swapping capabilities.
First, remote memories or magnetic disks are used to store application anddata by systems with limited or absent local mass storage space. Disklessworkstations and mobile terminals are both computer systems characterized bylimited or absent disk capacity.
199
Even if the memory is not a constraint, remote storage space is used as repos-itory of data shared among different users working on different machines, asin the case of file servers. Access to remote data can be controlled by networkfile systems such as NFS. However, mobile networks require suitable proto-cols to handle disconnected and weakly connected operations. To this purpose,dedicated file systems and file hoarding methods have been designed [13, 7,8]. File hoarding is the technique of preparing disconnections by caching crit-ical data. Differently from traditional caching, the cost of a miss (or failure)can be catastrophic if it occurs when the system is disconnected from the net-work. To identify critical data, LRU policies augmented with user-specifiedhoard-priority have been proposed as part of the CODA file system [13]. Here,priorities are used to offset the LRU age of an object. In addition, the user isgiven the possibility to interactively control the hoarding strategy (the so calledtranslucent caching concept). Automated hoarding methods have been alsorecently presented [8, 7].
Remote memories are also commonly used as swap areas to temporarilypark run-time data and application code when the total amount of availablesystem memory is not enough to contain user processes. In computer clustersremote swap areas are designed to replace local swap partitions for performancereasons. In fact, high speed links may provide faster access than local magneticdisks especially under certain workload conditions, due to the high rotationallatency [12, 6]. While for remote file systems the main issues is reliability, forremote swapping the performance of data transfer is the key point. For thisreason, simpler and more efficient supports have been proposed [5].
Remote swap areas can be also exploited by mobile devices, where localstorage space is limited and expensive [3]. However, network swapping inmobile devices does not come for free since they are much more bandwidth andenergy constrained than desktop PCs and workstations.
Remote swapping for handheld computing systems is a recent research topicthat has not been extensively studied so far. The problem of energy consumptionof network swapping in mobile devices has been faced by Hom et al. [9]. Theyproposed a compilation framework aimed at reducing the energy by switchingthe communication device on and off by means of specific instructions insertedat compile time based on a partial knowledge of the memory footprint of theapplication.
11.3 SWAP DEVICES
We refer to the page-based swapping support provided by the Linux OS.Linux performs a page swap in two situations: i) when a kernel daemon, ac-tivated once per second, finds that the number of free pages has fallen belowa given threshold; ii) when a memory request cannot be satisfied. The page
200
to be swapped-out is selected in a global way, independently from the pro-cess that made the request. The page replacement algorithm is based on anapproximation of least recently used (LRU) policy [4].
Modern operating systems equipping palmtops and PDAs make possible todefine heterogeneous support for swapping. Swapping can be performed bothlocally to the PDA and remotely, by exploiting server storage capabilities andnetwork connections. More than one swap units can be enabled at the sametime, with assigned priority. The unit with the highest priority is selected bydefault until it becomes insufficient.
11.3.1 Local Devices
On-board non-volatile memory is usually available in palmtop PCs to storethe bootloader and the filesystem. Magnetic disks can be added to extend filestorage capabilities. Swap can be made locally in palmtops as in desktop PCs.A dedicated partition can be defined in hard drives or flash memories, wherethe filesystem resides. Alternatively, some OS’s allow the user to define a swapfile that does not need a dedicated partition. Either way, the swap area comesat the price of decreasing the space available for actual storage purposes.
Compact Flash
Palmtop PCs are equipped with on-board flash memories, but additionalmemory chips can be installed as an expansion if an external slot is present.Memory Technology Device (MTD) drivers allow to define swap partitions orswap files on flash memories. However, being read-most devices, flash mem-ories are not the ideal support for swapping. Nevertheless, we evaluate theirswapping performance since they are always present in palmtop PCs, beingsometimes the only alternative to network swapping.
Hard Disk
Today’s technology made available hand-sized magnetic disks (called minior micro drives) suitable to be installed in palmtop computers. Currently theyprovide a storage capability up to 5GBytes. Like traditional hard disks (HD),micro drives provide a seek time much longer than the access time to sequentialblocks. For this reason, access to these kind of devices is usually performedin bursts whenever possible by exploiting on-board hardware buffers in orderto compensate for the initial transfer delay. The OS tries to limit the delaysby filtering disk accesses using software caches, whose size is limited by theavailable space in main memory. When a micro drive used as a swap device,this trade-off is even more critical, since increasing the memory space allocatedfor caching increases the number of swap requests.
201
11.3.2 Network Devices
In order to provide the performance required to fully exploit the channelbandwidth, remote swap files can be mapped in the main memory of a remoteserver. This is the choice we made for our experiments.
Network File System
NFS (Network File System) is used in a network to enable file sharing amongdifferent machines on a local area. The communication protocol is based on aUDP stack, while data transfers between NFS server and clients are based onRemote Procedure Calls (RPCs). The idea of using NFS to support networkswapping is relatively recent [16]. To this purpose, a remote file must be con-figured as a swap area. This is made possible by modern operating systems thatallow the user to specify either a device or a file as a swap unit.
Network Block Device
A Network Block Device (NBD) offers to the OS and to the applicationsrunning on top of it the illusion of using a local block device, while data arenot stored locally but sent to a remote server [5]. As in case of NFS, the virtuallocal device is mapped in a remote file, but the swap unit is viewed as a device,rather than as a file.
This is made possible by a kernel level driver (or module) that communicatesto a remote user-level server. The first time the network connection is set-up, aNDB user-level client negotiates with the NBD server the size and the accessgranularity of the exported file. After initialization, the user-level NBD clientdoes not take part to remaining transactions, that directly involve the kernel NBDdriver and the NBD server. No RPCs are required in this case, thus reducingthe software overhead. Latest releases of NBD driver use an user level networkcommunication, which affects the performance of the protocol, since data mustbe copied from the kernel to the user space address, but increases flexibility.Differently from NFS, the underlying network stack is TCP instead of UDP.This increases the reliability of network transfers, at the cost of increasing theprotocol overhead.
11.4 EXPERIMENTAL SETUP
We performed our experiments on a HP’s IPAQ 3600 handheld PC, equippedwith a Strong-ARM-1110 processor, 32MB SDRAM and 16MB of FLASH.Our benchmarks were executed on the palmtop on top of the Linux operatingsystem, Familiar release 6.0. The WNICs used to provide network connectivitywere a LUCENT (hereafter denoted by NICLucent) and a CISCO AIRONET350(NICCisco), while the AP connected to the remote swapping server was a
202
/*************** Benchmark 1 ***************/double A[ROW][COL];initialize(A,ROW,COL);t0 = time();read_by_column(A,ROW,COL);t1 = time();/*******************************************/
Figure 11.1. Pseudo-code of the benchmark used to characterize swap devices.
CISCO 350 Series base station [19, 18, 17]. The remote server was installed ona Athlon 4 Mobile 1.2 GHz notebook. For local swap experiments we used a340 MB IBM Microdrive (HD) and a 64 MB Compaq-Sundisk Compact FlashMemory (CF) [20, 21]. Power consumption of both WNICs and local deviceswas measured using a Sycard Card Extender that allowed us to monitor thetime behavior of the supply current drawn by the card. The current waveformswere then digitized using a National Instruments Data Acquisition Board PCI6024E [11] connected to a PC. A Labview (version 6.1) [10] software runningon the PC was used to coordinate the acquisition and bufferize current samplesto compute power and energy consumption.
The remote swap NBD server was instrumented in order to collect time-stamped traces of swapping activity during benchmarks execution.
11.5 CHARACTERIZATION OF SWAPPING COSTS
To characterize the inherent cost of a page swap we developed a suite ofbenchmarks accessing data structures much larger than the available main mem-ory, without performing any computation on them. This kind of benchmark issuitable to characterization swapping cost since the computation time is neg-ligible with respect of the time spent in swapping and the devices under char-acterization are always busy serving page requests. The pseudo-code of thebenchmark is shown in Figure 11.1. A large matrix is allocated and initial-ized and then read by column in order to maximize the number of page faults.Different benchmarks were generated by changing the number of columns androws of the matrix in order to change the number of page faults while keep-ing the total size of the matrix unchanged. This allowed us to cross-validateexperimental results and reduce characterization errors.
A second set of benchmarks was obtained by replacingthe read by column procedure with a write by column procedure, and usedto characterize the swapping cost in case of write-back.
203
Characterization Results
Experimental results are reported in Table 11.1 in terms of time, energy andpower required by each local and remote device to swap a page of 4096 bytes.Both read-only and write-back results are reported.
In general, write-back doubles the cost (in energy and time) of a read-onlyswap, since it involves two data transfers. As expected, local devices are moreefficient than WNICs and CF has an energy-per-page more than 10 times lowerthan all other devices.
It is also worth noting that, for a given WNIC, NBD provides greater perfor-mance than NFS, at the cost of a slightly higher power consumption. Since thetime reduction overcomes the additional power consumption, the energy perpage required by NBD is lower than that required by NFS.
Table 11.1. Power consumption and performance of local and remote swap devices.
Swap device Read-only Write-backType Mode Time Energy Power Time Energy Power
[ms] [mJ] [mW] [ms] [mJ] [mW]
CF local 4.1 0.201 49 8.2 0.402 49HD local 3.0 1.911 637 6.4 4.061 637
NICCisco NBD 7.0 5.934 848 14.0 10.319 735NICCisco NFS 8.5 6.123 720 14.6 10.516 720
NICLucent NBD 8.0 5.626 578 15.0 8.599 573NICLucent NFS 10.0 5.243 524 22.0 10.672 485
11.6 POWER OPTIMIZATION
In the previous section we have characterized swap devices in terms of timeand energy requirements per swap page. To this purpose we designed a setof benchmarks that simply accessed data structures much larger than the mainmemory without performing any computation on them.
Although useful for characterization purposes, the benchmarks of Figure 11.1are unrealistic for two main reasons. First, computation time is usually non-negligible, so that page requests are spaced in time according to a distributionthat depends on the workload and on the state of the main memory. Second, thetotal size of the data structures accessed by each application usually does notexceed the size of the main memory, or otherwise the performance degradationwould not be acceptable.
In most cases of practical interest, swapping is mainly needed after a contextswitch to bring in main memory data structures the first time they are used bythe active process. Moreover, in handheld devices there are often only a fewprocesses running concurrently, so that both main memory and peripherals are
204
Table 11.2. Power states of local and remote devices.
Device State Power Timeout WU-time WU-power[mW] [ms] [ms] [mW]
CF Read 107Write 156Wait 4.5
HD Read 946Write 991Wait 600Sleep 24 2000 4500 ± 1980 1067
NIC Receive 755(cisco) Transmit 1136
Wait 525(PSP/PSPCAM) 113 0/850 14/14 400
Power-Off 0 any 370 451NIC Receive 548
(lucent) Transmit 798Wait 407(PSP) 38 100 1 800
Power-Off 0 any 270 357
mainly used by a single process at the time. In this situation, the usage pattern ofswapping devices are significantly different from those used for characterizingswapping costs because of the presence of long idle periods between pageswaps.
Since swapping devices spend power while waiting for page requests, theeffective energy per page is larger than that reported in Table 11.1. On theother hand, idleness can be dynamically exploited to save power by puttingthe devices in low-power operating modes, or by turning them off. Dynamicpower management (DPM) significantly impacts the performance and energytrade-off offered by each device under bursty workloads.
In this section, we first analyze the DPM supports provided by each swappingdevice, then we show how to increase their effectiveness by means of softwareoptimization techniques aimed at reshaping the distribution of page requests.
11.6.1 DPM Support Evaluation
The DPM support provided by each swap device is schematically representedin Table 11.2. For each device, the key features of active and inactive operatingmodes are reported. Active modes are characterized only in terms of powerconsumption, while inactive modes are also characterized in terms of timeout tobe waited before entering the inactive state, wake-up time and wake-up power.
205
The data reported in the Table have been obtained by analyzing the currentprofiles provided by the measurement setup described in Section 11.4.
First of all we remark that the average power consumptions measured duringpage swaps (reported in Table 11.1) are not equal to the power consumptionsmeasured for the devices during read/receive or write/transmit. In fact, forinstance, a page swap across a wireless link entails the transmission of thepage request, a waiting time corresponding to the latency of the remote device,the reception of the page and, possibly, the write-back of a swapped-out page.The average swapping power comes from the weighted average of all thesecontributions.
The CF has no inactive states. This is because its power consumption in waitmode is negligible, making inactive low-power states useless. On the contrary,NICs and HDs consume a large amount of power while waiting for servicerequests, so that it is worth switching them to low-power inactive states duringlong idle periods.
The Sleep state of the HD has the lowest power consumption, but the highestwakeup cost in terms of power (higher than 1W) and time (in the order of severalseconds). Moreover, the wakeup time is highly unpredictable, its measuredstandard deviation being almost 2 seconds.
According to the IEEE802.11b standard, WNICs provide MAC-level DPMsupport that can be enabled via software [1]. The actual implementation ofthe DPM support depends on the WNIC. The protocol policy (PSP) consists inplacing the card in a low-power state called doze mode, in which it sleeps butwakes-up periodically to keep synchronized with the network and to check theaccess point (AP) for outstanding data. A polling frame bust be transmitted bythe card for each packet to be retrieved. PSP mode provides power savings ata cost of a noticeable performance hit. To increase performance, a variation ofthis policy is implemented by CISCO cards. They automatically switch fromPSP to CAM (Constant Awake Mode) when a large amount of traffic is detected.In this case no polling frame is needed between packets since the reception andtransmission happen in active mode.
Even if the power consumption in sleep state is low, it is not negligible.Moreover, the card is sensitive to broadcast traffic. A more aggressive policywould require to completely shut-off the card when no needed by any activeapplication in the system. Thus, more power can be saved, at the price of alarger wake-up delay needed by network re-association. OS-level policies canbe implemented to this purpose based on a power management infrastructurerecently developed for Linux OS [2]. This infrastructure is composed by apower manager module that handles requests from applications and keeps trackof their resource needs. On the other side, upon a request, the power managercan directly switch off a peripheral (WNIC in our case) if no other applicationsare using it. Switch off request may come from user applications through
206
/*************** Case study ****************/double dummy[2048][2048], C[128][128];double A[128][128], B[128][128];initialize(A,128,128);initialize(B,128,128);initialize(C,128,128);initialize(dummy,2048,2048); //swap outt0 = time();compute_product(A,B,C);t1 = time();/*******************************************/
Figure 11.2. Pseudo-code of the case study.
dedicated APIs or directly by another kernel module. We exploited this featureto let the NBD driver module switch on and off the card between swappingrequests.
The features of doze and power-off modes are reported for both WNICs inTable 11.2. We observe that the MAC-level DPM support of NICLucent is moreefficient that that of NICCisco, but the DPM policy is more conservative (thetimeout being 100ms).
11.6.2 Re-shaping of Swap Requests
The effectiveness of any DPM strategy strongly depends on workload statis-tics. Regardless of the DPM policy, the higher the burstiness of the workload,the higher the power savings. In fact, long idle periods can be effectivelyexploited to switch to the deepest inactive states, while long activity burstsamortize the cost of wake up.
Although caching and buffering can be performed by the OS to perform alow-level reshaping of page requests, a typical trace of swapping traffic showssmall bursts of a few pages followed by short periods of inactivity. Increasingthe granularity of page swaps could increase the burstiness of the workload, butalso increases the risk of preemptively swapping-in unused pages.
On the other hand, in many cases data pre-fetching could be deterministicallyperformed at the application-level in order to reshape swapping traffic. Thiscan be done by inserting dummy accesses to the data structures right beforethey are used. Dummy accesses generate bursts of page requests for two mainreasons: first, they are not delayed by any commutation, second, a single accessis sufficient to fetch an entire page.
207
11.7 CASE STUDY
We use matrix multiplication as a case study to evaluate the effectiveness ofthe DPM strategies implemented by the swap devices and to demonstrate thefeasibility of application-level reshaping of swap requests.
The pseudo-code of the case study is reported in Figure 11.2: it simplycomputes the product of two square matrices A and B and puts the result in a thirdmatrix C. The total size of the three matrices fits in main memory, but we use adummy matrix, exceeding the size of the physical memory, to force swappingactivity. Matrices A, B and C are first allocated and initialized, then the dummymatrix is initialized in order to swap A, B and C out from main memory. Inpractice, the initialization of the dummy matrix creates boundary conditionssimilar to those possibly caused by the execution of other applications. Thenwe monitor the execution time and the swapping energy caused by the executionof the compute product procedure.
The distribution of swap requests is shown in Figure 11.3. The expecteddistribution is also plotted for comparison. The large number of pages requestedat the beginning corresponds to the upload of the entire matrix B. In fact, the firstcolumn of B has to be read in order to compute the first entry of C. Since matricesare stored in memory by rows, reading the first column entails swapping theentire matrix. Subsequent page requests are spaced in time according to thetime required to compute 512 × 128 floating point products.
Comparing the actual requests with the theoretical needs, we observe thatthe OS swaps 8 pages at the time, thus increasing the opportunity for DPM.However, the total number of pages request by the OS is 104, while the threematrices fit into 96 pages.
that reads one matrix entry every 512 (i.e., one entry per page).
Case Study Results
Experimental results obtained by executing the case study with and withouttraffic reshaping are reported in Table 11.3. Based on the results reported inTable 11.1 we decided to use NBD for remote swapping. In Figure 11.5 wedetailed the comparison among different devices.
Each device was tested with and without DPM. The two WNICs were testedwith both MAC-level DPM (doze) and OS-level DPM (power-off). The DPMmode and the corresponding timeouts are reported in the first column. Theperformance of the CF and the CPU time obtained by running the applicationwith data available in main memory are also reported in the first two rows forreference. The DPM of the HD was enabled by default, so that data reported
To reshape swap requests we inserted dummy accesses to the three matricesbetween the computation of initial time t0 and the computation of the matrix pro-
access one per page,duct. Dummy accesses were performed by a routing, called
208
0 5 10 15 20 25 30Execution time [s]
0
25
50
75
100
Num
ber
of p
ages
0 5 10 15 20 25 3005
101520253035404550556065707580859095
100105
Theoretical needsPage swapped
Figure 11.3. Distribution of page requests.
on row HD are computed from previous characterization. All other data wereobtained from real measurements, by repeating each experiment 4 times.
Interestingly, even without DPM, the HD consumes more energy than WNICs.This is because of its higher power consumption when idle. When DPM is en-abled, WNICs become much more convenient than HD. In particular, the DPMof the HD is counterproductive both in terms of time and energy under this traf-fic conditions because of the large wakeup cost. On the contrary, MAC-levelDPM of NICCisco and NICLucent saves respectively more than 50% and morethan 80% of the swapping energy. If the power-off state is exploited, powersavings become of 85% and 94%, respectively, with negligible performanceloss.
When DPM policies are enabled, traffic reshaping provide further advantagesboth in terms of energy and execution time. For the HD, traffic reshaping makesDPM effective to save more than 60% of energy consumption. For WNICs,traffic reshaping provides additional energy savings while further reducing theperformance loss. The ratios between results obtained with and without trafficreshaping are reported in the table. In particular, when the power-off state isexploited, traffic reshaping leads to additional energy savings around 60%.
The overall effect of DPM and traffic reshaping makes network swappingmuch more energy efficient than local swapping on a HD (the overall energybeing almost 10 times lower) with a performance penalty of about 5%.
In Figure 11.4 we reported current profiles obtained by running the case studybenchmark for the Compact Flash, Cisco card without power management andmicrodrive. The Figure evidences the strong variance of microdrive’s powerprofile, due to the variability of the head position and speed at the shutdown
209
Table 11.3. Execution time and swapping energy required to run the case study of Figure 11.2with and without traffic reshaping.
Device Exec. time [s]Original Reshaped Ratio
Avg Std Avg Std
RAM 25 0 25.25 0.5 1.01CF 25.5 0.57 25.75 0.5 1.01HD 25.31 - 25.81 - 1.01
(PM ON) 37.75 5.91 27.75 0.96 0.73NICCisco 26 - 26.33 0.58 1.01
(PSPCAM) 30.67 2.16 27 1 0.88(PSP) 43.33 0.58 45 - 1.04(OFF) 28.75 0.5 26.0 0.82 0.93
NICLucent 30.25 0.5 28.5 1.0 0.94(PSP) 30.0 0 27.75 0.5 0.92(OFF) 30.0 0 28.25 0.5 0.94
Energy [mJ]Original Reshaped Ratio
Avg Std Avg Std
RAM - - - - -CF 0.14 0.003 0.16 0.02 1.12HD 15.20 - 15.50 - 1.01
(PM ON) 19.43 5.31 6.21 0.85 0.32NICCISCO 16.51 0.01 16.73 0.38 1.01(PSPCAM) 10.59 0.23 6.64 0.61 0.63
(PSP) 8.05 0.23 8.83 0.45 1.09(OFF) 2.47 0.09 0.89 0.051 0.36
NICLUCENT 13.60 0.56 12.70 0.51 0.93(PSP) 2.54 0.096 2.19 0.08 0.86(OFF) 1.76 0.08 0.72 0.045 0.41
instants. In fact, the two bottom traces in Figure are obtained running thesame benchmark. As a reference, we marked swapping activity intervals withuppercase latin letters.
11.8 CONCLUSION
In conclusions, our experiments demonstrate the feasibility and the energyefficiency of network swapping from wireless palmtop PCs. The effectivenessof the DPM support provided by WNICs makes them more efficient than localHDs and open the field to optimization strategies (like swap reshaping) thatmay further improve energy efficiency and performance.
210
0
5
10
15
20
100
125
150
175
0
100
200
0 10 20 30 40Time [s]
0
100
200
A B C D E F G
A B C D E F G
A B C D E F G
A B C D E F G
Compact Flash
Cisco WNIC
IBM uHD
IBM uHD
Cur
rent
[m
A]
Figure 11.4. Power profiles of the different swap devices during the execution of the case studyof Figure 11.2.
Figure 11.5. Comparison of energy consumptions reported in Table 11.3
References
[1] LAN/MAN Standards Committee of the IEEE Computer Society. Part 11:Wireless LAN MAC and PHY Specifications: Higher-Speed Physical LayerExtension in the 2.4 GHz Band, IEEE, 1999.
[2] A. Acquaviva, T. Simunic, V. Deolalikar, S. Roy, "Remote Power Controlof Wireless Network Interfaces," Proceedings of PATMOS, Turin, Italy,Sept. 2003.
211
[3] I. Bokun, K. Zielinski, "Active Badges–The Next Generation,"http://www.linuxjournal.com/article.php?sid=3047, Oct. 1998.
[4] D. Bovet, M. Cesati, "Understanding the Linux Kernel," OReally & Asso-ciates, Sebastopol, CA, Jan. 2001.
[5] P. T. Breuer, A. Marin Lopez, A. Garcia Ares, "The Network Block Device,"Linux Journal, Issue 73, May 2000.
[6] M. D. Flouris, E. P. Markatos, "The Network RamDisk: Using RemoteMemory on Heterogeneous NOWs," Cluster Computing, pp. 281-293,1999, Baltzer Science Publishers.
[7] G. H. Kuenning, G. J. Popek, “Automated Hoarding for Mobile Comput-ing,” Proc. of Symposium on Operating System Principles, pp. 264–275,Oct. 1997.
[8] G. Kuenning, W. Ma, P. Reiher, G. J. Popek, “Symplifying AutomatedHoarding Methods,” Proc. of MSWiM, pp. 15–21, Sept. 2002.
[9] J. Hom, U. Kremer, "Energy Management of Virtual Memory on DisklessDevices," Proceedings of COLP, Barcelona, Spain, Sept. 2001.
[10] National Instruments, "Labview User Manual," http://www.ni.com/pdf/manuals/320999d.pdf
[11] National Instruments, "NI 6023E/6024E/6025E Family Specifications,"http://www.ni.com/pdf/manuals/370719b.pdf
[12] T. Newhall, S. Finney, K. Ganchev, M. Spiegel, "Nswap: A Network Swap-ping Module for Linux Clusters," Proceedings of Euro-Par, Klagenfurt,Austria, August 2003.
[13] M. Satyanarayanan, "The Evolution of Coda," ACM TOCS, Vol. 20, Is-sue 2, Pages: 85–124, May 2002.
[14] A. Silbershatz, P. Galvin, G. Gagne, "Operating System Concepts, 6thEdition," Addison-Wesley, 2002.
[15] Sycard Technology, "PCCextend 140 CardBus Extender User’s Manual,"http://www.sycard.com/docs/cextman.pdf (1996)
[16] "Swapping via NFS for Linux," http://www.nfs-swap.dot-heine.de
[17] Cisco System, Cisco Aironet 350 Series Access Points,http://www.cisco.com/univercd/cc/td/doc/product/wireless/airo 350/accsspts/index.htm, 2003.
[18] Cisco System, Cisco Aironet 350 Series Wireless LAN Adapters,http://www.cisco.com/univercd/cc/td/doc/product/wireless/airo 350/350cards/index.htm, 2003.
[19] Agere, 802.11 Wireless Chip Set White Paper, http://www.agere.com/client/docs/multimode white paper.pdf, 2003.
212
[20] IBM, 340MB Microdrive Hard Drive, http://www.storage.ibm.com/hddredirect.html?/micro/index.html, 2003.
[21] Compaq, compact flash cards, http://www.hp.com/products1/storage/products/storagemedia/flash cards/index.html, 2003.
213
Chapter 12
ENERGY-EFFICIENT NETWORK-ON-CHIP DESIGN
Davide Bertozzi1, Luca Benini1, Giovanni De Micheli21University of Bologna;2Stanford University
Abstract Performance and power consumption of multi-processor Systems-on-Chip(SoCs) are increasingly determined by the scalability properties of the on-chipcommunication architecture. Networks-on-Chip (NoCs) are a promising solu-tion for efficient interconnection of SoC components. This chapter focuses onlow power NoC design techniques, analyzing the related issues at different lay-ers of abstraction and providing examples taken from the most advanced NoCimplementations presented in the open literature. Particular emphasis is given toapplication-specific NoC architectures, in that they represent the most promisingscenario for minimization of communication-energy in multi-processor SoCs.
Keywords: Network-on-Chip, Low Power, Micro-network Stack, Application-Specific
12.1 INTRODUCTION
The most critical factor in Systems-on-Chip (SoCs) integration will be relatedto the communication scheme among components. The challenges for on-chipinterconnect stem from the physical properties of the interconnection wires.Global wires will carry signals whose propagation delay will exceed the clockperiod. Thus signals on global wires will be pipelined. At the same time, theswitched capacitance on global wires will constitute a significant fraction ofthe dynamic power dissipation. Moreover, estimating delays accurately willbecome increasingly harder, as wire geometries may be determined late inthe design flow. Hence, the need for latency insensitive design is critical. Themost likely synchronization paradigm for future chips is globally-asynchronouslocally-synchronous (GALS), with many different clocks.
SoC design will be guided by the principle of consuming the least possiblepower. This requirement matches the need of using SoCs in portable battery-powered electronic devices and of curtailing thermal dissipation which canmake chip operation infeasible or impractical. Whereas computation and stor-
214
age energy greatly benefits from device scaling (smaller gates, smaller memorycells), the energy for global communication does not scale down. On the con-trary, projections based on current delay optimization techniques for globalwires [20] show that global on-chip communication will require increasinglyhigher energy consumption. Hence, communication-energy minimization willbe a growing concern in future technologies [40].
Energy considerations will impose small logic swings and power supplies,most likely below 1 Volt. Electrical noise due to cross-talk, electro-magneticinterference (EMI) and radiation-induced charge injection (soft errors) will belikely to produce data upsets. Thus, the mere transmission of digital values onwires will be inherently unreliable.
To cope with these problems, network design technology can be used to ana-lyze and design SoCs modeled as micro-networks of components (or Networks-on-Chip, NoCs). The SoC interconnect design analysis and synthesis is basedupon the micro-network stack paradigm, which is an adaptation of the pro-tocol stack [25] (Figure 12.1) used in networking. This abstraction is usefulfor layering micro-network protocols and separating design issues belonging todifferent domains.
SoCs differ from wide-area networks because of local proximity and be-cause they exhibit much less non-determinism. In particular, micro-networkshave a few distinctive characteristics, namely, energy constraints, design-timespecialization and low communication latency.
This chapter focuses on low power NoC design techniques, and analyzesspecific design issues related to the different layers of abstraction outlined inthe micro-network stack in a bottom-up way. The objective is to describe,for each layer, how the system interconnect is progressively abstracted andwhat the most relevant micro-network design issues are in order to come upwith an energy-efficient NoC architecture. Particular emphasis is given to cus-tomized, domain-specific NoCs, which represent the most promising scenariofor communication-energy minimization in the context of NoC-based multi-processor SoCs (MPSoCs). In most cases, specific solutions proposed in theliterature are outlined, even though it should be clear that many design issuesare open and significant progress in this area is expected in the near future.
12.2 PHYSICAL LAYER
Global wires are the physical implementation of the communication chan-nels. Traditional rail-to-rail voltage signaling with capacitive termination, asused today for on-chip communication, is definitely not well-suited for high-speed, low-energy communication on future global interconnect [16]. Reducedswing, current-mode transmission, as used in some processor-memory systems,
215
Figure 12.1. Micro-network stack
can significantly reduce communication power dissipation while preservingspeed of data communication [29].
In the case of a simple CMOS driver, low-swing signaling is achieved bylowering the driver’s supply voltage Vdd. This implies a quadratic dynamicpower reduction (because Pdyn = KV 2
dd). Unfortunately, swing reductionat the transmitter complicates the receiver’s design. Increased sensitivity andnoise immunity are required to guarantee reliable data reception. Differentialreceivers have superior sensitivity and robustness, but they require doubling thebus width. To reduce the overhead, pseudo-differential schemes have been pro-posed, where a reference signal is shared among several bus lines and receivers,and incoming data is compared against the reference in each receiver. Pseudo-differential signaling reduces the number of signal transitions, but also noisemargins with respect to fully differential signaling. Thus, reduced switchingactivity is counterbalanced by higher swings and determining the minimum-energy solution requires careful circuit-level analysis.
Dynamic voltage scaling has been recently applied to busses [23, 26]. In[26] the voltage swing on communication busses is reduced, even though signalintegrity is partially compromised. Encoding techniques can be used to detectcorrupted data which is then retransmitted. The retransmission rate is an inputto a closed-loop DVS control scheme, which sets the voltage swing at a trade-offpoint between energy saving and latency penalty (due to data retransmission).
The On-Chip Network (OCN) for low power heterogeneous SoC platformsillustrated in [9] employs some advanced techniques for low-power physical in-terconnect design. OCN consists of global links connecting clusters of tightly-connected IPs which are several millimeters long. By using overdrivers, clockedsense amplifiers and twisted differential signaling, packets are transmitted reli-ably with less than 600 mV swing. The size of a transceiver and the overdrivevoltage are chosen to obtain a 200 mV separation at the receiver end. A 5 mm
216
global link of 1.6um wire-pitch can carry a packet at 1.6GHz with 320ps wire-delay and consumes 0.35pJ/bit. On the contrary, a full-swing link consumes upto 3x more power and additional area of repeaters.
An on-chip serialization technique [10] is also used in OCN, thus significantlyreducing area. However, the number of signal transitions on a link is increasedsince the temporal locality between adjacent packets is removed. An ad-hocserialized low-energy transmission coding scheme was therefore designed as anattempt to exploit temporal locality between packets. The encoder generates a’1’ only when there is difference between a current packet and a previous packetbefore it is serialized. The decoder then uses this encoded packet to reconstructthe original input, using its previously stored packet. A 13.4% power savingis obtained for a multimedia application. The power overhead associated withthe encoder/decoder is only 0.4mW.
Nevertheless, as the technology trends lead us to use smaller voltage swingsand capacitances, the upset probabilities will rise. Thus the trend toward fasterand lower-power communication may decrease reliability as an unfortunateside effect. Reliability bounds as voltages scale can be derived from theoretical(entropic) considerations [19] and can be measured also by experiments on realcircuits.
Finally, another key physical-layer issue is synchronization. In fact, globalsynchronization signals (i.e., clocks) are responsible for a significant fractionof the power budget in digital integrated systems. Alternative on-chip synchro-nization protocols that do not require the presence of a global clock have beenproposed in the past [30, 11] but their effectiveness has not been studied indetail from the energy viewpoint.
In the OCN NoC [9], a programmable power management unit provides fourclocks with PLL; 1.6GHz for the OCN, 800 MHz for schedulers, 100MHz forprocessors and 50 MHz for peripherals. Those clock frequencies are scalableby software for power-mode control and also for optimal operation of eachapplication.
12.3 SYSTEM INTERCONNECT ARCHITECTURE
Designing the architecture for an on-chip interconnect requires choices at ahigher level of abstraction with respect to physical interconnect design, but alsowith a stronger impact on energy dissipation.
Traditional shared buses try to overcome their energy inefficiency by meansof bus splitting [5]. The bus is split into smaller segments and proper bridgesare inserted to ensure communication between any two adjacent segments whenneeded. Thus, the load capacitance charged and discharged at each bus accessis reduced. Most commercial shared buses make use of this solution, includingAMBA bus [1] and IBM CoreConnect [33]. They split the bus based on the
217
characteristics of the connected masters and slaves (e.g. high performance coresversus slow peripherals). More advanced bus specifications (such as AMBAMulti-Layer [34]) allow to group IP cores into clusters, and this can be donebased on their interaction during application execution.
As an example, OCN [9] exploits locality of IP cores by grouping theminto clusters, and a crossbar switch is used for intra-cluster packets, perform-ing buffer-less cut-through switching. A round-robin scheduling of the switchensures fairness and starvation-freedom to OCN. An n × n crossbar fabriccomprises n2 crosspoint junctions which contain NMOS pass-transistors. Ina conventional crossbar fabric, each input driver wastes power to charge twolong wires (horizontal and vertical) and 2n transistor-junction-capacitors. OCNemploys a crossbar partial activation technique. By splitting the crossbar fab-ric into 4 × 4 tiles, input and output wires can be divided into four. A gatedinput driver at each tile is activated only when the scheduler grants access tothe tile. The output signal does not propagate to other tiles to reduce the powerconsumption on the vertical wire. A 43% power saving is obtained in a 16×16crossbar switch fabric with a negligible area overhead.
12.3.1 Network topology
Energy considerations might affect the on-chip network topology selectionprocess, as showed by the architectural choices made in the design of recentlyproposed NoC solutions.
Again, the OCN case is very instructive. In fact, a star topology guaranteeingconstant and minimum switch hop counts between every communicating IP wasadopted in an early implementation [10]. However, a 1-level flat star topologyresults in a number of capacitive global wires that may cause long latencyand large power dissipation. Therefore, the most recent solution consists of ahierarchical SoC composed of clusters of tightly, star-connected IPs.
Octagon [21] on-chip communication architecture consists of 8 nodes and12 bi-directional links connected according to an octagonal topology. In thisway, communication between any pairs of nodes can be performed by at mosttwo hops. Moreover, Octagon exhibits higher aggregate throughput than ashared bus or crossbar interconnect, a simple, shortest past routing algorithmand less wiring than a crossbar interconnect. Octagon and OCN are examplesof network topologies that try to provide the highest degree of connectivitybetween network nodes while trying to minimize the number of hops, thereforetargeting high-performance and low-power NoC realizations.
Power-aware topology selection is briefly discussed in [31] with respect to theSoCIN NoC architecture. A mesh topology is compared with a torus one: theformer exhibits lower costs, while the latter reduces message latency. To avoidthe long wrapping-around links, with a very high associated capacitive load,
218
a folded torus topology can be used [13]. Such approach reduces the wiringlengths and the power consumption while allowing to improve the operatingfrequency of network channels.
A more detailed comparison between the power efficiency of a mesh and afolded torus topology is addressed in [13]. The power has been decomposedinto the power per hop (traversal of input and output controllers) and powerper wire distance travelled. The analysis shows that if wire transmission powerdominates per-hop power, the mesh is more power efficient. For the 16 tilenetwork considered in [13], the wire transmission power was estimated to besignificantly greater than per-hop power, however the power overhead of thetorus was small (less than 15%), and was counterbalanced by the benefits of itslarger effective bandwidth.
12.4 DATA LINK LAYER
The data-link layer abstracts the physical layer as an unreliable digital link,where the probability of bit upsets is non null (and increasing as technologyscales down). Furthermore, reliability can be traded off for energy [19]. Themain purpose of data-link protocols is to increase the reliability of the link upto a minimum required level, under the assumption that the physical layer byitself is not sufficiently reliable.
An effective way to deal with errors in communication is to packetize data. Ifdata is sent on an unreliable channel in packets, error containment and recoveryis easier, because the effect of errors is contained by packet boundaries, anderror recovery can be carried out on a packet-by-packet basis.
For the realization of on-chip micro-networks, several error recovery mech-anisms developed for macroscopic networks can be deployed, but their energyefficiency should be carefully assessed in this context. As a practical example,consider two alternative reliability-enhancement techniques: error-correctingcodes and error-detecting codes with retransmission. A set of experimentsinvolved applying error correcting and detecting codes to an AMBA bus andcomparing the energy consumption in four cases [12]: 1) original unencodeddata; 2) single-error correction, 3) single-error correction and double-error de-tection, 4) multiple-error detection. Hamming codes were used. Note that incase 3, a detected double error requires retransmission. In case 4, using (n, k)linear codes, 2n − 2k errors patterns of length n can be detected. In all cases,some errors may go undetected and be catastrophic. Using the property of thecodes, it is possible to map the mean time to failure (MTTF) requirement intobit upset probabilities, and thus comparing the effectiveness of the encodingscheme in a given noisy channel (characterized by the upset probability) inmeeting the MTTF target.
219
4600 46 0.46 0.0046 4.6e-5 4.6e-7 4.6e-9
MTTF (days)
Figure 12.2. Energy efficiency for various encoding schemes
The energy efficiency of various encoding schemes varies: we summarizehere one interesting case, where three assumptions apply. First, wires arelong enough so that the corresponding energy dissipation dominates encod-ing/decoding energy. Second, voltage swing can be lowered until the MTTFtarget is met. Third, upset probabilities are computed using a white Gaussiannoise model [18]. Figure 12.2 shows the average energy per useful bit as afunction of the MTTF (which is the inverse of the residual word error proba-bility). In particular, for reliable SoCs (i.e., for MTTF = 1 year), multiple-errordetection with retransmission is shown to be more efficient than error-correctingschemes. We refer the reader to [12] for results under different assumptions.
Another important aspect affecting the energy consumption is the media ac-cess control (MAC) function. Currently, centralized arbitration schemes arewidely adopted [1, 14] for the serialization of bus access requests. Unfortu-nately, central arbiters are instance-specific and therefore poorly scalable. Infact, the energy cost of communicating with the arbiter, and hardware complex-ity of the arbiter itself scale up more than linearly with the number of bus masters.The selection of a specific arbitration algorithm impacts both performance andpower consumption [32]. Alternative multiplexing approaches, such as code di-vision multiplexing, are actively investigated for on-chip communication [27].However, research in this area is just burgeoning, and significant work is neededto develop energy-aware media-access-control for future micro-networks.
Arbitration mechanisms are required also in the implementation of NoCswitches to address contention resolution problems such as: prioritizing oneout of multiple input channels whose packets have to be directed to the same
220
output channel or multiplexing multiple virtual channels onto the same physicaloutput link.
12.5 NETWORK LAYER
At the network layer, packetized data transmission can be customized by thechoice of switching and routing algorithms. The former establishes the type ofconnection while the latter determines the path followed by a message throughthe network to its final destination.
In circuit switching data and control are separated: control is provided to thenetwork just to set up a connection over which all subsequent data is transportedin a connection-free fashion.
On the contrary, packet-switched on-chip networks naturally offer best ef-fort services, as contention takes place at the granularity of individual packets.Packet arrival cannot be predicted and contention has to be resolved dynami-cally: power-hungry data storage is required at the routers for this purpose, andthe provision of guarantees is complicated. However, a better link utilization isachieved and error control is made easier.
The most promising packet switching technique for NoC application is worm-hole switching. It was originally designed for parallel computer clusters [17]because it achieves the minimal network delay and requires fewer buffers. Inwormhole switching, each packet is further segmented into flits (flow controlunit). The header flit reserves the routing channel of each switch, the body flitswill then follow the reserved channel, the tail flit will later release the channelreservation.
One major advantage of wormhole switching is that it does not require thecomplete packet to be stored in the switch while waiting for the header flit tobe routed to the next stages. Wormhole switching not only reduces the store-and-forward delay at each switch, but it also requires much less buffer spaces.Because of these advantages, wormhole switching is an ideal candidate tech-nique for on-chip interconnect networks [13], although deadlock and livelockare potential problems that need to be taken care of [17, 15].
Routing algorithms can be static (packets injected into the network alreadyinclude routing information and only minimum header processing is requiredat the switches) or dynamic (routing decisions are dynamically taken at theswitches). These latter policies allow packet routes to adapt to network condi-tions, and therefore trade-off the energy savings obtained in this way with theincreased switch complexity and related energy dissipation. Next, a compari-son between energy efficiency of routing techniques is provided as an exampleof network-level design decisions for low power.
Contention-Look-Ahead Routing A contention-look-ahead routing schemeis the one where the current routing decision is helped by monitoring the adjacent
221
switches, thus possibly avoiding or reducing blockages and contention in thecoming stages.
A contention-aware routing scheme is described in [22]. The routing decisionat every node is based on the “stress values” (the traffic loads of the neighbors)that are propagated between neighboring nodes. This scheme is effective inavoiding “hot spots” in the network. The routing decision steers the packets toless congested nodes.
To solve the contention problems in wormhole switching schemes, acontention-look-ahead routing algorithm can be used, which “foresees” thecontention and delays in the coming stages using a direct connection from theneighboring nodes. The major difference from [22] is that information is han-dled in flits, and thus large and/or variable size packets can be handled withlimited input buffers. Furthermore, because it avoids contention between pack-ets and requires much less buffer usage, the latter contention-look-ahead routingscheme can greatly reduce the network power consumption.
At every intermediate stage, there may be many alternate routes to go tothe next stage. We call the route that always leads the packet closer to thedestination a profitable route. Conversely, a route that leads the packet awayfrom the destination is called misroute [17]. In mesh networks, profitable routesand misroutes can be distinguished by comparing the current node ID with thedestination node ID.
Profitable routes will guarantee a shortest path from source to destination.Nevertheless misroutes do not necessarily need to be avoided. Occasionally,the buffer queues in all available profitable routes are full, or the queues are toolong. Thus detouring to a misroute may lead to a shorter delay time. Underthese circumstances, a misroute may be more desirable.
It is interesting to compare the contention-look-ahead routing algorithm withdimension order routing – a routing scheme that always routes the packets on onedimension first, upon reaching the destination row or column, then switches tothe other dimension until reaching the destination. Dimension ordered routingis deterministic and guarantees shortest path, but it cannot avoid contention.
The contention-look-ahead routing will reduce the power consumption onthe buffers because it can “foresee” the contention in the forthcoming stages andshorten the buffer queue length. On the contrary, dimension-ordered routingalways steers the packets along the shortest path, while contention-look-aheadrouting may choose the misroute when contention occurs and therefore hasa larger average hop count per packet. This translates to more power on theinterconnect.
Finally, the contention-look-ahead routing switch needs more logic gatesthan dimension-ordered routing. However, simulation results show that with16 RISC processors on a 4x4 mesh interconnect, contention-look-ahead rout-
222
Figure 12.3. Cache and Memory Energy Decrease as Packet Payload Size Increases
ing reduces the total network power by about 15% with 16-flit buffers. Thereduction is more significant with larger buffer sizes.
12.6 TRANSPORT LAYER
At the transport layer, algorithms deal with the decomposition of messagesinto packets at the source and their assembly at destination. The choice ofinformation decomposition into packets or flits, as well as the choice of packetsize can heavily impact energy efficiency. Next, we will use the shared-memoryMPSoC as a case study to analyze the packet size trade-offs both qualitativelyand quantitatively. The system architecture consists of an on-chip intercon-nect which provides connectivity to nodes composed by a RISC processor, itscaches, a local memory reachable by means of a local bus and the network in-terface. The MPSoC power consumption originates from three sources: 1) thenode processor power consumption, 2) the cache and shared memory powerconsumption, and 3) the interconnect network power consumption. We willstart first from the cache and memory analysis.
Cache and memory power consumption Whenever there is a cache miss,the cache block content needs to be encapsulated inside the packet payloadand sent across the network. In shared-memory MPSoC, the cache block sizecorrelates with the packet payload size. Larger packet sizes will decrease thecache miss rate, because more cache content can be updated in one memoryaccess. Consequently, both cache energy consumption and memory energyconsumption will be reduced. This relationship can be seen from Fig. 12.3.It shows the energy consumption of cache and memory under different packetsizes. The energy in the figure is normalized to the value of 256Byte, whichachieves the minimum energy consumption.
223
Figure 12.4. Network and Total MPSoC Energy Consumption under Different Packet PayloadSizes
Interconnect network power consumption The power consumption ofpacketized dataflow on MPSoC network is determined by the following threefactors: 1) the number of packets on the network, 2) the energy consumed byeach packet on one hop, and 3) the number of hops each packet travels. Wesummarize these effects and list them below:
1 Packets with larger payload size will decrease the cache miss rate andconsequently decrease the number of packets on the network.
2 Larger packet size will increase the energy consumed per packet, becausethere are more bits in the payload.
3 Larger packets will occupy the intermediate node switches for a longertime, and cause other packets to be re-routed to non-shortest paths. Thisleads to more contention that will increase the total number of hops neededfor packets traveling from source to destination.
Actually, increasing the cache block size will not decrease the cache missrate proportionally. Therefore, the decrease of packet count cannot compensatefor the increase of energy consumed per packet caused by the increase of packetlength. Larger packet size also increases the hop counts on the datapath. Fig.12.4a shows the combined effects of these factors. The values are normalizedto the measurement of 16Byte. As packet size increases, energy consumptionon the interconnect network will increase.
The total energy dissipated on the MPSoC is shown in Fig. 12.4b. It clearlydecreases as packet size increases. However, when the packets are too large, asin the case of 256Byte in the figure, the total MPSoC energy will increase. Thisis because when the packet is too large, the increase of interconnect networkenergy will outgrow the decrease of energy on cache and memories.
224
West
East
South
North
Decoder
Inputs
ControllerFrequency
Voltage
PC
North
West
East
South
Outputs
North to South & East
Instruction Memory
Core
Crossbar
Core−ports
ExampleCommunication
Pattern
LocalConfig.Line
(a) ASOC Core Interface
producer consumerP1 P2
wordsmemory
write operation read operation
Shared memory
counter
(b) Producer-consumer pair
Figure 12.5. Communication-based power management
12.7 SYSTEM AND APPLICATION LAYERS
In the context of highly integrated on-chip multi-processors, lowering supplyvoltage of the cores reduces power quadratically but also results in a perfor-mance degradation which can be tolerated only if it does not impact performancebeyond a critical, application-dependent threshold. Given the key role playedby on-chip communication with respect to MPSoC performance, the conceptof communication-based power management (CBPM) has been introduced. Itconsists of integrating the system-level power management functionality intothe communication architecture, which binds the system components together,thus eliminating the need for separate power management entities. Second, dueto its connectivity, the communication architecture can gather information (suchas the execution states of system components) required to make power man-agement decisions. Finally, since the communication architecture schedulesinter-component communications, it can control the timing of a component’spower modes, thus regulating the component’s (and therefore the system’s)power profile.
Multiple implementations of this concept are feasible [38, 6], and two rele-vant examples will be hereafter described. The first one is represented by theAdaptive System-on-Chip (ASOC) illustrated in [6], which has been used tobuild a backbone for power-aware signal processing cores [39]. ASOC abilityto provide dynamic voltage and frequency scaling is due to the architectureof the network interface of the cores, illustrated in Figure 12.5(a) [39]. Thecore interface uses a synchronized global communication schedule to managecommunications through each tile. The instruction memory holds a list of thecommunication patterns required at run-time. A program counter (PC) fetchesthese patterns in succession and a decoder converts them into switch settings for
225
a crossbar, that routes data between the local core and neighboring tiles (North,East, South or West). Moreover, at each core, frequency and voltage are au-tomatically adjusted. A subsystem uses up/down counters to track the datatransfer rate between core and interconnect. Blocked or unsuccessful transferscause the count to increase, while successful transfers decrease the value. Ifthe core input port is blocked consecutively, the core is running too slowly withrespect to its predecessors. If the core output port is consecutively blocked, thecore is running too quickly for its successors. In either case, these counters sendtrigger signals to the core configuration unit to increase or decrease the coreclock. The new frequency setting automatically selects a new supply voltagevalue.
A similar approach can be applied to pipelined signal processing applica-tions, wherein a sequence of computation stages exchange the results of theirprocessing in a pipelined fashion. From an hardware viewpoint, the systemmight consist of cascaded producer-consumer pairs communicating by meansof a shared memory. If producer and consumer are not well synchronized,energy-inefficient synchronization mechanisms could be triggered. For in-stance, if the consumer expects input data to process while the producer is notready to output the result of its computation yet, the consumer keeps pollingon a semaphore until its input data is available in the shared memory. Thismechanism wastes significant amounts of energy, and should be avoided asmuch as possible. A solution to keep producer-consumer pairs synchronizedis reported in Figure 12.5(b). Shared memory can be abstracted as a queueand a memory access counter keeps track of the queue level. When a lowerthreshold is crossed, it means that the producer is too slow or the consumertoo fast, and frequency/voltage scaling can be applied for balancing data pro-duction or consumption rate. The opposite holds when an upper threshold iscrossed. Counter monitoring might be carried out by a proper power man-agement hardware module connected to the bus, with the ability to program(through a register) the clock frequency generator of the cores. This couldbe done continuously or periodically at discrete times, in order to amortizethe frequency switching cost. Worst-case power savings with respect to staticfrequency selection (power-optimized for a particular application) amounts to12%.
12.8 APPLICATION-SPECIFICNETWORKS-ON-CHIP
Customizing MPSoC architectures and tailoring them to a specific appli-cation domain is a very promising approach to system energy minimization.It takes its steps from the optimization techniques used in some SoC designmethodologies, that explicitly target applications from a specific domain: their
226
main features (control flow, data organization and type of processing) are eval-uated and exploited for a power-aware architecture customization [8].
For customized NoCs to be successful, however, developers must select theappropriate domain-specific architecture and map the system’s communicationrequirements onto it [7]. This is a non-trivial task, in that an optimized net-work instance has to be derived by analyzing the application communicationrequirements, and by comparing a number of alternative interconnect solutions.
Moreover, reusing the components of a given NoC architecture across dif-ferent designs (and therefore network instances) becomes feasible provided thenetwork building blocks (network interfaces, switches, switch-to-switch links)are designed as soft macros.
Some NoC architectures proposed in the literature were built around thisconcept [6, 4, 3, 2]. Only two of them are mentioned for the sake of brevity.Quality of Service NoC (QNoC) [3] is a NoC framework wherein QoS and costmodel for communications in SoCs are first defined, and related NoC archi-tecture and design process are then derived. SoC inter-module communicationtraffic is classified into 4 classes of service: Signaling (control signals), Real-Time, RD/WR (for short data access) and Block-Transfer (for large data bursts).By analyzing the communication traffic of the target SoC, QoS requirements(in terms of delay and throughput) for each of the four service classes are de-rived. A customized QNoC architecture is then created by modifying a genericnetwork architecture (two-dimensional planar mesh, fixed shortest-path multi-class wormhole switching). The customization process minimizes the networkcost (in area and power) while maintaining the required QoS and works asfollows: the SoC modules are placed so as to minimize spatial traffic density,unnecessary mesh links and switching nodes are removed, and bandwidth isallocated to the remaining links and switches according to their relative load sothat link utilization is balanced.
Finally, Xpipes NoC architecture [2] is a library of highly parameterizablenetwork components which are design-time tunable and composable to get cus-tomized domain-specific architectures. Xpipes has been designed with high-performance in mind, and this has been achieved by means of deeply pipelinedswitches, pipelined links to decouple link throughput from link delay, virtualoutput buffering. The network interface implements OCP standard signalingand the look-up tables required by static routing algorithms. The network inher-ently provides best-effort services and targets multi-gigahertz heterogeneousMPSoCs, wherein irregular network topologies with links of uneven lengthmight be required.
Next, an instructive case study about application-specific NoC instances andtheir potentials for energy savings is reported, leveraging the Xpipes synthesisflow.
227
au cpumed rast
adsp
idct,etc
upsamp bab risc
sram1 sram2
600600.540
40190
670 500
250
173
32
0.5910
vu
sdram
(a) MPEG4 core graph
pad
pred samp
mem
memstripe
acdc
vop
Armscaninv up
voprec
vld
16
94
313
16
49
27
500
313 300
353
357
362362362
70idctiquan
runle dec
(b) VOPD core graph
Figure 12.6. Core Graphs of Video Processing Applications
12.8.1 Case study
A core graph representation of the application is the input to Xpipes-basedsynthesis flow (called NetChip). The design and generation of a customizedNoC is achieved by means of two tools: SUNMAP, which performs the networktopology mapping and selection functions, and ×pipesCompiler, which per-forms the topology generation function. SUNMAP produces a mapping of coresonto various NoC topologies that are defined in a topology library. The map-pings are optimized for the chosen design objective (such as minimizing area,power or latency) and satisfy the design constraints (such as area or bandwidthconstraints). SUNMAP uses floorplanning information early in the mapping pro-cess to determine the area-power estimates of a mapping and to produce feasiblemappings (satisfying the design constraints). The tool supports various routingfunctions (dimension ordered, minimum-path, traffic splitting across minimum-paths, traffic splitting across all paths) and chooses the mapping onto the besttopology from the library of available ones. A design file describing the chosentopology is input to the ×pipesCompiler, which automatically generates theSystemC description of the network components (switches, links and networkinterfaces) and their interconnection with the cores. A custom hand-mappedtopology specification can also be accepted by the NoC synthesizer, and thenetwork components with the selected configuration can be generated accord-ingly.
NetChip was applied to two different video processing applications: VideoObject Plane Decoder (VOPD - mapped onto 12 cores), MPEG4 decoder (14cores). These are high-end video-processing applications and the hardware-software partitioning of the applications is presented in [35, 36]. The coregraphs of these applications is presented in Figure 12.6. The maximum linkbandwidth for the NoCs is conservatively assumed to be 500 MB/s.
The results of mapping VOPD onto various topologies are presented in Fig-ure 12.7. As seen from Figure 12.7(a), the butterfly topology (4-ary 2-fly) has theleast communication delay out of all topologies, the least number of switches,
228
Msh Trs Hyp Cls Bfly0
1
2
3
AvgNoHops
(a) Avg hop delay
Mesh Torus Hyp Cls Bfly0
5
10
15
20
25
30
35
40
45
SWs
SwsLnks
&
Lnks
(b) Resource Util
Msh Trs Hyp Cls Bfly0
20
40
60Area(mm2)
(c) Design Area
Msh Trs Hyp Cls Bfly0
100
200
300
400
500
Power(mW)
(d) Design Power
Figure 12.7. Mapping Characteristics of VOPD
up
iQuantiDCTAC/DCPredict
scan
inverse
decoderlength
runVLD
samp
s1
− repeaters3 − 5x5s2 − 4x4s1 − 3x3
s1s2
s2
s3s2
s2s3s2
s1s2s1
PaddingMemVOP
coreARM
MemStripe
reconstrVOP
(a) Mesh NoC
samp
PredictAC/DCiDCT iQuant
scaninverse
decoderlengthrun
VLD
up
− repeater
s1 − 3x3s2 − 3x2s3 − 2x3
S2S3
Padding
S2 S1
Mem
Stripe
Mem
VOPcoreARM
S2
reconstrVOP
(b) Appln Specific NoC
1.6 1.8 2 2.2 2.4 2.626
28
30
32
34
36
38
40
42
44
Avg P
ack L
at (
Cy)
CustMesh
BW (GB/s)
(c) VOPD Avg Lat
Figure 12.8. Mesh and Custom Mappings of Video Object Plane Decoder
but has more links when compared to mesh, torus or hypercube. The largepower savings achieved by the butterfly network (Figure 12.7(d)) is attributedto the fact that there are fewer switches and smaller number of hops for commu-nication. Moreover, all the switches are 4x4, while the direct topologies have5x5 switches. The average link length in the butterfly network (obtained fromfloorplanner) was observed to be longer than the link lengths (around 1.5×) ofdirect networks. However, as the link power dissipation is much lower than theswitch power dissipation, we get large power savings for the butterfly network.The smaller number of switches and smaller switch sizes also account for thelarge area savings achieved by the butterfly network. Thus, butterfly is the besttopology for VOPD. The performance gains for the butterfly over other topolo-gies may be surprising, but careful inspection shows that the butterfly networktrades-off path diversity for network switches with average hop delay. On thecontrary, the same kind of analysis shows that a mesh topology is more suitablefor MPEG4 than other topologies.
12.8.2 Generating Custom Topologies
For custom topologies, the mapping and generation phases of the tool canbe skipped and ×pipesCompiler can be directly invoked on the input design.A custom hand-tuned NoC for the VOPD is presented in Figure 12.8(b). In
229
the VOPD, about half the cores communicate to more than a single core. Thismotivates the configuration of this custom NoC, having less than half the numberof switches than the mesh NoC. NetChip area and power reports relative to thecustom NoC were automatically obtained. Significant area (5x) and powerimprovements (2x) were noticed with the custom NoC as fewer, smaller sizeswitches are used with respect to the mesh network.
SystemC simulation of the NoC models allowed to assess their performance.The variation of average packet latency (for 64B packets, 32 bit flits and 7 cycleswitch delay) with link bandwidth is showed in Figure 12.8(c). Application-specific NoC has lower packet latency as the average number of switch and linktraversals is lower. Moreover, the latency increases more rapidly for the meshNoC with decrease in bandwidth. With the custom NoC, an average of 25%savings in latency (measured at the minimum plotted BW value) is achieved.
12.9 CONCLUSIONS
This chapter focuses on low power design techniques for NoC-based gi-gascale MPSoCs. Several open problems were described at various layers ofthe communication stack, and the basic strategies to effectively tackle themwere sketched. Finally, the large potentials for energy savings provided by theimplementation of customized, domain-specific NoCs have been discussed.
230
References[1] P. Aldworth, “System-on-a-Chip Bus Architecture for Embedded Applications,” IEEE International
Conference on Computer Design, pp. 297-298, 1999.
[2] “×pipes: a Latency Insensitive Parameterized Network-on-chip A rchitecture For Multi-ProcessorSoCs”, M.Dall’Osso, G.Biccari, L.Giovannini, D.Bertozzi, L.Benini, Int. Conf. on Computer Design,pp.536-541, October 2003.
[3] E.Bolotin, I.Cidon, R.Ginosar, A.Kolodny, "QNoC: QoS architecture and design process for Networkon Chip", Journal on Systems Architecture, Special Issue on Networks on Chip, December 2003.
[4] I.Saastamoinen, D.S.Tortosa, J.Nurmi, "Interconnect IP Node for Future System-on-Chip Designs",IEEE Int. Work. on Electronic Design, Test and Applications, pp.116-120, January 2002.
[5] C.T. Hsieh, M. Pedram, ”Architectural Energy Optimization by Bus Splitting,” IEEE Trans. CAD,Vol.21, issue 4, pp.408-414, April 2002.
[6] J.Liang, S.Swaminathan, R.Tessier,”aSOC: A Scalable, Single-Chip Communication Architecture,”IEEE Int. Conf. on Parallel Architectures and Compilation Techniques, pp.37-46, October 2000.
[7] S.Murali, G.De Micheli, "Bandwidth-Constrained Mapping of Cores onto NoC Architectures", De-sign Automation and Testing in Europe, 2004, pp.20896-20901.
[8] L.Bisdounis, C.Dre, S.Blionas, D.Metafas, A.Tatsaki, F.Ieromninon, E.Macii, P.Rouzet, R.Zafalon,L.Benini "Low-Power System-on-Chip Architecture for Wireless LANs," IEE Proc.-Comput. Digit.Tech., Vol.151, no1, January 2004.
[9] K. Lee, S.J. Lee, S.E. Kim, H.M. Choi, D. Kim, S. Kim, M.W. Lee, H.J. Yoo, "A 51mW 1.6GHzOn-Chip Network for Low Power Heterogeneous SoC Platform", IEEE Int.Solid-State Circuits Con-ference, pp.1-3, 2004.
[10] S.J. Lee et al., "An 800MHz Star-Connected On-Chip Network for Application to Systems on a Chip", IEEE Int.Solid-State Circuits Conference, pp.468-469, February 2003.
[11] W. Bainbridge, S. Furber, “Delay insensitive system-on-chip interconnect using 1-of-4 data encod-ing,” IEEE International Symposium on Asynchronous Circuits and Systems, pp. 118-126, 2001.
[12] D. Bertozzi, L. Benini and G. De Micheli, “Low-Power Error-Resilient Encoding for On-chip DataBusses,” DATE, International Conference on Design and Test Europe Paris, 2000, pp. 102-109.
[13] Dally, W.; Towles, B.; “Route Packets, Not Wires: On-Chip Interconnection Networks” 38th DesignAutomation Conference, 2001. Proceedings
[14] B. Cordan, “An efficient bus architecture for system-on-chip design,” IEEE Custom Integrated Cir-cuits Conference, pp. 623–626, 1999.
[15] Dally, W.J; Aoki, H. “Deadlock -free adaptive routing in multicomputer networks using virtualchannels” IEEE Trans. on Parallel and Distributed Systems, April 1993
[16] W. Dally and J. Poulton, Digital Systems Engineering, Cambridge University Press, 1998.
[17] J. Duato, S. Yalamanchili, L. Ni, Interconnection Networks: an Engineering Approach. IEEE Com-puter Society Press, 1997.
[18] R. Hegde, N. Shanbhag, “Toward Achieving Energy Efficiency in Presence of Deep SubmicronNoise,” IEEE Transactions on VLSI Systems, pp. 379–391, vol. 8, no. 4, August 2000.
[19] R. Hegde, N. Shanbhag, “Toward achieving energy efficiency in presence of deep submicron noise,”IEEE Transactions on VLSI Systems, pp. 379–391, vol. 8, no. 4, August 2000.
[20] R. Ho, K. Mai, M. Horowitz, “The Future of wires,” Proceedings of the IEEE, January 2001.
[21] Karim, F.; Nguyen, A.; Dey, S. “On-chip Communication Architecture for OC-768 Network Proces-sors” 38th Design Automation Conference, 2001. Proceedings
[22] E. Nilsson “Design and Implementation of a Hot-Potato Switch in a Network on Chip” Master ofScience Thesis, LECS, Royal Institute of Technology
[23] Li Shang and Li-Shiuan Peh and Niraj K. Jha, “Dynamic Voltage Scaling with Links for PowerOptimization of Interconnection Networks,” HPCA - Proceedings of the International Symposiumon High Performance Computer Architecture, Anaheim, February 2003, pp. 91-102.
[24] Singh, J. P.; Weber, W.; Gupta, A,; “SPLASH: Stanford Parallel Applications for Shared-Memory”Computer Architecture News, vol. 20, no. 1
[25] J. Walrand, P. Varaiya, High-Performance Communication Networks. Morgan Kaufman, 2000.
231
[26] F. Worm, P. Ienne, P. Thiran and G. De Micheli, “ An Adaptive Low-power Transmission Scheme forOn-chip Networks,” ISSS, Proceedings of the International Symposium on System Synthesis, Kyoto,October 2002, pp. 92-100.
[27] R. Yoshimura, T. Koat, S. Hatanaka, T. Matsuoka, K. Taniguchi, “DS-CDMA wired bus with simpleinterconnection topology for parallel processing system LSIs,” IEEE Solid-State Circuits Conference,pp. 371-371, Jan. 2000.
[28] Ye, T. T.; Benini, L.; De Micheli, G.; “Packetized On-Chip Interconnect Communication Analysisfor MPSoC” Design Automation and Test in Europe, DATE 2003 Proceedings
[29] H. Zhang, V. George, J. Rabaey, “Low-swing on-chip signaling techniques: effectiveness and robust-ness,” IEEE Transactions on VLSI Systems, vol. 8, no. 3, pp. 264-272, June 2000.
[30] H. Zhang, M. Wan, V. George, J. Rabaey, “Interconnect architecture exploration for low-energyconfigurable single-chip DSPs,” IEEE Computer Society Workshop on VLSI, pp. 2-8, 1999.
[31] C.H. Zeferino, A.A. Susin, "SoCIN: A Parametric and Scalable Network-on-Chip," Symposium onIntegrated Circuits and Systems Design SBCCI’03, pp. 169-174, September 2003.
[32] F. Poletti, D. Bertozzi,L. Benini,A. Bogliolo, "Performance Analysis of Arbitration Policies forSoC Communication Architectures," Journal on Design Automation for Embedded Systems, Kluwer,pp. 189-210, 2003.
[33] IBM CoreConnect bus architecture,”http://www-3.ibm.com/chips/products/coreconnect”
[34] AMBA Multi-Layer AHB and AHB-Lite,”http://www.arm.com/products/solutions/AMBAAHBandLite.html”
[35] E.B.Van der Tol, E.G.T.Jaspers,"Mapping of MPEG-4 Decoding on a Flexible Architecture Platform",SPIE 2002, pp. 1-13, Jan, 2002.
[36] E.G.T.Jaspers, et al.,"Chip-set for Video Display of Multimedia Information", IEEE Trans. on Con-sumer Electronics, Vol.4 5, No. 3, pp. 707-716, Aug, 1999.
[37] R.H. Havemann, J.A. Hutchby, ”High-Performance Interconnects: An Integration Overview”, Pro-ceedings of the IEEE, Vol.89, no5, pp.586-601,May 2001
[38] K. Lahiri, A. Raghunathan, S. Dey ,"Communication Architecture Based Power Management forBattery Efficient System Design", Proc. ACM/IEEE DAC, pp.691-696, 2002.
[39] A. Laffely, J. Liang, R. Tessier, W. Burleson ,"Adaptive System on Chip (aSoC): a Backbone forPower-Aware Signal Processing Cores", Int. Conf. on Image Processing, pp.105-108 (III), 2003.
[40] V. Raghunathan, M.B. Srivastava, R.K. Gupta ,"A Survey of Techniques for Energy Efficient On-ChipCommunication", DAC 2003, pp.900-905, June 2003.
232
Chapter 13
SYSTEM LEVEL POWER MODELING AND
SIMULATION OF HIGH-END INDUSTRIAL
NETWORK-ON-CHIP
Andrea Bona , Vittorio Zaccaria and Roberto Zafalon
STMicroelectronics
Abstract Today’s System on Chip (SoC) technology can achieve unprecedented
computing speed that is shifting the IC design bottleneck from computation
capacity to communication bandwidth and flexibility. This chapter presents an
innovative methodology for automatically generating the energy models of a
versatile and parametric on-chip communication IP (STBus). Eventually, those
models are linked to a standard SystemC simulator, running at BCA and TLM
abstraction level. To make the system power simulation fast and effective, we
enhanced the STBus class library with a new set of power profiling features
(“Power API”), allowing performing power analysis either statically (i.e.: total
avg. power) or at simulation runtime (i.e.: dynamic profiling). In addition to
random patterns, our methodology has been extensively benchmarked with the
high-level SystemC simulation of a real world multi-processor platform (MP-
ARM). It consists of four ARM7TDMI processors accessing a number of
peripheral targets (including several banks of SRAMs, Interrupt’s slaves and
ROMs) through the STBus communication infrastructure. The power analysis
of the benchmark platform proves to be effective and highly correlated, with
an average error of 2% and a RMS of 0.015 mW vs. the reference (i.e. gate
level) power figures. The chapter ends presenting a new and effective
methodology to minimize the Design of Experiments (DoE) needed to
characterize the above power models. The experimental figures show that our
DoE optimization techniques are able to trade off power modeling
approximation with characterization cost, leading to a 60% average reduction
of the sampling space, with 20% of maximum error.
Keywords: Network-on-Chip power analysis, communication based low power design,
system-level energy optimization.
233
13.1 INTRODUCTION
Embedded computing systems are on the way to provide a number of
new services that will arguably become common practice in the next few
years. The most important of these are (i) multimedia (audio/video
streaming) capabilities in personal communicators, (ii) huge computing
power (especially from clusters of processors) and storage size, (iii) high rate
accessibility from mobile terminals.
Today’s System on Chip (SoC) technology can achieve unprecedented
computing speed that is shifting the IC design bottleneck from computation
capacity to communication bandwidth and flexibility.
SoC’s designers need to leverage on pre-validated components and IPs
such as processor cores, controllers and memory arrays. Design
methodology will further support IP re-use in a plug-and-play fashion,
including buses and hierarchical interconnection infrastructures.
SoCs will have to provide a functionally correct, reliable operation under
data uncertainty and noisy signaling. The on-chip physical interconnection
will be a limiting factor for both performance and energy consumption, also
because the demand for component interfaces will steadily scale-up in size
and complexity.
In this chapter, we will present a thorough methodology for automatically
building the energy model of a Network-on-Chip (NoC) IP at the
BCA/Transaction level, in order to allow power profiling of an entire
platform since the very early stages of the system design, often when only a
software model of the system does exist.
The chapter is organized as follows: Section 13.2 introduces a short
background on Network-on-Chip. Section 13.3 illustrates the STBus
versatile interconnect IP as an industrial example of NoC infrastructure.
Section 13.4 introduces the overall NoC power characterization and
estimation framework while Section 13.5 goes into details about our NoC’s
energy model. Section 13.6 presents the Design of Experiment policy and
Section 13.7 reports a significant set of figures about the model validation
and the experimental results, including a real-world platform simulation
case. Eventually, Section 13.8 presents a new and effective methodology to
minimize the Design of Experiments (DoE).
13.2 BACKGROUND
Although the main concepts and the terminology of Network-on-Chip
design has been introduced quite recently [1][2][3], both the industrial and
research communities have been starting to realize the strategic importance
of shifting the design paradigm of high-end digital IC from a deterministic,
234
wire-based interconnection of individual blocks and IPs, to a thorough
communication-based design methodology [4][7][9], aiming to face with
data packets and non-deterministic communication protocols in next
generation’s SoCs.
With the advent of 90nm and 65nm CMOS technology, the challenges to
fix the Network-on-Chip (NoC) issue “by design”, will need:
– To provide a functionally-correct, reliable operation of the
interconnected components by exploiting appropriate network infrastructure
and protocols, i.e. interconnections to be intended as “on chip micro-
network” [5][6][7], which is an adaptation of the OSI protocol stack [18].
– To achieve a fluid “flexibility vs. energy-efficiency” system
exploration, allowing an effective network centric power management
[8][11][12]. Unlike computation energy in fact, the energy for global
communication does not scale down with technology shrinking [3][4]. This
makes energy more and more dominant in communications.
Reaching those goals will be crucial to the whole semiconductor industry
in the next future, in order to face with the escalating range of signal
integrity and physical wiring issues, who are making the target IC reliability
harder and exponentially expensive to achieve. As of today, there is a limited
availability of tools able to consistently support this emerging design
methodology. Indeed, some high level models for functional/performance
system simulations (i.e. Bus Cycle Accurate and Transaction) are smoothly
coming up [13] across the design community. However, power predictability
of NoCs still remains an open issue.
Although NoC’s power estimation has been partially addressed in [10],
its low level modeling (i.e. gate and device level) and the extremely slow
simulation (i.e. 1000 cycle/s) makes it definitely unsuitable to face with any
system level SW/HW exploration task, which might easily need for
simulation speeds larger than 100 Kcycle/s.
13.3 ON-CHIP NETWORK: STBUS INTERCONNECT
STBus is versatile, high performances interconnect IP, allowing to
specify the communication infrastructure in terms of protocol, interface and
parametric architectures [14][15]. It comes with an automated environment
(STBus generation kit) suitable to support the whole design flow, starting
from the system-level parametric network specification, all the way down to
the mapped design and global interconnect floor-plan [16]. The protocol
modes supported by STBus are compliant with VSIA standard [19]. In fact,
they can scale up from Peripheral, to Basic and to Advanced mode,
conventionally named Type-1, Type-2 and Type-3, respectively. In this
work, we focus on the last 2 protocols (i.e. Type-2 and Type-3) since they
235
better fit with the high demanding communication resources required by
modern SoCs. More specifically, Type-2 supports pipelined split
transactions, where each transaction is composed by a pair of send and
receive packets (packet: a sequence of atomic messages called cells). On top
of the above features, Type-3 allows to manage out-of-order packet delivery.
The datapath’s width can range between 32, 64 and 128 bits.
The STBus architecture builds upon the node module, configurable
switch fabrics who can be instantiated multiple times to create a hierarchical
interconnect structure. The topology of the switch fabric can be selected by
choosing the number of resources dedicated to the request and the response
packets; for example a shared bus interconnect has only 1 request and 1
response resources at a time, while a full cross-bar has as many request and
response resources as the number of initiators and targets connected to the
node. Eventually, type converter and size converter modules can be adopted
to interface heterogeneous network domains working under different
protocols (i.e. Type-1, 2 and 3) and/or different data-path widths.
13.4 ENABLING ENERGY EXPLORATION FOR NOC
When dealing with multi-processors embedded systems, characterized by
tens of masters and slaves connected through a complex communication
infrastructure, energy estimation and optimization become of utmost
importance. As a matter of fact, although more effective than traditional
buses, NoCs are expected to make a relevant contribution to the area budget,
due to the growing complexity of packet routing and transaction
management policies affecting the interconnection’s control-path, and to the
switch fabric in charge of supporting the high speed data packet delivery.
Such a complexity has a cost in terms of energy consumption that should
be traded-off with the performance benefits. Network structures achieving
lower packet’s congestion (i.e. higher performance), are usually
characterized by larger data-path complexity in terms of number of
simultaneous routing resources available for packet broadcasting. For
example, a shared bus communication node can be slower (i.e. higher
congestion), yet less power consuming than a full crossbar switch-box, or,
the slot-reservation arbitration policy may overcome the limitation of Time
Division Multiple Access (TDMA) policy in case of asymmetric workloads in
a multi-processors platform. These questions need to account for energy
metric during the design exploration in order to find out the optimal platform
configuration to meet the performance constraints at minimum energy.
Exploration and optimization for SoC design are rapidly evolving
towards the analysis of abstract description models that mimic the main
operations of the system under analysis, including speed and power
236
behavior. According to the SystemC modeling scenario depicted in [13], the
abstraction levels that can be used to model the function/power/performance
of a communication-based system are the Functional un-timed level, the
Transaction level (TLM), the Bus Cycle Accurate level (BCA) and the Pin
Accurate – Cycle Accurate level (PA-CA). In short, while the Functional
level does not give any insight on the timing figures of the system, the
Transaction level only gives coarse time hints (e.g. total read/write time
slot), with no structural information on actual wires or pins. The BCA level
achieves cycle-accurate timing estimates, yet functionally accurate at the
boundaries, while the PA-CA goes down to a clock cycle timing with
structural pin-accurate description, at the expense of a much slower
simulation. In this chapter we introduce a consistent methodology for
automatic energy model’s building to fit most of the above abstraction levels
(i.e. Transaction, BCA, PA-CA), suitable to support the NoC’s power
estimation since the very early stages of the design exploration, when only a
C/C++ model of the system is usually available.
Eventually, the system simulation (developed in SystemC, in our case)
will rely on high-level profiling statistics to figure out the energy cost, by
means of an appropriate library of energy views and a dedicated procedural
interface (API). In the following, we will explain how the STBus energy
models are based on a set of parametric, analytic equations that are
individually accessed by the simulator to compute the eventual energy
figures (either statically or at simulation runtime).
13.4.1 Energy Characterization Flow
The energy macro-model of the whole STBus interconnection is
partitioned into sub-components, corresponding to each micro-architectural
block of the interconnection fabrics that are node, type-converter and size-
converter. For sake of simplicity, in this chapter we will show the results of
the node component. However, the same automatic flow is currently applied
to all of the components of STBus architecture. The proposed model relies
on the bus utilization rate, i.e. the number of cells traveling across the bus, as
well as on the interconnection topology (i.e. the number of masters/targets),
which need to be pre-characterized, once and for all, through an accurate
gate-level simulation for each target technology.
The power characterization flow consists of 4 major steps depicted in
Figure 13-1.
237
STBus Generation Kit
CoreConsultant/Design Compiler
SynthesisAbstract
Network
Topology
Gate-level Netlist
Testbench
GenerationVCS/PowerCompiler
Gate-level power est.
Power & High-level stats
Model Characterization
Power
Models
DB
Figure 13-1. STBus Power Characterization Flow
As already mentioned in section 3, the STBus Generation Kit allows the
designer to automatically synthesize a gate-level netlist starting from a
system-level parametric network specification. This is done by inferring the
corresponding RTL code and, then, synthesizing all the way down to the
mapped design [16]. Thus, an extensive set of gate-level power simulations
is launched within a Test-bench Generation suite, specifically tuned to fulfill
the many requirements imposed by the STBus protocols and, at the same
time, to sensitize the node under a wide range of traffic workloads. Actually,
the test-benches can be configured in terms of average latency per master
request and slave response and type of operations to be performed on the
bus. The operations can be divided in two categories (Load and Store) as
they can play with different operand sizes (from 1 to 32 bytes).
The last step of the flow in Figure 13-1 is the Model Characterization,
where each of the coefficients is computed to fit the high-level model (ref. to
next section 5 for details). The final models (one for each component and
target technology) are stored into a centralized Power Model Database. Sure
enough, the choice of experiments, the length of each simulation and the
test-benches adopted during the characterization campaign are crucial knobs
to be optimized before running the characterization flow, by means of a
suitable Design of Experiments (DoE: see section 6).
238
13.4.2 Hooking up the energy models into the system simulation
The STBus Generation Kit supports the generation, among the others, of
the SystemC model of each component, ready to be plugged into the target
SystemC simulation platform. The current release of the STBus Generation
Kit is compliant with BCA SystemC v2.0 descriptions [13]. In evolution, the
support for TLM is planned soon, according to the STBus roadmap. The
overall SystemC power estimation flow is outlined in Figure 13-2. To make
the system simulation environment fast an effective, an ad-hoc API has been
developed (SystemC Power API), together with a consistent library of
functions allowing to enhance the basic SystemC capabilities with a power
profiling feature, providing power analysis either statically (i.e.: total avg.
power) or at simulation runtime (i.e.: dynamic profiling). The latter is done
by computing a moving average power on a given time window (e.g. ten
clock cycles).
Energy Enhanced
SystemC Node
STBus Generation Kit
Abstract
Network
Topology
SystemC Power API
Power
Models
DB
Power
Models
DB
SystemC NodePower
Profile
Figure 13-2. Power Enhanced SystemC Simulation
By deriving the SystemC node classes and hooking them up to the
specific SystemC Power API, we achieve the energy flow enhancement. As a
matter of fact, energy-enhanced SystemC nodes provide an extremely fast
procedural interface to retrieve each set of model’s coefficients out from the
power model database as well as an effective power analysis during the
SystemC simulation run.
239
13.5 STBUS ENERGY MODEL
In this section, we introduce the power model for a generic configuration
n of a node. The configuration of an STBus node identifies a specific
instance out from the design space S:
>=<= TypedpsCprprrqrtinnS L ,,,,,,,| (1)
where i is the number of initiators, t is the number of targets, rqr is the
number of request resources, rpr is the number of response resources, p is
the type of arbitration policy (STBus has 7 arbitration policies), CL is the
output pin capacitance (range: CLmin= 4 Standard Loads ; CLmax=1 pF), dps is
the data-path size (range: 32, 64 and 128 bit) and Type is the protocol mode
(Type-2 and 3, in this case).
Based on an extensive experimental background, we recognize a fairly
linear relationship between node energy and the rate of sent and received
packet cells across all of the interconnection node’s ports. Such a behavior
matches with a set of random configuration samples across the entire design
space and it has been confirmed during the model validation phase (see
section 7).
The energy model for a generic configuration n of the STBus node is the
following:
( ) ( ) clkTCnPnE ⋅⋅= (2)
where P(n) is the average power consumption of the node during a
simulation of C clock cycles, with a clock period of Tclk. The power
consumption P(n) is a linear combination of three contributions, according
to the following equation:
( ) ( ) ( ) ( )C
rnP
C
rnPnBnP r
recs
sent ⋅+⋅+=(3)
where B(n) is the average base cost depending on the specific
configuration n of the node, Psent(n) is the additive power cost due to cell
sent from the masters to the slaves and rs is the total number of cells sent,
Preq(n) is the power cost due to each packet cells received by the masters, rr
is the total number of cells received by the masters and C is the number of
clock cycles. In essence, the power model characterization consists in
determining the value of the coefficients B(n), Psent(n) and Preq(n) for each
240
specific configuration n of the node. As formerly mentioned, this task is
performed by means of a polynomial regression over the set of experiments
given by DoE (see section 6). Although linear regression has been
successfully used to build the model’s coefficients so far, any higher order
models can be easily adopted should a better accuracy become an issue. The
experimental setup is generated with the goal of properly stressing rs and rr
over the whole range of variation. The total avg. switching activity coming
out from the test-benches is kept at 0.5. As far as the interconnection
capacitive load “CL“ is concerned, our model supports a linear interpolation
between CLmin and CLmax in order to provide a quite accurate estimation of the
switching power under the actual instance’s output load.
From a global viewpoint, the characterization campaign of STBus across
the whole design space may easily become a huge computing task. The
computational effort to power characterize STBus is similar or even larger
than the characterization of an industrial size ASIC library. The whole
comprehensive STBus design-space, in fact, would lead to more than
3.4*105 individual configurations to be characterized (i.e. RTL synthesis +
gate-level simulation + power measure). Such a number comes out from the
product of all the possible combinations of the STBus design subspaces (i.e.
8 initiators, 8 targets, 8 request and 8 response resources, 7 arbitration
policies, 2 load capacitances, 3 data path sizes and 2 types of protocols).
Running an exhaustive characterization is far to be feasible in a reasonable
time, even by leveraging on distributed computers. We decided to adopt a
response surface method approach to solve this problem. In this approach,
only a selected set of configurations are synthesized and characterized,
making the remaining set of coefficients derivable by accessing an
appropriate set of models (either analytic or look-up table) obtained through
response surface methods. Although this approach may lead to some
inaccuracy with the energy estimation process, the global accuracy can be
taken well under control while allowing a remarkable saving in
characterization effort.
13.6 OPTIMAL DESIGN OF EXPERIMENTS
The fundamental theory on statistical design has been largely
consolidated during the last twenty years or so, for a wide variety of
applications [20]. In this context, the Design of Experiments is based on the
convergence analysis of some peculiar quality figures such as average
power and average prediction error. Converging to the average power
figure let us to identify the minimum length necessary for each simulation,
by considering when the power consumption gets close to a steady value,
given an arbitrary acceptance threshold (see the power–time curve of Figure
241
13-3). On the other hand, the minimum number of experiments (i.e.:
synthesis + simulations) needed to safely probe the design space and
characterize the specific model, strongly depends on the target accuracy (i.e.
max prediction error) as well as on the acceptable characterization effort.
The regression analysis to fit the model’s coefficients is performed on the
raw characterization data. Therefore, the QoR can be analytically measured
by the prediction correlation coefficients (R and R2) and the Root Mean
Square error (RMS). Eventually the minimum number of experiments is
identified by considering both the RMS steady state and the absolute error
over a set of significant benchmarks.
13.6.1 Convergence Analysis: Average Power
The minimum simulations length necessary for the model
characterization has to be identified through a convergence analysis. While
the minimum simulations length of the testbench would not affect the actual
power consumption, it is crucial to make sure that the circuit under analysis
can always reach a steady state functional activity before measuring the avg.
power consumption. To identify the correct simulation length, we minimized
a cost-function that is a product of the simulation time and a measure of the
derivative of the power consumption. The cost function is the following:
( ) ( )( )tP
tPttC
∆⋅= 2(4)
where t is the simulation time, P(t) is the power consumption measured at
time t and P(t) is the difference between P(t) and P(t-1). Figure 13-3 shows
the average behavior of the cost function for all the possible configurations
of shared bus, Type-3, 32 bit width nodes. As can be seen, after 5000 ns the
difference between power values does not pay for the increased simulation
time. Thus, 5000 ns have been selected as the simulation length for all the
characterization experiments.
242
Accuracy - Time Cost Function
300000
500000
700000
900000
1100000
1300000
0 2000 4000 6000 8000 10000
Simulation Time [ns]
Co
st
fun
cti
on
Figure 13-3. Avg. Power vs. Simulation-Time convergence analysis for a given STBus
node’s configuration
The derivative (i.e. differential ratio) has been sampled every 1000 ns
and, then, normalized to the related power values in order to give a
percentage variation.
13.6.2 Convergence Analysis: Model Accuracy
According to previous section 5, the model’s coefficients are resulting
from the polynomial regression over a given set of experiments. Those
experiments are generated according to the DoE’s policy, by stochastically
changing the number of data packets sent/transmitted across the bus and the
operation modes. The goal is to find out the minimum number of
experiments necessary to meet the required accuracy. Given a set of
representative STBus nodes, we perform their characterization with an
increasing number i of experiments. For each set of i calibration
experiments, the Root Mean Square error (RMS) is evaluated. Figure 13-4
shows that, for i>160, the RMS for all the configurations of the design space
gets close to the respective asymptotic values, with a maximum value
bounded to 0.01 mW. The minimum number of experiments to proceed with
the characterization of the STBus nodes has been defined accordingly.
243
RMS Error
0
0.002
0.004
0.006
0.008
0.01
0.012
0 50 100 150 200
n. of experiments
RM
S [
mW
]
8x8 Shared Bus - Type 3 - 32 bit 8x1 Shared Bus - Type 3 - 32 bit
1x8 Shared Bus - Type 3 - 32 bit 1x1 Shared Bus - Type 3 - 32 bit
Steady State
Figure 13-4. Power model’s RMS Error vs. Number of calibration experiments, under four
different initiators/targets configurations
13.7 STBUS POWER MODEL VALIDATION AND
EXPERIMENTAL RESULTS
We present hereafter the results obtained from the validation phase of the
proposed power macro modeling. In addition to the validation carried out by
applying an extensive set of synthetic test-benches, we extended the test by
running a realistic application, featuring mission-mode SystemC simulations
of a multi-processors platform. All the characterization and experimental
results presented in this chapter are targeted to STMicroelectronics
HCMOS9 ASIC library, featuring 8 metal layers and 0.13 µm MOS channel
length, operating at 1.2V nominal supply voltage.
13.7.1 Random pattern validation
We carried out a synthetic validation by applying a uniform set of
stochastically generated Verilog test-benches, similar to those used during
the calibration phase (section 6.2). In Figure 13-5 we illustrate the scatter
plot between the model estimation and the reference power measurement
(coming from detailed gate-level power analysis). The average error is 1%
with a correlation R of 96%.
244
Estimated Power
Measured P
ow
er
Figure 13-5. Scatter plot of Measured vs. Estimated power consumption, for a set of synthetic
benchmarks
13.7.2 Mission mode validation through SystemC co-simulation
To extensively validate our methodology into a real world simulation
platform, we decided to assess the robustness of the power model by
correlating the power estimation coming from a high-level SystemC
simulation with respect to the gate-level power measure of the synthesized
STBus node subject to the input stream generated at runtime by SystemC.
The multi-processor platform is outlined in Figure 13-6. The architecture
consists of four ARM7TDMI processors accessing a number of targets
(including several banks of SRAMs, Interrupt’s slaves and ROMs) through
the STBus communication infrastructure, configured as a 4 initiators, 3
targets, Type-3, shared bus, 32 bit, fixed priority request arbitration policy,
dynamic priority response arbitration policy.
0
245
SystemC
Wrapper
ARM 7 TDMI
ISS
RTEMS OS
SystemC
Wrapper
ARM 7 TDMI
ISS
RTEMS OS
8x8 shared bus – 32 bit – Type 3
STBUS Interconnect
ROM
SystemC
Wrapper
ARM 7 TDMI
ISS
RTEMS OS
SystemC
Wrapper
ARM 7 TDMI
ISS
RTEMS OS
SystemC
Wrapper
ARM 7 TDMI
ISS
RTEMS OS
SystemC
Wrapper
ARM 7 TDMI
ISS
RTEMS OS
SystemC
Wrapper
ARM 7 TDMI
ISS
RTEMS OS
SystemC
Wrapper
ARM 7 TDMI
ISS
RTEMS OS
…Shared RAM
…
Private #1 RAM Private #4 RAM…
Figure 13-6. Multi-processor platform including four ARM7TDMI processors connected
through STBus
Indeed, a remarkable amount of SW layers are intended to be executed
on top of this HW platform, including a distributed real-time operating
system who runs on each individual processor (RTEMS), and a class of
multi-tasking DSP applications, featuring intensive integer matrix
computations.
Bus Traffic Rate
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 10 20 30 40 50 60 70 80 90
Time - 10 KCycles
Rate
Sent Cells Received Cells
Figure 13-7. Data packet rate monitored across the STBus on the target multiprocessor
platform
As far as the simulation framework is concerned, each processor’s ISS
has been encapsulated with a SystemC wrapper, in charge of managing the
interface protocol with the STBus communication node. The whole SW
benchmark has total execution duration of 1 Million clock cycles, including
the RT-OS booth strap (the initial 200 Kcycles) and the execution of the
246
DSP application SW. In Figure 13-7, the data cell’s statistics (i.e. rate of
cells sent/received per time unit) across the STBus is reported. The overall
SystemC/Verilog co-simulation flow is depicted in Figure 13-8.
VCS
Initiator 1
Initiator n
Cosimulation
Enhanced Node
SystemC
STBus
Node
Cosimulation
Enhanced Node
SystemC
STBus
Node
Target 1
Target m
Syste
mC
Co-simulation
Testbench
Generator
…
…
# Cosimulation interchange file - Generated for shared_2x1ICN.Node_1# This file has been automatically generated by a SystemC simualtion of STbus# STBus is a propietary IP of STMicroelectronics
.dt init_0_data .vector 32 0x0 0 init_0_data.dt init_0_add .vector 32 0x1 0 init_0_add.dt init_0_req .vector 1 0x2 0 init_0_req.dt init_0_eop .vector 1 0x3 0 init_0_eop.dt init_0_be .vector 8 0x4 0 init_0_be.dt init_0_opc .vector 8 0x5 0 init_0_opc
# Cosimulation interchange file - Generated for shared_2x1ICN.Node_1# This file has been automatically generated by a SystemC simualtion of STbus# STBus is a propietary IP of STMicroelectronics
.dt init_0_data .vector 32 0x0 0 init_0_data.dt init_0_add .vector 32 0x1 0 init_0_add.dt init_0_req .vector 1 0x2 0 init_0_req.dt init_0_eop .vector 1 0x3 0 init_0_eop.dt init_0_be .vector 8 0x4 0 init_0_be.dt init_0_opc .vector 8 0x5 0 init_0_opc
Co-simulation Interchange
Format File. ST proprietary
Verilog
Synthesized
STBus Node
Verilo
g
Power
Compilersaif
statsAST - Power
Macro Model
∆
Figure 13-8. SystemC/Verilog co-simulation flow
During the SystemC simulation, initiators and targets generate a trace of
“mission mode” transactions, monitored through a specific feature of the
STBus node. In fact, the node has been enhanced in order to gather the full
signals stream out from the SystemC simulation session. The eventual trace
file carries comprehensive print-on-change information, sampled on a clock
cycle basis. The co-simulation file is then applied to drive the gate-level
Verilog simulation (VCS [16]) and, then, feeding the detailed power analysis
of the mapped netlist (PowerCompiler [16]).
In Figure 13-9 we compare the power predicted by SystemC when
running the system simulation (“Power estimated”) vs. the reference power
measured by Power Compiler at gate-level (“Power measured”). Please
notice that absolute power numbers are hidden for technology
confidentiality.
The system level estimation proves to be highly correlated to the
reference power figure, with an average error of 2% and a RMS of 0.015
mW.
247
Cosimulation - Power Report
0 1 2 3 4 5 7 8 9 10
100K Clock Cycles
Po
we
r [m
W]
Power Measured[mW]
Power Estimated[mW]
Figure 13-9. Estimated vs. Measured average power in STBus
13.8 LOW EFFORT, HIGH ACCURACY POWER
MACRO MODELING
Due to the huge domain size, the optimization of the Design of
Experiments (DoE) to be adopted when characterizing the power macro-
model is a key methodology issue to eventually ensure the task’s feasibility.
We decided to adopt a response surface method approach to allow this
problem to be manageable and actually solvable through an automatic tool
flow. Indeed, although this methodology has been developed to cope with
STBus, the described solution can be easily applied to a number of generic
parametric IP as well as to third party’s on chip Bus architectures.
13.8.1 Surface reconstruction methods
Let us consider Equation 1 and Equation 3 (Section 5). As can be seen,
the coefficients are functions over the dominium S, which can be considered
the cartesian product of two subspaces:
S = G × X (5)
G = g | g = <i, t > (6)
X = x | x = < rqr, rpr, p, CL, dps, Type> (7)
248
The coefficients B, Psent and Prec can be seen as a function of two
variables:
f(n) = f(g,x) (8)
The variable g belongs to a discretized space called ‘grid’, over the set of
possible pairs <initiator,target>, while variable x represents the remaining
parameters, according to Equation (7).
By considering x fixed, we have that each coeffiecient is described by a
surface over the set G. Experimentally, the surface shows to have a very
smooth behavior; as an example Figure 13-10 shows a representative
behavior of the coefficient B.
Energy / Sent Cell
Initiators Targets
Power
Figure 13-10. An example surface for the base cost
The problem of fast characterization of the entire design space can be
thought as reconstructing a surface by directly characterizing only a small
subset of points Gs ⊂ G. The given methodology must assure that for an
increasing ratio z=|Gs|/|G|, (i.e., the characterization effort of the power
model library), the approximation error decreases. Ideally, for z=1, the error
should be 0.
Our approach, can be decomposed in three fundamental steps:
1. Choice of a representative set Xs ⊂ X to be used as a training set for the
evaluation of the different reconstruction methods.
249
2. Automatic optimization of the surface reconstruction methods over the
space G × Xs. The output of this stage will be a combination of
algorithm and sub-grid Gs suitable to minimize fitting error and
characterization effort.
3. Perform the actual characterization of Gs× X.
The considered surface reconstruction methods are twofold.
• Regression–based algorithms: Analytic approximation of a surface,
usually a polynomial expression, in which each of the coefficients is
fitted to minimize an overall square error. The procedure to fit these
parameters is often called Least Square Approximation. Three regression
methods have been evaluated: linear, quadratic and cubic.
• Interpolation–based algorithms: Surface approximation by using
multiple piecewise polynomials or more generic functions. Being based
on look-up tables, the response surface always fits with all of the
training set points. The following interpolation algorithms have been
analyzed: cubic, bicubic, bilinear and linear grid data. Algorithms
belonging to the “spline” interpolation class have been experimentally
rejected due to a worse accuracy.
As far as the sub-grid constraints are concerned, the regression based
methods do not enforce any limitation on the sub-grid topology Gs , while
interpolation methods have several topology constraints, for which the
reader is referred to the specific literature [20].
13.8.1.1 Choice of a reference training set Xs ⊂ X
The training set Xs is composed by a random selection within X such that
each value of the parameters rqr, rpr, p, CL, dps, Type (see eq. 5) is
evaluated at least once. In this way it is possible to uniformly visit the
characterization space. More specifically in our design case, considering 8
request and 8 response resources, 7 arbitration policies, 2 load capacitances,
3 data path sizes and 2 protocols, our method allows to reduce the training
set from 3.4*105 down to 30 configurations, without any significative
degradation in accuracy.
13.8.1.2 Surface Reconstruction Algorithm Benchmarking
Each of the afore mentioned algorithms has been benchmarked by
varying the ratio z=|Gs|/|G| and by sampling the design space G in order to
250
find an optimal solution Gs. The search for the optimal solution is driven by
the following cost function:
C(Gs) = σ (Gs)η(Gs) (9)
Where σ(Gs) is a guess of the characterization effort and η(Gs) is the
maximum error between the predicted coefficients and the actual
coefficients.
Then, for each value of z=|Gs|/|G|, we find out the optimal reconstruction
algorithm and the corresponding grid Gs to be adopted for the final
characterization campaign.
13.8.2 Experimental results
In this paragraph we show the experimental evaluations supporting the
proposed methodology. The experimental flow aims at selecting the optimal
grid Gs together with the best reconstruction algorithm for a given grid size
|Gs|. The analysis is performed on a subset of the entire design space
configurations (Xs ⊂ X), as explained in section 8.1.1.
For each algorithm and grid size |Gs|, an iterative random search is
performed to find the best grid, by optimizing the cost function specified in
Equation 9. While driving the grid-optimization with the full cost function
C(Gs) (see eq. 9), we will use the maximum error η(Gs) to discriminate
between candidate reconstruction methods, given a fixed grid-size.
13.8.2.1 Combined Analysis
To compare the various methods we are going to focus our attention on
the best surface reconstruction method for each category (regression and
interpolation). As far as the regression is concerned, the quadratic algorithm
shows to be the most promising for small grid-sizes, even better than the
interpolation-based methods. On the other side, regarding interpolation, the
cubic grid data technique is the most accurate in terms of maximum error.
Accordingly, Figure 13-11 shows the best pair <σ, η> (see eq. 9) for each
grid size |Gs|, with respect to Quadratic Regression (shortened as
‘Regression’) and Cubic Grid Data Interpolation (shortened as
‘Interpolation’). Moreover, the figure shows the Pareto curve highlighting
points that are not dominated in both directions by any other point. This
curve is useful to screen out the solutions that optimize both the merit figures
and, at the same time, meet specific constraints. For example, a quadratic
regression method should be used if the characterization effort is a
dominating constraint. In fact, in this case we have only 10% of the original
251
characterization effort at the expense of a 33% maximum error. For medium-
high characterization effort, the Cubic Grid Data interpolation has shown to
reduce the maximum error more than the other methods. It leads to a
maximum error of about 18% at the expense of 62% of the original
characterization effort.
Accuracy/Effort Trade-off
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 10% 20% 30% 40%
Maximum Error
Chara
cte
rization e
ffort
Interpolation Regression
Pareto Curve
Figure 13-11. Characterization effort vs. Maximum error Trade-off
13.9 CONCLUSIONS
An innovative methodology for automatically generating the energy
models of a versatile and parametric on-chip communication infrastructure
(STBus) has been presented in this chapter. The methodology aggressively
targets correlated power estimation with efficient SystemC simulation,
running at BCA and TLM abstraction level. Among other synthetic
benchmarks, the NoC’s power models validation has been extensively
addressing the high-level SystemC simulation of a real world multi-
processor platform (MP-ARM), which includes four ARM7TDMI
processors accessing a number of peripheral targets (including several banks
of SRAMs, Interrupt’s slaves and ROMs) through the STBus
communication infrastructure. All the characterization and experimental
252
results presented in this chapter are targeted to STMicroelectronics
HCMOS9 ASIC library, featuring 8 metal layers and 0.13 µm MOS channel
length, operating at 1.2V nominal supply voltage.
The synthetic validation between the model estimation and the reference
power figures (i.e. gate-level power measure of the synthesized NoC) shows
an average error of 1% and correlation R of 96%. The power analysis of
the MP-ARM benchmark proves to be highly effective and correlated, with
an average error of 2% and a RMS of 0.015 mW vs. the reference power.
Eventually, we presented a new and effective methodology to minimize the
Design of Experiments (DoE) needed to characterize a set of innovative
energy macro models of Network-on-Chip architectures. We have shown
that, by properly combining regression (polynomial) and interpolation (table-
based) techniques, this methodology aims to reduce the overwhelming
complexity of sampling the huge, non-linear design space involved with the
system operations of an high-end parametric on-chip-network module. The
experimental figures show that our DoE optimization techniques are able to
trade off power modeling approximation with model building cost, leading
to an average 62% reduction of the sampling space with a maximum error of
18%.
Acknowledgements
The authors are grateful to dr. C.Pistritto and his CMG/OCCS team in
Catania, for their valuable and synergic support to achieve the leap
enhancement of making the STBus Design Flow truly "power aware".
References
[1]J.Duato, S.Yalamanchili, L. Ni, “Interconnection Networks: an Engineering Approach”,
IEEE Computer Society Press, 1997.
[2] K. Lahiri, S.Dey et al.,”Efficient Exploration of the SOC Communication Architecture
Design Space”, Proc. of ICCAD-2000, Nov. 2000, S.Jose`, USA.
[3] W. Dally, B. Toles, “Route Packets, not Wires: On-Chip Interconnection Network”,
Proceedings of 38th DAC 2001, June 2001, Las Vegas, USA.
[4] A. Sangiovanni Vincentelli, J. Rabaey, K. Keutzer et al., “Addressing the System-on-a-
Chip Interconnect Woes Through Communication-Based Design”, Proceedings of 38th
DAC 2001, June 2001, Las Vegas, USA.
[5] F. Karim, A. Nguyen et al., “On Chip Communication Architecture for OC-768 Network
Processors”, Proceedings of 38th DAC 2001, June 2001, Las Vegas, USA.
[6] K. Lahiri, S.Dey et al.,”Evaluation of the Traffic Performance Characteristics of System-
on-Chip Communication Architectures”, Proc. 14th Int’l Conference on VLSI Design
2001, Los Alamitos, USA.
[7] L. Benini, G. De Micheli, “Network on Chip: A New SoC Paradigm”, IEEE Computer,
January 2002.
253
[8] T. Ye, L. Benini, G. De Micheli, “Analysis of power consumption on switch fabrics in
network routers”, Proceedings of 39th DAC 2002, June 2002, New Orleans, USA.
[9] S. Kumar et al., “A network on chip architecture and design methodology”, International
Symposium on VLSI 2002.
[10] H.-S. Wang, X. Zhu, L.-S. Peh, and S. Malik, “Orion: A Power-Performance Simulator
for Interconnection Networks”, International Symposium on Microarchitecture, MICRO-
35, November 2002, Istanbul, Turkey.
[11] T. Ye, G. De Micheli and L.Benini, “Packetized On-Chip Interconnect Communication
Analysis for MPSoC”, Proceedings of DATE-03, March 2003, Munich, Germany, pp.
344-349.
[12] J.Hu and R. Marculescu, “Exploiting the Routing Flexibility for Energy/Performance
Aware Mapping of Regular NoC Architectures”, Proceedings of DATE-03, March 2003,
Munich, Germany, pp. 688-693.
[13] T. Grotker, S. Liao, G. Martin and S. Swan, “System Design with SystemC”, Kluwer
Academic Publishers, 2002.
[14] “STBus Communication System: Concepts and Definitions”, Reference Guide,
STMicroelectronics, October 2002.
[15] “STBus Functional Specs”, STMicroelectronics, public web support site,
http://www.stmcu.com/inchtml-pages-STBus_intro.html, STMicroelectronics, April 2003.
[16] Synopsys Inc., “Core Consultant Reference Manual”, “Power Compiler Reference
Manual” and “VCS: Verilog Compiled Simulator Reference Manual”, v2003.06, June
2003.
[17] C. Patel, S. Chai, S. Yalamanchili, and D. Schimmel, “Power-constrained design of
multiprocessor interconnection networks," in Proc. Int. Conf. Computer Design, pp. 408-
416, Oct. 1997.
[18] H.Zimmermann, “OSI Reference Model – The ISO model of architecture for Open
System Interconnection”, IEEE Trans. on Communication, n 4, April 1980.
[19] VSI Alliance Standard, “System-Level Interface Behavioral Documentation Standard
Version 1”, Released March 2000.
[20] Box, George E. P. and Draper Norman Richard. Empirical model-building and
response surfaces, John Wiley & Sons New York, 1987
254
Chapter 14
ENERGY-AWARE ADAPTATIONS FOR END-TO-END VIDEO STREAMING TO MOBILE HANDHELDDEVICES
Shivajit Mohapatra1, Nalini Venkatasubramanian1, Nikil Dutt1,Cristiano Pereira2, Rajesh Gupta2
1University of California, Irvine;2University of California, San Diego
Abstract Optimizing user experience for streaming video applications on handheld devicesis a significant research challenge. In this chapter, we propose an integrated end-to-end power management approach that unifies low level architectural optimiza-tions(CPU, memory, registers), OS power-saving mechanisms(dynamic voltagescaling) and adaptive middleware techniques(admission control, transcoding, net-work traffic regulation). Specifically, we identify interaction parameters betweenthe different levels and optimize them to reduce power consumption. With knowl-edge of device configurations, dynamic device parameters and changing systemconditions, the middleware layer selects an appropriate video quality and finetunes the architecture for optimized delivery of video. Performance results in-dicate that architectural optimizations that are cognizant of user level parame-ters(e.g. transcoded video quality) can provide energy gains as high as 57.5%for the CPU and memory, when compared to the baseline case that does notemploy any energy optimization. Middleware adaptations to changing networknoise levels can save as much as 70% of energy consumed by the wireless net-work interface. Our approach to multiple-level and end-to-end management ofpower/performance has been implemented in a framework, called FORGE. Weshow how FORGE can substantially enhance the user experience in a mobilemultimedia application.
Keywords: low-power optimization, cross-layer adaptation, power-aware middleware, FORGEproject
14.1 MOTIVATION
Limiting the energy consumption is an important design goal for mobiledevices. Designers have explored techniques for minimizing energy usage of
255
most components, from CPU, network, display to peripherals of a mobile sys-tem platform. On the other hand, rapid advances in processor and wirelessnetworking technology are ushering in a new class of multimedia applications(e.g. video streaming/conferencing) for mobile handheld devices. Multimediaapplications have distinctive Quality of Service(QoS) and processing require-ments which tend to make them extremely resource-hungry. Moreover, thedevice specific attributes(e.g form factor of handhelds) significantly influencethe human perception of multimedia quality. As a result delivering high qual-ity realtime multimedia content to mobile handheld devices remains a difficultchallenge.
The difficulty here is due to the fact that energy efficient delivery of mediacontent with good quality attributes requires tradeoffs across various layers ofsystem implementation and functionality - from application to system softwareto networking. Since the optimal energy conditions can change dynamically,these optimizations should also allow for dynamic adaption of system func-tionality and its performance. In order to dynamically adapt to device mobility,systems need to have a high degree of “network awareness" (e.g. congestionrates, mobility patterns etc.) and need to be cognizant of a constantly changingglobal system state. Efforts are underway to exploit multimedia specific char-acteristics to enable a range of energy optimization techniques that adapt to, andoptimize for, changes in application data (video stream), OS/Hardware (CPU,Memory, Reconfigurable logic), network (congestion, noise, node mobility),residual energy (battery) and even the user environment (ambient light, sound).
These issues have been aggressively pursued by researchers and numerousinteresting power optimization solutions have been proposed at various crosscomputational levels. For instance, a sampling of optimizations across designdomains are: system cache and external memory access optimizations [1, 15],dynamic voltage scaling(DVS) [29, 4], of the CPU, dynamic power manage-ment of disks and network interfaces(NICs) [8, 3], efficient compilers and ap-plication/middleware based adaptations for power management [22]. Interest-ingly, power optimization techniques developed for individual components ofa device have remained seemingly incognizant of the strategies employed forother components. While focussing their attention to a single component, re-searchers make a general assumption that no other power optimization schemesare operational for other components. However, the cumulative power gains forincorporating multiple techniques can be potentially significant. This requirescareful evaluation of the trade-offs involved and the customizations required forunified operation [21]. The interaction between different layers is even moreimportant in distributed applications where a combination of local and globalinformation helps and improves the control decisions (power, performance andQoS trade-offs) made at runtime.
256
For the mobile multimedia applications, Fig. 14.1 presents the different com-putation levels in a typical handheld computer and shows the cross layer inter-actions for optimized power and performance deliverance.
Server
Clientn
Clienti
Client1
NetworkCard
Display Cache Memory RegFilesCPU
H/W
Operating SystemDVS Scheduler
NetworkManagement
Transcoding AdmissionControl
ApplicationsVideo Player Other Tasks
Middleware
Figure 14.1. Abstraction Layers in Distributed Multimedia Streaming
The FORGE project aims to study the tradeoffs between power, performanceand Quality of Service requirements across the various computational layers [6].The goal of FORGE is to develop and integrate hardware based architecturaloptimization techniques with high level operating system and middleware ap-proaches (Fig. 14.1), for improvements in power savings and the overall userexperience, in the context of video streaming to a low-power handheld device.Multimedia applications heavily utilize the biggest power consumers in mod-ern computers: the CPU, the network and the display(Fig. 14.1). Therefore, inFORGE, we aggregate the hardware and software techniques that lead to powersavings for these resources. To maximize power gains for a CPU architecture,we identify the predominant internal units of the architecture that contribute topower consumption. We use higher-level knowledge of the application such asquality and encoding parameters of the video stream to optimize internal cacheconfigurations, CPU registers and the external memory accesses. Similarly,we utilize hardware/design level data (e.g. cache configuration) and user-levelinformation (video quality perception) to optimize middleware and OS compo-nents for improved performance and power savings - through effective videotranscoding, power-aware admission control and efficient network transmis-sion. We reduce the power consumption of the network card by switching itto the “sleep" mode during periods of inactivity. An efficient middleware isused to control network traffic for optimal power management of the networkinterface. To maximize the user experience, we have studied video quality andpower trade-offs for handheld computers. These results drive our optimizationefforts in FORGE at each computing level.
14.2 RELATED WORK
Let us briefly review the optimization techniques used at various levels, suchas architecture, OS, middleware and application in the context of multimedia
257
applications. We then examine the relationship of FORGE with prior andongoing approaches in power aware middleware.
14.2.1 Architectural Adaptations
To provide acceptable video performance at the hardware level, efforts haveconcentrated on analyzing the behavior of the decoder software and devisingeither architectural enhancements or software improvements for the decodingalgorithm. Until recently it was believed that caches can bring no potentialbenefit in the context of MPEG (video) decoding. In fact, due to the poor lo-cality of the data stream, many MPEG implementations viewed video data as“un-cacheable" and completely disabled the internal caches during playback.However, Soderquist and Leeser showed that video data has sufficient localitythat can be exploited to reduce cache-memory traffic by 50 percent or morethrough simple architectural changes [28]. A different way of improving cacheperformance by reordering frame traversal was proposed in [9]. Register filereconfiguration was applied in [1]. [16] proposes a technique for combining twohardware adaptations (architecture adaptation and dynamic voltage scaling) toreduce energy in multimedia workloads. The algorithm presented chooses be-tween one of the two adaptations or a combination, depending on their relativeperformance. This approach is similar to ours, in that architectural optimiza-tions are combined with dynamic voltage scaling (DVS). However, instead of aframe-based adaption founded on profiling and prediction, we target tuning anarchitecture through available architectural parameters to specific video qualityrequirements. We apply the optimizations globally (for the entire period that amedia of constant quality levels are played), rather than at frame granularity.
14.2.2 Operating System & Middleware Adaptations
Most power optimization efforts at the operating system level, have been fo-cussed on techniques like dynamic voltage scaling(DVS) [29, 25, 18], anddynamic power management (DPM) [17, 7]. DVS exploits the fact the CMOSlogic used in most current processors has a voltage dependent maximum operat-ing frequency. So when used at lower frequencies, the processor can operate ata correspondingly lower voltage, thereby saving battery power. The challengehere is to accurately predict workload execution times for future jobs. Whileworkloads can be predicted heuristically for best-effort applications [29], orbased on worst case execution times of real-time applications [25], worst-casebased approaches will almost certainly result in sub-optimal solutions, whereasheuristic predictions can cause timing violations for multimedia tasks. In theGRACE project, the authors suggest using an aggregate statistical demand ofapplications to adjust frequency/voltage for the processor [31]. DVS techniquesfor reducing energy in MPEG decoding has been studied in [20]. Additionally,
258
scheduling techniques like DSRT [5] have been studied to deliver real-timeguarantees.
At the OS/middleware levels, another primary focus has been to optimizenetwork interface power consumption [8, 2, 3]. A thorough analysis of powerconsumption of wireless network interfaces has been presented in [8]. ECOSys-tem [32] is an OS level prototype that incorporates energy allocation and ac-counting mechanisms for various power consuming devices. ECOSystem usesthe Currentcy [33] model which is an abstraction for formulating energy awarepolicies.
Chandra et al. [2] have explored the wireless network energy consumptionof streaming video formats like Windows Media, Real media and Apple QuickTime. Chandra and Vahdat have explored the effectiveness of energy awaretraffic shaping closer to a mobile client [3]. In [26], Shenoy suggests performingpower friendly proxy based video transformations to reduce video quality inreal-time for energy savings. They also suggest an intelligent network streamingstrategy for saving power on the network interface. FORGE uses a similarapproach, but models a noisy channel. Caching streams of multiple qualitiesfor efficient performance has been suggested in [10].
PowerScope [12] is an interesting tool that maps energy consumption to pro-gram structure. It first profiles the power consumption and system activity ofa computer and then generates an energy profile from this data. Odyssey [22]presents an applications aware adaptation scheme for mobile applications. Inthis approach the system monitors resource levels, enforces resource alloca-tion and provides feedback to the applications. The application then decideson the best possible adaptation strategy. In our approach we try to integratethe the positive aspects of all the three levels: OS, middleware and applica-tion. Application based adaptation will therefore enhance the performance ofour framework. However, applications have to be specifically designed forthe framework. JouleTrack [27] is a web-based energy measurement tool forprofiling software energy consumption of applications based on StrongArmprocessor.
14.2.3 Cross-Layer Adaptation Frameworks
For efficient coordination and management of cross-layer adaptations, it is cru-cial to develop efficient resource allocation mechanisms. Q-RAM [19] modelsQoS management as a constraint optimization problem for maximizing sys-tem utility while guaranteeing minimum resources to each application. Pup-peteer [11] presents a middleware framework that uses transcoding to achieveenergy gains. Using the well defined interface of applications, the frameworkpresents a distilled version of the application to the user, in order to draw en-
259
ergy gains. EQoS [24] formulates energy-aware QoS adaptation as a constraintoptimization problem and solves it using heuristic algorithms.
The GRACE project [31, 30] uses cross-layer adaptations for maximizingsystem utility at lower energy costs. They suggest both coarse grained and finegrained tuning through global co-ordination and local adaptation of hardware,OS and application layers. The coarse/global adaptations are expensive andless frequent and occur only when global system changes are triggered (e.gtask-set changes). The local adaptations are for the local variation in the ex-ecution of tasks. In GRACE, the global and local coordinators exist on thelocal device and perform the necessary adaptations. GRACE first tries to de-liver highest utility for each application and then optimizes the energy usingdynamic voltage scaling. In contrast, FORGE uses a proxy based distributedmiddleware approach, that integrates cross-layer(architecture, OS, middleware,application) adaptations on the local device with distributed adaptations suchas adaptive traffic shaping and transcoding at the proxy for energy gains. Whileadaptations in GRACE are limited to the local mobile device, our frameworkdesign uses a distributed middleware layer to exploit global system knowledge(e.g. device mobility patterns, network noise levels etc.) to facilitate effectivepower management (e.g. wireless NIC). Moreover, we adopt an end-to-end ap-proach to power optimization, where residual battery power of a mobile devicealso drives the adaptations. GRACE on the other hand provides a best-effortapproach to energy optimization. Additionally, FORGE tries to tune architec-tural level parameters (e.g. cache configurations) to perform optimally for thecurrently executing application. The distributed middleware co-ordinates theadaptations at each level based on a rule-base and control information from theproxy.
14.3 SYSTEM MODEL
Our system model for a wireless mobile multimedia distributed system is shownin Fig. 14.2. The system entities include a multimedia server, a proxy serverthat utilizes a directory service, a rule base for specific devices and a videotranscoder, an ethernet switch, the wireless access point and users with low-power wireless devices. The multimedia servers store the multimedia contentand stream videos to clients upon receipt of a request. The users issue requestsfor video streams on their handheld devices. All communication between thehandheld device and the servers are routed through the proxy server, that cantranscode the video stream in realtime. The middleware executes on both thehandheld device and the proxy, and performs two important functions. On thedevice, it obtains residual energy availability information from the underlyingarchitecture and feeds it back to the proxy and relates the video stream parame-ters and network related control information to lower abstraction layers. On the
260
proxy, it performs feedback based power aware admission control and realtimetranscoding of the video stream, based on the feedback from the device. It alsoregulates the video transmission over the network based on the noise level andthe video stream quality. Additionally, the middleware exploits dynamic globalstate information(e.g mobility info, noise level etc.) available at the directoryservice and static device specific knowledge (architecture, OS, video qualitylevels) from the static rule base, to optimally perform its functions. The rateat which feedback are sent by the device is dictated by administrative policies(e.g. periodic feedback). Moreover, we assume that network connectivity ismaintained at all times.
P C
Switch Access Point
Proxy
SServer
noise
W A N W I R E D E T H E R N E TW I R E L E SS
C
C
U S
E R
S
Directory Service
Rule base
Transcoder
Figure 14.2. System Model
In the rest of the chapter, we present important research challenges encoun-tered at each level and discuss approaches that involve both distributed proxybased adaptations coupled with coordinated cross-layer energy optimizationsat the device.
14.4 HARDWARE/ARCHITECTURAL LEVELOPTIMIZATIONS
The architectural optimizations are particularly important because of the useof microelectronic system-on-chip components used in multimedia platforms.Since most multimedia applications spend a significant amount of time access-ing and transforming audio and video data, the design of the memory subsystemarchitecture, and compiler support for exploiting the specialized memory struc-tures are critical for meeting the performance, power and cost budgets of suchapplications.
Since the memory subsystem will dominate the cost (area), performance andpower, we have to pay special attention to how it can benefit from customiza-tion. For example, the memory can be selectively cached; the cache line sizecan be determined by the application; the designer can opt to discard the cachecompletely and choose specialized memory configurations such as FIFOs andstream buffers. The exploration space of different possible memory architec-tures is vast, and there have been attempts to automate or semi-automate thisexploration process [13].
261
CPU
MemoryDisplay
Networkcard
a b
DataCache
RegisterFile
Functional Units
Clock
Figure 14.3. Main Components of a Handheld Device (a) and CPU Detail (b)
14.4.1 Hardware-level Optimizations for HandheldDevices
There are three major sources of power consumption in a handheld devicesuch as a Compaq iPAQ 3650 for which we indicate the corresponding powernumbers: display (approximately 1W for full backlight), network hardware(1.4W) and CPU/memory (1-3W, with the additional board circuits). Each ofthese subsystems also provide opportunities for controlling the power dissi-pation. In case of the display (LCD), the main energy drain comes from thebacklight, which is a predefined user setting and therefore has a limited degreeof controllability by the system (without affecting the final utility). The networkinterface allows for efficient power savings if cognizant of the higher level pro-tocol’s behavior and will be explored in a subsequent section. Out of the threecomponents mentioned above, the CPU coupled with the memory subsystemposes the biggest challenge. The dependence on the input data to be processed,the quality of the code generated by the compiler and the organization of itsinternal architecture make predicting its power consumption profile very hardin general; nevertheless, very good power saving results can be obtained byutilizing the knowledge of the application running on it and through extensiveprofiling of a representative data input set from the application’s domain. Inthe rest of this section, we focus our attention on the possible optimizations atthe CPU level for a multimedia streaming application (e.g MPEG-1).
We identified the subcomponents of the CPU (Fig. 14.3(b)) that consumethe most power and observed the power distribution inside the CPU for MPEGdecoding. By running the decoder process in a power simulator (Wattch) forvideos of various types and by measuring the relative power consumption ofeach unit in the CPU we generate the internal processor power distribution. Weconclude that:
• The relative power contribution of the internal units of the CPU do not varysignificantly with the nature or quality of the video played. A possible reasonfor this is the symmetrical and repetitive nature of MPEG decoding, whoseprocessing is done on fixed size blocks or macroblocks.
262
• The units that show an important contribution to the overall power con-sumption and are amenable for power optimization are: caches, register filesand functional units. Cache behavior greatly affects the memory performanceand hence power consumption, so we optimize the entire memory subsystemin an integrated way.
We briefly discuss these components, their impact on overall power con-sumption and how it can be affected by these architectural choices:• Caches/Memory: cache configurations are determined by their size, numberof sets, and associativity. The size specifies how large a cache should be, whilethe associativity/number of sets control its internal structure. We identify thatmost power gains for MPEG are possible through reconfiguration of the datacache and its effect on the memory traffic, thus amplifying the effect of poweroptimizations through cache reconfiguration.• Frame Traversal: Decompressing MPEG video in its implied order doesnot leave space for exploiting the limited locality existent between dependentmacroblocks. By just changing the frame traversal order algorithm based onthe existing locality, faster decompression rates and higher power savings areachieved via reduced memory accesses [9]. Our proxy-based approach allowsfor a transparent on-the fly traversal reordering at the proxy server. In additiondynamic voltage scaling provides further savings for MPEG streaming as itallows tradeoffs for transforming the frame decoding slack time (CPU idle time)into important power savings. We discuss DVS and investigate the implicationsof DVS on other power optimizations in the system. All these parameters whenfine-tuned for a specific video quality, will provide the best operating point(forpower and performance) for a specific video stream.
14.4.2 Quality-driven Cache Reconfiguration
Power consumption for the cache depends on the runtime access counts:while hits result in only a cache access, misses add the penalty of accessingthe main memory (external). Fortunately, in most applications the inherent lo-cality of data means that cache miss rate is relatively low and so are accessesto external memory. However, MPEG decoding exhibits a relatively poor datalocality, which, when combined with the large data sets exercised by the algo-rithm, leads to an increase in the cache memory-traffic. In order to find the bestsolution point, we resort to extensive simulation and profiling with data that isrepresentative of the video domain. Internal CPU caches are characterized bytheir size(S), number of sets(NS), line size(LS) and associativity(A). Ourcache reconfiguration goal is optimizing energy consumption for a particularvideo quality level Qk. In general, cache power consumption for a particularconfiguration and video quality is given by the function Ecache,k(S, A). Byprofiling this function for the entire search space (S, A) of available cache
263
1 2 4 8 16 32
64
32
16
8
4
1.1
1.2
1.3
1.4
1.5
1.6
1.7
CacheSize
Cache Associativity
Total Energy (J)
Figure 14.4. Cache Energy Variation on Size and Associativity
configurations, we generate a cache energy variation graph shown in Fig. 14.4.Depending on the video quality Qk played, there will be one optimal operatingpoint for that video quality: (Sopt
k , Aoptk ). We found out that for all video
qualities an optimized operating point exists and it improves cache power con-sumption by up to 10-20% (as opposed to a suboptimized configuration). Thistechnique effectively fine-tunes the organization of the cache so that it perfectlymatches the application and the data sets to be processed, yielding importantpower savings.
14.5 OS/MIDDLEWARE LEVEL OPTIMIZATIONS
Gains in power reduction and performance improvement from architecturaloptimizations can be further amplified if the low-level architecture is cognizantof the exact characteristics of the streamed video. An adaptive middlewaresoftware at a proxy can dynamically intercept and doctor a video stream toexactly match the video characteristics for which the target architecture hasbeen optimized. It can also regulate the network traffic to induce maximalpower savings in a network interface. Additionally, with knowledge of thevideo stream the operating system can employ an optimized dynamic voltagescaling of the CPU.
14.5.1 Integrated Dynamic Voltage Scaling
For a given supply voltage, V, and clock frequency f, the dynamic powerdue to digital CMOS varies linearly with frequency and quadratically with thesupply voltage (which is also the switching voltage). This relationship can beused at the application level [4]. In our case, for MPEG decoding, frames areprocessed in a fraction of the frame delay (Fd = 1/frame rate). The actualframe decoding time D depends on the type of MPEG frame being processed(I, P, B) and is also influenced by the cache configuration (S, A) and DVSsetting (f, V ). We assume a buffer based decoding, where the decoded frames
264
are placed in a temporary buffer and are only read when the frame is displayed.This allows us to decouple the decoding of the frame from the displaying part;decoding time is still different for different frames, we therefore assume anaverage D for a particular video stream/quality. The difference between theaverage frame delay and actual frame decoding time gives us the slack timeθ = Fd − D. We can then perform DVS, where we slow down the CPU touse up the slack time. Cache configuration also slightly influences the framedecoding time (due to the cache misses, which translate into external memorytraffic), extreme values proving very inefficient. An optimized cache combinedwith DVS yields the best power saving results. Determination of the bestoperating point for the DVS/cache reconfiguration requires simulation of theapplication with the power aware system software that has direct influence onthe technology parameters. This is discussed next.
14.5.2 Power Aware Operating System Architecture
We view the notion of power ’awareness’ in the application and OS as acapability to carry out a continuous dialogue between the application, the OS,and the underlying hardware. This dialogue establishes the functionality andperformance expectations (or even contracts, as in real-time sense) within theavailable energy constraints. We describe here our implementation of a specificservice, namely the task scheduler, that makes the OS power aware. The sched-uler architecture is composed of two software layers and the OS kernel. Onelayer interfaces applications with operating system and the other layer makespower related hardware “knobs” available to the operating system. Both layersare connected by means of corresponding power aware operating system ser-vices as shown in Figure 14.5. At the topmost level, embedded applicationscall the API level interface functions to make use of a range of services thatultimately makes the application energy efficient in the context of its specificfunctionality. The API level is separated into two sub-layers. The PA-API layerprovides all the functions available to the applications, while the other layer pro-vides access to operating system services and power aware modified operatingsystem services (PA OS Services). Active entities that are not implementedwithin the OS kernel are also be implemented at this level (threads created withthe sole purpose of assisting the power management of an operating systemservice).
We call this layer the power aware operating system layer (PA-OSL). Tointerface the modified operating system level and the underlying hardware level,we define a power aware hardware abstraction layer (PA-HAL). The PA-HALprovides access to the power related hardware parameters in a way that makesit independent of the hardware.
265
POSIX
HARDWARE
API Level
Application Level
Hardware Level
OS Level
PA-HALOS HAL
PA-OSL
Applications
Scheduler
DeviceDrivers
MemoryManager
Services
PA OSOSKernel
PA-API
Figure 14.5. Power Aware Operating System Architecture
14.5.3 Middleware based Network Traffic Regulation
We now describe a proxy-based traffic regulation mechanism to reduce energyconsumption by the device network interface. Our mechanism (a) dynami-cally adapts to changing network(e.g noise) and device conditions(e.g. residualbattery energy). (b) accounts for attributes of the wireless access points (e.g.buffering capabilities) and the underlying network protocol (e.g. packet size).(c) uses the proxy to buffer and transmit optimized bursts of video along withcontrol information to the device. However, even though packets are transmit-ted in bursts by the proxy, the device receives packets that are skewed over timeFig. 14.6; this cuts power savings, as the net sleep time of the interface is re-duced. The skew is caused due to the ethernet access protocol(e.g CSMA/CD)and/or the fair queueing algorithms implemented at the wireless access points.Our mechanism optimizes the stream, such that optimal video bursts sizes aresent for a given noise level, thus maximizing energy savings without perfor-mance costs.
Wireless network interface(WNIC) cards typically operate in four modes:transmit, receive, sleep and idle. We estimated the power consumption ofthe Cisco Aironet 350 series WLAN card to have the following power con-sumption characteristics: transmit(1.68W), receive(1.435W), idle (1.34W) andsleep(0.184W) which agree with the measurements made by Havinga et al.in [14]. This observation suggests that considerable energy savings can beachieved by transitioning the network interface from idle to sleep mode during
266
periods of inactivity. The use of bursty traffic was first suggested by Chan-dra [2, 3] and control information was used for adaptation in [26].
We analyze the above power saving approach using a realistic network frame-work(Fig. 14.6), in the presence of noise and AP limitations [21]. The proxymiddleware buffers the transcoded video and transmits I seconds of video in asingle burst along with the time τ=I for the next transmission as control infor-mation. The device then uses this control information to switch the interface tothe active/idle mode at time τ + γ × DEtoE, where γ is an estimate betweenzero and one and DEtoE is the end-to-end network delay with no noise.
t t
P CHTTP/TCP/IP
RTP/UDP/IP802.11b
C
CWired
Wired wireless
User NUser 1
Proxy
Access Point
Wirelessdevice
packets
Figure 14.6. Wireless Network
We acknowledge that a QoS aware preferential service algorithm at the accesspoint can impact power management significantly. The above analysis can beused by an adaptive middleware to calculate an optimal I(burst length) for anygiven video stream and noise level. Note that energy overhead for buffering thevideo packets is not affected by using our strategy because the number of readand write memory operations remain unchanged irrespective of the memorybuffer size.
In the previous section, we demonstrated how low level architecture canbe optimized using high level information. In this section, we presented twomiddleware techniques that can be used to compliment the low-level hardwareoptimizations, lower energy consumption of the NIC and improve the overallutility of the system. We now introduce a middleware based adaptation schemefor backlight power savings in handheld devices.
14.5.4 Reducing Backlight Power Consumption
The backlight accounts for considerable energy overheads in a low-powerdevice. However, potentially large energy savings are realizable by operatingthe device at a lower backlight intensity levels. We explore a more aggressiveapproach to brightness compensation and device backlight control for stream-ing video. Furthermore, the adaptation is shifted away from the low-powerdevice and performed at a network proxy server, obviating the need for thedecoder on the device to be modified. We have found that aggressive bright-ness compensation is possible for streaming video as compared to still images,
267
without considerably impacting the video quality. This is because small defects(introduced due to aggressive compensation) that might be noticeable in a stillimage are less discernable in streaming video where several frames (images) aredisplayed on the screen every second. We also propose an effective brightnesscompensation algorithm for optimized power savings [23]. In this approach,we introduce middleware based adaptation schemes which integrate our com-pensation algorithm to achieve low power backlight operation for streamingvideo content to mobile handheld devices. Our experiments indicate that thisapproach can provide power reductions of up to 60% of the power consumptionattributed to the backlight, depending on the chosen adaptation scheme and thecharacteristics of the streamed video.
We assume that the proxy server has access to a database of profiled lumi-nosity values for various video streams and device specific parameters (e.g.number of backlight levels, average luminosity at each level etc.), a rule baseto determine compensation values and a video transcoder(Fig. 14.7); and low-power wireless devices capable of displaying streaming MPEG video content.All communication between the handhelds and the multimedia server are routedthrough the proxy server that can change the video stream in real-time.
Figure 14.7. Model for Backlight Adaptation
Each device/client has an application layer where the video stream is de-coded and a middleware layer which routes the information flowing to andfrom the video decoder application. The client middleware layer has accessto system parameters such as the backlight levels, the current battery level andinformation identifying the type and make of the handheld (e.g. iPAQ, Jor-nada etc). In addition to accessing these system parameters, the middlewarelayer on the client can change these parameters (e.g. operating backlight level)through API calls to the underlying OS. The middleware on the proxy performsthe dynamic adaptation of the streaming video content (brightness compensa-tion) and communicates control information to the client middleware (operatingbacklight levels) through the low bandwidth control stream. The proxy main-tains a database of information about the videos available at the server and
268
information specific to different handheld types such as the number, luminousintensity and average power consumption of the backlight levels. Additionally,the proxy also employs a static rule base which specifies conditions which de-termine values for backlight and video compensation. The database and certainparameters of the rule base are populated by extensive profiling and subjectiveassessment of videos on different handhelds.
14.6 APPLICATION LAYER ADAPTATION
Improving the service lifetimes of low-power mobile devices through effectivepower management strategies can facilitate optimization of user experience forstreaming video on to handheld devices. To achieve this, a system should beable to dynamically adapt to global system changes, so that the entire duration ofa requested video is streamed to the user at the highest possible quality, whilemeeting the power constraints of the user’s low-power device. We achievesuch an optimal balance between power and performance, by introducing anotion of “Utility Factor UF " for a system, and optimizing the UF for thesystem. This approach precludes the system from aggressively optimizing forpower at the expense of performance and vice-versa; thereby providing anoptimized operating point for the system at all times. UF is a measure of “usersatisfaction" and we specify it as follows: given the residual energy Eres ona handheld device, a threshold video quality level (QA : QMAX ≥ QA ≥QMIN) acceptable to the user, and the time of the video playback T, the UF
of the system is non-negative, if the system can stream the highest possiblequality of video to the user such that the time, quality and the power constraintsare satisfied; otherwise UF is negative. Let PV ID denote the average powerconsumption rate of the video playback at the handheld and QPLAY be thequality of video streamed to the user by the system. Using the above notation,we define UF as follows:
UF =
QPLAY − QMIN IFF PV ID ∗ T < ERES
QPLAY ≥ QA
−1 OtherwiseOur experiments to determine video transcoding levels that affect the video
quality against increased energy consumption indicate the following major con-clusions:• It is hard to programmatically identify video quality parameters( a combina-tion of bit rate, frame rate and video resolution) that produced a user perceptiblechange in video quality and/or a noticeable shift in power consumption in hand-helds.
• For all the video streams on handheld devices, it was enough to use justthree standard intermediate formats(e.g SIF(320x240), Half SIF(340x160) andQuarter SIF(160x120)) for frame resolution values. Other resolutions did not
269
Table 14.1. Energy-Aware Transformations for Compaq Ipaq 3650 with bright backlight, Cisco350 Series Aironet WNIC card. (Q1) Terrible, (Q2) Bad, (Q3) Poor, (Q4) Fair, (Q5) Good, (Q6)Very Good, (Q7) Excellent, (Q8) Like Original
Quality Parameters Avg. Power Avg. Power(WinCE) (Linux)
(Q8) SIF, 30fps, 650Kbps 4.42W 6.07W(Q7) SIF, 25fps,450Kbps 4.37W 5.99W(Q6) SIF, 25fps, 350Kbs 4.31W 5.86W(Q5) HSIF, 24fps, 350Kbps 4.24W 5.81W(Q4) HSIF, 24fps, 200Kbps 4.15W 5.73W(Q3) HSIF, 24fps, 150Kbps 4.06W 5.63W(Q2) QSIF, 20fps, 150Kbps 3.95W 5.5W(Q1) QSIF, 20fps, 100kbps 3.88W 5.38W
produce a perceptible quality change or power uptake compared to the nearestSIF encoded video with similar bit and frame rates.
Based on these conclusions, we identified eight dynamic video stream trans-formation parameters (Table 14.1) for our proxy-based realtime transcoding anduse the profiled average power consumption values to perform our adaptations.
14.7 SUMMARY
It has been pointed out by several researchers that power optimization acrossvarious levels of system functionality and implementation (architecture, OS,middleware, application) can lead to much greater savings than the case whenthese are individually optimized for power. The challenge is how these op-timizations can be coordinated across layers; what is the right architecturalframework that allows this optimization to occur simultaneously and even dy-namically? To answer this question, this paper proposes a proxy-based middle-ware solution to accommodate optimizations across diverse clients with limitedcomputation and battery power by controlling the amount of needed computa-tion and communication to the client device. We showed how such adaptation inthe middleware can be used to improve energy efficient delivery of multimediacontent in the case of streaming video. User perception of video also plays avital role in deciding the proxy-based video transformations and in identifyingarchitectural tuning “knobs". However, identifying the various video qualitiesremains a highly subjective aspect of the study. Identifying video quality levelsobjectively/programmatically still remains an open research challenge. In prac-tice however, the widespread deployment of such a unified power managementframework for mobile devices would require a set of APIs (programming inter-faces) to be implemented at the various computational layers; this API should fa-
270
cilitate effective communication between the various levels. Recent approachestowards power management suggest a more open and flexible architecture formobile devices that allows higher layers to make informed adaptations at lowerlayers and vice-versa. A prototype implementation of the framework is cur-rently underway as a part of the FORGE(http://www.ics.uci.edu/ forge) project.
Acknowledgments
This work was supported by funding from an ONR MURI Grant N00014-02-1-0715 and NSF NGS award ACI-0204028.
References
[1] Azevedo, Ana, Cornea, Radu, Issenin, Ilya, Gupta, Rajesh, Dutt, Nikil,Nicolau, Alex, and Veidenbaum, Alex (2001). Architectural and compilerstrategies for dynamic power management in the copper project. In Inter-national Workshop on Innovative Architecure.
[2] Chandra, Surendar (2002). Wireless Network Interface Energy Consump-tion Implications of Popular Streaming Formats. In Multimedia Computingand Networking.
[3] Chandra, Surendar and Vahdat, A. (2002). Application-specific NetworkManagement for Energy-aware Streaming of Popular Multimedia Formats.In Usenix Annual Technical Conference.
[4] Choi, Kihwan, Dantu, Karthik, Chen, Wei-Chung, and Pedram, Massoud(2002). Frame-Based Dynamic Voltage and Frequency Scaling for a MPEGDecoder. In International Conference on Computer Aided Design.
[5] Chu, H. H. and Nahrstedt, Klara (1999). "Cpu Service classes for multime-dia applications". In International Conference on Multimedia Computingand Systems.
[6] Cornea, Radu, Dutt, Nikil, Gupta, Rajesh, Mohapatra, Shivajit, and et. al.(2003). ServiceFORGE: A Software Architecture for Power and QualityAware Services. In FME Symposium.
[7] Douglis, Fred, Krishnan, P., and Marsh, B. (1994). Thwarting the powerhungry disk. In WINTER USENIX conference.
[8] Feeney, L.M. and Nilsson, M (2001). Investigating the Energy Consumptionof a Wireless Network Interface in an Ad Hoc Networking Environment.In IEEE Infocom.
271
[9] Feng, Wu-chi and Sechrest, S. (1996). Improving data caching for softwarempeg video decompression. In IS&T/SPIE Digital Video Compresssion:Algorithms and Technologies.
[10] Flinn, J. and Satyanarayanan, M. (1999a). Energy-Aware Adaptations forMobile Applications. In In Symposium on Operating Systems Principles.
[11] Flinn, Jason, de Lara, Eyal, Satyanarayanan, M., Wallach, Dan S., andZwaenepoel, Willy (2001). “Reducing the energy usage of office appli-cations". In IFIP/ACM International Conference on Distributed SystemsPlatforms.
[12] Flinn, Jason and Satyanarayanan, M. (1999b). PowerScope: a tool for pro-filing the energy usage of mobile applications. In Second IEEE Workshopon Mobile Computing Systems and Applications.
[13] Grun, P., Dutt, N., and Nicolau, A. (2003). "Memory architecture ex-ploration for programmable embedded systems". In Kluwer AcademicPublishers, Norwell, MA.
[14] Havinga, Paul J. M. (2000). Mobile Multimedia Systems. PhD thesis,University of Twente.
[15] Hughes, C. J., Srinivasan, J., and Adve, S. V. (2001a). Saving energywith architectural and frequency adaptations for multimedia applications.In IEEE/ACM International Symposium on Microarchitecture.
[16] Hughes, C. J., Srinivasan, J., and Adve, S. V. (2001b). Saving energywith architectural and frequency adaptations for multimedia applications.In IEEE/ACM International Symposium on Microarchitecture.
[17] Irani, S., Shukla, S., and Gupta, R. (2002). “Competitive analysis ofdynamic power management strategies for systems with multiple powersaving states". In Design Automation and Test in Europe.
[18] Kumar, Pavan and Srivastava, Mani (2000). Predictive Strategies for Low-Power RTOS Scheduling. In International Conference on Computer De-sign.
[19] Lee, C., Lehoczky, J., Siewiorek, D., Rajkumar, R., and et.al (1999). AScalable solution to the multi-resource QoS problem. In Real-Time SystemsSymposium.
[20] Mesarina, M. and Turner, Y. (2002). A Reduced Energy Decoding ofMPEG Streams. In Multimedia Computing and Networking.
[21] Mohapatra, Shivajit, Cornea, Radu, Dutt, Nikil, Nicolau, Alex, andVenkatasubramanian, Nalini (2003). Integrated power management forvideo streaming to mobile handheld devices. In ACMMM.
[22] Noble, B. D., Satyanarayanan, M., D.Narayanan, J.E.Tilton, and Flinn, J.(1997). Agile Application-Aware Adaptation for Mobility. In In Symposiumon Operating Systems Principles.
272
[23] Pasricha, Sudeep, Mohapatra, Shivajit, and et. al. (2003). "Reducing back-light power consumption for streaming video applications on mobile hand-held devices". In ACM/IEEE/IFIP Workshop on Embedded Systems forReal-Time Multimedia, 2003.
[24] Pillai, P., Huang, H., and Shin, K. G. (2003). Energy-Aware quality ofService adaptation. In Technical Report CSE-TR-479-03, Univ. of Michi-gan.
[25] Pillai, P. and Shin, K. G. (2001). Real-Time Dynamic Voltage Scaling forLow-Power Embedded Operating Systems. In In Symposium on OperatingSystems Principles.
[26] Shenoy, Prashant and Radkov, Peter (2003). Proxy-Assisted Power-Friendly Streaming to Mobile Devices. In Multimedia Computing andNetworking.
[27] Sinha, Amit and Chandrakasan, Anantha (2001). "jouletrack - a web basedtool for software energy profiling". In Design Automation Conference.
[28] Soderquist, Peter and Leeser, Miriam (1997). Optimizing the data cacheperformance of a software MPEG-2 video decoder. In ACM Multimedia,pages 291–301.
[29] Weiser, M., Welch, B., Demers, A., and Shenker, S. (1994). Schedulingfor Reduced CPU Energy. In In Symposium on Operating Systems Designand Implementation.
[30] Yuan, W. and Nahrstedt, K. (2004). Process Group Management in Cross-Layer Adaptation. In Multimedia Computing and Networking.
[31] Yuan, W., Nahrstedt, K., Adve, S., Jones, Doug., and Kravets, Robin(2003). Design and Evaluation of a Cross-Layer Adaptation Framework forMobile Multimedia Systems. In Multimedia Computing and Networking.
[32] Zeng, H., Ellis, C., Lebeck, A., and Vahdat, A. (2002). "Ecosystem: Man-aging energy as a first class operating system resource". In ArchitecturalSupport for Programming Languages and Operating Systems.
[33] Zeng, H., Ellis, C., Lebeck, A., and Vahdat, A. (2003). Currentcy: Unifyingpolicies for resource management. In USENIX.
273
top related