Optical PacketOptical Packet--Switched Switched WDM Networks: WDM Networks:
a Cost and Energy Perspectivea Cost and Energy Perspective
Rodney S. Tucker
ARC Special Research Centre for Ultra-Broadband Information Networks (CUBIN)
University of Melbourne
Acknowledgements: Jayant Baliga, Kerry Hinton, Rob Ayre, Gangxiang Shen, Wayne Sorin
Australian Research Council, Cisco
SummarySummary
Optical packet-switched networks
Network architectures- Point-to-point WDM IP networks- IP networks with optical grooming- Optical burst switching- Optical packet switching
CAPEX and network architectures- Scaling
Energy consumption and network architectures- Core and access networks
Disclaimers: - Numbers given here are approximate - YMMV- OPEX not included
E-mail me for copies of these slides
Impact on network growth
Edge Routers Optical PacketOptical Packet--Switched WDM NetworkSwitched WDM Network
1 hop
Is this simple model realistic?
IP Packets Core Router
OXC
Patent(s) Pending and Copyright © LumetaCorporation 2007
The Reality: Map of the InternetThe Reality: Map of the Internet
Not Included:• Enterprise Networks• Content Distribution Networks• Data Centres
Number of Hops in the InternetNumber of Hops in the Internet
0 5 10 15 20 250
0.02
0.04
0.06
0.08
0.1
Number of Hops, k
[]
Pr
Hk
=
Source: P. Van Mieghem,“Performance Analysis of Computer Systems and Networks”, Cambridge (2006)
2006 Data
Optical Burst
Switching (OBS)
Evolution of Optical PacketEvolution of Optical Packet--Switched NetworksSwitched Networks
Capacity
Time
Point-to-Point WDM (P2P)
Optical Circuit
Switching (OCS)
Stage 1
CAPEX? Energy Consumption? Optical
Packet Switching
(OPS)
““ConventionalConventional”” WisdomWisdom
1. Point1. Point--toto--Point WDM NetworkPoint WDM Network
Router provides sub-wavelength groomingby statistical multiplexing
LightpathFiber O/E/O Converters
Core Router
O/E/O Converters
Access Routers
Edge Routers
Core Router
1 hop
OXC
2. Optical IP Network2. Optical IP Network
2. Optical IP Network2. Optical IP Network
Lightpath
OXC/ROADM
Core Router
Fiber
• Optical bypass • Sub-wavelength grooming
O/E/O Converters
O/E/O Converters
Access Routers
3. Waveband IP Network3. Waveband IP Network
MUX
• Waveband grooming• Optical bypass • Sub-wavelength grooming
Lightpath
FiberWaveband
OXC/ROADM
Access Routers
O/E/O ConvertersCore
Router
Consider three network architectures:
Cost and Scalability of Optical NetworksCost and Scalability of Optical Networks
Compare CAPEX
Parthiban et al., 2003
Number of users: 10 million
P2P WDM Optical IP Waveband IP
Input ParametersInput Parameters
Capacities:Router: 5 – 90 Tb/sOXC: 1000 portsLightpath: 40 Gb/sFiber: 250 lightpaths
250 λ
1 λ = 40 Gb/s
1000 1000
Average Access rate per user: 1 - 100 Mb/s10 million users
Component costs [Ferreira 02, Sengupta 03]OXC, router – chassis, portFiber, amplifiers, lightpath terminations
IgnoreAccess network, search engines, data centers, etc.Optical impairment cost – e.g. regeneratorsProtection & restorationMultiple domainsCost reductions as technology maturesOPEX
Results Results –– Network CostNetwork Cost
1
102
104
1010 100
Average Access Rate (Mb/s)
Cos
t per
Use
r ($)
Parthiban et al., 2003
P2P WDM
Optical IP
Waveband IP
Today’s Internet (~ 100 kb/s) (Access rate per user in a busy period)
Slope = 1
Cumulative Internet Capex (North America)
0
10
20
30
40
50
60
70
80
90
2000 2005 2010
TotalCore and
Metro
Access
Optical transport
Year
Cum
ulat
ive
Cap
ex $
Bill
ions
Based on data from Nemertes Research, 2007
~$400 / user
Sanity CheckSanity Check
150 million users
Nemertes, November 19, 2007: “User demand could outpace network capacity by 2010$137 billion global infrastructure investment needed”
• Optical IP network – Can save costs in today’s network– Optical bypass reduces router ports
• Waveband IP network– Eliminates the bottleneck in number of ports and lightpaths– Least costly for high access rates
• There is no such thing as a free lunch– If you want more bandwidth, you will have to pay for it
ObservationsObservations
Energy Consumption of the NetworkEnergy Consumption of the Network
Power In
• Greenhouse Impact
• Managing “Hot Spots”- Getting the energy in- Getting the heat out
• Energy-limited capacity bottlenecks
Why worry about energy consumption?
• OPEX
Hot spot
Metro
Core
Edge Edge
Curb Curb Curb Curb
Core Core
Access
Core
ONU ~ 5-10W
OLT - 100W
CRS-1 ~ 10 kW / rack
12816 Edge ~ 4 kW
0.1 - 1000 Mb/s to the user
Fibre Amps
WDM
Passive Optical Network
Packet over
Sonet
Energy Model of Simple IP NetworkEnergy Model of Simple IP Network
Baliga et al., 2007
1000
Pow
er (W
/use
r)
% o
f Ele
ctric
ity S
uppl
y
Baliga et al., 2007
0
20
0.5
25
5
Average Access Rate (Mb/s)
15
4 862
1.0
10
20 hops
Power Consumption of IP NetworkPower Consumption of IP Network
Total
Core
Access +Metro
SDH/WDM Links
Today’s Internet (~ 100 kb/s)
2008 Technology
10000
Pow
er (W
/use
r)
% o
f Ele
ctric
ity S
uppl
y
Baliga et al., 2007
0
150
2.5
200
50
Average Access Rate (Mb/s)
100
40 806020
5.0
7.5
10
Total
Core
Access +Metro SDH/WDM
20 hops
Power Consumption of IP NetworkPower Consumption of IP Network
2008 Technology
• Access network dominates energy consumption at low rates– Standby mode?
• Core network dominates at higher rates– Reduce hop count?
• What is the bottleneck in the core? Speed or energy?
– Optical packet switching
• Optical transport (WDM) consumes relatively little energy< 5% of energy> 25% of CAPEX
• Annual CAPEX / Annual energy OPEX > 2
Observations / QuestionsObservations / Questions
1
3
2
OpticsElectronics
1
F
1
K
1
Switch Fabrics
F
Buffers
F Demutiplexers F
Multiplexers
Fibers
Forwarding Engine
J
1
SwitchFabric
J
O/E Converters
Reduced bit rate
Electronic RouterElectronic Router
Router Throughput
Pow
er c
onsu
mpt
ion
(W)
Source: METI, 2006, Nordman, 2007
Power Consumption in RoutersPower Consumption in Routers
10
100
1,000
10,000
100,000
1,000,000
P = C2/3
where P is in Wattswhere C is in Mb/s
10 nJ/bit
100 nJ/bit
11 Pb/s
P ~ 10
1 Tb/s1 Gb/s1 Mb/s
?
Ener
gy p
er B
it(n
J)Energy per Bit in RoutersEnergy per Bit in Routers
1
10
100
1,000
10,000
100,000
10 nJ/bit
100 nJ/bit
0.1
1 nJ/bit
Router Throughput
1 Pb/s1 Tb/s1 Gb/s1 Mb/s
?
Year
Source: K. Brill, The Uptime Institute
Heat LoadHeat Load
1 nJ/bit
Air Cooling Limit
Switch Fabric
Buffer
I/O
Switch Control
Data Plane Control Plane
Line Card
Energy in HighEnergy in High--End Electronic RouterEnd Electronic Router
Routing Engine
Routing Tables
Power supply
inefficiency
Fans and blowers
35%11%32%7% 5% 10%Fraction of Total
Source: G. Epps, Cisco, 2007
Buffer
O/E
Buffer
O/E Forwarding Engine
Energy/bit 0.7 nJ 1.0 nJ0.5 nJ3.2 nJ 3.5 nJ1.1 nJ
Forwarding Engine
Total
Metro + AccessCore
0.1 1 10 100
Ener
gy p
er b
it (J
)
10-6
10-3
10-8
10-5
10-7
10-4
20 hops
~1 μJ/b
WDM Links
Average Access Rate (Mb/s)
Network Network EnergyEnergy Consumption per BitConsumption per Bit
Energy in Electronic Integrated CircuitsEnergy in Electronic Integrated Circuits
CMOS GatesCwire
⎡ ⎤= + ⎣ ⎦∑ ∑ 21Energy2gate wireE C V
=Power Energy x Bit Rate
CMOS IC
Diversion: Diversion: CapacitanceCapacitance of the Core Networkof the Core Network
30 F (North America)NetworkC ≈
300 F (Global)NetworkC ≈
Source
→ = × 8~ 150 million users 200 nF 1.5 10NetworkC x
50% efficiency
Energy per bit per user: 212E C V⎡ ⎤= ⎣ ⎦∑
= = → ≈∑1 μJ2, 200 nF2
V E C
V Destination
Capacitance per user
C1 C2
Energy Consumption in Access NetworksEnergy Consumption in Access NetworksNEC CM7710T
Splitter PON
FTTN
PtP
WiMAX
Cabinet
Edge Node
Cisco 12816
NEC CM7700S
ZyxelVES-1616F-34Cisco
4503
NEC VF200F6
NEC GM100
Axxcelera ExcelMax CPE
AxxceleraExcelMax BTS
Cabinet
Baliga et al., OThT6
Access N/W
Average Access Rate (Mb/s)
Baliga et al., OThT6
Energy Consumption in Access NetworksEnergy Consumption in Access Networks
• Wireless access consumes more energy than optical access• PON FTTH is “greener” than FTTN• “Standby mode” shows significant potential (OThT6)
Ener
gy p
er b
it (J
)WiMAX
FTTN
PtP
PON10-7
10-5
10-8
10-6
1 10 100 1000
Stage 1
Time
Optical Burst Switching
(OBS)
Point-to-Point WDM
(P2P)
Stage 2Optical Packet
Switching (OPS)
Capacity
Evolution of Optical PacketEvolution of Optical Packet--Switched NetworksSwitched Networks
Optical Circuit
Switching (OCS)
CAPEX and Energy Barrier
All-optical sub-wavelength grooming
Burst Assembly Router
Core Router
Edge Router
Stage 2 Evolution Stage 2 Evolution –– OCS to OBSOCS to OBS
Access Routers
Edge Node
OXC
ElectronicsWDM Link
Optical Burst Switches
Optical Burst Switched NetworkOptical Burst Switched Network
t
Optical Burst Switch No Buffering
IP Packets
Burst Assembly
Router
IP Packets (duration < μs)
Data Bursts (duration < ms)
Headers
Offset t
C. Qiao and M. Yoo, JHSN, 1999
Optical Burst SwitchingOptical Burst Switching
Source Destination
D
AT
A
Offset
Burst length< ms
Header
Time
Distance
Switch 1 3 2 4 5
Nor
mal
ized
Cos
t
20
With wavelength conversion
10-6 10-5 10-4 10-3 10-2 10-1
5
0
10
Average Path Blocking Probability
15
Normalized Cost: OBS vs. IPNormalized Cost: OBS vs. IP
Waveband IP
OBS
Without wavelength conversion
1 Gbit/s Access Rate
Optical IP
Parthiban et al., OFC 2005
Why is OBS more Costly?Why is OBS more Costly?
Waveband
MUX
OBSDEMX
Waveband
Wavelengths Wavelengths
Bursts Bursts
• Requires increased number of lightpaths for a given blocking probability
• Switch technology requires fast reconfiguration time: More costly per port than “slow” OXC’s (MEMS etc.)
Burst
WavebandWBS
Waveband Route
λ
t
Pro’s:• Requires fewer OXC ports
Con’s:• OXC ports must be wideband• Requires waveband (i.e. multi-channel) wavelength conversion• Dispersion issues
Waveband
Solution: Waveband Burst Switching (WBS)Solution: Waveband Burst Switching (WBS)
Y. Huang et al., OFC 2004, Pathiban et al., OFC 2006
Average Path Blocking Probability
Nor
mal
ized
Cos
t
5
Optical IP
Waveband IP
10-2 10-1
WBS (WC) + Deflection Routing + Burst Segmentation
WBS with wavelength conversion
Waveband Burst Switching (WBS)Waveband Burst Switching (WBS)
3
1
7
Parthiban et al., OFC 2006
1 Gbit/s Access Rate
10-5 10-4 10-3 10-2 10-1
Stage 1
Stage 2
Time
Optical Burst
Switching (OBS)
Optical Packet
Switching (OPS)
Point-to-Point WDM
(P2P)
Optical Circuit
Switching (OCS)
Stage 3Capacity
Evolution of Optical PacketEvolution of Optical Packet--Switched NetworksSwitched Networks
• Will a viable optical buffering technology emerge?
• Can OPS reduce energy consumption?
Optical Packet Switches
With Buffering
Access Router
Optical PacketOptical Packet--Switched NetworkSwitched NetworkThe “Holy Grail” of Optical Networking
OXC
Demutiplexers
Optical Packet Switch (OPS)Optical Packet Switch (OPS)
1
F
1
K
Wavelength-Interchanging
Cross Connect + Header
Replacement
F
Output Buffers
1
Forwarding Engine
Input SynchronizersMutiplexers
Electronics Optics
12
Delay
Delay
Delay LineVariable
Delay Line
Cross Point
Recirculating Loop
Staggered Delay Line
Optical Buffer StructuresOptical Buffer Structures
Delay
Delay
Control
Cross Point
Cross Point
.
Optical Fiber BuffersOptical Fiber Buffers
Cisco CRS-1 with 1000 ports, 250 ms buffering per port
OPS with 1000 ports, 250 ms buffering per portoptical fiber delay lines
Total buffer capacity of 104 Gigabits
~ 103 RAM chips
Cost < US$ 50k
Buffer power dissipation < 1 kW
Total fibre length = 40 Gm
150 times distance from Earth to Moon!
Total fibre length = 1000 km~5 μs (~20 packets) buffering per port
Loss per port = 0.2 dB
Loss per port = 33,000 dB
Optical fiber delay lines
Planar waveguide or slow light delay lines
(0.1 dB/cm)
Reduced Buffer SizeReduced Buffer Size
Minimum feasible buffer size?
Loss Energy
Tucker, PS 2007
Holographic BuffersHolographic Buffers
Input image
Output image
Reference beam
Orlov et al., Proc IEEE, 2004Psaltis, CLEO 2002Ashley et al. IBM J. Res. Dev., May 2000
Read speeds up to 10 Gb/s
Holographic Medium: Photorefractive material (e.g LiNbO3, polymers)
Write speeds up to 1 Gb/s
Storage density
Retention time
Access Time Write Time
∞ > 500 μs1/λ3 ~ 50 μs
Show stopper
Comparison of Optical Buffer TechnologiesComparison of Optical Buffer Technologies
Challenge
Technology Fiber Planar WG, Slow Light
Optical Resonator Holographic CMOS
Access Time
Structure-dependent
Structure-dependent Small ~ 50 μs 200 ps
Retention Time > 500 μs < 5 μs 1-100 ns ∞ 64 ms
Capacity (Packets) > 2,000 < 20 << 1 ∞ ∞Energy/bit ~ 1 fJ ~ 1 pJ ~ 1 pJ ~ 1 pJ ~ 1 fJ
Physical Size
Very Large Medium Medium Small Very
Small
Chirp Sensitivity No Small Large Large No
Tucker, PS 2007
Optical Interconnects
ChipsAWG’s
Wavelength Converters
Packet Switch CrossPacket Switch Cross--Connect TechnologiesConnect Technologies
AWG-Based CMOS
SOA-Based
~10 pJ/bit ~5 pJ/bit
>100 pJ/bit
3-stage Clos
(2 pJ/bit)
(1 pJ/bit)
(1.5 pJ/bit)
(10 pJ/bit per gate)
Tucker, JLT 2005, OSN 2008
Projected Energy Consumption
Benes
Benes Array
• No viable optical buffering technology in sight
• Optical switch fabrics may become competitive with CMOS
• Not clear whether optical packet switching will solve the energybottleneck problem
Observations on Optical Packet SwitchingObservations on Optical Packet Switching
Summary
• Optical bypass reduces CAPEX and energy consumption- Promising future for optical cross-connects and ROADMs
• Energy bottleneck in routers is looming- More significant than the so-called “electronic speed bottleneck”
• Can optical packet switching overcome the energy bottleneck?- Optical buffering is currently a show-stopper
• Think “Energy per Bit”