alpstein - cesnet...the alpstein goals 5 key characteristics how to reach more bandwidth reserves...

ALPSTEIN Building SWITCH’s 2nd optical backbone

CEF Workshop, Prague, 15.9.2014

Felix Kugler

[email protected]

© 2014 SWITCH

• ALPSTEIN intro

• The project plan

• Fiber footprint

• The new optical core

• Beyond the core

• Roll-out

• Closing

Agenda

2

© 2014 SWITCH 3

AgiLe Photonic Scalable TErabIt Network

ALPSTEIN ?

© 2014 SWITCH

...and what’s that all about?

SWITCHlan’s heaviest modification

since SWITCHlambda (2001-2005)

– topological extensions and modifications

– complete rebuild of the optical transport

system

– deployment of first 100G routers

4

© 2014 SWITCH

The ALPSTEIN goals

5

Key characteristics how to reach

more bandwidth reserves more optical channels

higher bitrate per channel

more flexibility

tunable components

photonic switching

sophisticated NMS

better geographical coverage

more reliability

additional, geo-diverse physical paths to key sites

optical switching at all branching points

protection & restoration mechanisms

(but beware of excessive complexity!)

high sustainability

reasonable economics

promising road map

vendor stability

© 2014 SWITCH

Why a lot more can be done today

6

2005

2014

shrink, shrink! sophisticated modulation formats

ALPSTEIN shall

benefit from all

these nice things!

optical switching (ROADMs)

this absolutely needs two fibers!

tunable optics

© 2014 SWITCH

• ALPSTEIN intro


• Fiber footprint


• Beyond the core

• Roll-out

• Closing

Agenda

7

© 2014 SWITCH

Project timeline

8

t

WTO

proc.

installation &

migration

tech

specs

topology modifications in

the core backbone

technology survey with

selected vendors

2011 2012 2013 2014 2015 2016

detailed

planning

operational phase with

continuous upgrades

ve

nd

or

de

cis

ion

no

w

off

icia

l s

tart

of

pro

jec

t

© 2014 SWITCH

Planned SWITCHlan backbone 2015

9

some minor topology

modifications are likely!

~phase 1

3Q2014

phase 2 until

mid 2015

phase 2 until

mid 2015

© 2014 SWITCH

The ALPSTEIN procurement

10

Project ALPSTEIN

RFQ “100G router

platform”

RFQ “optical transport

platform”:

• hardware

• software

• vendor services

selection of

CH local support

GATT/WTO GATT/WTO

• as an option of

the RFQ

• alternative ways

SWITCH reserved the

right to select a CH

supporter of its choice

© 2014 SWITCH

vendor decision: ECI, Cisco

Project timeline, zoomed in

11

conversion to uni-directional fiber infrastructure

footprint extension

topology optimization

test roll-out new optical layer procurement

process

refined

design

roll-out design

roll-out design

roll-out design

IP Layer continuous upgrades of production routers

introduction of a new 100G capable router platform procurement

process

2012 2013 2014 2015

now official project start

© 2014 SWITCH

• ALPSTEIN intro


• Fiber footprint


• Beyond the core

• Roll-out

• Closing

Agenda

12

© 2014 SWITCH

Footprint extensions 2013

13

backbone footprint extensions

first city rings

© 2014 SWITCH

9 fiber modifications for phase 1

14

fiber swaps and conversion to HWDM

© 2014 SWITCH

• ALPSTEIN intro


• Fiber footprint


• Beyond the core

• Roll-out

• Closing

Agenda

15

© 2014 SWITCH

Roll-out of the new DWDM core

16

Lausanne

bypass core backbone

extension 1Q2015

© 2014 SWITCH

ALPSTEIN deployment phase 1

17 in operation since Sep 11, 2014 !

© 2014 SWITCH

ALPSTEIN final deployment 2015

18

2600km of backbone fiber

81 usable lambdas

30 nodes

~ 2.5t of gear

© 2014 SWITCH

• Apollo: a modern platform (2012)

• key parameters of our installation:

– “photonic switching” at all fiber crossings (any lambda to any exit)

– no OTN switching; sub-lambda services generated on routers

– 81/88 lambdas available in our current setup

– “dispersion compensated” – allows mixed operation of

• low cost “standard” 10G waves (“on/off keying”)

• 100G coherent waves

– 1000km “photonic” reach

• two geo-diverse paths between any core node pair

– Raman-free design

– ROADMs 4D and (soon) 8D, ”flexible grid ready”

– tunable lasers and filter arrays

• rapid service turn up

• few spare parts

• permit flexible path protection or restoration

– two fiber system

– tolerant to fiber sharing with off-C-band waves

A new DWDM system: ECI Apollo

19

© 2014 SWITCH

Basic DWDM node building block

20

line-side ROADMs compensation amplification

inband

management

By all means, we tried to avoid 80 lambda OADMs for all degrees in a node – that’s huge!

• to build a pure layer 3 network with

routers in all nodes

• re-use same channels throughout

the network

• to support dynamic photonic services,

• express channels to distant nodes,

• future flexgrid services

© 2014 SWITCH

a typical 3D static

node…

for dynamic nodes add:

– add/drop ROADM

– tunable filter arrays for

conventional

transponders

– splitters for coherent

transponders

DWDM nodes

backdoor

access

© 2014 SWITCH

• inline amplification (ILA):

– no OADMs

– single amplifiers per direction

• dynamic nodes come in many different setups • collocated:

– everything in a single room

• distributed:

– at least on degree in a different location

– needs <degree> fiber pairs between locations

• dual add/drop:

– redundant build-out of the dynamic add/drop part

– requires an extra degree

DWDM nodes (2)

22

© 2014 SWITCH

Backbone link matrix (phase 1)

23

• static blue network • static “alien” 10G lambdas to next

hop routers (unprotected)

• DWDM optics plugged into routers

– low cost (no transponders, no

tunable lasers)

– reliable (independent of NMS)

– goes well with existing router base

• dynamic red network – 10G and 100G waves (protected)

– use of transponders, white optics

towards clients

– transparent lambdas between any PoP

pair

– flexible (directly reach any node)

– scalable

– reliable (restoration, protection)

Actually, we build two core backbone networks!

© 2014 SWITCH

Channel plan

24

• 50GHz spacing today, ready for superchannels

• reflects the two networks we build

• blue network uses same lambdas throughout

the net (mostly ch57)

• red network: balance between OOK and

coherent lambdas will shift over time

• amber network: reserved for distance-limited

dynamic channels, permits re-use of lambdas

[only rough ideas]

10

0G

+ c

oh

ere

nt w

ave

s

10

G O

OK

wa

ve

s

4 channels reserved

for aliens to

next hop routers

8 channels reserved

for future city &

regional networks

rest of channels

dedicated to

dynamic lambdas

small guard band

between OOK and

coherent lambdas

1529.55nm 60

1532.68nm 56

1535.82nm 52

1563.86 17

4

8

71

© 2014 SWITCH

• most alien channels are lit by plugins in our routers

• Xenpak and X2 were dominant on our favorite router platform

– unpleasant availability and pricing of plugins

– we try to avoid buying them since many years!

• use of SFP+ & suitable adapters instead!

– colored SFP+ plugins will have a long life – conservation of value!

– programmable to fit host platform

– weired problem with adapters now solved: compatibility, bit errors

• SFP+ performance is not (yet?) 100% up to Xenpak/X2 levels

– lower sensitivity

– smaller dispersion tolerance

About colored plugins

25

© 2014 SWITCH

• ALPSTEIN intro


• Fiber footprint


• Beyond the core

• Roll-out

• Closing

Agenda

26

© 2014 SWITCH

Beyond the core

saving fiber lease cost is the goal – more than ever!

• share fibers between core backbone and access links

• share with suitable partners (e.g. mutual backup paths)

27

• small sites daisy chained on their own

dedicated single fiber

• backbone and regional network are fully

isolated

• this setup will be quite incompatible with

modern DWDM systems

• DWDM (extended C-band) and other optical

systems (O/S/L-bands) share a fiber pair

• passive optical filters @fiber junctions separate

the services

• restricted by max span attenuation

• careful power management needed to avoid

interference

up to now... …after migration

© 2014 SWITCH

HMUX splitters

28

DWDM + OSC regional links CWDM short local LR links currently unused

• two year evolution from a cascade of CWDM OADMs to a sophisticated optical filter with 4 windows

• 3rd generation of “HMUX”-splitters is now widely deployed

• precise cut-over frequencies

• very low loss

compatible with • operational C-band DWDM system incl. OSC

• half of the “standard” CWDM colors

• long-range optics LR4 & LR10

© 2014 SWITCH

HMUX splitter performance

29

• production seems tricky, so careful

selection of the best pieces – we

suspect high reject rate

• measurements: – C-band attenuation <0.9dB

– neighbor channel suppression >25dB

– cutoff frequency variation <3nm

overlay of S- and C-band port attenuation

© 2014 SWITCH

1HE rack mount carrier holding 3 front loadable trays, each with

– 2x HMUX 4-window splitter or any combination of CWDM filters

HMUX splitter mechanics

30

two optical modules fit into one metal tray

© 2014 SWITCH

HWDM operation

31

• about 20 HMUX installations up to now

• careful documentation is paramount!

– separate maps for non-DWDM optical layer

in addition to DWDM system’s docs

• minimize interference!

– HMUX filters are not perfect – about >23dB

of isolation; crosstalk adds to noise level

– non-linear effects (Raman effect) if power

levels get high ?

– we keep O/S-band signals as low as

possible (tx-side attenuators)

© 2014 SWITCH

HWDM use cases

backup network access

– avoid single point of failure

– avoid full blown 2nd PoP

– tradeoff: HWDM allows

emergency 10G connectivity at

a low price point

backup paths for partner networks

– provides transparent optical channels

outside C-Band – SpaaS ?

– high isolation

– coexist with SWITCH backup paths

– requires agreement on power levels

32

access to small sites along backbone

– drastically reduces cost

• only little extra fiber to be leased

• low cost optics will do

1

1 2

2

3

3

© 2014 SWITCH

Phase 1 deployment schedule

35

XX Sorrento or BTI DWDM node

XX next generation DWDM node

CE

LS

BE

BS

EZ

AG

RF

LG

LO

VS MY

EL

GE

FR

EZ

AG

RF

LG

LO

VS MY

EL

GE

CE

LS

BE

BS

FR

X X

SI

© 2014 SWITCH

Installation

36

• ECI DE: collect gear from various sources and pack per installation site

• shipment from ECI DE to our partner Deltanet

• preparation “@home”: – assembly of chassis

– basic configuration of node controllers

– fitting of power cables

– results in a substantially reduced transport volume

– reduces installation time on site!

• installation on site (0.5..2 days/site) – mount all chassis

– connect & configure power feeds

– carefully plug the fiber patches

– Ethernet connectivity for ECI chassis & rectifiers

– integrate local chassis into NMS

– beautify cabling

© 2014 SWITCH

• 2..3 hops per day

• ECI + SWITCH staff at all nodes

• Apollo setup procedure

– systematically check optical paths from transponders towards exits

– compare to simulation – debug if necessary

– fine tune certain critical power levels within the node

– measure in-band OSNR values for future reference

– bring up optical services

• SWITCH staff to

– monitor network performance, make sure there is no impact on our users

– connect Apollo client ports to routers, bring up services

– short function test on site

– in some cases, considerable reconfiguration work is required simultaneously

Migration

37

© 2014 SWITCH

• SWITCH’s innovation cycle driven by need, not funding periods

• the intension is to live with the new DWDM system a long time

– with upgrades and new forthcoming vendor components

– with appropriate optical add-ons from other sources (“multi-vendor”)

• design & roll-out gradually, not by fork-lift – very closely assisted by SWITCH staff

– build know-how about the new system

– continuously adjust the design to the latest requirements

• hot migration without new fibers while clients stay connected

– acquisition of a few new permanent fiber links instead of tons of temporary ones

– these backbone extensions facilitated the migration process as well

– migration still rather tricky !

• fiber sharing – to save lease cost

– to foster cooperation with our regional partner networks

Recalling some key points…

39

alpstein - cesnet...the alpstein goals 5 key characteristics how to reach more bandwidth reserves...

Documents