gÉant fibre and lighting designing the next generation gÉant optical layer

connect • communicate • collaborate

GÉANT fibre and lightingDesigning the next generation GÉANT optical layer

Guy Roberts, DANTE13 September, 2010


Contents

GÉANT network PoPs, architecture, fibre types Wavelength services, regeneration and channel planning Growth projections

Fibre footprint Infrastructure analysis - route diversity Diversity case studies

40/100G 40G trial 100G and coherent, PM-QPSK and fibre types

GÉANT3 RFI: re-engineer the photonic layer? RFI process and outcomes


GÉANT Network


GÉANT network

Local campus networks link to NRENs which interconnect via GÉANT backbone

Transfer rates of up to 10Gbps across 50,000 km of network infrastructure

25 Points of Presence (PoPs), 44 routes and 18 dark fibre routes


Dark fibre topology

5

LON

PARVIE

COP

BRA

DUB

FRA

AMSBRU

GEN

MAD

BUD

MIL LJU

PRA

ZAGBAR

12,000 route km

Two dark fibre rings – Western Ring and Eastern Ring

Western Ring:UK-BE-NL-DE-CH-FR

Eastern Ring:SK-HU-HR-SI-AT

Rings interconnected: CH-IT-ATDE-CZ-SK

DF spurs/loops to:ES, IE, DK

Western Ring

EasternRing


A big European mesh...

GÉANT backbone is only part of the bigger picture

When NREN connectivity is overlaid can see full complexity

CBF – increasing role?

This is needs updating for: Rediris Garr Hungarnet

Updates please!


GÉANT Fibre

6 fibre providers: Level3 Colt Interoute TeliaSonera Invitel (previously

Memorex and Pantel) Global Crossing

G.655 (E-LEAF)• All Western ring• Madrid, vienna...

G.652 (SMF)• Routes to Copenhagen• Eastern ring (?)

below 200

200-399

400-599

600-799

800-999

1000-1199

1200-1399

1400-1599

1600-1799

1800-1999 over

2000

4

3

5 5

1 1 1

0 0 01

PoP-PoP distance (km)

below 50

50-59 60-69 70-79 80-89 90-99 100-109 110-120 over 120

14 1724

44

31

18

54 7

Hut-hut distance (km)

Some long routes!


Alcatel LightManager

Hybrid networking was introduced in GÉANT2

~100 x 10 Gb/s multi-hop wavelengths currently deployed

Lambda services delivered using Alcatel’s 1626LM equipment

1241km unregenerate reach on G.652 Frankfurt-Copenhagen

Attenuation up to 28dB between amplifiers

Point-to-point only – no ROADM functions used

40 Gb/s field trial successfully completed, these transponders now to be used for IP: Amsterdam-Frankfurt Frankfurt-Geneva

1626LM


Managed wavelength service

POP A

POP BPOP D

GÉANT

• 10G and 40G SDH clients• Static routing and OADM• Full rate 10GE LanPhy

POP C

Manually patched at intermediate sites


GÉANT 10G lambda traffic matrix(Feb 2010)

10

AMS FRA GEN PAR LON COP PRA MIL MAD LJU ZAG BRA BUD VIE BRU DUBAMS * FRA 4 * GEN 16 5 * PAR 2 2 4 * LON 4 5 4 4 * COP 3 4 2 0 2 * PRA 2 3 0 0 0 0 * MIL 0 2 5 0 0 0 0 * MAD 4 3 6 0 0 0 0 0 * LJU 0 0 0 0 0 0 0 0 0 * ZAG 0 0 0 0 0 0 0 0 0 1 * BRA 0 0 0 0 0 0 1 0 0 0 0 * BUD 0 0 0 0 0 0 1 0 0 2 2 1 * VIE 0 0 0 0 0 0 2 4 0 4 3 2 6 * BRU 2 0 0 0 2 0 0 0 0 0 0 0 0 0 * DUB 0 0 0 1 4 0 0 0 0 0 0 0 0 0 0 *

All end-to-end wavelengths: Wavelength services IP wavelengths GÉANT+ wavelengths


GÉANT: 100 x 10Gb/s wavelengths

All lambdas....

Michael Enrico

Diagram is out of date, will need to note that colour scheme refers to project applications or maybe add legend to this effect, not all waves are self-provided, purple=LHCOPN related, black=IP trunk/access/backup, blue=GEANT Plus trunks, dashed lines are now solid black, TR still on 2.5G lines


Wavelength exhaust?

System is designed for up to 40 wavelengths on all routes.

G.655 fibre requires spacing of 100GHz for first 20 wavelengths with 50GHz infill available for next 20 wavelengths.

Alcatel have modified their wavelength spacing rules over the past 4 years as they refine their modelling tools.

Currently fibre is approx. 50% full on Western Ring, 19 wavelengths on Amsterdam-Frankfurt and Frankfurt-Geneva spans.

Capacity planning projections suggest that design limit of 40 channels will be exhausted in lifetime of GÉANT3 project on some routes (if no 40/100G channels are deployed).


Dark fibre footprint


DANTE has an ongoing exercise to enter all GEANT dark fibre and leased wavelength routing into a Geographic Information System.

Currently using Google Earth – data is being distributed to NRENs

Fibre providers supply data of various formats and quality

Main goal is to identify shared risk groups (often very tricky to be certain about these; “intelligence” on deals between operators is often useful as well – swaps, purchases, etc)

Note that shared routes does not necessarily mean shared huts

For completeness this should (but does not yet) include NREN routes

Two case studies presented

Dark fibre routing


Fibre footprint, fibre types

fibre, routing

GÉANT dark fibre routes


4 dark fibre routes, 4 wavelengths and 1 CBF route all terminate in Frankfurt

DF routes go to: Amsterdam, Copenhagen, Prague, Geneva

Difficult to keep track of routing of shared routing risks. This could be a shared trench, shared duct or shared cable – these are not easy to distinguish

Frankfurt city authorities limit the number of roads available for fibres – increases risk of shared routes

Case study 1: Frankfurt metro


Case study 1 - Frankfurt metro

GÉANT: Frankfurt dark fibre metro routes

Shared risk?


Major fibre cut between Geneva and Basel - has happened at least once

Resulted in loss of both GEN-FRA and GEN-MIL

IP traffic between Geneva and Milan (and S&E of there) will reroute around a very long loop:

Geneva-Paris-London-Brussels-Amsterdam-Frankfurt-Prague-Bratislava-Vienna/Budapest...

If providing restorable wavelengths then substantial regen would be required to make the restoration path feasible

18

Case Study 2: LHC OPN and Geneva


To Paris

To Madrid

To Frankfurt

To Milan

Shared risk?



LHCOPN procured 3rd party leased wavelengths

Analysis of LHCOPN performed by Michael Enrico (back in 2007) showed that leased wavelengths had some shared risks

For example, up to 8 wavelengths may share a routing risk between Geneva and Basel

Ways to improve resilience levels (3 examples, may be others): add a (GEANT) fibre route between Marseille (+ 3-d ROADM) and Milan add a third diverse fibre route out of Geneva to Milan - this will allow for

restorable wavelength services use a CBF route (ensuring it is based on diverse fibre) to introduce a 2nd

diverse lambda route between Geneva and Milan - this will allow for restorable sub-wavelength services and improve all round IP resilience


SRLG analysis

DEFrankfurt

Basel

T1 GRIDKA

T1

Zurich

CNAF

DK

Copenhagen

NL

SARA

UK

London

T1

BNL

T1FNAL

CH

NY

Starlight

MAN LAN

FR

Paris

T1

IN2P3

Barcelona

T1

PIC

ES

Madrid

T1

RAL

ITMilan

Lyon

Strasbourg/Kehl

GENEVA

AtlanticOcean

VSNL N

VSNL S

AC-2/Yellow

Stuttgart

T1 NDGF

T0

HamburgT1SURFnet(Between CERN and BASEL)

Following lambdas run in same fibre pair:CERN-GRIDKACERN-NDGFCERN-SARACERN-SURFnet-TRIUMF/ASGC (x2)USLHCNET NY (AC-2)Following lambdas run in same (sub-)duct/trench:(all above +)CERN-CNAFUSLHCNET NY (VSNL N) [supplier is COLT]Following lambda MAY run in same (sub-)duct/trench as all above:USLHCNET Chicago (VSNL S) [awaiting info from Qwest…]

T1

TRIUMF T1

ASGC

???

Via SMW-3 or 4 (?)



40G and 100G


Alcatel 40 Gb/s p-DPSK

Technology/cost challenge led to gong delay between 10G and 40G: Alcatel & Lucent spent 10+ years investigating 40G solutions

Designed to have reach that is similar to existing 10G and no guard bands required

Partial DPSK uses filtering to fit into 50 GHz channel spacing – easy to mix with existing 10G wavelengths

Problems experienced by JANET with PMD are not seenso far on GÉANT fibres

Not clear if 40G has a future (even as an interim step) –is market moving more rapidly to 100G than envisaged sayeven 6 months ago?

P-DPSK


40G wavelength trial

CHIT

DE

40G SDH analyser

40G pass-through

40G

PIC

40G PIC

40G trial Frankfurt-Geneva-Milan

Juniper T1600 SDH STM-256 PICs

Ran stability tests with Xena 10G Ethernet testers & Monitored PM data

Stable operation observed


100 Gb/s - PDM-QPSK andcoherent detection

Alcatel has developed 100G using PDM-QPSK and coherent detection

Polarization Division Multiplexing – Quadrature Phase Shift Keying

Other vendors are also using similar approaches for 100G

Coherent detection (mixing with local oscillator at receiver) allows phase information to be retained, this enables digital signal processing (DSP) to compensate for chromatic dispersion.

Eliminates the need for dispersion compensation fibre modules (DCM).

Detection of polarization modes at the receiver also allows DSP to compensate for polarization mode dispersion (PMD).

Complex Forward Error Correction (FEC) mechanisms now able to provide up to 10dB or more receiver margin, but can add latency.


GÉANT3 RFI:Re-engineer the photonic layer?


GÉANT RFI process

GÉANT has been engaging equipment vendors in an RFI process

The goal is to understand technology options for GÉANT3

Particular focus on the WDM/OTN layer

Vendors have been provided with fibre characteristics in the Western ring of the GÉANT network

Respondents have been asked to provide designs based on fibre data and capacity assumptions described in ‘Reference Network’ – see next slide


Reference network for RFIApply to this reference network:

Full mesh of 3x10G (excl. Bru & Lux)Full mesh of 1x 40G (ditto)Full mesh of 1x100G (ditto)Combination of these

28

LON

BRU AMS

FRA

GENPAR

COP

LUX

G.655441km

G.655290km

G.6521122km

G.6521241km

G.655641km

G.655737km

G.655818km

G.655658km


All coherent transmission?

GÉANT network designed for 10G transponders, 100G coherent technology offers performance improvements

Should we re-engineer parts of the common photonic layer to take advantage of coherent technology? Addition of some new ILA sites (where huts have previously been “skipped”)

Removal of dispersion compensation fibre

The addition of gain equalisers in some ILA sites

This will require a significant up-front capital investment to replace 10G transponders with 100G muxponders – cost benefit analysis required.

RFI includes questions to help this process – decision also depends on other issues such as capacity growth projections and architecture choices.


Coherent transmission – RFI results

Easier for green field rollout

No discrete 10G coherent! Only 4/10x10G muxponders so 10G traffic matrix needs to be commensurate

How will this fit in with GÉANT fibre footprint?

30

Also used Raman!

0

5

10

15

20

25

30

35

DCM- mixed noDCM- mixed noDCM- SMFonly

Regen Reqs (vendor A)

40G coh

100G

0

5

10

15

20

25

Vendor A Vendor B Vendor B - ng100G

Regen Reqs (vendor A vs. B)(noDCM, mixed fibre)

40G coh

100G


ROADMS, to use or not to use?

Are ROADMs suitable to the GÉANT backbone?

Pros: Rapid service delivery Restoration using GMPLS Reduced regeneration cost on multi-hop routes

Cons: Complex and expensive Large floor space requirements Where 10x10G muxponders used all 10 sub-wavelengths have to be

demuxed (like old PDH) at add-drop sites Potential benefits of reduced regeneration is limited for GÉANT due to very

long routes Restoration requires a lot of extra spare regeneration capacity to support very

long restoration paths


Summary

It is important to share European fibre footprint information to reduce risk Common GIF system helpful for this process Role for CBF?

ROADMs much more appealing for GÉANT than in “pre-coherent era”

RFI results show a significant variation in reach between vendors Not sure why yet (better xponders?, optimistic/pessimistic and/or more/less

thorough modelling???)

The move to coherent technology – an opportunity exists to re-engineer the photonic layer - up-front investment needs cost-benefit analysis

All coherent transmission scheme certainly looks appealing but only if expected (lambda) traffic matrix is suitable long-haul lambdas (not just POP to neighbouring POP) and enough of them (e.g. 10G channels may come in chunks of 4)

32

gÉant fibre and lighting designing the next generation gÉant optical layer

Documents

gant gant fibre

gbs wavelengths

eastern ring western

dark fibre rings western

g endtoend wavelengths

dark fibre routes

copenhageneastern ring

gbs multihop wavelengths