www.geant.org1 |

www.geant.org

Guy RobertsSenior Network Architect

GÉANT network evolution – an update

www.geant.org

GÉANT DWDM network

Based on Infinera DTNxsystem – Network elements provide large capacity point to point circuits across defined routes

OTN and GMPLS makes solution is easy to operate and very fast to turn up new services with high availability

GÉANT Point-to-Point – 10G and 100G lambda services as well as our IP trunks are provided on this infrastructure

GÉANT Network – current situation

2 |

DTN-X in operation with excellent results since 2012

www.geant.org

Current GÉANT cost pain points

3

Problems with current architecture:

• OTN layer expensive

• OTN chassis running out of slots in central PoPs

• OTN useful for protection switching and multihop but traffic is 70% unprotected and next hop

• OTN chassis are DC powered full rack cost is very high in some PoPs

• Proliferation of IP/MPLS

Slide by Sebastiano Buscaglione

OTN

www.geant.org

GÉANT Network Traffic

4 |

Network Traffic

Average volumes during 2017 were 3.13PB per day for the IP/MPLS network, average daily rate of 289Gbps

Average volumes including Lambdas are 4.79PB day or an average data rate of 444Gbps

Science traffic growth: 43%

Internet traffic growth : 26%

www.geant.org

Drivers of disruption

Content providers (FAANGs) are experiencing huge growth in traffic

volume. They are driving innovation to ensure that their business models scale.

• The need for simple centralized controllers is driving software and hardware disaggregation.

• Started with OpenFlow, now content providers are pushing disruption through mechanisms such as

TIP, OpenConfig etc.

• Hardware commoditisation drives economies of scale - merchant silicon packet switching (e.g

Broadcom Jericho II)

• These changes are now moving from the datacentre into the Wide Area Network.

• This will lead to both opportunities and competitive price pressures on NRENs.

What challenges/opportunities are there for Research Networks?

5

www.geant.org

Merchant Optics

• DWDM traditionally resides on DWDM transponders. The new generationof interfaces can now be integrated into switches/routers.

• Acacia

• DWDM DP-QPSK/8QAM/16QAM/64QAM, 100G up to 600G per channel

• Reach: 2500km reach in CFP2 package, 4000km reach in for AC400

• Up to 1.2Tbps with AC1200

• Where is optimal location for CFP2?

• Juniper/Cisco would like to sell 100G DWDM on their routers

• Heavy reading white paper Feb 2017 suggests that external transponders/muxponders are cheaper due to cheap QSFP 100G interfaces and higher cost of using DWDM on routers consuming valuable router slots.

• Integration of DWDM into routers blurs layer independence and Heavy Reading claims that big content providers are opting to keep DWDM off their routers.

DCI solution brings gains of low cost DWDM while maintaining layer separation6

www.geant.org

Data Centre Interconnects

• Next gen of commodity pluggable optics has excellent performance

• Data centre style 1 RU stackable form factor.

• Around 6 times reduction in cost over traditional telecoms equipment architectures

• Significant increase in density and reduction in power consumption

• Modular – easy to scale up

• Easy upgrade path to new technology

7

Loss of equipment integrity – not designed to be highly available as per ETSI etc.

• No internal hardware redundancy

• No in-service upgrades

• Restricted temperature operation

www.geant.org

A cost comparison DCI vs. carrier grade DWDM/OTN

8

100G 200G 300G 400G 500G 600G 700G 800G 900G 1T 1.1T 1.2T 1.3T 1.4T 1.5T

DCI 1 DCI 2

• Blue line is carrier-grade DWDM with OTN. Grey/Orange are DCI boxes

• HW cost of delivering capacity between adjacent PoPs over a fibre link (€/Bps)

• Analysis by Sebastiano Buscaglione and Matthew Gordon, August 2017

www.geant.org

DCI costs comparison

• HW cost of capacity between adjacent PoPs over a fibre (€/Bps) for a range of DCI

• High variation in cost between DCI solutions – prices compressed towards lower level in tender process

• Analysis by Sebastiano Buscaglione and Matthew Gordon, August 20179

100 200 300 400 500 600 700 800 900 1000 1100 1200

www.geant.org

GÉANT transmission: Short-term solution

10Slide by Sebastiano Buscaglione

• Use DCI to provision bandwidth for high capacity IP/MPLS trunks

• DCIs are up to 6x cheaper than carrier grade transmission

• Keep OTN layer for link management and lambda provisioning greatly simplifying DCI role and minimising risk

• Integration of DCI allows for growth offset and generate enough spares

OTN

www.geant.org

Medium-term solution: Open Line system.

Current closed interop model

11

Long-term vision. But open standards and management under development

Medium term solution. Open access, single management plane for OLS

www.geant.org

• Technology is moving faster in the packet and transponders than the amplifiers and WSS.

• Alien waves allow transponders from multiple vendors to operate on a single line system.

• Still benefit from a single vendor providing end-to-end optical management: Channel & span equalization, DCN connectivity (OSC), ALS, Alarm reporting ect.

12

Open Line System

ILAROADM ROADMILA

OLS gives flexibility

www.geant.org

DCI equipment testing in GÉANT

Optics First Vendor Second Vendor

Acacia AC400 Equipment: Facebook VoyagerStatus: Cambridge lab testing

GÉANT field trial Lon-BruNREN evaluation by PSNC and UNINETT

Equipment: ADVA FSP3000Status: Testing under consideration.

Acacia CFP2 ACO Equipment: Juniper DWDM CFP2 100G ACO Status: Cambridge lab testing

GÉANT field trial Mil-Mar

Equipment: Coriant Groove G30Status: Lab testing in Cambridge

Acacia CFP2 DCO Equipment: Vendor in DWDM stealth modeStatus: GÉANT field trial on Mil-Mar

Equipment: Juniper DWDM CFP2 100G DCO Status: Testing expected mid 2018

Acacia AC1200 Equipment: Coriant Groove G30Status: To be tested in Cambridge lab

NA

Ciena WaveServer Equipment: WaveServer AIStatus: Cambridge lab testing

Field trial on CESNET fibre Prague to Vienna

NA

Infinera Equipment: XT3300/3600Status: Compatible with GÉANT DWDM system

NA

• GÉANT is testing DCI boxes

• Lab testing and field trials

• Additional testing by NRENs

www.geant.org

AW Field Trials/ Lab conclusions

What have we learnt from the testing/evaluation so far?

• We like the products that we tested, in particular the optical performance is very good.

• Performance of latest generation of DWDM pluggables is very good –8QAM will work on most (all?) links in GÉANT network and 16QAM on several.

• Potential for very large cost savings in the network.

• Optical power management is critical in an alien wave environment.

• Software selectable modulation makes planning much easier.

• An optical modelling tool will help scale up number of AWs.

• Turning up AWs is best done in small slow steps

14

www.geant.org

DCI choice

• GÉANT has chosen the Coriant Groove G30 product.

• I RU stackable

• 4 sleds, each up to 4 x 100G

• Optics are based on Acacia CFP2 ACO• 200G up to 1000km with 16 QAM modulation

• 150G up to 2000km with 8 QAM modulation

• 100G up to 5000km with DP-QPSK

• Client side is QSFP28

• Next generation to support AC1200 with 600Gbps using 64QAM

15

www.geant.org

Fibre IRU – the opportunity

• In funding cycle 2019-2022, the IRU budget is at least 16 M€ (likely to rise to 50M€), out of total 4-year SGA budget of 128 M€

• European Commission have set the following objectives

• Go beyond the state-of-the-art by restructuring the backbone network through

exploration and procurement of long-term IRUs and associated equipment to

increase the footprint, stimulating the market in cross-border communications

infrastructure whilst decreasing the digital divide and reducing costs

• Improve the minimum service level of the smaller European NRENs and their

users by ensuring connectivity speeds of 100 Gbps (where technically and

economically feasible)

www.geant.org17 |

• 2018 design has been in place since 2012…. With very little change

• Fibre ‘core’ in yellow

• Leased capacity in black

• Spectrum in green

• PoPs blue circle

GÉANT topology

www.geant.org

What are we doing? Regional studies

1818

IrelandUK

UK/IrelandLed by Guy Roberts

Belarus Latvia Estonia Lithuania

Russia

Finland

Sweden NorwayDenmark

Baltic - Led by Ieva Muraškienė

Poland

Greece Hungary Bulgaria

Romania Serbia

South East Europe - Led by Pavel Skoda

Albania

Slovakia

Israel

Cyprus Turkey

Macedonia

Montenegro

Croatia

France Croatia

Central Europe region - Led by Sebastiano Buscaglione

PolandGermany

Italy

Switzerland

Austria CzechRepublic

Slovenia

Malta Holland Belgium UK

Spain

Hungary

Luxembourg

+ Nordunet

Portugal

Spain

Iberian PeninsulaLed by Emanuel Massano

France

www.geant.org

• The network design will use a mix of dark fibre, spectrum and leased capacity

• To find the optimal solution GÉANT is carrying out cost modelling based on the total cost of ownership for the IRU

• This is proving to be complex as the cost of different technical solutions are not easy to compare

• Also complicated by the different rebates from the EC and how these can be claimed

• Ultimate result will provide cost transparency for our governance and NREN shareholders

Cost modelling

19 |

www.geant.org

Candidate GÉANT topology

20 |

• Future architecture extends fibre to more regions

• Cost of fibre everywhere is beyond the budget available

• Negotiation and modelling required tounderstand how to reduce the cost

• Solution becomes affordable if we share NREN fibre

• Dark Fibre candidates in green

• Spectrum sharing options in orange

• Leased wavelengths shown in black

www.geant.org

Telecom Infra Project

• Facebook has initiated the Telecom Infra Project which aims to achieve open market for networking equipment.

• Built around components: APIs, line equipment, subsystems, software.

• Operators are the leaders, via the 'Operator Council' (OC).

• The sub-project relevant to optical transmission is Open Optical Packet Transport (OOPT)

• Facebook believes that the new architecture will be stackable boxes and software to replace hardware redundancy.

• TIP is developing an optical modelling tool to open up the optical infrastructure.

21

www.geant.org

Summary

• New generation of data-centre type transponders will deliver the performance GÉANT needs over our existing line system.

• Testing has confirmed several DCI boxes on the market with excellent pricing cf. traditional carrier grade equipment.

• We have chosen the Coriant Groove G30 DCI equipment as an interim technology for transponders between now and the network refresh in 2020/21

• GÉANT will receive around 40 million Euros for fibre IRUs in our next funding cycle starting January 2019, this will be used to strengthen our fibre footprint.

• We will move to a flexgrid ROADM based Open Line System to light this new fibre.

• Sharing fibre with the NRENs is a key to maximising the European dark fibre footprint.

22

www.geant.org23 |

Thank you

www.geant.org

Any questions?

© GEANT Limited on behalf of the GN4 Phase 2 project (GN4-2).

The research leading to these results has received funding from

the European Union’s Horizon 2020 research and innovation programme under Grant

Agreement No. 731122 (GN4-2).

www.geant.org

TNC

The GÉANT community's flagship conference.

Regular attendance of over 700 participants from all across the world.

Bringing together decision makers, networking and collaboration specialists, and identity and access management experts from all major European networking and research organisations, universities, worldwide sister institutions, as well as industry representatives.

tnc18.geant.org

24 |