Pan-European Research Networks:

Past, Present & Future

Michael Enrico

Network Engineering & Planning

DANTE Ltd

Workshop on “The Internet Protocol & Optical Networking”

Grasmere, Cunbria, UK

23-25 September 2002

Agenda

• DANTE in Context• Who? What? Where? How?• Who pays?

• History (of pan-Euro backbone R&E Networks)

• The early days (EuropaNET)• TEN-34 & TEN-155

• Today• The GN1 project and GÉANT

• Tomorrow• What place for Optical Networking?

Who is ?

•Delivery of

•Advanced

•Networking

•Technology to

•Europe

Who is ?

• Legal status: “not-for-profit” Limited Company registered in the UK

• Governance:• Board of Directors• NREN Policy Committee

• Started in 1993• Based in Cambridge, UK• Currently ~ 22 people (most in Cambridge)• Broad European makeup of staff

• 11 non-UK

A Brief History...

In the Beginning (pre 1992)...

EARN

• Some links to EARN (BitNet)

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

JANET

RENATER

DFN

GARRRedIRIS

NorduNET

• There were NRENs &...

• Ad hoc bilateral links

EuropaNET (1992)

• RARE contracted EuropaNET from Unisource• Not subsidised by the EC• “Competition” with fledgling E-Bone

• E-Bone later followed the commercial route

• Some suspicion from commercial providers• Would EuropaNET restrict itself to R&E?

• Typical backbone circuits ~E1• Used Telebit routers• Offered a datagram service (X.25, IP & CLNS)• Finished in October 1995

IBDNS (1995)

• International Backbone Data Network Service

• Contracted from BT• Used Cisco routers• Typical NREN accesses ~few M

• JANET access was 8M at the end

• Finished early 1997

TEN-34 & TEN-155 (1997-2001)

• IP Backbone built on managed ATM network• DANTE owned (Cisco) routers• TEN-34 (Early 1997)

• Typical backbone circuit ~16M (e.g. London-Paris)• Odd 34M circuit in backbone

• TEN-155 (Late 1998)• Started with single ring of STM-1• Finished up with main ring at 4 x STM-1• Typical NREN accesses at STM-1• Finished end of 2001

http://www.dante.net/ten-155/ten155net.gif

January 2000 Topology

USA

Israel

Cyprus

JapanNACSIS

January 2000Topology

155 Mbps

10 Mbps34/45 Mbps

TEN-155 Services

• IP transit service for European NRENs• Connectivity to North American R&E Networks

• Abilene, ESnet & Canarie• via DANTE POP in New York

• DANTE World Service• Connectivity to UUnet in NY (by additional

subscription)

• European Commodity Service• “Peerings” with InfoNet

• Managed Bandwidth Service (MBS):• end-to-end ATM CBR & UBR PVCs

• Advanced IP services: e.g. Multicast

A pan-European R&E network with a dual remit:

• To provide production network services to the European R&E community

• To provide a platform for networking research

GN1 Project:

GN1 Project:

• Planned & procured during 2001• Switched on Nov/Dec 2001• 10 Gbps in 8 locations, 9 circuits• 2.5 Gbps in 7 locations, 12 circuits• 3 main providers: COLT, T-Systems & Telia• Connects 31 NRENs• Connectivity to North America:

• 3 x 2.5Gbps• Connects Abilene, Canarie & ESnet• GTR(E)N POP in New York

GÉANT Topology(Today)

PLUS:(not shown)•3 x 2.5G to NY•STM-1 to Japan•DWS peerings•NREN Accesses

• Generous funding of Research Networks by the EC & ...

How so BIG?

36

2,000

20,000

100,000

200,000

1

10

100

1,000

10,000

100,000

1,000,000

96 97 98-99 99-00 00-01

Eur

o / M

bps / Y

ear

average offer price

lowest offer price

Typical GÉANT POP

• A single Juniper M160 router• (except Frankfurt which needs two!)

• Ancillary management/monitoring equipment:• In-POP (switched) FE & GE LAN• Terminal server router (with ISDN & POTS access)• Two Sun workstations

• Housed in 3 or 4 racks or a small suite in (mostly) carrier-neutral datacentres

• Most POPs in capital city• Exceptions: DE (Frankfurt), IT (Milan), CH (Geneva),

PL (Poznan)

A POP(underconstruction!)

Network Management Service

• NMS & NOC outsourced to a company in Paris

• “Second Line” support performed by DANTE Operations team (based in Cambridge)

• Each NREN has an Access Port Manager (APM)

Customer Services on GÉANT

• (Best effort) IP transit for European NRENs

• DANTE World Service (chargeable extra)• Native Multicast• Premium IP• L2 VPN (using MPLS)

DANTE World Service

• Commodity IP Transit service for NRENs• Chargeable extra• Not all NRENs make use of it• Some use it as a back-up to their own

commodity access• 2 providers: Global Crossing & Telia (was

KPNQ!)• Numerous “peering” points around Europe

• Each at STM-16

• Routing policy: “hot potato”• Subscription levels allowed: up to half of access

New Services

• Less than Best Effort (Scavanger)• Currently undergoing tests

• MBS Replacement?• Need to be able to provide support for projects

(e.g. GRIDs, 6net, testbeds, etc)• Could use Premium IP but what about traffic

isolation?• Could use MPLS L2 VPN but what about BW

guarantees?

Extending the Reach of GÉANT

• EUMEDCONNECT• Providing IP connectivity to EUMEDIS NRENs• Phase 1 (feasibility study) now complete• Phase 2 (procurement, implementation &

operation) to start shortly• Lifetime up to summer 2006

• CAESAR• Extending IP connectivity to Latin America• Still in feasibility stages

A Bit about IP Routing

• IGP is OSPF• TE performed via OSPF metrics• Plan to migrate to IS-IS (to allow for IPv6

support)

• EGP is BGP4• Full mesh of iBGP peerings (no Route

Reflectors)• eBGP peerings with all external peers

• Multicast extensions (MBGP)• Each router maintains a full Internet

routing table

IP Multicast Service

• Access to the service:• Via primary access port• Another possibility is via a GRE tunnel (currently nobody!)• Support PIM-SM v2 only

• 3 Rendezvous Points (RPs) in network• Source distribution via MSDP peerings• Looking at SSM• Operational procedures (rolling out now)

• Goal: fully supported service as for unicast• Plus specific monitoring

• Troubleshooting • Extension of the trouble ticket systems to multicast

incidents

More information on Multicast Setup

http://www.dante.net/nep/GEANT-MULTICAST

Premium IP Service

• Aims to provide international Virtual Leased Line based on DiffServ

• The Premium IP service is an end-to-end service (site to site) crossing multiple administrative domains

• Defined on the basis of the DiffServ EF PHB which is required to offer:•Bandwidth, low loss, upper bounded

delay and jitter

Premium IP: VLL Service

UK

DE

NREN Janet

DFN

AS20965

GÉANT backbone

AS786

AS680

A

B

RegionalNetwork

RegionalNetwork

SE

FR

streams < 5 % of the BW access

Rate limitation is applied on the NREN access

•Premium IP packets are tagged with DSCP code =46.•Rate Limitation is applied per aggregate on the NREN’s access, based on the total demanded bandwidth towards the destination.•Admission Control is made manually based on the Sum of bandwidth already booked on the destination access.

The Optical Networking bit...

GÉANT’s Optical Networking Today

• DANTE does not operate it’s own transmission infrastructure

• GÉANT Connectivity:• International Leased Lines• Unprotected SDH circuits for 622M or less• Unprotected “lambdas” for 2.5G and 10G

• Often 2.5G “lambdas” are really SDH circuits

• “White light” (G.957/691) “Thin SDH” interfaces

• Use SR or IR POS interfaces on routers

GÉANT’s Optical Networking Tomorrow?

• Two drivers:• Support for higher bandwidths:

what happens when NRENs start to subscribe to 10G?• Support for projects/user groups:

how do we extract arbitrary bandwidth “channels” from 2.5/10/40G core circuits?

Support for Higher Bandwidths

• Will router vendors offer 40G interfaces (soon)?

• Next generation “40G capable”• What will be the form of these interfaces?

• STM-256 on white light?

• Will xWDM interfaces become incorporated into routers?

• Already appearing in SDH switches• Does this help us at all?

• Will operators offer coloured interfaces?

Support for Projects/User Groups• Just use Layer 3 Techniques? (some say

yes!)• Simplify L2• Use 40G if possible• Are the routers really up to it (2.5G or 10G

tunnels?)

Operator

Network

• Add L2 switch/mux capability ourselves?

Support for Projects/User Groups

Keep POS i/f STM-n

Channelized modePOS STM-n i/f

STM-m(<n)

Operator

Network

• Add L1 capability ourselves?• Offer BoD? (some say no - leave it to

operators!)

Evolution of GÉANT?

Operator

Network

Light channel, λ bundle or even managed fibre?

Nature of i/f?

Optical Testbed Possibilities

• Some European NRENs now have access to dark fibre:

• CESNET (Czech Republic)• PSNC (Poland)• SWITCH (Switzerland)

• PSNC has been trying out Alcatel DWDM equipment

Optical Testbeds

Optical Testbed Possibilities II

• ASTON Project EoI (FP6):• A Step Towards the provision of Optical

Networking• Has come out of TF-NGN activity on optical

networks• Information gathering, knowledge distribution• Find partners to do Europe-wide testing of

technologies & methodologies• Create feedback loops with manufacturers &

providers• Results used for engineering new network

architectures & provisioning models

Conclusions

• Research Networking Community has High Demands

• Early Adopters• Some European NRENs are starting to do

their own transmission• Probably not going to happen on pan-

European scale (at least for foreseeable future)

• However, feeling is that GÉANT and European NRENs will go more optical…

• …we’re just not quite sure how!

Thank You&

Questions?