Connecting to the new Internet2 Network

What to Expect…

Steve CotterRick Summerhill

FMM 2006 / Chicago

2

Outline

• Network Infrastructures

• Services on The Infrastructures

• Description of Circuit Services

• Control Plane

The Network

4

Optical Topology

5

The Network

• Made up of four architecturally distinct infrastructures:

• the core infrastructure• the IP infrastructure• the Multiservice Switching infrastructure• the HOPI Testbed infrastructure

• Initially, circuit services across different architectures will be kept separate to maintain integrity

• As network matures, level of integration of infrastructures will increase

6

Services

8

9

IP Services

• Continuation of IP services offered on Abilene plus commercial peering and commodity transit

• 9 router locations: NYC, DC, Chicago, Atlanta, Houston, KC, LA, Salt Lake City, Seattle

10

IP Services

• Connector builds out to PoP and I2 will carry connection to the nearest backbone router

• 10Gbps connection over core infrastructure• <10Gbps may go over multiservice switching

infrastructure

• Additional “opt-in” services will require separate BGP peerings, I2 suggests connect with:

• VLANs enabled (Ethernet connections)• Frame Relay DLCIs set up for SONET connections

11

Commercial Peering Service

• As configured for beta trial, we anticipate using separate peerings between I2 and participating connectors

• Use VLANs or Frame Relay DLCI• Connect at 10Gbps to commercial exchange points

in Chicago, Palo Alto (PAIX), and possibly Ashburn

• Ongoing trial - NTAC and I2 staff are seeking community input on how to best offer this service

12

Commodity Transit Service

• Offer optional Level3 commodity through IP connection at reduced rates

• Level3 will also allow direct 1GE connections to Level3 commodity network

• Will add other transit providers in the near future

• Separate BGP peering requiring VLANs or DLCIs

• NTAC Working Group providing input

13

MPLS Services

• Internet2 network staff and the IU NOC will work with connectors to implement MPLS tunnels through the Internet2 Network on a case-by-case basis

14

15

• Purpose is to allow experimentation in implementing and using dynamic circuits

• Experimental in a number of ways:• Footprint limited Internet2 Network footprint• Uses experimental control plane software to create

circuits and to interconnect with other domains using their control software

• Allows other networks to interconnect with it to test their interdomain circuit capabilities

HOPI Test Service

16

HOPI Test Services

• Connect at I2 PoP with HOPI access point• Physical connection can have one or more

sub-circuits over 1G or 10G interface• Kinds of connections expected to change over time

as other devices added• Initially, use existing HOPI Force 10 Ethernet

switches

17

HOPI Test Services

• Connections across HOPI are p-2-p Ethernet VLAN based circuits in increments of 100 Mbps

• User input devices must support 802.1q VLAN capabilities

• Requests for bandwidth can me made using:• GMPLS-style Peer Mode, GMPLS-style UNI Mode,

Web Service API or email/phone

18

HOPI Test Services

• HOPI may be used to:• Test dynamic services developed by other domains

• Likely interconnect HOPI to test labs and with other organizations (regionals)

• Test applications prior to using them on dynamic or static circuit services (i.e. performance measuring software for dynamic networks)

• For more information, contact Rick Summerhill rrsum@internet2.edu

19

20

Circuit Service

• Major aspects of the circuit service:• Physical connection between Internet2 and

user device (type and speed)• Circuit bandwidth (multiples of STS-1s)

Note: Speed of interfaces do not have to match

21

Static Circuit Service Overview

• Longer lasting point-2-point circuits across network by connecting to Infinera or Ciena gear

• Infinera via 10GE or OC-192 interfaces• Across network can be either Ethernet or SONET framed

• Ciena via 1GE, 10GE or SONET OC-48, OC-192 interfaces

• Ethernet frames encapsulated in SONET using GFP• SONET can be channelized or not (OC-48 or OC-48c)• Across Network is always SONET in multiples of OC-1

• Provisioned by Internet2 NOC in coordination with user’s NOC.

22

23

Multiservice Switch Description

• Each Ciena Multiservice Switch has two or more trunk side SONET circuits connecting it to other switches

• Client side can be Ethernet or SONET - 1GE, 10GE or OC-48, OC-192 interfaces

• Ethernet frames encapsulated in SONET using GFP

• SONET can be channelized or not (OC-48 or OC-48c)

24

25

26

Dynamic Circuit Service Overview

• Shorter period (minutes to months ) point-2-point circuits across Network

• Connected to Ciena Multiservice Switching infrastructure

• Connect as single circuit to be switched or multiple circuits multiplexed over the physical connection

• Will use a set of waves on the core infrastructure dedicated to the dynamic wave service

27

Dynamic Circuit Ethernet Requirements

• All Ethernet connectors must be capable of supporting 9K (MTU) payload frames.

• Ethernet participants may be tagged with VLANs or untagged, and VLANs may be switched internally on the transport.

• That is, a VLAN tag on one end need not be the same as a VLAN tag on the far end.

• Physical connections that use Ethernet must support 802.3x (flow control).

• Physical connections using Ethernet VLANs must support 802.1q (VLAN).

28

Dynamic Circuit Details - Ethernet

• Provided as a p-2-p Ethernet circuit• Frames are encapsulated into SONET streams

using Generic Framing Protocol (GFP). • Encapsulation and decapsulation done at

ingress/egress • Streams divided into one or more OC-1 channels

using Virtual Concatenation (VCAT)• It is strongly recommended that all Ethernet

connectors support IEEE 802.1p (flow control)• Circuits between networks passed as GFP

encapsulated SONET

29

• All SONET connectors must support VCAT and LCAS

• All SONET connectors providing Ethernet services must support GFP

Dynamic Circuit SONET Requirements

30

Dynamic Circuit Details - SONET

• Provided in multiples of OC-1s• Incoming SONET can be channelized or not (i.e. OC-

48 or OC-48c)

• Connections across the Ciena MS infrastructure are always SONET

Multiservice Switching Infrastructure Control Plane

32

Multiservice Switch Control Plane Overview

• Control plane software will:• Manage the set up of dynamic

circuits• Keep track of bandwidth• Allow reservation of future

bandwidth• Authenticate users requesting

bandwidth• Report on network status

• Additional software integrated with Ciena software to provide additional capabilities

33

Dynamic Circuit Control Plane

• Automated services are expected in the near future

• Will use DRAGON-style control plane capabilities• Control plane channels will be transported via IP,

and connectors will be offered several options for implementation and/or interaction with the dynamic control plane

• Initial deployments of control plane software will take place on HOPI test service before moving to the production network

Dynamic Circuit Service Provisioning

35

Dynamic Circuit Service Provisioning

• Uses control plane software to set up circuits• Initially only I2/NOC staff after requested by user• Eventually software added to allow circuits to be

created by user request• For cross-domain circuits, think of I2 Network

circuits as segments of longer p-2-p circuits• Cross-domain coordination via email/phone or

software to do automatically • Contact the NOC for circuit setup, NOC engineer

will be assigned to coordinate with other networks to create the required paths across the network.

36

• Cross-domain connections require appropriate control plane software

• Internet2 will provide experimental software to regionals

Dynamic Circuit Service Provisioning

37

Connecting Dynamic Circuits to Users

• Scenario #1: Two regionals and their users

• Physical connections to regionals made – we expect this will typically be an Ethernet connection that supports VLANS

• Each regional creates VLAN circuit to its user and makes connection to Internet2, then segments joined to create end-2-end circuit

38

39

• Typically regional provides its own circuit multiplexing for its user and provides circuit to I2

• Multiplexed connection sent across I2 network and delivered to end regional

• Circuits set up and managed by control plane software

• May be done automatically across regional domains if using compatible control plane software

Connecting Dynamic Circuits to Users

40

Connecting Dynamic Circuits to Users

41

• Scenario #2: Connecting to similar services provided by other national or international networks

• By either direct connections or at exchange points• Control plane interface is similar to the I2/RON

model

• I2 working on user authorization and network information sharing with other networks

Connecting Dynamic Circuits to Other Networks

42

43

44

Connecting Dynamic Circuits to Other Networks

• Using these interconnections and appropriate manual and automatic control of circuit switching, Internet2 will be able to be a partner in creating circuits from users in the United States to users connected to other international core networks

• Today, working to define service definitions and info sharing policies to enable this

45

Applications

• TeraGrid

• High definition videoconferencing

• eVLBI Radio telescopes

• Remote medicine

• IP load shifting

• File transfer – Phoebus / VFER

Control Plane Rollout

47

Control Plane Rollout – Current Plan

• Initially, Ciena control plane will be used to create circuits by NOC from user requests

• Feb 2007 – web form available to request connection, initially filled by NOC

• Web form to interface with control plane software to make connections without NOC intervention

• Feb/Mar 2007 – software made available to regionals for them to provide switched circuits to users

48

An Example of How to Connect to HOPI and the Internet2 Network - Phase 1

• Campus connects through RON using static VLANs and deploys VLSR on PC connected to switch (GMPLS control plane)

• Ethernet based• Connect to HOPI control plane

49

Phase 2

• Add NARB (could be same PC)• Separates the campus domain from HOPI

domain• Now have separate control planes

50

Phase 3

• When ready, RON implements GMPLS control plane

51

Phase 4

• Move to the Multiservice Switching Infrastructure on the Internet2 Network

• There are many other possible alternatives

52 Steve Cotter