1

Applications for dynamically shared GMPLS networks

Outline Quick summary of CHEETAH project "Business" orientation Technical details of CHEETAH

Malathi VeeraraghavanUniversity of Virginiamvee@virginia.edu

Sept. 24, 2007

Router-to-router leased Ethernet-SONET-Ethernet or SONET circuits (red)

Backbone network (e.g., Abilene)

Enterprise networks

WAN-access router

Regional (metro) network

Enterprise networks

WAN-access router

Regional (metro) network

e.g., CD-CIs

e.g., T640s

if colocatedin same PoP

if not colocatedin same PoP

Server-to-server circuits (rather than router-to-router)

Cheetah studies: Focused on the use of circuits from server

to server Since servers only have Ethernet NICs, the

circuits were all Ethernet-SONET-Ethernet circuits

Focused on enabling dynamic sharing of circuits Leased lines between servers would likely be

unjustifiable (from cost perspective)

High-speed justification

For router-to-router circuits, "high-speed" is required because of aggregation

For server-to-server, our justification was for file transfers Higher the rate, faster the transfer

Applications we developed for experimentation with GMPLS networks

Given that a significant % of file transfers involve the Web, we experimented with two Web based file-transfer applications:

Simple Web client to Web server transfers Goal: Use GMPLS network without changing Web

client or Web server software Problem: GMPLS networks need to stretch end-to-

end Web proxy servers located at core-network PoPs

Goal: If GMPLS network can be only deployed in the core initially, deploying proxies allows even non-connected end hosts located in enterprises to benefit from core GMPLS network's high speeds.

Quick summary of Cheetah project

Deployed a wide-area experimental SONET GMPLS network Three PoPs: Raleigh, Atlanta, ORNL Intercity OC192s purchased from NLR and ORNL Colo services purchased from MCNC, SLR, ORNL GbE interface cards for server connectivity Located 2 to 3 servers + GMPLS switch (SN16000) at each PoP

Developed core software RSVP-TE client for the server Circuit-TCP for transport protocol on circuits

Modified Web applications to interface with the RSVP-TE client to request circuit setup before transfers, and release after

Ran our Cheetah core software on HOPI Interconnected Cheetah to HOPI testbed

Outline check

Outline Quick summary of CHEETAH project "Business" orientation

Revenues: potential market - applications Costs

Technical details of CHEETAH

Business orientation

Choose applications [for development, experimentation and demonstration] after taking into account "business" considerations

So we started by classifying applications suitable for different types of GMPLS network deployments and services

Services & applications (for "dynamic circuit" networks)

Leased lines TCP/IPCoarse Grained Sharing (CGS)

Fine Grained Sharing (FGS)

• Coarse Grained Sharing• High-bandwidth circuits, AND • "Long" holding times• Need Book-Ahead (BA) support in the control-plane

(scheduling or advance reservations)

• Fine Grained Sharing • Moderate-BW circuits, and/or • Short holding time• Immediate-Request (IR) mode sufficient in the control-plane.

GMPLS networks

10

Services & applications (for "dynamic circuit" networks)

DCS-network scope

Bandwidth-sharing modes

Dynamic circuit services in the core network ONLY

Dynamic circuit services are intraregional

Dynamic circuitservices involveregional and corenetworks

Coarse Grained Sharing

• ISP router-to-router Long-Distance (LD) leased lines

• ASP server-to-server LD lines?

• Disaster-recovery (DR)

• Server-replication

• WAN access-link rate change

• Business interconnect

• eScience applications• Video-conferencing• Distance-learning

Fine Grained Sharing

• Web services (proxy, CDN)

• IPTV/video distribution (CDN)

• Inter-SMTP server

• Software-on- the-web

• Backup-storage

• WAN access-link rate change

• Business interconnect

• Video-telephony

Row/column headings: define service typesEntries in the body cells: applications

Blue: router-to-router

One sample point

To support the case for providing GMPLS network based dynamic circuit services between PoPs MCI network has 2500 PoPs throughout

North America and 2000 around the globe! Are there SMTP servers, CDN servers and

other applications servers that need interconnectivity?

Video and Content Delivery Network (CDN)

The rise of You-tube and video is often cited as a reason for growth in bandwidth and network equipment sales

CDN example providers: Akamai CDN servers placed in PoPs Requests from clients served from

closest CDN server Use high-speed GMPLS networks in the

core to move files between CDN servers

Catch?

Akamai does a trade with regional Research-and-Education Networks (RENs)

Place servers in regional REN PoP Regional REN pays for collocation costs

(power, space, remote hands-and-eyes) Regional REN gains by cutting the

required rate for the circuit it purchases for IP connectivity from core IP service provider

"Dynamic CDN"

CDN service is comparable to "leased line" service

A web service provider enters into an agreement with a CDN provider to serve out its content

What about small-to-moderate sized enterprises? Can they recruit CDN servers located at a few PoPs if

they expect a sudden surge of traffic to their web servers (e.g., slashdot phenomenon)?

If so, use dynamically setup high-speed circuit to copy over the whole web structure (esp. with databases) to dynamically recruited CDN servers

Storage

Three types of applications: Disaster recovery (DR): backup of critical

data Server replication: e.g., of web servers (to

allow for quick switchover in case of failures)

Backup storage: of ordinary enterprise users' data

DR and server replication Typically, only these two types require

network connectivity outside the enterprise Small-to-moderate sized enterprises only

require intraregional DCS services (if used) general rule of thumb: 75-mile distance of backup

site hence listed in column 2 of services/applications

classification table Fortune-500 companies with multiple locations

require DCS across regional AND core hence listed in column 3 of services/applications

classification table

DR and server replication Requirements

Few endpoints or users initiating these apps. Few transfers a day

Is IP-routed network sufficient?

Backup storage, on the other hand

"Backup storage" application If a new "storage ASP" emerged, which sold backup

storage services for "all" data in enterprises, then given

the large number of employees, who could initiate backup at any time if they want to save

an important file as they make modifications,

could justify needing high-speed DCS networks Is bandwidth cheaper than HR costs to hire

engineers to maintain backup storage at each enterprise?

"Blue" vs "black" applications in table

Applications Listed in Blue

Listed in Black

Endpoints Router-to-router

Server-to-server

Target market for "encroachment"

Leased line services

IP services

Volume and price Low volume;High per-unit price

High volumeLow per-unit price

Router-to-router circuits

Services (Verizon): Provide network administrator web portal access to

explicitly request an increase in leased-line rate e.g., if GbE interface used, but rate capped with

VLAN rate-limiting, allow for rate limit to be increased (signaling if leased line realized through SN16000s).

Software that reads SNMP MIBs to monitor usage on leased line, and automatically issue signaling request for bandwidth increase

Both ideas: aggregate traffic based increase/decrease requests

Per-transfer based increases

Even if link is lightly loaded, a single file transfer delay can be reduced by increasing the bandwidth of the bottleneck (lowest-rate) link. e.g., an enterprise has an OC3 WAN access link.

Even if this link is lightly loaded, this becomes the max. rate that any single file transfer can enjoy.

By dynamically increasing this rate for a few seconds, user can enjoy a higher transfer rate.

Need tools to determine if WAN access link is the bottleneck link on an end-to-end path, and then increase rate.

Costs

Started by seeing Internet2 fee structure http://www.internet2.edu/network/fees.html

Why GMPLS in core network?

Because high-speed interface cards cost less in SONET switches than in IP routers

For high switching capacity nodes, which are mainly required in the core.

What is the major component of cost?

Service provider costs: Same for IP-routed and SONET networks

HR costs Bandwidth costs

Differ: Equipment costs:

Mainly line card costs

If bulk of the costs are in HR and bandwidth, then equipment cost differentials become less significant

Summary Opportunity to increase potential market for GMPLS

switches We have access to three GMPLS testbeds on which we

can test applications and gain experience with R&E users Internet2's DCS, HOPI, Cheetah

Choose application(s) carefully with due consideration of business aspects

Looking for support: Student HR support to implement "glue" software to

make applications run on GMPLS networks, and to build usage base

Cheetah testbed annual maintenance charges

Outline check

Outline Quick summary of CHEETAH project "Business" orientation

Revenues: potential market - applications Costs

Technical details of CHEETAH

Cheetah concept Hybrid architecture: an IP-routed network AND a GMPLS network Use dynamically setup circuits for file transfers Send small files on IP-routed path and use GMPLS network only

for large files: call-setup overhead

SONET

switch

SONET

switch

EthernetInterface

SONETInterface

EthernetInterface

SONETInterface

Circuit gateway

(1) (2) (3) (4) (5)

(1)-(5): RSVP-TE PATH messages(6)-(10): RSVP-TE RESV messagesEthernet-EOS-Ethernet CHEETAH circuit

NIC1

NIC2End host

NIC1

NIC2End host

Messages through Internet

Circuit gateway

(10) (9) (8) (7) (6)

IP-routed network

CHEETAH: Circuit-switched High-speed End-to-End ArcHitecture

GMPLS network

CHEETAH End-host Software

NIC 1

NIC 2

OCS Client

Routing Decision

RSVP-TE client Application

CHEETAH software

TCP/IP

C-TCP/IP

End Host

NIC 1

NIC 2

OCS Client

Routing Decision

RSVP-TE clientApplication

CHEETAH software

TCP/IP

C-TCP/IP

End Host

IP-routed network

SONET circuit-switched network

CircuitGateway

CircuitGateway

Circuit-TCP: TCP minus congestion control

Optical connectivity service (uses DNS servers)

Determines which path to use: IP-routed or Circuit

CHEETAH End-host Software

End Host

bwlib Sig_proc

OCS Client

CAC

Data-planeConfiguration

RSVPD

RSVP-TE messages

Configuration file

read

DNSserver

circuitrequestor

Parsing/Construction

RSVP-TE client software architecture

Connection Admission Control:check if bandwidth is availableon the UNI from the host to the switch (multiple VLANs)

Configure IP routing and ARP table since remote host is reached directly on the newly setup circuit

30

CHEETAH testbed

Atlanta

SN16000

ORNL, TN

OC192

Zelda1/2/3

3xGbE

GbE

Zelda4/5

Wukong/Wuneng

OC192

Raleigh, NC

SN16000

SN16000

•Long-distance OC192s purchased from NLR and ORNL•Collocation services purchased from MCNC in NC, SLR in Atlanta•Zeldas and wukong/wuneng: Linux Dell PCs

31

Interconnection of CHEETAH toUS-wide HOPI experimental testbed

ATL

SN16k

NxGbE

SN16k

SN16k

ORNLOC192

Zelda1/2/3

GbE

GbE

Zelda4/5

Wukong/Wuneng

Washington HOPI

Force10

PC3

10GbE

OC192

NYC HOPI Force10

PC3

PC3

PC3

PC3 Chicago HOPI

Force10

Seattle HOPI

Force10

LA HOPI Force10

10GbE

10GbE10GbE

10GbE

NC

HOPI

CHEETAH

HOPI: Hybrid Optical/Packet Infrastructure: Internet2 supported testbedForce10 E600s used to dynamically setup and release VLANs (virtual circuits)

32

10GbE 10GbE GbE

Force10

Tech. transfer: CHEETAH control plane software modified for HOPI

CCPM: CHEETAH Control-Plane Module OSPFD RSVPD Force10 programming module

CCSA: CHEETAH Client System Agent RSVPD CHEETAHD Circuit-requestor

Circuit setup procedure losa-pc1:

Use circuit-requestor to initiate setup to sttl-pc1 sends PATH meesage

losa-CCPM: Route computation, CAC, VLAN ID assignment

sttl-CCPM: Route extract, Local CAC and VLAN ID check

sttl-pc1: Configures VLAN, programs ARP and route tables Sends back RESV message

sttl-ccpm: Programs sttl-Force10 for that VLAN

losa-ccpm: Programs losa-Force10 for that VLAN; sets rate

policing

losa-pc1: Configures VLAN, programs ARP and route tables

LOSA10GbE 10GbE GbE

Force10

CCSA

CCSA

CCPM

pc1

pc2

pc3

Internet

STTLCCSA

CCSA

CCPM

pc1

pc2

pc3

33

Setup a circuit from losa-pc1 to sttl-pc1

34

Automatic configuration on the end host

35

Setup multiple circuits to the same remote end host

Request exceeding the available bandwidth is rejected.

Internet2's new Dynamic Circuit Services (DCS) network

Yellow nodes: Ciena CD-CI SONET switches

Blue nodes: Juniper T640 IP routersCourtesy: Rick Summerhill (2006)

Testbeds

Three "GMPLS" wide-area testbeds are available for testing and demonstrating new applications for GMPLS networksCheetahHOPI Internet2's DCS network

Control messages via Internet

Application: WebFT

Web serverWeb client

Web Server (e.g. Apache)

CGI scripts (download.cgi &

redirection.cgi

URLResponse

WebFT sender

OCS API RD API

RSVP-TE API

C-TCP API

Web Browser(e.g. Mozilla)

WebFT receiver

RSVP-TE API

C-TCP API Data transfers via a circuit

OCS daemon

RD daemon

RSVP-TE daemon

RSVP-TE daemon

Cheetah end-host software APIsand daemons

Cheetah end-host software APIsand daemons

PROBLEM: Need GMPLS networks to be deployed withinregional and enterprise networks, not just the core

39

Application: circuit-aware web proxy servers

IP-routed networkIP-routed network

Core-only GMPLS network

Core-only GMPLS network

Web client Web server

Original HTTP messages HTTP

messages

HTTP and ICP messages

HTTP messages

squidsquid

• A web proxy software package: squid• "Circuit-aware" by integrating RSVP-TE & CTCP• Dynamic circuit setup triggered by web client request

• Use of circuits transparent to human users• Use Internet path while circuit is being setup