isp design fundelmentals-6up

7/21/2019 ISP Design Fundelmentals-6up

1/8

Copyright 1998, Cisco Systems, Inc. All rights reserved. Printed in USA.Presentation_ID.scr 1

1Presentation_ID 1999,CiscoSystems,Inc.

Introductions to ISPDesign FundamentalsIntroductions to ISP

Design Fundamentals

2Presentation_ID 1999,CiscoSystems,Inc. www.cisco.com

AgendaAgenda

Rational Behind ISP Network Design

Point of Presence Topologies

Adding Services to the Architecture

Impact of Services on the Network


Rational Behind ISPNetwork Design

Rational Behind ISPNetwork Design

Layers upo n Layers upon Layers

upo n L ayers ...

Layers upo n Layers upon Layers

upo n L ayers ...

3Presentation_ID 1999,CiscoSystems,Inc. www.cisco.com 4Presentation_ID 1999,CiscoSystems,Inc. www.cisco.com

The Free On-line Dictionaryof Computing

The Free On-line Dictionaryof Computing

Architecture:Design; the waycomponents fit together;

it may also be used for anycomplex system, e.g. software

architecture, networkarchitecture


Network Design andArchitecture

Network Design andArchitecture

can be critical

can contribute to the successof the network

can contribute to the failure

of the network


No amount of magicknobs will save a

sloppily

designed networkPaul FergusonConsulting Engineer,

Cisco Systems

Fergusons Law ofEngineering

Fergusons Law ofEngineering


2/8



What Is a Well-DesignedNetwork?

What Is a Well-DesignedNetwork?

One that takes into considerationsome main factors

3 Topological/protocol hierarchy

3 Redundancy

3 Addressing aggregation (IGP and BGP)

3 Scaling

3 Policy implementation (core/edge)

3 Management/maintenance/operations

3 Cost8Presentation_ID 1999,CiscoSystems,Inc. www.cisco.com

One Must Acknowledge thatOne Must Acknowledge that

Two different worlds exist

3One world revolves around privateorganizational networks and anotherconcerns the global Internet

Growth in the Internet is faster thanany other technology introduced to

the public-at-large


Source: Forbes Magazine July 7th 1997

Internet

CellPhone

PC

TV

Radio

MicrowaveMicrowave

VCR

Airplane

TelephoneTelephone

Car

Electricity

Technology AdoptionTechnology Adoption


If yo u re not scared yet ,

you don t und ers tand the

prob lem!

Mike ODellChief Scientist,

UUnet

Scaling is the #1 Problem onthe Internet

Scaling is the #1 Problem onthe Internet


Core Influences to ISP DesignCore Influences to ISP Design

Modular Design

Functional Design

Tiered/Hierarchical Design

Multiple Levels of Redundancy

Routing Protocol Hierarchy

Build for IP Forwarding First - thenadd services


Modular DesignModular Design

Backbone linkto another PoP

Backbone linkto another PoP

Nx64customer

aggregation layer

Nx64 leased line circuit deliveryChannelised T1/E1 circuits

T1/E1 leased line circuit deliveryChannelised T3/E3 circuits

NetworkOperations

Centre

Consumer

DIALAccess

OtherISPs

Consumer Cable

andxDSL Access

Networ k

Core

ISP Services(DNS,Mail,News,

FTP,WWW)

HostedServices

NxT1/E1 customeraggregation layer

Organize the Networkinto separate andrepeatable modules

3Backbone

3POP

3Hosting Services

3 ISP Services

3Support/NOC


3/8



Functional DesignFunctional Design

One Boxcannot do everything! (no materhow hard people have tried in the past)

Each router/switch in a network has awell-defined set of functions.

The various boxes each with afunction interact with each other.

ISP Networks are a systemsapproachto design.


Tiered/Hierarchical NetworkDesign

Tiered/Hierarchical NetworkDesign

Access Layer

Distribution

Layer

OtherRegions

OtherRegions

Other

Regions

Core Flat - MeshedTopologies havenot scaled.

Hierarchy is usedin network designsto scale thenetwork.


Intra-POP Interconnect

Border

Backbone

Access

Multiple Levels of RedundancyMultiple Levels of Redundancy

Triple Layered POPRedundancy

3 Lower-level failures are better

3 Lower-level failures may triggerhigher-level failures

3 L2: Two of everything at

3 L3: IGP and BGP provideredundancy and load balancing

3 L4: TCP re-transmissionsrecovers during the fail-over

POP IntraconnectPOP Intraconnect


Multiple Levels of RedundancyMultiple Levels of Redundancy

PoP

BackbonePeer

Networks

ResidentialAccess

LocationAccess

Objectives -

3 As little user visibility of a fault as possible

3 Minimize the impact of any fault in any part of the

network.

3 Network needs to handle L2, L3, L4, and Router

failure


Hierarchy of RoutingProtocols

BGP4

and OSPF/ISIS

FDDI

Other ISPs

CustomersLocal

IXP

BGP4 Static/BGP4

BGP4


WarningWarning

Beware Block Diagram/Slideware

Design Gurus! They have gottenpeople and networks into trouble- including Cisco


4/8



Point of PresenceTopologies

Point of PresenceTopologies

19ISP/IXPWorkshops 1999,CiscoSystems,Inc. www.cisco.com 20Presentation_ID 1999,CiscoSystems,Inc. www.cisco.com

PoP DesignPoP Design

Core Backbone

Routers

POP

Interconnect

Medium

NeighboringPOP

NeighboringPOP

Dedicated Access PSTN/ISDN

Core 1 Core 2

SW 1 SW 2

Access 1 Access 2 NAS 1 NAS 2

External BGP Peering


45 Mb/s

HSSI

FDDI

InternetInternetBackboneBackbone

Early Internet POPArchitecture - NSPEarly Internet POPArchitecture - NSP

3Backbone trunksat 45 Mb/s

3Shared mediainterconnectwithin POP:

FDDI, Ethernet, Switched

Ethernet

3ConventionalT3 backbone

Internet router22Presentation_ID 1999,CiscoSystems,Inc. www.cisco.com

155 Mb/s

POS/ATM

XX


3Backbone trunksat 155 Mb/s

Packet over SONET OC3

ATM OC3

3Switched interconnectwithin POP:

Switched FDDI/Fast Ethernet

ATM OC3

3Advanced OC3 backboneInternet router

Internet POP Architecture -96/97



622 Mb/s

POS/ATM

622 Mb/s ATM

1 Gb/s Ethernet

Duplex Fast E, 155/622 Mb/s ATM, or

1 Gb/s Ethernet


3Backbone trunks at

622 Mb/sPacket over SONET OC12

ATM OC12

3Switched interconnect

within POP:ATM at OC3 AND OC12

Ethernet Channel

Gigabit Ethernet (early 98)

POSIP (late 98)

3Gigabit OC12 backbone

Internet router




SRP Rings - High Speedof SDH combined with fastfailover and redundancy

3High bandwidth3Reduced port counts

3Reduced complexity

3Proactive self healing

Backbone

7xxx

GSRGSR

7xxx

7xxx

7xxx

7xxx

leased l ine aggregation




5/8



Large POPs - add a 3rd layerLarge POPs - add a 3rd layer


3

Problem: port density!3 Solution: buy more routers!

3 Customer routers connect toaggregation routers

Packet over SONET OC3

ATM OC3

3 Aggregation routers connect tobackbone routers

3 Scales nicely

3 X CRs to Y ARs to Z BRs

3...where X>Y>Z

3Be careful not tooversubscribe!

OC3

OC12

OC48


FDDI100Mbps

POSN x 155

SRP

2 x 622Mbpsor 2x2.5GB

Fast/Gig Ethernet100/1000Mbps

POP Interconnect SummaryPOP Interconnect Summary


Key Design Principles

Interconnection forManagement, Security,and Accounting services

3Netflow Devices -

FlowCollector

3Syslog collector for all

network devices

3SNMP collector (PC BasedUNIX)

3Security Auditing Tools(NetSonar)

POS

POS & ATM for Core Backbone

GSRGSR

75077507

Customer and Services

Management

&

Accounting


ISP routing Architectures - IPISP routing Architectures - IP

IGP = EIGRP, IS-IS,or OSPF

3 a lmo s t a l wa y s IS-IS or OSPF

3 IS-IS, single level (usually L2)

3 OSPF, either single area or BB/POPareas

BGP = all routers in full mesh

3 mesh accomplished with routereflectors, confederations, actual full

mesh

All routers have all routes, so

services could go anywhere


ISP routing Architectures -IP+MPLS

ISP routing Architectures -IP+MPLS

IGP = EIGRP, IS-IS,or OSPF

3 mus t be IS-IS or OSPF to use MPLSTE

BGP = only edge routers need full routes

3 full-mesh of edge routers using

aforementioned mechanisms

3 packets are forwarded via LDP

labels, not IP destination address

Where to put your services?

3 cannot hang a cache service off of arouter that doesnt have full routes!


Adding Services to the

Architecture

Adding Services to the

ArchitectureCause and EffectCause and Effect

3015030925_04F9_c1 1999,CiscoSystems,Inc.


6/8



Services?How many Services?

Services?How many Services?

Edge (one-time) services

Voice over IP

MPLS VPNs

CDNs

VPDNs

Managed services

DialDSLcable

Per-hop services

MPLS packet forwarding

DiffServ, other QoS

Multicast Services

Most network services are applied at the edge!


Ask the Right QuestionsAsk the Right Questions

What is the value of the service?3Technical merit

3Cost savings

3Marchitecture

What is the cost of the service?

3Equipment?

3Training people to support it?

3Network buildouts/topology changes?


Impact of Services onthe Network

Impact of Services onthe Network

3315030925_04F9_c1 1999,CiscoSystems,Inc. 34Presentation_ID 1999,CiscoSystems,Inc. www.cisco.com

Who Knows?Who Knows?

What will be the impact on existing trafficloads/patterns?

Can the network deliver the performancethat your customers/applications desire?delay? jitter (delay variation)?

Make sure to add capacity as you addservices - bandwidth is a must.


Deployment of New ServicesDeployment of New Services

Is more of a business decision

The technical aspect is to ensurecontinued network performance

scalability and stability

Try to keep services within your AS

3end2end control

3less likelihood of failure/flaps


Deploying New ServicesDeploying New Services

Dont feed the hype fire

Look beforeyou leap!

Dont deploy new technologies andservices just for the sake of it; havevalid business and technical reasons


7/8




Usually a Service requires a TCP/UDPtermination (I.e. TCPs three wayhandshake)

Termination should happen out side

of the pr imary f low path

Otherwise, the network is thendesigned around the single service.



CPECPE POPPOP CORECORE

MultiplePOP Services

Interconnect Leased Core

ISDN

POTS

Lease LineCable

xDSL

Access

Primary Packet Flow

Service

TerminateHere


TransparentRedirection of a Flow

in the POP

TransparentRedirection of a Flow

in the POPFactors that went into the

design of WCCP

Factors that went into the

design of WCCP

39Presentation_ID 1999,CiscoSystems,Inc. www.cisco.com 40Presentation_ID 1999,CiscoSystems,Inc. www.cisco.com

Design Objectives for the ISPDesign Objectives for the ISP

Transparent Redirec t ionof a IP flowbased on source, destination, and/orport number.

Transparent In tegrat ion- norebuilding the POP to add this

service.

Failed open - if the service fails, itshould not effect the core IP servicenor any other services.





ISDN

POTS

Lease LineCable

xDSL

Access

Primary Packet Flow

WCCPService

Clusters

Not to effect the primary packet flow of thePOP - if not redirected - then is CEF/dCEFSwitched!





ISDN

POTS

Lease LineCable

xDSL

Access

WCCP

ServiceClusters

Work with the multi-level L2/L3 redundancyof the ISP POP. Equal paths in the IGP +CEF leads packet asymmetry.

Input PortWCCP

Redirect


8/8

isp design fundelmentals-6up

Documents