Campus NetworkBest Practices:Core and Edge Networks

Dale SmithUniversity of Oregon/NSRCdsmith@uoregon.eduThis document is a result of work by the Network Startup Resource Center (NSRC at http://www.nsrc.org). This document may be freely copied, modified, and otherwise re-used on the condition that any re-use acknowledge the NSRC as the original source.

Campus Network RulesMinimize number of network devices in any pathUse standard solutions for common situationsBuild Separate Core and Edge NetworksProvide services near the coreSeparate border routers from coreProvide opportunities to firewall and shape network traffic

Core versus EdgeCore network is the core of your networkNeeds to have reliable power and air conditioningMay have multiple coresAlways route in the coreEdge is toward the edges of your networkProvide service inside of individual buildings to individual computersAlways switch at the edge

Minimize Number of Network Devices in the PathBuild star networksNot daisy chained networks

Edge Networks (Layer 2 LANs)Provides Service to end usersEach of these networks will be an IP subnetPlan for no more than 250 Computers at maximum Should be one of these for every reasonable sized buildingThis network should only be switchedAlways buy switches that are managed no unmanaged switches!

Edge Networks

Make every network look like this:Fiber link to core router

Edge Networks ContinuedBuild Edge network incrementally as you have demand and moneyStart Small:Fiber link to core router

Edge Networks ContinuedThen as you need to add machines to the network, add a switch to get this:Fiber link to core router

Edge Networks ContinuedAnd keep adding switches to get to the final configurationFiber link to core router

Edge Networks ContinuedAnd keep adding switches to get to the final configurationFiber link to core router

Edge Networks ContinuedResist the urge to save money by breaking this model and daisy chaining networks or buildings togetherTry hard not to do this:Fiber link to core routerLink to adjacent buildingLink to another building

Edge Networks ContinuedThere are cases where you can serve multiple small buildings with one subnet.Do it carefully. Two basic models:Switch in core locationCat5e or fiberFiber link to core routerCat5e or fiberFiber circuits to small buildingsCopper or fiber link to core router

Selected Layer 2 TopicsCollision versus Broadcast DomainVLANsARP how it worksDHCP - How it worksSpanning TreeLink AggregationFailure modes100 Mbs and Gigabit Duplex mismatch

Collision vs. Broadcast DomainSimilar issues affects performance of LANHubs (Repeaters)Every packet goes to every port, irrespective of destination of packetEvery port is half duplexCan only be one packet in transit two transmitters = Collision

Collision vs. Broadcast DomainHubOnly One Packet at a timeEvery packet (even unicast) goes to every portHub Hubs/Repeaters

Collision vs. Broadcast DomainHubTwo Transmitters = CollisionHubCollision Hubs/Repeaters

Collision vs. Broadcast DomainSwitchesSwitches learn where hosts are eavesdropping on traffic and building a forwarding tableSwitches forward packets to correct port Can only be many packets in transitBroadcasts must go to all ports

Collision vs. Broadcast DomainSwitchMany packets can be in flight store and forwardUnicast Packets go to intended destinationSwitch Switches

Collision vs. Broadcast DomainSwitchBroadcasts go to all ports (notice this looks like the hubs picture some slides ago)Switch Switches

Collision vs. Broadcast DomainSwitchSwitches need to know about multicastSwitch Switches

VLANsVirtual LANs reduce scope of broadcast domain and separate trafficTagging identifying the VLAN associated with a packet. Ports are configured as Tagged or untagged.Trunking Carrying traffic for multiple VLANs on a single link. Must use tagging.

VLANsTagging on Trunks must tagSingle link carrying 3 VLANS

ARPAddress Resolution ProtocolBuilds a mapping of IP address to Ethernet AddressARP ProtocolBroadcast ARP Request (who has this IP?)Owner of IP address in ARP Request issues ARP replyPathology: anyone can issue an ARP reply at any time

ARP10.0.0.100:00:11:00:00:aa10.0.0.200:00:11:00:00:bb10.0.0.300:00:11:00:00:cc

DHCPDynamic Host Configuration ProtocolUsed to assign IP address and provide basic IP configuration to a host.Simple protocolClient broadcasts a DHCP DISCOVERServer(s) unicast back a DHCP OFFERClient selects an offer and sends a REQUESTServer sends back a DHCP ACK to clientManaged switches can block rogue DHCP

Spanning TreeEliminates loops in Layer 2 networksSeveral flavorsOriginal Spanning Tree 802.1DRapid Spanning Tree (RSTP) 802.1wMultiple Spanning Tree (MSTP) 802.1s and 802.1Q-2003Modern managed switches can do all of the aboveLots of discussion about this Tuesday

Link AggregationBonds multiple channels together to provide more bandwidthIssues:CompatibilityHow traffic is scheduled 3 separate links aggregated as one

Failure ModesARP spoofingLoops in your networkRogue DHCP serversDuplex mis-match100Mbs late collisions and CRC1000Mbs cant establish linkNeed managed switches to correct these

Core Network

Routing versus SwitchingLayer 2 versus Layer 3Routers provide more isolation between devices (they stop broadcasts)Routing is more complicated, but also more sophisticated and can make more efficient use of the network, particularly if there are redundancy elements such as loops

Switching versus RoutingThese links must be routed, not switched

Core NetworkReliability is the keyremember many users and possibly your whole network relies on the coreMay have one or more network core locationsCore location must have reliable powerUPS battery backup (redundant UPS as your network evolves)GeneratorCore location must have reliable air conditioningAs your network evolves, core equipment should be equipped with dual power supplies, each powered from separate UPSBorder routers separate from CoreFirewalls and Traffic Shaping DevicesIntrusion DetectionIntrusion PreventionNetwork Address Translation

Core NetworkAt the core of your network should be routers you must route, not switch.Routers give isolation between subnetsA simple core:

Border RouterFirewall/ Traffic ShaperCore RouterAll router interfaces on a separate subnetCentral Servers for campusFiber optic links to remote buildings

Where to put Servers?Servers should be on a high speed interface off of your core routerServers should be at your core location where there is good power and air conditioningBorder RouterFirewall/ Traffic ShaperCore RouterAll router interfaces on a separate subnetFiber optic links to remote buildingsServersin core

Border RouterConnects to outside worldRENs and Peering are the reason you need themMust get Provider Independent IP address space to really make this work right

CampusNetworkRENInternetExchange

Putting it all Together

Notes on IP AddressingGet your own Public IP address space (get your V6 block when you get your V4 one)Make subnet IP space large enough for growthUse DHCP to assign addresses to individual PCsUse static addressing for switches, printers, and servers

More Complex Core DesignsOne Armed Router for CoreCore RouterFiber Optic LinksFiber Optic LinksCore ServersVLAN Trunk carrying all subnetsCore Switch

Complex Core DesignsFirewall/Traffic ShaperMultiple Core Routers

Alternative Core DesignsWireless Links versus FiberBorder RouterCore RouterFiber Optic LinksREN switchFirewall/Traffic ShaperWireless LinksCore Servers

Layer 2 and 3 SummaryRoute in the coreSwitch at the edgeBuild star networks dont daisy chainBuy only managed switches re-purpose your old unmanaged switches for labs

Questions?

This document is a result of work by the Network Startup Resource Center (NSRC at http://www.nsrc.org). This document may be freely copied, modified, and otherwise re-used on the condition that any re-use acknowledge the NSRC as the original source.

Symbols to use for diagrams

**