matěj grégr resilient network design - uninettresilient.pdf · resilient network design ... hsrp,...

/36 © 2011 Brno University of Technology, Faculty of Information Technology, Matěj Grégr, [email protected]

nes. .fitat

RESILIENT NETWORK DESIGN Matěj Grégr

http://www.fit.vutbr.cz/research/groups/nes@fit/


nes. .fitat

Campus Best Practices - Resilient network design

Campus Best Practices documents share knowledge within several technical areas (physical infrastructure, campus networking, wireless, security, etc.)

Campus Best Practice Documents are available at: http://www.terena.org/activities/campus-bp/bpd.html

Resilient network design is described mainly in: Recommended Resilient Campus Network Design, March 2010 (CBPD114,

the Czech Republic)

Recommended configuration of switches in campus networks, May 2010 (UFS105, Norway)


http://www.terena.org/activities/campus-bp/bpd.html






nes. .fitat

Resilient network design

Enterprise campus requires highly available and secure network infrastructure, to support business solutions such as voice, video, wireless, and mission-critical data applications.

Resiliency—Ability to provide non-stop business communication with rapid sub-second network recovery during abnormal network failures or even network upgrades.

The goal of resilient topology is to eliminate downtime and convergence time during crashes and device upgrades



nes. .fitat

Network design ①

Single broadcast domain

Single security domain

No backup

Central switch performance



nes. .fitat

Network design ②

Routers can separate network

Smaller broadcast domains

Possibility to control traffic path

Routers are pretty expensive

Number of ports in an ordinary router is limited



nes. .fitat

Resilient Campus design

Servers and users access ports

Aggregation traffic from access layer

High speed switching/routing



nes. .fitat

Access layer

Entry point for clients into the network

Provides Layer 2 (VLAN) connectivity between users

High port density

PoE

Security mechanism – 802.1x

QoS classification



nes. .fitat

Access layer

Problems:

Access switches are single points of failure in a network

• Redundant connection for end users is very expensive

• Resiliency has to be integrated into the device

• Redundant supervisor, power outlet …

Recommendations:

Disable Etherchannel and trunk negotiation for end users

• Prevents VLAN hopping attacks

Enable edge ports (PortFast) for access ports



nes. .fitat

Server access layer redundancy

Servers need redundant network connection

Possible solutions: Link aggregation protocol (LACP, PAgP)

Ethernet card bonding

Server virtualization

Service load balancing



nes. .fitat

Link aggregation (Etherchannel)

Allows combination of several physical links to one logical channel

Load balancing MAC, IP, IP+TCP/UDP

Simplify configuration only logical port configuration is necessary

Simplify other protocol operation STP sees only Etherchannel link

Logical view Physical view



nes. .fitat

Link aggregation protocol

Two signaling protocols exist (PAgP, LACP)

LACP – IEEE 802.3ad is recommended, PAgP is Cisco proprietary

Modes:

Active/Passive: request/response channel establishment

On/Off: static configuration

Recommendation:

Use static configuration (mode on): dynamic configuration delays channel establishment



nes. .fitat

Etherchannel configuration

Prerequisites to established channel

Same speed/duplex

Same mode (trunk – same vlans enabled, access – same access vlan)

Same STP cost and mode (edge/non-edge)

Cisco Switch1(config)# interface range gi 0/1 - 2

Switch1(config-if-range)# channel-group 1 mode active

HP Switch1(config)# trunk g1-g2 trk1 lacp



nes. .fitat

Ethernet card bonding

Useful, when server does not support aggregation protocol

Linux - supported in kernel

Windows – supported in Ethernet card drivers

Several modes:

backup

transmit load balancing: load balance in transmit direction

adaptive load balancing: rewrite MAC addresses, different peers use different MAC address, no switch support is necessary



nes. .fitat

Server virtualization

Virtualization brings simplification in network resilient design

Virtual servers are connected through Virtual Switch

Virtual switch – redundant connection to resilient network

All virtual servers have resilient connection to Internet



nes. .fitat

Server Load Balancing

SLB provides a virtual server IP address to which clients can connect, representing a group of real physical servers in a server farm

Load balancing: according to: L4 - L7 information

Software implementation

Hardware implementation

• Cisco Application Control Engine modul needed

Advantages: Reduced server load

Higher security – real IP address is not visible

Downtime elimination if more servers are used



nes. .fitat

SLB modes

Dispatched mode

Every server in server farm has own real IP together with virtual IP address (secondary IP or loopback IP) of whole server farm

Traffic redirection: packet with virtual IP is put in Ethernet frame with MAC address of real server

All servers in SLB farm have to be in same IP subnet

Directed mode

Every server has only own real IP address

Servers do not know virtual IP address of whole server farm

NAT is used – virtual IP address of the farm is translated to real IP address of a server



nes. .fitat

Distribution layer

Purpose is to provide L2 distribution through switched network

Topology contains loops (needed for redundancy)

L2 loop protocol (STP) is needed

Gateway redundancy, high availability

Packet filtering,



nes. .fitat

Spanning tree protocol

Necessary for loop elimination

802.1D

Original version has very long convergence time > 30s

Did not support VLAN – support added in 802.1t, extend BID, now integrated into 802.1D

Recommendation is to use RSTP (802.1w), RPVSTP+ or MSTP (802.1s)



nes. .fitat

Rapid Spanning Tree (RSTP)

IEEE 802.1w

Convergence time < 1s

Backward compatibility with 802.1D

Several Cisco 802.1D improvements were integrated into 802.1w standard (UplinkFast, BackboneFast …)

Configuration:

Cisco: Switch(config)# spanning-tree mode rapid-pvst

HP: Switch(config)# spanning-tree force-version rstp-operation



nes. .fitat

STP load balancing

Scenario:

Port-priority or port cost can be used

Example: Left(config)# interface gi1/6

Left(config)# spanning-tree vlan 200 port priority 112

Recommendation:

Useful is to set higher priority on undesired port instead of setting lower priority on desired port



nes. .fitat

MSTP

STP or RSTP support only one STP tree for all VLANs

RPVSTP+ (Cisco proprietary): STP tree per every VLAN

Main idea of MSTP:

Administrator can configure several STP instances

VLANS are mapped to instances

MSTP internally use RSTP



nes. .fitat

MSTP configuration

Switch(config)# spanning-tree mode mst

Switch(config)# spanning-tree mst configuration

Switch(config-mst)# name region1

Switch(config-mst)# revision 1

Switch(config-mst)# instance 1 vlan 100

Switch(config-mst)# instance 2 vlan 200

Configuration needed for every switch

Increase complexity

Proprietary solution:

Use VTPv3



nes. .fitat

VTPv3

More flexible protocol – can distribute “any” database

Better authentication

VTP can be turned on/off per port

Client can not rewrite database as it was common in previous versions

Server/client/transparent switch for databases VLAN, MST, Unknown (another database in future)

Primary/secondary server Primary server (only) can modify a database

• Only one server in a domain

Secondary server: backup server, could be primary



nes. .fitat

VTPv3 – MSTP configuration

Configure VTPv3 Switch(config)# vtp version 3

Switch(config)# vtp domain NAME

Switch(config)# vtp mode server mst

Switch(config)# end

Switch# vtp primary mst

MSTP configuration similar to previous slide

MSTP config is distributed within VTPv3 domain



nes. .fitat

Gateway redundancy

Historic attempts Proxy ARP, ICMP Router Discovery Protocol, routing support in

end station

Does not scale well, software support is needed

Solution: Redundancy using virtual router

Virtual IP, virtual MAC

No host configuration needed

Proprietary solutions HSRP, GLBP

Standard solution VRRP

Internet,

Backbone, etc.

Virtual Router

ForwarderBackup in

Standby



nes. .fitat

VRRP

Open standard IETF RFC 3768 – version 2

IETF RFC 5798 – version 3 (IPv4 + IPv6)

VRRP Group – virtual router with virtual IP address

Virtual MAC address - 0000.5e00.01xx - last byte is group number

Master router Highest priority

IP address same as virtual IP (IP address owner) – always win master role

Backup router Other routers in VRRP group



nes. .fitat

VRRP configuration

Cisco configuration SwitchA(config)# interface vlan10

SwitchA(config-if)# ip address 10.1.10.5 255.255.255.0

! Virtual IP for vrrp group 10

SwitchA(config-if)# vrrp 10 ip 10.1.10.1

! Priority for router in group 10 (standard priority is 100)

SwitchA(config-if)# vrrp 10 priority 150

! Preempt delay

SwitchA(config-if)# vrrp 10 preempt delay minimum 380

HP configuration hp (config)# vlan 223

hp (vlan-224)# vrrp vrid 1

hp (vlan-224-vrid-1)# owner

hp (vlan-224-vrid-1)# virtual-ip-address 10.1.10.5 255.255.255.0

hp (vlan-224-vrid-1)# enable

Recommendation Use the first IP address from subnet for Master router

Set preempt-delay-time to let routing protocol converge



nes. .fitat

Gateway Load Balancing Protocol

HSRP, VRRP may have inactive routers in a group Standby and Other routers: cannot be used by end station (do not have

virtual IP/MAC)

Possible solution: several HSRP/VRRP groups with end stations distributed among them

Static configuration!

GLBP goal is to utilize all routers equally Several members of GLBP group should participate in packets

switching/routing

GLBP solution Virtual IP per group

max. 4 virtual MAC addresses per group



nes. .fitat

GLBP – fundamental concept

GLBP group contains two types of members Active Virtual Gateway Active Virtual Forwarder

Active virtual gateway (AVG) Router with highest priority (highest IP address) There is only one AVG per group Assign virtual MAC addresses of other members of the GLBP group Reply to virtual IP ARP requests

• GLBP group controller: end device requesting virtual IP address obtains some of the assigned virtual MAC addresses

Active virtual forwarder (AVF) Max. 4 AVF per group, other routers are in backups AVF are responsible for assigned virtual MAC/IP address AVG is also AVF Communication is done via Hello messages every 3 sec, multicast

address 224.0.0.102 UDP 3222



nes. .fitat

GLBP operation



nes. .fitat

GLBP load balancing techniques

Weighted load-balancing algorithm

Based on weighting parameter

Per-host (Host-dependent load-balancing algorithm)

End station has always the same AVF

Round-robin load-balancing algorithm (default)

Virtual MAC addresses are rotated per ARP request



nes. .fitat

Core layer

High speed routing

Fast convergence is necessary

Aggregate links from distribution layer

Try to avoid any packet manipulation, (access lists and filtering), which would slow down the switching of packets.

Smaller campus can combine core and distribution layer functions



nes. .fitat

Is the core layer necessary?



nes. .fitat

Summary

Resilient network design eliminates downtime and convergence time in a network

If is it properly deployed

Always depends of “How much money do You have”



nes. .fitat
