brkdct-1927 - bridging in the data center with or without spanning tree

7/28/2019 BRKDCT-1927 - Bridging in the Data Center With or Without Spanning Tree

http://slidepdf.com/reader/full/brkdct-1927-bridging-in-the-data-center-with-or-without-spanning-tree 1/55

BRKDCT-2927_c2 © 2009 Cisco Systems, Inc. All rights reserved. Cisco Public 1

Bridging in the Data Center With or Without Spanning Tree

BRKDCT-2927



© 2009 Cisco Systems, Inc. All rights reserved. Cisco PublicBRKDCT-2927_c2 2

Overview

Transparent bridging data plane

Spanning Tree Protocol (control plane)

How it works, how it fails

Stability features

Application to DC design

Future of bridging




Transparent Bridging

Data Plane


http://slidepdf.com/reader/full/brkdct-1927-bridging-in-the-data-center-with-or-without-spanning-tree 4/55© 2009 Cisco Systems, Inc. All rights reserved. Cisco PublicBRKDCT-2927_c2 4

Ethernet

Physical Layer

coax cable, repeater, hubs

Broadcast medium

Any frame seen by the whole LAN, unmodified

Plug and play (literally!)No cooperation expected from the host

Protocols were developed with Ethernet behavior in mind

Set the Expectations…



Transparent Bridging

Layer 2:

Terminate Layer 1Can take decisions based on frame content

Transparent to Ethernet clients implies:

Create a broadcast domain

Forward frames unmodified

Be plug and play

Looks Like Ethernet for End Devices



Bridges Segment the Collision Domain

repeater

By Terminating Layer 1

B CA

B CA

bridge: less collisions, full-duplex possible



Bridges Filter Frames

Bridges learn MAC addresses independently

Build a filtering database (not a routing table!)

Increase overall bandwidth available

By Taking Decisions Based On Frame Content

B CA

A,B C

Dst: B



Why Not a Routing Table?

Frames with unknown destinationaddress *must* be flooded

=> need support for flooding

There is no cooperation from the hosts

No hierarchy in the MAC addressesNo subnet

Only host routes would be possible => not scalable



Extreme Hierarchical Network Example

Routers: 3 summary routes per devices

Bridges: 4 billion host routes per devices

4 Billion Hosts

3 2

“ l a y e r s ”

It might beacceptable tohave 4 billionroutes here…

But not here



Forwarding Decision

Routing:

If an entry exists in the forwarding table, forward

Else, drop

Bridging:

If an entry exists in the filtering database, drop

Else, flood

Fundamental Difference Between Routing and Bridging

optimization



No Routing Table… Consequences

Routing: Notion of location associated to addresses

Equal Cost Multipathing (ECMP),

Reverse Path Forwarding Check (RPFC)

B

Bridging: flooding requires a tree

B

A

A

To BTo A

R1 R2

B1 B2



Bridgingloop

Failure domain ≈ bridging domain

Failure to Provide a Tree Is Catastrophic

A loop will result in network wide flooding

Can have an impact on CPU (low end platforms)

No Time To Live (TTL) field in frames



So Why Bridging?

Some protocols require it

IP uses it: subnet concept linked to Layer 2

172.28.192.1 .2 .3 .4




Extend a Subnet across Devices

For port density (not enough port on device)

For provisioning flexibility (add devices withoutchanging L3 network configuration)

For redundancy (NIC teaming)

Virtual machine mobility

.6 .7 .3172.28.192.1 .4 .2 .5




Section Summary

Bridging is complementary to routing

Bridging is flexible

Bridging main weaknesses are:

Failure domain = bridging domain (not scalable)

A tree is required => no multipathing

Those limitations are causes by historic constraints inthe data plane

STP not mentioned yet (control plane)




Spanning Tree Protocol

Control Plane




STP Goals

Enforce a tree (at all time)

Spanning eventually

In a plug and play fashion

Notify learning function of topology changes




STP Information

Bridges exchange information usingBridge Protocol Data Units (BPDUs)

This information can be compared

Bridges propagate a “degraded” version of the bestinformation they ever received

A:1

0

B:2

0

C:3

0A

13

A “better” than B,B “better” than C

1010

12




STP Strategy For Building a Tree

The bridge with best information is the root

A bridge keeps its best path to the root forwarding

Alternate paths to the root are blocked

A

B C

A

Root bridge

(best information inthe network)

Designated Port

(best information onthis segment)

Root Port(best path to the root)

Alternate Port

(alternate path to the root)

Designated port

Root port

Alternate port




STP Stability: What Can STP Do Wrong?

Failure to create a “spanning” topology

Loss of connectivity. Local issue, simple to troubleshoot,

similar to most L3 failures.

Failure to create a “tree” topology, i.e. introduce a loop

The real issue!

Failure to notify the learning function

Temporary black holing for some addresses




How Can STP Open a Loop?

Fundamental difference bridging vs. routing:

Router: not control message => no forwarding

Bridge: no control message => no blocking

A port that fails to receive BPDUs goesdesignated (forwarding)

Most STP failures are related to BPDUsbeing lost or not acted upon

Extra care must be taken before puttinga designated port to forwarding




Unidirectional Link Failure

A link only transmit traffic in one direction

BPDUs are dropped

Clockwise loop open

A

B C

A

13

10 10

12

BPDU lost because of

unidirectional link failure

BPDU ignored by

B (worseinformation)

loop

BPDUs lost

Designated port

Root port

Alternate port




“Brain Dead” Bridge

C does not process BPDUs (CPU)

C still forwards traffic (ASIC)

Traffic loops in both directions

A

B C

A

A

BPDUs ignoredand not relayed

loop

Designated port

Root port

Alternate port

BPDUs Ignored




Layer 2 Features and STP Enhancements

EtherChannels

BPDUguard, RootGuard

Dispute mechanism

Bridge Assurance




B

EtherChannel

Bundle several physical links into a logical one

No blocked port (redundancy not handled by STP)

Per frame (not per-vlan) load balancing

Control protocols like PAgP (Port Aggregation Protocol)and LACP (Link Aggregation Control Protocol) handlethe bundling process and monitor the health of the link

Limited to parallel links between two switches

Minor Change In the Data Plane

A

B

AChannel looks like asingle link to STP

Designated port

Root port

Alternate port




Rootguard/BPDUguard

Rootguard: prevents a port from accepting better info

BPDUguard: shut down a port that receives a BPDU

Not stability features per se

Enforce security policy

Restrict STP’s freedom

Trade off stability/connectivity

Enforce a Policy




Dispute Mechanism

There can only be a one designated port on a LAN

RSTP (Rapid Spanning Tree) and MST (Multiple

Spanning Trees) advertise a role in their BPDUs

A designated port with “worse” information is a problem

A

B C

A

Designated:13

10 10

BPDU lost because of

unidirectional link failure

Worse designatedBPDU: B detects adispute

Disputed port

Designated:12

Protects Against Unidirectional Link

No

loop!

Designated port

Root port Alternate port




Dispute Mechanism

A channel is a single logical link from STP’s perspective

A single BPDU is sent on a single physical port

Protects Against Bundling Errors

half loop

po1

BA

p1 & p2 not bundled on Bp1 & p2

bundled on B

p1

p2

po1

B:2

0

A:1

0

p1

p2

po1 disputed

D e s i g n

a t e d : 1 0

D e s i g n a t e d : 12 Worse designatedBPDU: A detects adispute

Without

Dispute

Mechanism

With

Dispute

Mechanism




Bridge Assurance

Identify and configure network ports vs. edge ports

On p2p network ports:

Send periodic BPDUs, regardless of role

Expect periodic BPDUs, regardless of role

If no BPDU is received, the port goes inconsistent

B:20

A:10

Designated:10

Root:12

Worse root BPDU:does not trigger dispute

Network port sendsperiodic BPDUs

Network port:

expects BPDUs

Edge port: does

not expect BPDUs

Designated port

Root port

Alternate port

Edge port




Bridge Assurance

Introduce a behavior closer to L3:

A network port with no peer does not transmit traffic

A

B C

A

A

The Ultimate Brain Dead Detection Mechanism

Bridge AssuranceInconsistent ports

(no BPDU received)

“brain dead” bridgeDesignated port

Root port

Alternate port




STP Features at Work

Data Center Network Design Examples




Redundancy Handled by STP

Aggregation

Access

Data Center Core B

L

R

N

E

BPDUguard

Loopguard

Rootguard

Network port

Edge port

- Normal port type

B

RR

N N

N N N

N NNN

N N

N N NRRRRRR

--

B

E

B

E

B

E

B

E

Layer 3

Layer 2 (STP + Bridge Assurance)

Layer 2 (STP + BA + Rootguard)

Layer 2 (STP + BPDUguard)

L L

E

Backup

Root

Backup

Root

HSRP STANDBY

HSRP STANDBY

Root Root

HSRP ACTIVE

HSRP ACTIVE

Protecting Against Access Failures




Protecting Against Access FailuresWhere Can a Loop Be Open?

The access layer is blocking the loops

A loop can only be open if an access bridge

puts both its uplinks to forwarding:

Network portN

R

Root GuardDesignated port

Root port

Alternate port

This port couldintroduce a loop

N

NN

RN R

N

N

Aggregation

Access

An Uplink Must Go to Designated




An Uplink Must Go to DesignatedRole For a Loop to Occur

Only root ports and designated ports can be forwarding

There is at most one root port per bridge

This means that a loop can only be open if an access uplink takes the designated role

Aggregation

Access Access

Aggregation

loop loop

At least onedesignated uplink

Designated port

Root port

Alternate port





Protecting Against Access FailuresDesignated Silent Access Uplink

Uplink is designated

Uplink does not send BPDUs

Bridge Assurance prevents the loop

N

NN

RN

Network portN

R Root GuardDesignated port


R

N

N

Bridge Assurance blocks

the aggregation port

Designated port (problem)





g gDesignated Access Uplink, Worse BPDU


Uplink sends worse designated information

Dispute mechanism prevents the loop

N

NN

RN

Network portN



R

N

N

Dispute mechanism blocks

the aggregation port w

o r s

e






g gDesignated Access Uplink, Better BPDU


Uplink sends better designated information

Root Guard forbids this scenario

N

NN

RN

Network portN



R

N

N

Root Guard blocks the

aggregation port b

e t t e r


Protecting Against Access Failures Two




g gRoot Access Uplinks…

Two root port on a bridge would be a severe bug

There is a limit to what can be done in the control plane

N

NN

RN

Network portN


Root port

Alternate port

R

N

N

Root port (problem)




VPCdomain

Virtual Port Channel (vPC)

Introduces some changes to the Data Plane

Provides load balancing

Does not rely on STP for redundancy

Limited to pair of switches

VPCdomain

Redundancy

handled by STP

Redundancy

handled by vPC

STP view of

vPC

Blockedport




vPC Data Center Example

VPCdomain

Aggregation

Access

Data Center

Core B

L

R

N

E

BPDUguard

Loopguard

Rootguard

Network port

Edge port

- Normal port type

B

RR

N N

N N N N N NRRRRRR

--

B

E

B

E

B

E

B

E E

NN

N

L

Layer 3

Layer 2 (STP + Bridge Assurance)

Layer 2 (STP + BA + Rootguard)

Layer 2 (STP + BPDUguard)

Backup

Root

Backup

Root

HSRP STANDBY

HSRP STANDBY

Root Root

HSRP ACTIVE

HSRP ACTIVE




Fixing “STP” Problems

By Fixing the Data Plane




Mac-in-Mac (802.1ah) Model

Introduced for Service Providers

Create more services

Solve Mac Address Table scalability issues

WX

Z

Y

A BA B A B

A ->X

B ->Y

A ->XB ->Y

X YA B

XY

BackboneEdge

Bridge Backbone

Bridge

User space Backbone space User space

Backbone Edge Bridge Provider Bridge




Mac-in-Mac Scalability

Backbone Edge Bridges (BEB) are able to:

map mac addresses between user and backbone spaces

encapsulate/decapsulate frames

BEB only need to learn a subset of the mac addresses

Backbone Bridges are regular bridges They only see backbone space addresses

Now, let’s assume that the backbone bridges are not

bridges but new “special” devices…




Application: Routing Backbone Frames

Backbone addresses are limited in number =>

They can be propagated by a control protocol

A routing table is possible in the backbone!

WX Y

A B


To X To Y

ECMP, RPFC etc… now possible in the backbone

Next generation bridge

Addi TTL




Adding a TTL

Frames are encapsulated unchanged in anew frame format in the backbone

The encapsulation can carry a TTL

A Link state protocol allows determining the exact hop count

WX Y

A B


To X To Y A ->X, TTL 2

AB ABXY 2AB ABXY 1

U i T h l i




Upcoming Technologies

By introducing a new data plane in the “backbone”, theadvantages of Layer 3 can be added to Layer 2

The backbone addresses are not seen by L2 users,they represent a location, aggregating several devices

The plug and play aspect of L2 can be maintained

XPCA

User space Backbone space

Global PC A address = X.A

Backbone Address(location)

Mac Address

(ID)

D t C t Eth t (DCE) TRILL




Data Center Ethernet (DCE) TRILL

Goal: replace current transparent bridging model

Add multipathing

Introduce L3-like stability for bridging

New frame format, using a compactbackbone address to

minimize overhead.

Note: DCE offers other properties (like lossless

Ethernet) not relevantto this presentation

DCE/

TRILL

(Transparent Interconnection of Lots of Links)

C l i




Conclusion

L2 desirable for its flexibility (as a complement to L3)

Transparent bridging has some scalability issues

Several stability features have been developed in thecontrol plane => they will never be enough to match L3

The final solution will be injecting L3 elements in thedata plane

References Related Sessions




References, Related Sessions

BRKDCT-2961, Evolution of Hierarchical Network Design for theData Center

BRKDCT-2981, Overview of L2MP technologies Data Center Design—IP Network Infrastructure

http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DC_3_0/DC-3_0_IPInfra.html

Interested in Data Center?





Discover the Data Center of the Future

Cisco booth: #617

See a simulated data center and discover the benefits includinginvesting to save, energy efficiency and innovation.

Data Center Booth

Come by and see what’s happening in the world of Data Center –demos; social media activities; bloggers; author signings

Demos include:

Unified Computing SystemsCisco on Cisco Data Center Interactive Tour

Unified Service Delivery for Service Providers

Advanced Services






Data Center Super Session

Data Center Virtualization Architectures, Road to Cloud Computing (UCS)

Wednesday, July 1, 2:30 – 3:30 pm, Hall D

Speakers: John McCool and Ed Bugnion

Panel: 10 Gig LOM

Wednesday 08:00 AM Moscone S303

Panel: Next Generation Data Center

Wednesday 04:00 PM Moscone S303

Panel: Mobility in the DC Data

Thursday 08:00 AM Moscone S303

Please Visit the Cisco Booth in theWorld of Solutions




Data Center and Virtualization

DC1 – Cisco Unified Computing System

DC2 – Data Center Switching: CiscoNexus and Catalyst

DC3 – Unified Fabric Solutions

DC4 – Data Center Switching: CiscoNexus and Catalyst

DC5 – Data Center 3.0: AccelerateYour Business, Optimize Your Future

DC6 – Storage Area Networking: MDS

DC7 – Application Networking Systems:WAAS and ACE

World of Solutions

See the technology in action

Complete Your OnlineSession Evaluation




Session Evaluation

Give us your feedback and youcould win fabulous prizes.Winners announced daily.

Receive 20 Passport points for each session evaluation youcomplete.

Complete your session

evaluation online now (open abrowser through our wirelessnetwork to access our portal) or visit one of the Internet stations

throughout the ConventionCenter. Don’t forget to activate your Cisco Live Virtual account for access to

all session material, communities, and

on-demand and live activities throughout

the year. Activate your account at the

Cisco booth in the World of Solutions or visitwww.ciscolive.com.




Appendix:LoopGuard




LoopGuard

A:10

p1

B:20

p2

A:?

p1

B:20

p2

A:?

p1

B:20

p2

p1 designated

(sends best info)

?

p1 stops sending

BPDUs

p2 ages out p1’s

info and becomes

designated

p2’s transition to forwarding

prevented by LoopGuard

?

A:30

p1

B:20

p2

A:30

p1

B:20

p2

p1 starts sending

worse information

p2 becomes

designated

transition authorized by

LoopGuard

A:10

A:30

B:20

Transition a Port to Designated Under Scrutiny…

brkdct-1927 - bridging in the data center with or without spanning tree

Documents