10/11/06 CS/ECE 438 - UIUC, Fall 2006 1

Switching Hardware

10/11/06 CS/ECE 438 - UIUC, Fall 2006 2

Switch Design

Champaign

Effingham

Springfield

Bloomington

St. Louis

Indianapolis

Chicago

10/11/06 CS/ECE 438 - UIUC, Fall 2006 2

Switch Design

Champaign

Effingham

Springfield

Bloomington

St. Louis

Indianapolis

Chicago

10/11/06 CS/ECE 438 - UIUC, Fall 2006 2

Switch Design

Champaign

Effingham

Springfield

Bloomington

St. Louis

Indianapolis

Chicago

How should wedesign Champaignto accommodate

traffic flows?

10/11/06 CS/ECE 438 - UIUC, Fall 2006 2

Switch Design

Champaign

Effingham

Springfield

Bloomington

St. Louis

Indianapolis

Chicago

How should wedesign Champaignto accommodate

traffic flows?

10/11/06 CS/ECE 438 - UIUC, Fall 2006 3

Contention

10/11/06 CS/ECE 438 - UIUC, Fall 2006 3

Contention

Bridges: same collision domain If an output port is busy when forwarding packet

from input port, cause collision

10/11/06 CS/ECE 438 - UIUC, Fall 2006 3

Contention

Bridges: same collision domain If an output port is busy when forwarding packet

from input port, cause collision Switches: different collision domain

Use CSMA/CD before sending packet onward

10/11/06 CS/ECE 438 - UIUC, Fall 2006 3

Contention

Bridges: same collision domain If an output port is busy when forwarding packet

from input port, cause collision Switches: different collision domain

Use CSMA/CD before sending packet onward Buffer packets

When output port is busy When multiple packets are destined for same

output port

10/11/06 CS/ECE 438 - UIUC, Fall 2006 4

Switch Design

Input Port

Input Port

Input Port

Input Port

Input Port

Input Port

Output Port

Output Port

Output Port

Output Port

Output Port

Output Port

SwitchFabric

10/11/06 CS/ECE 438 - UIUC, Fall 2006 5

Contention – Output PortBuffering

B

A

irate Bobwriting complaintletter in triplicate

Alicewaiting to return

penny given in error

trying to buy food you

standard checkout lines customer service

1x6 switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 6

Contention – Input PortBuffering

standard checkout lines customer service

B

Airate Bob

writing complaintletter in triplicate

Alicewaiting to return

penny given in error

trying to buy food

you1x6 switchhead-of-line

blocking

10/11/06 CS/ECE 438 - UIUC, Fall 2006 6

Contention – Input PortBuffering

standard checkout lines customer service

B

Airate Bob

writing complaintletter in triplicate

Alicewaiting to return

penny given in error

trying to buy food

you1x6 switchhead-of-line

blocking

cashiers are standing by!

10/11/06 CS/ECE 438 - UIUC, Fall 2006 7

Contention – Output PortContention

B

irate Bobwriting complaintletter in triplicate

Alice waiting to return penny given in error you

A

waiting to complainabout head-of-line blocking

standard checkout lines customer service

1x6 switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 8

Contention – BufferingCapacity

B

you

buffering capacityper output

is finite

A

(others turned away at door)

standard checkout lines customer service

1x6 switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 9

Contention – Back Pressure

Let the receiver tell the sender to slow down Propagation delay requires that the receiver

react before the buffer is full Typically used in networks with small

propagation delay

switch 1 switch 2

“no more, please”

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

SwitchSwitch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch1

Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch2 1

Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch3 2 1

Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch4 3 2

1Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch5 4 3

12Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch6 5 4

123stop Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch 12347 6 5

stop Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch 123458 7 6

stop Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch 1234569 8 7

Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch 1234569 8

7Discard:

Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch9

123456

8Discard:

Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 10

Contention – Back Pressure

NOTE Propagation delay requires that switch 2 exert

backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).

Switch

9Discard:

123456Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 11

Switch Design Goals

Throughput Number of packets a switch can forward

per second Scalability

How many input/output ports can itconnect

Cost Per port monetary costs

10/11/06 CS/ECE 438 - UIUC, Fall 2006 12

Special Purpose Switches

Problem Connect N inputs to M outputs

NxM (“N by M”) switch Often N = M

Goals High throughput

Best is MIN(sum of inputs, sum of outputs) Avoid contention Good scalability

Linear size/cost growth

10/11/06 CS/ECE 438 - UIUC, Fall 2006 13

Switch Design

Ports handle complexity Forwarding decisions Buffering

Simple fabric Move packets from inputs to outputs May have a small amount of internal

buffering

10/11/06 CS/ECE 438 - UIUC, Fall 2006 14

Switch Design Goals

Throughput Main problem is contention Need a good traffic model

Arrival time Destination port Packet length

Telephony modeling is well understood Until faxes and modems

Modeling of data traffic is new Not well understood Will good models help?

10/11/06 CS/ECE 438 - UIUC, Fall 2006 15

Switch Design Goals

Contention Avoid contention through intelligent buffering Use output buffering when possible Apply back pressure through switch fabric Improve input buffering through non-FIFO

buffers Reduces head-of-line blocking

Drop packets if input buffers overflow

10/11/06 CS/ECE 438 - UIUC, Fall 2006 16

Switch Design Goals

Scalability O(N) ports Port design complexity O(N) gives O(N2)

for entire switch Port design complexity of O(1) gives

O(N) for entire switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 17

Switch Design

Crossbar switches Banyan Networks Batcher Networks Sunshine Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 18

Crossbar Switch

Every input port is connected to everyoutput port NxN

Output ports Complexity scales as O(N2)

10/11/06 CS/ECE 438 - UIUC, Fall 2006 19

Crossbar Switch

Output Port

Output Port

Output Port

Output Port

10/11/06 CS/ECE 438 - UIUC, Fall 2006 20

Knockout Switch

Assumption: It is unlikely that N inputs will have packets destined for

the same output port Pick L from N packets at a port

Output port maintains L cyclic buffers Shifter places up to L packets in one cycle Each buffer gets only one packet Output port uses round-robin between buffers Arrival order is maintained

Problem Hot spots

Output ports scale as O(N)

10/11/06 CS/ECE 438 - UIUC, Fall 2006 21

Knockout Switch

Output port design Packet filters

Recognize packets destined for a specific port

Concentrator Selects up to L packets from those destined for this

port Discards excess packets

Queue Length L

10/11/06 CS/ECE 438 - UIUC, Fall 2006 22

Knockout Switch

R R

R

D

R

RD

R2x2

randomselector

Delayunit

Choose L of N

Ex: 2 of 4

10/11/06 CS/ECE 438 - UIUC, Fall 2006 22

Knockout Switch

R R

R

D

R

RD

R2x2

randomselector

Delayunit

Choose L of N

Ex: 2 of 4

What happensif more than L

arrive?

10/11/06 CS/ECE 438 - UIUC, Fall 2006 22

Knockout Switch

R R

R

D

R

RD

R2x2

randomselector

Delayunit

Choose L of N

Ex: 2 of 4

What happensif more than L

arrive?

1 2 3 4

10/11/06 CS/ECE 438 - UIUC, Fall 2006 22

Knockout Switch

R R

R

D

R

RD

R2x2

randomselector

Delayunit

Choose L of N

Ex: 2 of 4

What happensif more than L

arrive?

12

34

10/11/06 CS/ECE 438 - UIUC, Fall 2006 22

Knockout Switch

R R

R

D

R

RD

R2x2

randomselector

Delayunit

Choose L of N

Ex: 2 of 4

What happensif more than L

arrive?

Discard1

234

10/11/06 CS/ECE 438 - UIUC, Fall 2006 22

Knockout Switch

R R

R

D

R

RD

R2x2

randomselector

Delayunit

Choose L of N

Ex: 2 of 4

What happensif more than L

arrive?Choice

1Choice

2

Discard1

34

10/11/06 CS/ECE 438 - UIUC, Fall 2006 23

Self-Routing Fabrics

Idea Use source routing on “network” in switch Input port attaches output port number as

header Fabric routes packet based on output port

Types Banyan Network Batcher-Banyan Network Sunshine Switch

10/11/06 CS/ECE 438 - UIUC, Fall 2006 24

Banyan Network

A network of 2x2 switches Each element routes to output 0 or 1

based on packet header A switch at stage i looks at bit i in the

header

01

10/11/06 CS/ECE 438 - UIUC, Fall 2006 24

Banyan Network

A network of 2x2 switches Each element routes to output 0 or 1

based on packet header A switch at stage i looks at bit i in the

header

0100100

10/11/06 CS/ECE 438 - UIUC, Fall 2006 24

Banyan Network

A network of 2x2 switches Each element routes to output 0 or 1

based on packet header A switch at stage i looks at bit i in the

header

0100100

10/11/06 CS/ECE 438 - UIUC, Fall 2006 25

Banyan Network

10/11/06 CS/ECE 438 - UIUC, Fall 2006 25

Banyan Network

001

011

110

111

10/11/06 CS/ECE 438 - UIUC, Fall 2006 25

Banyan Network

001

011

110

111

10/11/06 CS/ECE 438 - UIUC, Fall 2006 25

Banyan Network

001

011

110

111

001

011

110

111

10/11/06 CS/ECE 438 - UIUC, Fall 2006 25

Banyan Network

001

011

110

111

10/11/06 CS/ECE 438 - UIUC, Fall 2006 25

Banyan Network

001

011

110

111

001

011

110

111

10/11/06 CS/ECE 438 - UIUC, Fall 2006 25

Banyan Network

001

011

110

111

10/11/06 CS/ECE 438 - UIUC, Fall 2006 25

Banyan Network

001

011

110

111

001

011

110111

10/11/06 CS/ECE 438 - UIUC, Fall 2006 25

Banyan Network

001

011

110

111

001

011

110111

10/11/06 CS/ECE 438 - UIUC, Fall 2006 26

Banyan Network

Perfect Shuffle N inputs requires log2N stages of N/2 switching

elements Complexity on order of N log2N

Collisions If two packets arrive at the same switch destined

for the same output port, a collision will occur If all packets are sorted in ascending order upon

arrival to a banyan network, no collisions willoccur!

10/11/06 CS/ECE 438 - UIUC, Fall 2006 27

Batcher Network

Performs merge sort A network of 2x2 switches

Each element routes to output 0 or 1 based onpacket header

A switch at stage i looks at the whole header Two types of switches

Up switch Sends higher number to top output (0)

Down switch Sends higher number to bottom output (1)

10/11/06 CS/ECE 438 - UIUC, Fall 2006 28

Batcher Network

D

D

D

U

D

D

10/11/06 CS/ECE 438 - UIUC, Fall 2006 28

Batcher Network

D

D

D

U

D

D

7

3

6

1

10/11/06 CS/ECE 438 - UIUC, Fall 2006 28

Batcher Network

D

D

D

U

D

D

Sort

7

3

6

1

10/11/06 CS/ECE 438 - UIUC, Fall 2006 28

Batcher Network

D

D

D

U

D

D

Sort

7

3

6

1

7

3

6

1

10/11/06 CS/ECE 438 - UIUC, Fall 2006 28

Batcher Network

D

D

D

U

D

D

Sort

7

3

6

1

7

3

6

1Merge

7

3

6

1

10/11/06 CS/ECE 438 - UIUC, Fall 2006 28

Batcher Network

D

D

D

U

D

D

Sort

7

3

6

1

7

3

6

1

3

7

6

1

Merge

7

3

6

1

10/11/06 CS/ECE 438 - UIUC, Fall 2006 28

Batcher Network

D

D

D

U

D

D

Sort Merge

6

3

1

7

7

3

6

1

7

3

6

1

3

7

6

1

Merge

7

3

6

1

10/11/06 CS/ECE 438 - UIUC, Fall 2006 28

Batcher Network

D

D

D

U

D

D

Sort Merge

6

3

1

7

7

3

6

1

7

3

6

1

3

7

6

1

1

7

3

6

Merge

7

3

6

1

10/11/06 CS/ECE 438 - UIUC, Fall 2006 29

Batcher Network

10/11/06 CS/ECE 438 - UIUC, Fall 2006 29

Batcher Network

D

D

D

U

D

D

10/11/06 CS/ECE 438 - UIUC, Fall 2006 29

Batcher Network

U

U

U

D

U

U

D

D

D

U

D

D

10/11/06 CS/ECE 438 - UIUC, Fall 2006 29

Batcher Network

U

U

U

D

U

U

D

D

D

U

D

D

D

D

D

D

D

D

D

D

D

D

D

D

10/11/06 CS/ECE 438 - UIUC, Fall 2006 30

Batcher Network

How it really works Merger is presented with a pair of sorted lists,

one in ascending order, one in descending order First stage of merger sends packets to the

correct half of the network Second stage sends them to the correct quarter

Size N/2 switches per stage log2N x (1 + log2N)/2 stages Complexity = N log2

2N

10/11/06 CS/ECE 438 - UIUC, Fall 2006 31

Batcher-Banyan Network

Idea Attach a batcher network back-to-back with a banyan

network Arbitrary unique permutations can be routed without

contention Two packets destined for same output port still collide!

Sunshine Switch Like a knockout switch

Can handle up to L packets per output port Recirculates overflow packets

If more than L packets arrive for any output port in one cycle

10/11/06 CS/ECE 438 - UIUC, Fall 2006 32

Sunshine Switch

Elements Multiple Banyan networks

Enables multiple packets per output port Delay Box

Excess (K) packets are recirculated and resubmitted to the switch Batcher network

N new packets K delayed packets

Trap Identifies packets destined for banyan Identifies excess packets

Selector Routes multiple packets for same output on separate banyans

10/11/06 CS/ECE 438 - UIUC, Fall 2006 33

Sunshine Switch

Batcher Trap Selector

L Banyans

Delay

Inputs

n n+k n+k

nnn

nn

n

kk

10/11/06 CS/ECE 438 - UIUC, Fall 2006 34

Invariants

Batcher All packets are sorted by destination

Trap Compares line i with line i+L, marks i+L for

recirculation if address equal Selector

Another batcher network, sorting based on therecirculation bit

Banyan Alternate every L lines to each Banyan

10/11/06 CS/ECE 438 - UIUC, Fall 2006 35

Sunshine Switch

Can packets circulate for ever? Priority bit is used to favor older packets Priority bit also ensure packet order is

preserved through the switch