10/11/06 CS/ECE 438 - UIUC, Fall 2006 1
Switching Hardware
10/11/06 CS/ECE 438 - UIUC, Fall 2006 2
Switch Design
Champaign
Effingham
Springfield
Bloomington
St. Louis
Indianapolis
Chicago
10/11/06 CS/ECE 438 - UIUC, Fall 2006 2
Switch Design
Champaign
Effingham
Springfield
Bloomington
St. Louis
Indianapolis
Chicago
10/11/06 CS/ECE 438 - UIUC, Fall 2006 2
Switch Design
Champaign
Effingham
Springfield
Bloomington
St. Louis
Indianapolis
Chicago
How should wedesign Champaignto accommodate
traffic flows?
10/11/06 CS/ECE 438 - UIUC, Fall 2006 2
Switch Design
Champaign
Effingham
Springfield
Bloomington
St. Louis
Indianapolis
Chicago
How should wedesign Champaignto accommodate
traffic flows?
10/11/06 CS/ECE 438 - UIUC, Fall 2006 3
Contention
10/11/06 CS/ECE 438 - UIUC, Fall 2006 3
Contention
Bridges: same collision domain If an output port is busy when forwarding packet
from input port, cause collision
10/11/06 CS/ECE 438 - UIUC, Fall 2006 3
Contention
Bridges: same collision domain If an output port is busy when forwarding packet
from input port, cause collision Switches: different collision domain
Use CSMA/CD before sending packet onward
10/11/06 CS/ECE 438 - UIUC, Fall 2006 3
Contention
Bridges: same collision domain If an output port is busy when forwarding packet
from input port, cause collision Switches: different collision domain
Use CSMA/CD before sending packet onward Buffer packets
When output port is busy When multiple packets are destined for same
output port
10/11/06 CS/ECE 438 - UIUC, Fall 2006 4
Switch Design
Input Port
Input Port
Input Port
Input Port
Input Port
Input Port
Output Port
Output Port
Output Port
Output Port
Output Port
Output Port
SwitchFabric
10/11/06 CS/ECE 438 - UIUC, Fall 2006 5
Contention – Output PortBuffering
B
A
irate Bobwriting complaintletter in triplicate
Alicewaiting to return
penny given in error
trying to buy food you
standard checkout lines customer service
1x6 switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 6
Contention – Input PortBuffering
standard checkout lines customer service
B
Airate Bob
writing complaintletter in triplicate
Alicewaiting to return
penny given in error
trying to buy food
you1x6 switchhead-of-line
blocking
10/11/06 CS/ECE 438 - UIUC, Fall 2006 6
Contention – Input PortBuffering
standard checkout lines customer service
B
Airate Bob
writing complaintletter in triplicate
Alicewaiting to return
penny given in error
trying to buy food
you1x6 switchhead-of-line
blocking
cashiers are standing by!
10/11/06 CS/ECE 438 - UIUC, Fall 2006 7
Contention – Output PortContention
B
irate Bobwriting complaintletter in triplicate
Alice waiting to return penny given in error you
A
waiting to complainabout head-of-line blocking
standard checkout lines customer service
1x6 switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 8
Contention – BufferingCapacity
B
you
buffering capacityper output
is finite
A
(others turned away at door)
standard checkout lines customer service
1x6 switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 9
Contention – Back Pressure
Let the receiver tell the sender to slow down Propagation delay requires that the receiver
react before the buffer is full Typically used in networks with small
propagation delay
switch 1 switch 2
“no more, please”
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
SwitchSwitch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch1
Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch2 1
Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch3 2 1
Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch4 3 2
1Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch5 4 3
12Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch6 5 4
123stop Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch 12347 6 5
stop Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch 123458 7 6
stop Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch 1234569 8 7
Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch 1234569 8
7Discard:
Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch9
123456
8Discard:
Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 10
Contention – Back Pressure
NOTE Propagation delay requires that switch 2 exert
backpressure at high-water mark rather whenbuffer completely full. Backpressure is thustypically only used in networks with smallpropagation delays (e.g., switch fabrics).
Switch
9Discard:
123456Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 11
Switch Design Goals
Throughput Number of packets a switch can forward
per second Scalability
How many input/output ports can itconnect
Cost Per port monetary costs
10/11/06 CS/ECE 438 - UIUC, Fall 2006 12
Special Purpose Switches
Problem Connect N inputs to M outputs
NxM (“N by M”) switch Often N = M
Goals High throughput
Best is MIN(sum of inputs, sum of outputs) Avoid contention Good scalability
Linear size/cost growth
10/11/06 CS/ECE 438 - UIUC, Fall 2006 13
Switch Design
Ports handle complexity Forwarding decisions Buffering
Simple fabric Move packets from inputs to outputs May have a small amount of internal
buffering
10/11/06 CS/ECE 438 - UIUC, Fall 2006 14
Switch Design Goals
Throughput Main problem is contention Need a good traffic model
Arrival time Destination port Packet length
Telephony modeling is well understood Until faxes and modems
Modeling of data traffic is new Not well understood Will good models help?
10/11/06 CS/ECE 438 - UIUC, Fall 2006 15
Switch Design Goals
Contention Avoid contention through intelligent buffering Use output buffering when possible Apply back pressure through switch fabric Improve input buffering through non-FIFO
buffers Reduces head-of-line blocking
Drop packets if input buffers overflow
10/11/06 CS/ECE 438 - UIUC, Fall 2006 16
Switch Design Goals
Scalability O(N) ports Port design complexity O(N) gives O(N2)
for entire switch Port design complexity of O(1) gives
O(N) for entire switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 17
Switch Design
Crossbar switches Banyan Networks Batcher Networks Sunshine Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 18
Crossbar Switch
Every input port is connected to everyoutput port NxN
Output ports Complexity scales as O(N2)
10/11/06 CS/ECE 438 - UIUC, Fall 2006 19
Crossbar Switch
Output Port
Output Port
Output Port
Output Port
10/11/06 CS/ECE 438 - UIUC, Fall 2006 20
Knockout Switch
Assumption: It is unlikely that N inputs will have packets destined for
the same output port Pick L from N packets at a port
Output port maintains L cyclic buffers Shifter places up to L packets in one cycle Each buffer gets only one packet Output port uses round-robin between buffers Arrival order is maintained
Problem Hot spots
Output ports scale as O(N)
10/11/06 CS/ECE 438 - UIUC, Fall 2006 21
Knockout Switch
Output port design Packet filters
Recognize packets destined for a specific port
Concentrator Selects up to L packets from those destined for this
port Discards excess packets
Queue Length L
10/11/06 CS/ECE 438 - UIUC, Fall 2006 22
Knockout Switch
R R
R
D
R
RD
R2x2
randomselector
Delayunit
Choose L of N
Ex: 2 of 4
10/11/06 CS/ECE 438 - UIUC, Fall 2006 22
Knockout Switch
R R
R
D
R
RD
R2x2
randomselector
Delayunit
Choose L of N
Ex: 2 of 4
What happensif more than L
arrive?
10/11/06 CS/ECE 438 - UIUC, Fall 2006 22
Knockout Switch
R R
R
D
R
RD
R2x2
randomselector
Delayunit
Choose L of N
Ex: 2 of 4
What happensif more than L
arrive?
1 2 3 4
10/11/06 CS/ECE 438 - UIUC, Fall 2006 22
Knockout Switch
R R
R
D
R
RD
R2x2
randomselector
Delayunit
Choose L of N
Ex: 2 of 4
What happensif more than L
arrive?
12
34
10/11/06 CS/ECE 438 - UIUC, Fall 2006 22
Knockout Switch
R R
R
D
R
RD
R2x2
randomselector
Delayunit
Choose L of N
Ex: 2 of 4
What happensif more than L
arrive?
Discard1
234
10/11/06 CS/ECE 438 - UIUC, Fall 2006 22
Knockout Switch
R R
R
D
R
RD
R2x2
randomselector
Delayunit
Choose L of N
Ex: 2 of 4
What happensif more than L
arrive?Choice
1Choice
2
Discard1
34
10/11/06 CS/ECE 438 - UIUC, Fall 2006 23
Self-Routing Fabrics
Idea Use source routing on “network” in switch Input port attaches output port number as
header Fabric routes packet based on output port
Types Banyan Network Batcher-Banyan Network Sunshine Switch
10/11/06 CS/ECE 438 - UIUC, Fall 2006 24
Banyan Network
A network of 2x2 switches Each element routes to output 0 or 1
based on packet header A switch at stage i looks at bit i in the
header
01
10/11/06 CS/ECE 438 - UIUC, Fall 2006 24
Banyan Network
A network of 2x2 switches Each element routes to output 0 or 1
based on packet header A switch at stage i looks at bit i in the
header
0100100
10/11/06 CS/ECE 438 - UIUC, Fall 2006 24
Banyan Network
A network of 2x2 switches Each element routes to output 0 or 1
based on packet header A switch at stage i looks at bit i in the
header
0100100
10/11/06 CS/ECE 438 - UIUC, Fall 2006 25
Banyan Network
10/11/06 CS/ECE 438 - UIUC, Fall 2006 25
Banyan Network
001
011
110
111
10/11/06 CS/ECE 438 - UIUC, Fall 2006 25
Banyan Network
001
011
110
111
10/11/06 CS/ECE 438 - UIUC, Fall 2006 25
Banyan Network
001
011
110
111
001
011
110
111
10/11/06 CS/ECE 438 - UIUC, Fall 2006 25
Banyan Network
001
011
110
111
10/11/06 CS/ECE 438 - UIUC, Fall 2006 25
Banyan Network
001
011
110
111
001
011
110
111
10/11/06 CS/ECE 438 - UIUC, Fall 2006 25
Banyan Network
001
011
110
111
10/11/06 CS/ECE 438 - UIUC, Fall 2006 25
Banyan Network
001
011
110
111
001
011
110111
10/11/06 CS/ECE 438 - UIUC, Fall 2006 25
Banyan Network
001
011
110
111
001
011
110111
10/11/06 CS/ECE 438 - UIUC, Fall 2006 26
Banyan Network
Perfect Shuffle N inputs requires log2N stages of N/2 switching
elements Complexity on order of N log2N
Collisions If two packets arrive at the same switch destined
for the same output port, a collision will occur If all packets are sorted in ascending order upon
arrival to a banyan network, no collisions willoccur!
10/11/06 CS/ECE 438 - UIUC, Fall 2006 27
Batcher Network
Performs merge sort A network of 2x2 switches
Each element routes to output 0 or 1 based onpacket header
A switch at stage i looks at the whole header Two types of switches
Up switch Sends higher number to top output (0)
Down switch Sends higher number to bottom output (1)
10/11/06 CS/ECE 438 - UIUC, Fall 2006 28
Batcher Network
D
D
D
U
D
D
10/11/06 CS/ECE 438 - UIUC, Fall 2006 28
Batcher Network
D
D
D
U
D
D
7
3
6
1
10/11/06 CS/ECE 438 - UIUC, Fall 2006 28
Batcher Network
D
D
D
U
D
D
Sort
7
3
6
1
10/11/06 CS/ECE 438 - UIUC, Fall 2006 28
Batcher Network
D
D
D
U
D
D
Sort
7
3
6
1
7
3
6
1
10/11/06 CS/ECE 438 - UIUC, Fall 2006 28
Batcher Network
D
D
D
U
D
D
Sort
7
3
6
1
7
3
6
1Merge
7
3
6
1
10/11/06 CS/ECE 438 - UIUC, Fall 2006 28
Batcher Network
D
D
D
U
D
D
Sort
7
3
6
1
7
3
6
1
3
7
6
1
Merge
7
3
6
1
10/11/06 CS/ECE 438 - UIUC, Fall 2006 28
Batcher Network
D
D
D
U
D
D
Sort Merge
6
3
1
7
7
3
6
1
7
3
6
1
3
7
6
1
Merge
7
3
6
1
10/11/06 CS/ECE 438 - UIUC, Fall 2006 28
Batcher Network
D
D
D
U
D
D
Sort Merge
6
3
1
7
7
3
6
1
7
3
6
1
3
7
6
1
1
7
3
6
Merge
7
3
6
1
10/11/06 CS/ECE 438 - UIUC, Fall 2006 29
Batcher Network
10/11/06 CS/ECE 438 - UIUC, Fall 2006 29
Batcher Network
D
D
D
U
D
D
10/11/06 CS/ECE 438 - UIUC, Fall 2006 29
Batcher Network
U
U
U
D
U
U
D
D
D
U
D
D
10/11/06 CS/ECE 438 - UIUC, Fall 2006 29
Batcher Network
U
U
U
D
U
U
D
D
D
U
D
D
D
D
D
D
D
D
D
D
D
D
D
D
10/11/06 CS/ECE 438 - UIUC, Fall 2006 30
Batcher Network
How it really works Merger is presented with a pair of sorted lists,
one in ascending order, one in descending order First stage of merger sends packets to the
correct half of the network Second stage sends them to the correct quarter
Size N/2 switches per stage log2N x (1 + log2N)/2 stages Complexity = N log2
2N
10/11/06 CS/ECE 438 - UIUC, Fall 2006 31
Batcher-Banyan Network
Idea Attach a batcher network back-to-back with a banyan
network Arbitrary unique permutations can be routed without
contention Two packets destined for same output port still collide!
Sunshine Switch Like a knockout switch
Can handle up to L packets per output port Recirculates overflow packets
If more than L packets arrive for any output port in one cycle
10/11/06 CS/ECE 438 - UIUC, Fall 2006 32
Sunshine Switch
Elements Multiple Banyan networks
Enables multiple packets per output port Delay Box
Excess (K) packets are recirculated and resubmitted to the switch Batcher network
N new packets K delayed packets
Trap Identifies packets destined for banyan Identifies excess packets
Selector Routes multiple packets for same output on separate banyans
10/11/06 CS/ECE 438 - UIUC, Fall 2006 33
Sunshine Switch
Batcher Trap Selector
L Banyans
Delay
Inputs
n n+k n+k
nnn
nn
n
kk
10/11/06 CS/ECE 438 - UIUC, Fall 2006 34
Invariants
Batcher All packets are sorted by destination
Trap Compares line i with line i+L, marks i+L for
recirculation if address equal Selector
Another batcher network, sorting based on therecirculation bit
Banyan Alternate every L lines to each Banyan
10/11/06 CS/ECE 438 - UIUC, Fall 2006 35
Sunshine Switch
Can packets circulate for ever? Priority bit is used to favor older packets Priority bit also ensure packet order is
preserved through the switch