Download - High-Speed Backplane Interconnect
-
7/28/2019 High-Speed Backplane Interconnect
1/49
High-speed backplane interconnect
Vladimir Stojanovic
(with slides from J. Zerbe, P. Desai, R. Kollipara)
-
7/28/2019 High-Speed Backplane Interconnect
2/49
Outline
Inside the router
Backplane channel problem
What can backplane designer do about it
What can IC designer do about it
Scaling the system to 10-100Tb/s
-
7/28/2019 High-Speed Backplane Interconnect
3/49
Inside the Router
MAC
TM/
Fabric
IF
NPU
SerDes
Optics
SerDes
SerDes
Crossbar
Line Cards:8 to 16 per System
Switch Cards:2 to 4 per System
Passive
Backplane
MEM
MEM
MEM
MEM
Past OC-12
622 MHz LVDSparallel
GigE
1.25 Gbps serial
Present OC48
2.5 Gbps serial
10GigE
XAUI (3.125 Gbps) serial
-
7/28/2019 High-Speed Backplane Interconnect
4/49
OpticsMAC/
Framer
NPU/
TM
Switch
Fabric IF
XAUI
4, 3.125 Gbps
Serial Links
SPI4.2 CSIX
ProprietaryBackplane
8 to 16
of 1-3.2Gbps
Serial Links
Line Card:
Switch Card:32 to 64
Backplane
Serial Links
(1-3.2 Gbps)
Switch
Crossbar
IC
Serial Links in Networking Systems
-
7/28/2019 High-Speed Backplane Interconnect
5/49
Backplane interconnect path
Backplane via
Backplane connector
Line card
trace
PackageChip
Line card
viaBackplane trace
Packageto board
transition
There are many components on the signal path,
potential source of problems
-
7/28/2019 High-Speed Backplane Interconnect
6/49
RaSer X Link Features
PLL
Serializer Tx Link
20 bit
ParallelInterface
1-10 Gbps
RefClk
Tx
Eq
Deserializer and
CDR
Rx Link
1-10 Gbps
Rx
Eq
20 bit
Parallel
Interface
TX
RX
PLL
Process 0.13m CMOSPower 40mW / Gb
Area 1mm2
2-PAM Range 2 6.4 Gb/s
4-PAM Range 5 10 Gb/s
-
7/28/2019 High-Speed Backplane Interconnect
7/49
Impedance-controlled (CML) I/Os
Integrated terminations
Adjustable output-voltage/common mode
I/O Driver Scheme (Example)
50 W 50 WVtt
Zo = 50W
Zo = 50WRx
Tx
Vtt
-
7/28/2019 High-Speed Backplane Interconnect
8/49
System Issues
Goal Increase Router Throughput
Limitations
Backplane channel
Power
Mechanical/Physical density constraints
Backplane and linecard routing density
Connector pin density
Package I/O density
-
7/28/2019 High-Speed Backplane Interconnect
9/49
Outline
Inside the router
Backplane channel problem
What can backplane designer do about it
What can IC designer do about it
Scaling the system to 10-100Tb/s
-
7/28/2019 High-Speed Backplane Interconnect
10/49
Backplane Component Effects
PCB only
PCB + Connectors
PCB, Connectors,
Via stubs & Devices
-
7/28/2019 High-Speed Backplane Interconnect
11/49
Deterministic Noise
0 2 4 6 8 10
-60
-50
-40
-30
-20
-10
0
frequency [GHz]
Attenuation[d
B]
FEXT
NEXT
THROUGH
0 1 2 3
0
0.2
0.4
0.6
0.8
1
ns
pu
se
response
Tsymbol=160ps
Inter-symbol interference Dispersion (skin-effect, dielectric loss) - short
latency
Reflections (impedance mismatches connectors,
via stubs, device parasitics, package) long latency
-
7/28/2019 High-Speed Backplane Interconnect
12/49
XTALK and reflections
Far-end XTALK (FEXT)
Desired signal
Near-end XTALK (NEXT)
Reflections
Primary reflection sources are at the
connector/backplane transition
Grouped in time as a function of backplane length
-
7/28/2019 High-Speed Backplane Interconnect
13/49
-
7/28/2019 High-Speed Backplane Interconnect
14/49
Test Backplane Example
Dielectric material FR-4 Nelco Roger
6000 4350
Dielectric constant 4.2 4 3.6
Loss tangent (1 MHz) 0.016 0.005 0.0035
Loss tangent (1 GHz) 0.017 0.007 0.0035
Thickness 0.295" 0.299" 0.297"
FR4 Cross Section
Trace lengths: 1.5,9, 14, 20 and 32
Effective number of
signal layers: 13
Effective number of
total layers: 28
-
7/28/2019 High-Speed Backplane Interconnect
15/49
Backdrilling - A Solution to the Stub
Effect
-
7/28/2019 High-Speed Backplane Interconnect
16/49
Stub Effect Eye Pattern Analysis(2.5 Gbits/sec FR-4)
MAX STUB MIN STUB
-
7/28/2019 High-Speed Backplane Interconnect
17/49
MAX STUB MIN STUB
Stub Effect Eye Pattern Analysis(5.0 Gbits/sec FR-4)
-
7/28/2019 High-Speed Backplane Interconnect
18/49
MAX STUB MIN STUB
Stub Effect Eye Pattern Analysis(12.0 Gbits/sec FR-4)
-
7/28/2019 High-Speed Backplane Interconnect
19/49
Connector design
GBX
Teradyne
-
7/28/2019 High-Speed Backplane Interconnect
20/49
Connector Density
Teradynes GbX Connector
Differentia
l
Pairs/inch
Card
Pitch
Bandwidth/linear inch
(at 6.25 Gbps)
5 pair 69 1.25" min.(30 mm)
431 Gb
4 pair 551.00" min.
(24.7mm)343 Gb
3 pair 41.80" min.
(20 mm)256 Gb
2 pair 27.5.575" min.
(14 mm)171 Gb
-
7/28/2019 High-Speed Backplane Interconnect
21/49
Reducing Crosstalk within the
Connector
Cross talk is
reduced in the
mating interfaceby surrounding
each pair with a
ground shield
D/C Shield
B/P ShieldMated pair
-
7/28/2019 High-Speed Backplane Interconnect
22/49
Backplane Connector Considerations
Many connector types: Teradyne: VHDM, HSD, GbX,
Tyco: HS3, HMZd,
FCI: Metral 2000, 3000, 4000,
3M/Harting: HSHM,
ERNI: ERmetZd, ErmetXT,
Issues
Loss, impedance profile, crosstalk, skew Foot print: routability, pin density, via impedance
Single-ended and differential
Press-fit and SMT
-
7/28/2019 High-Speed Backplane Interconnect
23/49
10Gbps Test Package Design Example
Ceramic BGA
Wire-bonded
4-Layer
1 mm pitch
Source: Designed for Rambus by Kyocera
-
7/28/2019 High-Speed Backplane Interconnect
24/49
-
7/28/2019 High-Speed Backplane Interconnect
25/49
Outline
Inside the router
Backplane channel problem
What can backplane designer do about it
What can IC designer do about it
Scaling the system to 10-100Tb/s
-
7/28/2019 High-Speed Backplane Interconnect
26/49
Loss : Equalize to Flatten Response
Channel is band-limited
Equalization : boost high-frequencies relative to lowerfrequencies
+
=
-
7/28/2019 High-Speed Backplane Interconnect
27/49
Receiver Linear Equalizer
Amplifies high-frequencies
attenuated by the channel
Digital or Analog FIR filter
Issues
Also amplifies noise!
Precision
Tuning delays (if analog)
Setting coefficients
Adaptive algorithms such asLMS
WL-1
DDD
WLW
1
+
H(s)
freq
-
7/28/2019 High-Speed Backplane Interconnect
28/49
Transmitter Linear Equalizer
Attenuates low-frequencies Need to be careful about output
amplitude : limited output power
If you could make bigger swingsyou would
EQ really attenuates low-
frequencies to match highfrequencies
Also FIR filter : D/A converter
Can get better precision than Rx
Issues How to set EQ weights?
Doesnt help loss at f
H(s)
freq
-
7/28/2019 High-Speed Backplane Interconnect
29/49
Transmit Linear Equalizer :
Single Bit Operation
0.0 0.3 0.6 0.9 1.2-0.3
-0.1
0.1
0.3
0.5
0.7
UnequalizedEqualization PulseEnd of Line
time (ns)
Volta
ge
-
7/28/2019 High-Speed Backplane Interconnect
30/49
Decision Feedback Equalization (DFE)
Dont invert channeljust remove ISI
Know ISI because already
received symbols
Doesnt amplify noise
Requires a feed-forward
equalizer for precursor
ISI
Reshapes pulse to
eliminate precursor
-
FIR filter
Decision (slicer)
FIR filter
Feed-forward EQ
Feed-back EQ
-
7/28/2019 High-Speed Backplane Interconnect
31/49
DFE Example
-
7/28/2019 High-Speed Backplane Interconnect
32/49
Transmit and Receive Equalization
Transmit and receive equalizers are
combined to make a range restricted DFE Tx equalizer functions as the feed-forward filter
Rx equalizer restricted in performance of loop
TAP SEL
LOGIC
TX
DATA
3
RX
DATA
-
7/28/2019 High-Speed Backplane Interconnect
33/49
Tx & Rx Equalization Ranges
TX Driver/Equalizer : 5 taps
1(pre)+1(main)+3(post)
RX Equalizer
5-17 taps after main
Pick any 5 taps
-
7/28/2019 High-Speed Backplane Interconnect
34/49
Pulse Amplitude Modulation
Binary (NRZ) is 2-PAM 2-PAM uses 2-levels to send
one bit per symbol
Signaling rate = 2 x Nyquist
4-PAM uses 4-levels to send2 bits per symbol
Each level has 2 bit value
Signaling rate = 4 x Nyquist
00
01
11
10
1
0
1
0
-
7/28/2019 High-Speed Backplane Interconnect
35/49
When Does 4-PAM Make Sense?
First order : slope of S21
3 eyes : 1 eye = 10db
loss > 10db/octave : 4-PAM
should be considered
0.0 1.0 2.0 3.0 4.0
Nyquist Frequency (GHz)
|H(f)|
-20db
-40db
-60db
-
7/28/2019 High-Speed Backplane Interconnect
36/49
Example : 5Gbps Over 26 FR4
With No Equalization
-
7/28/2019 High-Speed Backplane Interconnect
37/49
Example : 5Gbps Over26 FR4
Correct Tx Equalization
-
7/28/2019 High-Speed Backplane Interconnect
38/49
Example : 5Gbps Over26 FR4
Under Equalized
-
7/28/2019 High-Speed Backplane Interconnect
39/49
Example : 5Gbps Over26 FR4
Over Equalized
-
7/28/2019 High-Speed Backplane Interconnect
40/49
26 FR4 Bot 3.125Gbps, 2P noEQ
-
7/28/2019 High-Speed Backplane Interconnect
41/49
26 FR4 Bot 3.125Gbps, 2P w/EQ
-
7/28/2019 High-Speed Backplane Interconnect
42/49
26 FR4 Bot 6.4Gbps, 2P w/3G EQ
-
7/28/2019 High-Speed Backplane Interconnect
43/49
26 FR4 Bot 6.4Gbps, 2P w/EQ
-
7/28/2019 High-Speed Backplane Interconnect
44/49
26 FR4 Top 6.4Gbps, 2P w/EQ
-
7/28/2019 High-Speed Backplane Interconnect
45/49
26 FR4 Top 6.4Gbps, 4P w/EQ
-
7/28/2019 High-Speed Backplane Interconnect
46/49
26 Nelco6k-cb Top 10Gbps, 4P
-
7/28/2019 High-Speed Backplane Interconnect
47/49
26 Nelco6k-cb Top 6.4Gbps, 2P
S li h h h
-
7/28/2019 High-Speed Backplane Interconnect
48/49
Scaling the router throughput
System (Tot. throughput ~2.5Tb/s) 8-16 Line Cards
-
7/28/2019 High-Speed Backplane Interconnect
49/49
Scaling the router throughput
System (Tot. throughput ~100Tb/s) 100 Line Cards
1Tbs / LC 10Gbs links
4mW/Gbs Link power in 0.065um
Speedup ~1x
#links at switch card ~ 10k
Limitations #diff pairs at switch card 20k
Switch card power from links in 0.13um~10k*10Gbs*4mW/Gbs ~ 400W
Connector density 50diff pairs/inch: (tot length=20k/50= 400)
BP/LC routing pitch 0.050 Num. Layers (BP=13, LC=4) 5k diff pairs/layer = 250LC
routing width
Package ball pitch (1mm/200um)