100 gb/s: the high speed connectivity race is ons3.amazonaws.com/zanran_storage/€¦ · 100 gb/s:...
TRANSCRIPT
100 Gb/s: The High Speed Connectivity Race is On
Cathy Liu
SerDes Architect, LSI Corporation
Harold Gomard
SerDes Product Manager, LSI Corporation
October 6, 2010
2
Agenda
• 100 Gb/s Ethernet evolution
• SoC challenges and solutions to drive 4x25 Gb/s Ethernet
• High Speed Serial link system design
• Summary
3
Contemporary 100 Gb/s Ethernet with 10 Gb/s SerDes
100 Gb/s Media Access Controller (MAC)
10 x 10 Gb/s SerDes
100 Gb/s Physical Coding Sublayer (PCS)
Optional Forward Error Correction (FEC)
20 “virtual” lanes
10 “physical” lanes
2:1 bit multiplexer
10x10 GbE
Se
rDe
s
Se
rDe
s
Today’s 100 Gb/s Ethernet built from 10 lanes of 10 Gb/s SerDes
Challenges for 10 x 10 Gb/s implementation implies high cost
Uses many package and/or connector pins
Complicates board routing and more high speed signal layers
Limits ASIC and faceplate port densities
Large number of pairs cabling to terminate implies higher cost
Large outer diameter implies heavy, inflexible cable
20 differential pairs per interface
4
Industry Trend for High Speed Serial Link System
• Merging 25-28 Gb/s serial electrical links
– IEEE 100GbE, OIF CEI 28G-SR/25G-LR, 32GFC, Infiniband EDR
• Lower-cost, more manageable cable assemblies
• Common form-factors and connector technologies
4 x 25 Gb/s is the natural evolutionary step for 100 Gb/s Ethernet
10x10 GbE
Se
rDes
Se
rDes
4x25 GbE
Se
rDes
Se
rDes
5
Path from 10x10 Gb/s to 4x25 Gb/s: Challenges
Challenges
Lossy links and noisy environment
Need High speed circuit design at 25-28 Gb/s
Technology and implementation limitations
Large channel counts (~200) in SoCs 4x25 GbE
SerD
es
SerD
es
BladeSystem
10x10 GbE
SerD
es
SerD
es
- 32dB IL at 5GHz
- 30dB Xtalk
Legacy Channels already present significant challenges @ 10 Gb/s !
6
Path from 10x10 Gb/s to 4x25 Gb/s: Solutions
4x25 GbE
SerD
es
SerD
es
10x10 GbE
SerD
es
SerD
es
Addressed Challenge Solution Space
High-Loss Channel Alternative Signaling Format Beyond NRZ
High-Loss and Noisy Environment ADC + DSP-based SerDes Architecture
Noisy Environment FEC and Noise Cancellation
Technology and Circuit Limitations Innovative Circuit Implementation
Large Channel Counts Advanced SoC Integration
All Joint System-Design Optimization
7
SerDes Architecture Evolution
• Industry trends are to consider alternative signaling formats beyond NRZ like
Multi-level or Multi-band for 25G+ long reach application
• Low power and high speed interleaved ADC technology enables DSP
architecture to extend equalization capability
• Forward Error Correction (FEC) and crosstalk cancellation to further improve
SNR
ADC 1
ADC 2
ADC n
Interleaved
ADC
FF
E
Analog Filter
AGC
DF
E
FE
C
Sys
tem
In
terf
ac
e
Digital Domain
Timing
Recovery
Analog Front End
Reference: Liu et al, “Comparison of signaling and equalization schemes in high speed SerDes (10-25Gbps)” DesignCon 2007
8
Implementation Challenges for Building a 25G SerDes
Circuit innovations required to tackle 25G SerDes challenges
Challenge Requirements Constraint
Signal Integrity RL of 8dB @ 12.5GHz Low power
Tx Driver Bandwidth 18.75GHz Low power
Signal Dynamic Range
Scaling30dB Linearity
28nm CMOS device
headroom @ 0.85V
Ultra-low PLL Random Jitter 250fs (rms) Low power
Reference: Wu et al, “A 2x25Gb/s Deserializer with 2:5 DMUX for 100Gb/s Ethernet applications” ISSCC 2010
9
Circuit Innovation to Tackle Implementation Challenges (I)
• Source-Series Terminated (SST) driver with a bridge inductor
– CMOS differential circuit provides Ultra-high bandwidth for 25 Gbps signal processing
– Bridge inductor mitigates return loss degradation from I/O capacitance
SST driver simulations on LSI 16G-LR SerDes meet 8dB RL spec @ 25 Gbps
TX Return Loss Simulation of 16G-LR driverSST Driver with Inductor
Blue: without inductor
Black: with inductor and power mesh
Pink: with inductor
With Inductor
Without Inductor 8dB RL spec @ 25 Gbps
10
Circuit Innovation to Tackle Implementation Challenges (II)
• ADC-based SerDes for 25Gbps Serial link with alternative signaling
– 12.5Gs/s 6-bit low-power time-interleaved ADC enables DSP for 25G SerDes
– DSP provides powerful equalization to tackle ISI, crosstalk and reflection
ISI Correction
CDR
Gain Adaptation
Skew Adjustment
ADC Calibration
Data
Ali
gn
me
nt
Dec
od
ing
Bu
bb
le C
orr
ec
tio
n
11
High Channel Count SerDes Integration in ASIC
Challenges in high channel count
SerDes integration• High-speed Clocking within SoC
• Power Integrity of SoC
• High-speed Signal Integrity
Asymmetrical TX and RX
12
Challenges in high channel count SerDes integration- PLL Coupling in High Channel Count SerDes Integration
• LC-PLL Coupling is one of top challenges for high-speed clocking
• Advanced techniques are required for multiple LC-PLL placement– Minimum spacing between 2 LC-PLLs
– Design rule for substrate isolation and power mesh
Advanced LC-PLL placement achieves no injection lock coupling from other PLL
PLLB
Test Condition
PLLA @ 14.025G
p2p Jitter
8.5G 12.3ps
10.3125G 13.2ps
12.5G 12.9ps
14.025G 11.2ps
Powered off 12.6ps
PLLA Clock Out
PLLA = 14.025 GHz, PLLB = 8.5 GHz
-80
-70
-60
-50
-40
-30
-20
-10
0
1.600
E+09
2.000
E+09
2.400
E+09
2.800
E+09
3.200
E+09
3.600
E+09
4.000
E+09
4.400
E+09
4.800
E+09
5.200
E+09
5.600
E+09
PLLA Clock Out
Measured Jitter of PLLA
Variation limited to +/- 1psSpectrum of LSI 40nm 14G-LR SerDesPLLA Clock Out
PLLA=14.025 GHz, PLLB = 8.5 GHz
13
Challenges in high channel count SerDes integration
- Power Integrity in High Channel Count SerDes Integration
Power supply
Recommended
PKG DCAP PKG IR Drop (%)
Self-induced on-die
AC noise peak voltage (%)
CAP Value Spec Simulated Spec Simulated
VDDA 22nF (x2) 1.0 0.76 1.25 0.81
VDDREF 10nF (x2) 1.0 0.23 1.25 1.02
VDDA1P8 N/A 1.0 <0.5 1.25 0.5
Simulation Setup Simulated AC Noise
PI Analysis of SoC with LSI 14G-LR SerDes
PI analysis ensures IR drop and AC noise meet SoC design requirements
14
Challenges in high channel count SerDes integration - Package SI for High Channel Count SerDes Integration
• Package routing length can vary 12mm to 25mm for high channel count SoC
TX insertion loss TX return loss RX-TX cross talk
Package SI Kit provides powerful tool for High-Speed SI analysis
Package SI Analysis of SoC with LSI 14G-LR SerDes
0 1 2 3 4 5 6 7 8 9 10-20
-15
-10
-5
0
Frequency (GHz)
Magnitude (
dB
)
CW108040 Tx Device Sdd22
Return Loss: Worst Case Die over PVT
Port 0: Tx
Port 1: Tx
Port 2: Tx
Port 9: Tx
16GFC EpsilonT Sdd22 RL Compliance Spectrum
15
• Increased loss over given medium length
• System sizes do not shrink
• Fixed budget loss driven by a transceiverElectrical
• Increased xtalk at connector and package
• New BGA routing strategy required
• Port density vs. signal integrity Mechanical
• Dynamic power consumption increase
• Single hot spot ASIC thermal management
• Cannot solve with innovative circuits aloneThermal
Challenges in high channel count SerDes integration - Multiple dimensions
16
High-Speed System Design Considerations: SerDes
• SerDes targets compensation for non-ideal system characteristics
– DJ caused by frequency dependent loss and reflections
– Limited ability to address RJ from excessive crosstalk
LSI Proprietary
SerDes Features System Benefit
TX FIR de-emphasis Reduction in dispersion/loss
RX Linear EQ Reduction in dispersion/loss
RX Floating Tap DFE Compensates post-cursor ISI and reflections
RX DFE loops Improves immunity to Crosstalk
SerDes mitigates the need for advanced PCB material and
Interconnect Solutions…until now
RX Floating Tap DFE
kaka
Tx FIR RX Linear EQ
Transmitter Channel Receiver
+
17
• 10Gbps System Example with severe Crosstalk
High Speed System Design Considerations: Crosstalk
Linear EQ Only
7.5” Megtron-6 40” Megtron-6 7.5” Megtron-6
Linear EQ + DFE
Without Crosstalk With Crosstalk
Joint System Optimization required for crosstalk mitigation
TX Xcede
Connector
Xcede
ConnectorRX
18
High Speed System Design Considerations: Reflection
• 10Gbps System Example with severe Reflections
Mezz Connector
Mezz Card
SoC
SoC
Blade Card
Significant reflections at 5 GHz Nyquist Frequency
0 2 4 6 8 10 12 14 16 18 20-40
-35
-30
-25
-20
-15
-10
-5
0
Frequency (GHz)
Magnitude (
dB
)
Worst Case 5 inch Blade Channel with FCI GigArray Mezz Connector
Sdd21
Insertion Loss: Blade Channel
Sdd11
Return Loss: Blade Channel
Sdd21
Insertion Loss: Ideal 5-inch FR4 trace
5 inch Chip-to-Chip Blade Channel
19
High Speed System Design Considerations: Reflection
• 10Gbps System Example with severe Reflections
LSI Proprietary
1-tap Rx DFEFixed and Floating Rx DFE taps
reflections
5 inch Blade Channel 5 inch Ideal FR4 Trace Channel
Joint System Optimization required for reflection mitigation
20
• High density system applications require truncated search criteria
• Bound the problem by simulating conservative worst case scenarios
• Parameterize problem and limit permutations to only viable options
WC
Scenario SI simulation stress parameter SerDes Rx Stress Goal
1 Maximize IL and XTLK magnitudes Degrade SNR
2Maximize reflections using
short T-lines and/or long stubs
Increase ISI from
reflections
Stressing System simulation model with worst case SI scenarios
High Speed System Design Considerations: WC Scenario
21
25 Gb/s System Design Challenges
• Higher data rates lead to electrically longer channels
• Consequences: diminishing SNR’s and generally higher return loss!
Primary Impairments are More Accentuated at Higher Frequencies
0 2 4 6 8 10 12 14 16 18 20-80
-70
-60
-50
-40
-30
-20
-10
0
Frequency (Hz)
Magnitude (
dB
)
Insertion Loss
Total XTLK
10 Gb/s 25 Gb/s
0 2 4 6 8 10 12 14 16 18 20-20
-18
-16
-14
-12
-10
-8
-6
-4
-2
0
Frequency (Hz)
Magnitude (
dB
)
Return Loss
Less
SNR Higher Return Loss
Reference: Kollipara et al, “Practical Design Considerations for 10 to 25 Gbps Copper Backplane Serial Links,” DesignCon 2006
22
Issue Potential Solutions
Elevated Insertion LossUltra Low Loss Tangent PCB laminates
Ultra Low Profile Copper
Aggravated dispersion Lower Dielectric Constant
Elevated IO Return Loss Innovative circuit / inductors
Elevated Connector Return LossOptimized Mating Interface Impedance
Smaller Compliant Pins
Elevated Connector CrosstalkImproved Shielding
Optimized Footprint Topology
25 Gb/s System Design Issues
Joint System Optimization Required for 4x25 Gb/s
23
25 Gb/s Applications Power vs. System Margins Tradeoff
No SkewSystem Margin Focus
….at manageable power
Low margins
LSI margins
• LSI IP and System design focus on application margins
– 3-sigma design and beyond for some key parameter (latch offset)
– Analog and CML circuits for industry leading performance and margins
– Low Power Design but not at the expense of system margins
24
Summary
• 4 x 25 Gb/s is the natural evolutionary step for 100 Gb/s Ethernet
• SerDes architecture and circuit innovation required for 4 X 25 Gb/s
• SoCs require high quality clocking, Power Integrity and SI analysis
• 100G is a multi-dimensional problem that needs to be tackled holistically
• SerDes alone not a panacea for high speed system design challenges
• Joint System Optimization required for crosstalk and reflection mitigation
25 LSI