1

Level-1 Data Driver Card (L1DDC)

HEP 2014 8-10 May 2014Naxos

08/05/2014 HEP 2014, NAXOS

Panagiotis GkountoumisNational Technical University of Athens

HEP 2014, NAXOS 2

L1DDC• Level-1 Data Driver Card• Responsible for

– collecting L1 data from FE boards and sending them to the counting room– Receiving Slow Control (SC) information– receiving configuration data from the counting room and sending them to the FE

boards– sending timing control (TTC) signals to the FE boards– sending configuration data to the ADDC (Address Real Time) boards

08/05/2014

On-DetectorRadiation Hard Electronics

Off-DetectorCommercial Off-The-Shelf (COTS)

GBTXGBTIA

GBLD

PD

LD

Custom ASICs

Timing & Trigger

DAQ

Slow Control

Timing & Trigger

DAQ

Slow Control

FPGA

GBT GBT

Versatile Link

HEP 2014, NAXOS 3

FE

Connectivity for MMs

08/05/2014

L1DDC

FE FE FE FE

FE FE FE

Optical link

ADDC

4

• Board dimensions 100mmx50mmx18mm (LengthxWidthxHeight)• On detector – hostile radiation environment• Contains 3 radiation hard ASICs

– GBTx (GigaBit transceiver)– GBTIA (Trans-Impedance Amplifier)– GBLD (Laser Driver)

• Input voltage 24V• 3 Voltage levels

– 3.3V– 2.5V– 1.5V

• Linear LTM4619 DC-DC converter– 4.5V to 26.5V input – Dual output 0.8V to 5V @ 4Amps each

• Linear LT3080 LDOs– Wide input voltage 1.2V to 36V – output @1.1Amps (Dropout Voltage 350mV)

• Power consumption ~3.3Watts• Connectivity

– VTRX (CERN’s custom made optical transceiver) @ 4.8Gb/s– 8 Mini SAS 26 pins cables for the FE connection (@ 80Mb/s, 160Mb/s, 320Mb/s)– 1 Mini SAS 26 pins cable for the ADDC connection (@ 80Mb/s)– Low Voltage Power connector

08/05/2014 HEP 2014, NAXOS

Parts and connectivity

5

GBTx• Gigabit Transceiver ASIC – highly flexible link interface chip• Can be configured as:

– Bidirectional transceiver– Unidirectional transmitter– Unidirectional receiver

• Very high level of error correction from SEU’s and transmission errors

• Radiation Hard ASIC - 130nm CMOS technology• Fits in 17mmx17mm BGA package (400pins), 1mm pitch• 7 power domains

– Serializer (1.5V)– Deserializer (1.5V)– Clock manager (1.5V)– Phase shifter (1.5V)– Core digital (1.5V)– I/O (1.5V)– Fuses (3.3V)

• Max power consumption is 2 Watts (1.33Amps when all on)• Electrical caracteristics

– Drivers: SLVS signaling (VCM=200mV, +/-200mV)– Receivers: SLVS/LVDS signaling (VCM=1.25mV, +/-400mV)08/05/2014 HEP 2014, NAXOS

HEP 2014, NAXOS 6

FEModule

GBTx • Duplex local electrical serial links (E-

Links)• 1 elink consists of 3 signal lines

(differential pairs).– Data down– Data up– Clock

• GBTx supports up 40 E-Links• 8 E-Links/group • E-Link data rates are programmable

per group– 40E-Links@80Mb/s– 20E-Links@160Mb/s– 10E-Links@320Mb/s

• E-Link clock signals independently programmable @ (40MHz, 80MHz, 160MHz and 320MHz)

• 1 dedicated e-link @ 80 Mb/s for SCA• E-Links signal drivers and receivers not

actively used can be powered down08/05/2014

FEModule

Phase – Aligners + Ser/Des for E – Ports

E – Port

GBT-SCA

E – Port

E – PortE – Port

data

One 80 Mb/s port

GBTX

e-Link

clock

data-up

data-down

clocks

E – PortE – Port

FEModule

E – PortE – Port

HEP 2014, NAXOS 7

E-Links

08/05/2014

VMM

Com/nion

SCA

VMM

VMM

VMM

FE

RXCLK

RXDATA

TXDATA

RXCLK

RXDATA

TXDATA

Companion E-Link

SCA E-Link

VMM

VMM

VMM

VMM

• FE contains 2 ASICS• Companion ASIC - Collects

data from VMMs and transmits them to L1DDC

• SCA (Slow Control Adapter) - Configures the VMMs and monitors the voltage levels

• L1DDC is connected to each ASIC with 1 E-Link

• 2E-Links/FE - 6 Lines (6 differential pairs)

HEP 2014, NAXOS 8

E-Links• Group1-4 can run @

80Mb/s, 160Mb/s and 320Mb/s

• Group5 is used for the SCA connections and always runs @ 80Mb/s

• The dedicated E-Link @ 80Mb/s can be used for the ADDC configuration data

08/05/2014

L1DDC

GRO

UP1

C.ASIC3

C.ASIC4

SCA

SCA

…1

8

GRO

UP2

GRO

UP3

GRO

UP4

GRO

UP5

C.ASIC1

C.ASIC2

ADDC

C.ASIC7

C.ASIC8

C.ASIC5

C.ASIC6

Dedicated

HEP 2014, NAXOS 9

Mini SAS• Serial Attached SCSI (SAS)• Commercial cables

– Round– Flat, twin axial

• 26 positions• Shielded• 8 differential pair• Mini SAS 26 pins connector with cage• Fixed cable length for all the FEs

( AWG 28 or 30)

08/05/2014

10

Optical transceiver

08/05/2014 HEP 2014, NAXOS

• GBTIA- 2.5V @ 0.05Amps- Power consumption

0.12Watts- Bit rate 5Gb/s- Die size: 0.75 mm × 1.25

mm- 0.13-μm IBM CMOS8RF-

LM technology, a standard eight-metal-layer

• GBLD- Bit rate 5Gb/s- 2.5V @ 0.2Amps- Power consumption

0.5Watts

HEP 2014, NAXOS 11

GBT System

08/05/2014

Clock & Data Recovery. Recovers and generates a high speed clock to sample the incoming data stream

Programmable in frequency and phase

Decoder / Descrambler

Scrambler / Encoder

Serializer

12

Packet format (optical link)• Fixed packet length: 120bits

– Packet transmission rate: 1/25ns – Data transmission rate: 4.8 Gbps

• Header field 4 bits• Forward Error Correction (FEC)

field of 32bits is added resulting in a code efficiency of 70%

• 84bits for data transmission corresponding to 3.36Gb/s user bandwidth

• SC field 4 bits – Internal Control (IC) field 2 bits– External Control (EC) field 2 bits

• Finally only 80 bits are used for data 3.2Gb/s

08/05/2014 HEP 2014, NAXOS

HEP 2014, NAXOS 13

Packet format (optical link)8b/10b frame format• 12 x (8b/10b) words• The first one is the comma

character• 88/120=73% coding

efficiency • 3.52Gb/s user bandwidth• No error correction and

limited error detection

Wide frame format• 4.48Gb/s user bandwidth• No SEU protection

08/05/2014

8b/10b frame format

Wide frame format

HEP 2014, NAXOS 14

Encoding/decoding block diagram

08/05/2014

• Scrambler performs DC balance• Double interleaved Reed-Solomon

two-errors correcting code• A sequence of up to 16 consecutive

corrupted bits can be corrected• Transmission errors or SEUs

occurring in the Serializer (SER), GBLD, PIN-diode, GBTIA, Clock and Data Recovery (CDR) and De-serializer (DES) will be corrected in the decoding operation

• Encoding & decoding can be done in a single machine clock cycle – minimal impact on the transmission latency

HEP 2014, NAXOS 15

Position of L1DDC in MMs

FEs

L1DDC

08/05/2014

ADDC

LM2

LM1

• Requirement: not to lose more than one plane in case of L1DDC failure

• Every L1DDC is connected with 8 FEs of the same side and plane

• Fixed length of cables to all FEs

• Placement of L1DDC and ADDC only in the first plane

HEP 2014, NAXOS 16

Position of L1DDC

08/05/2014

FEs

HEP 2014, NAXOS 17

Ground pins

L1DDC prototype

08/05/2014

GBTxLTM4619

Ground pins

Mini SAS 26p

HEP 2014, NAXOS 18

L1DDC prototype

08/05/2014

Power

HEP 2014, NAXOS 19

Calculations• MMs

– Per plane• 2 L1DDC/16 FEs

– Per Wedge• 8 L1DDC/64 FEs

– Per Sector• 16 L1DDC/128 FEs

– Per 1 Wheel• 256 L1DDC/2048 FEs

– Per 2 Wheels• 512 L1DDC/4096 FEs

08/05/2014

Sum 1024 L1DDC & 3.38KWatts

• TGCs– Per plane

• 2 L1DDC/6 FEs– Per Wedge

• 8 L1DDC/24 FEs– Per Sector

• 16 L1DDC/48 FEs– Per 1 Wheel

• 256 L1DDC/ 768 FEs – Per 2 Wheels

• 512 L1DDC/1536 FEs

20

Backup Slides

08/05/2014 HEP 2014, NAXOS

21

Available space

08/05/2014 HEP 2014, NAXOS