cms: gct2: concentrator card pre-manufacture review

CMS: GCT2: Concentrator Card: Pre-Manufacture Review ([email protected])11 August 2006 1

CMS: GCT2: Concentrator CardCMS: GCT2: Concentrator CardPre-Manufacture ReviewPre-Manufacture Review

11th August 200611th August 2006

version 2 (Draft)version 2 (Draft)


What is the concentrator ?What is the concentrator ?

Last processing Last processing stage before stage before calorimetry data calorimetry data sent to Global sent to Global TriggerTrigger

– 9U VME64x card

– 2 DPMCs mounts for electron-leaf cards

– 2x3 Samtec cables to interface to jet-wheel cards

– 1 DPMC mount for Global Trigger interface


TriggerTrigger

Electron η+ datafrom Leaf DPMC

Jet η+ datafrom

Wheel Card

ElectronV4 FPGA

JetV4 FPGA

Global Trigger DPMC

7 x dual channelSerdes links

Electron η- datafrom Leaf DPMC

Jet η- datafrom

Wheel Card

Sorted Et and jet count2 x 50 Diff Pairs

Via Samtec J2 & J3

Electron data2 x 160 Single EndedVia J11, J12, J21, J22

All paths 40 MHz DDR -> 80MHz

Sorted & unfinished jets2 x 150 Diff Pairs

Via Samtec J1 & J3

80 Single Ended

1) Iso Elec2) Non-Iso Elec3) Energy Sum4) Jet Counts

5) Forward Jet6) Central Jet7) Tau Jet

2 x 180 Single EndedVia fully populated 1/2 DPMC

Leaf

Leaf

Leaf

Leaf

Leaf

Leaf


Control & ReadoutControl & Readout

Electron η+ datafrom Leaf DPMC

ElectronV4 FPGA

JetV4 FPGA

Electron η- datafrom Leaf DPMC

2 x 34 Diff Pairs Via Samtec J2

V2 driving LVDSEXT

2 x 40 Single EndedVia J23

1) Iso Elec2) Non-Iso Elec3) Energy Sum4) Jet Counts

5) Forward Jet6) Central Jet7) Tau Jet

2 x 32 Single EndedCommV2 FPGA

40 MHz DDR -> 80MHz

100

V4V2

100DCI LVDS requires62.5mW per pair

SlinkVME

TTCrxClock Control

FMMUSB

Ethernet

Jet η+ datafrom

Wheel Card

Jet η- datafrom

Wheel Card

Leaf

Leaf

Leaf

Leaf

Leaf

Leaf


ImplementationImplementation

ProcessingProcessing– Two Xilinx Virtex4 FPGAs– XC4VLX100-FF1513

• Must concentrate large amount of data– Choose package with most I/O

• Integrated differential termination makes layout simpler

• High speed I/O provide reserve capability

CommunicationCommunication– Xilinx Virtex2 FPGA– XC2V3000-BF957

• Robust in 3.3V enviroment– VME 64x interface– Slink– TTCrx– Ethernet PHY & USB for future

Elec FPGAElec FPGA– Isolated Electrons– Non-Isolated

Electrons– Energy Sums– Jet Counts

Jet FPGAJet FPGA– Forward Jets– Central Jets– Tau Jets


Top SideTop Side

Jet Comm

+5V

Elec

DPMC: ElecLeaf(and one on other side)

DPMC +3.3V power

DPMC: GlobalTigger

Clock Distribution

Switching power supplies

Comm JTAG header

CPLD, System & MiscJTAG headers

RJ45-LVDS Out(e.g. for TTS)

J2: VME & Slink

J1: VME

Both sides of board

Top side only

Trigger data flow


Bottom SideBottom Side

Proms

DPMC: ElecLeaf(and one on other side)

DPMC: GlobalTigger

USB

Duplicated

Single

Clock distribution

Ethernet

RJ45-LVDS Out(e.g. for TTS) RJ45-LVDS In

(e.g. for TTS)

Readout path

Duplicated

Single


Board Specifications (I)Board Specifications (I)

Layers = 14Layers = 14– 7 signal

• 50 ohm single ended • track width• 100 ohm differential• track width/gap

– 2 ground– 5 power

Additional layersAdditional layers– May need at least one extra

power & ground layer• ExceptionPCB in UK• 20 layers possible• Need to look at stack up to

verify.


Board Specifications (II)Board Specifications (II)

Vias SpecsVias Specs– Min drill diameter = 0.3mm– Through via

• layers 1 to 14 • depth = 2.45mm• aspect ratio = 8.2:1

– Blind Top via • layers 1 to 8• depth = 1.3mm

– Blind Bottom via• layers 9 to 14• depth = 0.8mm

Thickness = 2.45 mmThickness = 2.45 mm– Recommended not to exceed

this otherwise• Through hole component

lead length too short• Aspect ratio on vias too

large

Card edgesCard edges– Board will be milled down to

1.6mm• Should this be 2mm ?


Power Supply (I)Power Supply (I)

Input powerInput power– From ECAL backplane, but with +5V.– 3 connectors (3M:MP2-SP10-51M1)

– Each connector has 5 pins power, 5 pins ground– 6.5A per pin => 32.5A per connector

– At present 2 of the 3 are used to supply 5V• 65A or 325W

– 1 connector is spare.


Power Supply (II)Power Supply (II)

Clock SystemClock System– Driven by linear regulators– 2.5V, 0.5A from LT1763CD

• QPLL– 3.3V, 3A from LT1764E

• TTCrx, SN65LVDS125, SN65LVDS125

Dual PMC sitesDual PMC sites– Leaf cards & GlobalTrigger card– Supplied by either:

• Datel LSM-10A switcher• TPS75933 linear, 3.3V, 7.5A

– Both situated under PMC site. Not ideal because

• Height of switcher is 9mm• Power dissipation of linear 5A

from 5V = 8.5W• Could easily add temp sensor

to exiting I2C chain.

FPGAs and rest of boardFPGAs and rest of board– Main power from 6 Datel LSM-10A

switchers– 3 of the switchers provide core

volatges for each FPGA• 2 x 1.2V• 1 x 1.5V

– 3 of the switchers provide I/O power to the FPGAs

• 2 x 2.5V• 1 x 3.3V

– FPGA Proms are powerwed from LT1763CD, 1.8V, 0.5A


JTAG ChainJTAG Chain

Header Misc

CPLD

Header System

DPMC: ElecPos

DPMC: ElecNeg

DPMC: GlobalTrig

PMC: Spare

V4 & PROM: Elec

V4 & PROM: Jet

TTCrxSlink

Ethernet

Header Comm

HeaderCPLD

VME Voltage

Error

WheelPos

WheelNeg

Control

Red = 3.3VBlue = 2.5V x8


CLK_DIFF_A (CLK40)

CLK_DIFF_B (CLK80)

CLK_DIFF_C (CLK80)

CLK_DIFF_D (CLK40)

Clock Distribution (I)Clock Distribution (I)

TTCrxCLK160

QPLL

SN65LVDS125Cross-Point Switch

SN65LVDS108

HeaderDual 0.1”

Destinations:3 x FPGAs3 x DPMCs2 x Wheel Cards

SN65LVDS108

SN65LVDS108

SN65LVDS108

Out: 0

Out: 1

Out: 2

Out: 3

In: 0

In: 1

In: 2

In: 3

CLK80

CLK40

CLK40DES1

HeaderDual 0.1”

Comm FPGA

Destinations:1 x PMC1 x two 50 ohm SMB1 x header 0.1” dual


Clock Distribution (II)Clock Distribution (II)– At present everything driven by

40MHz & 80MHz QPLL clock, including VME.

• Comm FPGA cannot configure the clock without leaving itself vunerable.

• The QPLL has searche mode, which may force DCMs to unlock.

– Solution:• Plan to hard code QPLL and

Cross-Point Switch configuration.

• No DCMs

– Backup:• Use 40MHz utility clock for

VME. Then bridge to QPLL clock domain.

– Clock traces are at present not length matched

• Suggest that at least the 6 traces to the Comm, Jet and Elec FPGA are length matched to within an inch.

– Would need an extra power/gnd layer

• Extra layer would also help routing around Samtec bolt holes.

– An extra layer would allow length matching up onto the leaf cards.


Interfaces (not VME) Interfaces (not VME)

RJ45RJ45– Require an RJ45 with LVDS outputs for Trigger Throttle System

• Added extra capacity• 2 x RJ45-LVDS outputs• 2 x RJ45-LVDS inputs

USB & EthernetUSB & Ethernet– USB uses Cypress CY7C68001

• Wrong connector at present (Type A rather than Type B)– Ethernet Uses Intel LXT971A– Both used on Source/IDAQ cards

• Can benefit from experience at Imperial College– Both request careful layout. Not yet checked

SlinkSlink– Will use ECAL transition card and DCI


Recommended changesRecommended changes

Move clock outputs (simple)

Replace caps with fusesfor +5V supply

Replace sense resistor with fuses on switchers.

What about linear and leaf supplies?

Change USB type A to type B

Can 30A flow through antipads

Match lengthclock traces

Mounting holes

for Samtec

Decoupling caps at connectorsp3v3, p2v5a, p2v5b, p5v, p1v2band 3.3V for all DPMC sites


No change recommendedNo change recommended

Linear dissipating

8.5W in confined space

Comm FPGAJTAG Hdrin awkward location

Switcher 9mmhigh. Caps on

leaf at least 1mm.

Maxclearance =

15mm. Depnds on leaf

Power supply power up rules for Xilinx devices


Global Trigger Card (I)Global Trigger Card (I)

– Susceptible to power supply noise

• SerDes power will be provided by local linear regulators

– Mounted on DPMC• If design revision necessary

cost and turnaround time should be substantially less.

– Clock distribution similar to that of Concentrator Card.

• Cross-point switches receives clk40 and clk80 from concentrator and an exetrnal source

• Distributed by four ICS83948I_147 ICs

• Clk 80 sent to SerDes chips• Clk40/80 sent to CPLDs


Global Trigger Card (II)Global Trigger Card (II)

XC2C256FT256

DS92LV16

DS92LV16

DPMC42

Tx or Rx

Tx or RxCable 0

Instantiate 8 times

- 7 for transmit - 1 for receive

Self contained powerfrom linear regulatorsLT1764E / 3.3VLT1763CD / 2.5VLT1763CD / 1.8V

Provides 2.5V and 3.3V logic level conversion.Also allows us to test DS92LV16 serdes units in loop back mode, albeit one at a time, should we encounter problems.


TestingTesting

Connectivity testConnectivity test– Can test connectivity of ~80% of board with either JTAG or

custom firmware.• Samtec connections

– Loopback with production cables

• PMC sites– DPMC test board (Matt Stettler)

• FPGA-FPGA connnections

Insitu testsInsitu tests– VME & Slink etc are probably best tested by final or test

firmware • E.g. for VME by writing/reading register many times• Alternative is a dedicated JTAG loopback system.

– Time consuming to construct


Schedule & StatusSchedule & Status

– PCB manufacture & assembly in September• Manufacture 3 PCBs• Assemble 1

– At present 1 month behind schedule• Also need GT DPMC card layout and manufacture


AppendixAppendix

Following slides list the signal counts Following slides list the signal counts and how they were obtainedand how they were obtained


Incoming dataIncoming data

Jet interface Jet interface – Arrives from 2 x wheel cards via high speed Samtec cable

assemblies• 240 LVDS signals from each Wheel card (40 MHz DDR -> 80Mhz)

– 200 for trigger path– 34 for control & readout– 2 for clk– 4 for jtag

Electron interfaceElectron interface– Arrives from 2 x leaf dual PMC cards mounted on concentrator card

• Only 206 out of 360 I/O used)– 160 for trigger path– 40 for control & readout– 2 for clk– 4 for jtag


Outgoing dataOutgoing data

Global Trigger interface Global Trigger interface – Tranmit to Global Trigger on 7 cables

• 2 unidirectional SerDes channels per cable• Each channel driven by NatSemi DS92LV16• Takes 16 bit parallel data at 80MHz. Transmits at 1.44 Gb/s

– Loopback testing possible with 1 extra cable– Mounted on Dual PMC

• All 360 I/O connected• Allows relatively fast & cheap modifications if problems exist with

high speed serial links

Slink to DAQSlink to DAQ– Signals connect to VME J2

• Uses ECAL transition card to host SLINK transmitter card


Jet Trigger: GuideJet Trigger: Guide

6 x Leaf (3 per Wheel)

Jet

2 x Wheel

All numbers in “bits” assuming 80 MHz data transmission on single-ended & differential pairs6 clustered jets (12) & Ht(13) = 6 clustered jets of 12 bits each and 13bits for Ht per leaf card H: ~3x160 = “Have” aprrox 160 bits from each of the 3 leaf cardsR: 3x85 = “Require” 85 bits from each of the 3 leaf cards (6x12+13)

What does this mean ?

How to understand the next slide

Spare capacity on bus

Bus at limit, although running at double capacity (160MHz data) should be possible in future

U21 x JetFinder

& Cntrl 6 clustered jets (12) & Ht(13)H: ~3x160, R: 3x85


Jet TriggerJet Trigger

Electron

Jet

GT

Leave at least 2bits per bus for BC0 (equivalent to 1 signal at 80MHz)

1) Iso Elec2) Non-Iso Elec3) Energy Sum4) LoopBack cable

5) Forward Jet6) Central Jet7) Tau Jet8) JetCounts

6 x Leaf (3 per Wheel)

Energy

Jet

U21 x JetFinder

& Cntrl

U12 x JetFinder

12 sorted jets (14 + 2 spare) JetCnt(60)

H: 2x280, R: 2x192

2 x Wheel 1 x ConcentratorEt(13), Exy(34)Ht(13)

H: 2x80, R: 2x60

Ctnrl (64), Et(13), Exy(34), Ht(13)H: ~3x160, R: 3x124

6 clustered jets (12+2 spare)JetCount(60)

H: ~3x160, R: 3x144

12 clustered jets (12+2 spare)JetCount(60)

H: ~3x320, R: 3x228

H: 160, R: 0 H: 160, R: 0

SerDes pair (40)H: 180, R: 160

SerDes pair (40)H: 180, R: 160

H: 2x20, R: 2x0

Double Compare 3x3 requires 2x3 pre clusters (18)

H: 120, R: 108

Et(13), Exy(34), Ht(13)Double Compare 3x3 requires

2x3 pre clusters (18)H: 400, R: 168

Next Leaf


Signal: From single Jet-WheelSignal: From single Jet-Wheel

Jet data sent to ‘”Jet’” FPGAJet data sent to ‘”Jet’” FPGA– Top 4 rank of central, forward and tau

jets. Hence 12 objects– 12 sorted jets (min 14 bits each)

• 5 bits phi• 3 bits eta (no need for sign)• 6 bits rank

– 9 unsorted jets (min 14 bits each)• 18 phi regions -> max 9 jets• 1 bit phi (each jet covers 2 phi)• 0 bits eta (events in the middle)• 10 bits Et• 1 bit tau veto• 2 bits spare

– Total = 147 signals @ 80MHz • 294 bits

– Available = 150 signals @ 80MHz

Jet data sent to “Elec” FPGAJet data sent to “Elec” FPGA– Et-total (13 bits)

• 12+1 bits mag + overflow– Et-missing (26 bits)

• x & y components• 12+1 bits mag + overflow

– Jet counts (36 bits)• 6 jet count regions each 5 bit• Alternative more flexible

system of 12 jet count regions of 3 bits each (not TDR)

– Ht (13 bits)• 12+1 bits mag + overflow

– Total = 38 + 7 signals @ 80MHz• 75 + 13 bits

– Available = 40 + 10 signals @ 80MHz


Signal: From single Elec-Leaf Signal: From single Elec-Leaf

Electron data sent to ‘”Elec” FPGAElectron data sent to ‘”Elec” FPGA– Top 4 rank of isolated and non-isolated electrons (min 14 bits each)– 8 electron objects (14 bits)

• 5 bits phi• 3 bits eta (no need for sign)• 6 bits rank

– To reduce the latency the FPGAs on the electron leaf card will not share data. Hence each FPGA will send 8 electron objects to the concentrator (i.e. concentrator receives 16 electron objects from each leaf card)

– Total = 112 signals @ 80MHz• 224 bits



Signal: Between Jet/Elec FPGAsSignal: Between Jet/Elec FPGAs

Data transmitted between V4 FPGAsData transmitted between V4 FPGAs– The 9 unsorted jets on the boundary between the two wheels are

turned into clusters in the “Jet” FPGA– These jets will contribute to Ht and the jet-counts being summed in

the “Elec” FPGA

– Ht (13 bits)• 12+1 bits mag + overflow

– Jet counts (60)• 12 types each 5 bits

– Total = 37 signals @ 80MHz• 73 bits



Signal: Control & ReadoutSignal: Control & Readout

ReadoutReadout– Max slink sustained rate = 200 MB/s– Assume no source generates more than 100MB/s

• 10bits @ 80MHz

ControlControl– Serial VME

• 2bits– L1A & BC0

• 2bits– Serial Fast Commands from TTC B channel (e.g. resync)

• 1bit– AsyncReset

• 1bit– Serial FastFeedback

• 1bit

Total Total – Required = 17 signals– Minimum available = 32 signals


Signal: GT interfaceSignal: GT interface

Global Trigger interface Global Trigger interface – GT receives 7 cables (2 unidirectional SerDes channels per cable)

• Each channel driven by NatSemi DS92LV16• Takes 16 bit parallel data at 80MHz. Adds 2 bits. Transmits at 1.44 Gb/s• Require 2 bits for powerdown/sync• 252 signals (7 x 2 x 18)

– Require 1 cable for loopback testing• Generates 16 bits parallel data• Require 4 bits for lock, refclk, powerdown and recovered clk• 40 signals (1 x 2 x 20)

– NatSemi chips do not have JTAG• Could use local loopback to test data lines only.• Require 2 bits for outenable and local loopback on all chips• 44 signals ((14 x 2) + 16

– Mounted on dual PMC • All 380 I/O connected, • Require at least 292 signals, perhaps 336

SlinkSlink– Signals connect to VME J2 and hence to ECAL transition card

cms: gct2: concentrator card pre-manufacture review

Documents