hcal/tridas. november, 2003 hcal tridas 1 tridas status drew baden university of maryland november...

HCAL/TriDas. November, 2003 HCAL TriDAS 1

Tridas Status

Drew BadenUniversity of Maryland

November 2003


HTR Rev 3 (2003)

Dual-LC O-to-E

VME

Deserializers

Xilinx XC2V3000

-4

Sti

ffen

er

s

SLBs (6)

TTC mezzanine

• Rev 3 – 30 boards made in March 2003– Changes from Rev 1 (2002)

• Full 48 channel capability– Rev 1 was “half HTR”

• FPGA package– Change from FBGA (1.00mm) to BGA

(1.27mm)• Added stiffeners• SLB placement

– Moved to front-panel daughterboards• Clocking

– TLK2501 (deserializer) “refclk”» Can use crystal oscillator or TTC80

– Fanout scheme» Receive all signals on 1 quad Cat6

– Board production changes:• New assembler, in-house X-ray, DFM

review, QC• Gold plated (Rev 1 was white-tin) for

better QC


HTR Testbeam 2003 Results

• Clocking issues– We believe our clocking problems were a combination of

• Front-end difficulties• Fiber cleanliness

– Front-end capability to use either TTC derived clock or crystal oscillator• TTC cleanup using Cypress robochip

– Not rad hard, but meets GOL 100ps jitter spec• Had more success with the crystal oscillator

– Possibly a very nasty EM environment as well– We advocate running with UPS powering FE boards and VME crates in H2

• QPLL implementation will be next– Fiber cleanliness

• We underestimated how critical this is. Will be more carefully done in the future

• Overall HTR implementation validated– Clocking, synching, data transmission, VME, etc.– But not for TPG part

• Checkout of connectivity in lab• Checking SLB this month• Firmware completed (as much as possible)• TPG is the “easy” part of validation for production


Changes to HTR for Rev4

• Moved 2 LC’s down to give more clearance for fibers

– Upper rear of card• Spread out routing of differential pairs for 6 SLB

and 2 FPGA system clocks• Removed hot swapping circuits

– Relax hot-swappable requirement - worry about noise

• Front-panel changes– Rotary switch, LEDs, change mux selects from

jumper to internal, etc• Miscellaneous changes

– Fixed what was found to be wrong with Rev3 board, add test points, terminate all unterminated I/O lines to SLB, VME byte swapping, other minor stuff for noise considerations

• Schedule– HTR Rev 4 is done – reviewing now. Ready for

next stage of pre-production– Will probably submit by Dec 1 to make 30 boards

• Will stuff 3, checkout, then stuff the rest, that should get us ready for Testbeam 2004 and Slice Tests


TODO - TPG

• HTR production can begin after:– SLB/HTR/Wisconsin check

• Check that we can maintain link• Setup ready…so far so good• Measure BER, etc.

– SLB/HTR connectivity check• Logic analyzer card on SLB site already

shows connectivity• Waiting for firmware for SLB to verify for

sure– Should be this week or next

• TPG Data validation– Will build a 6U VME board with sites for the

Wisconsin Vitesse receiver boards– Will fifo data and read out over VME– Use this to check data validity, try different

TPG tests, etc.– Plan to start on this board this week– Ready in a few months. Will use this for the

early slice tests to test TPG output.

FPGA

Clock Input


TTC receiver (TTC_umd)

• General purpose TTC receiver board• TTCrx ASIC and associated PMC connectors• Will be used to receive TTC signal by HTR, DCC, and Fanout boards• No signal receivers

– Copper/fiber receivers must be on the motherboard

– Signal driven through TTC connectors

• Tested successfully– Maryland, Princeton, BU, FNAL– Testbeam H2

• Production:– Need 1 per HTR (~260) + DCC (20) +

Fanout (10)– Need ~500 TTCrx for HCAL– Will layout test board for mass testing


HCAL Clock Fanout• HTR clocks provided by a single 9U VME board

– Chris Tully/Jeremy Mans from Princeton– Has fiber TTC input

• Signals fanned out over Cat6 twisted pair:– TTC stream

• To be used by each HTR and by DCC to decode commands & L1A

– BC0• To be used by SLBs to synchronize TPGs

– “40MHz” clock• To be used by FPGA and SLBs to maintain pipeline

– Comes from QPLL

– “80MHz” clean clock• To be used for deserializer REFCLK

– Comes from QPLL

• Changes– TTCrx reset circuitry added– VME capability added

• Status– Final layout in hand. Waiting till the last minute.– Production can start mid January


Clock Distribution

HTR

TTC fiber

TTC

CLK80

BC0

CLK40

distributionto 6 SLBs

and to 2 Xilinx

Brdcst<7:0>,BrcstStr, L1A

O/E

BC0

TTCTTC

TTC

FPGA

..

..

TestPoints forRxCLKand RxBC0

..

..

..

..

80.18 MHz

..

..

..

..

TTCrx

to Ref_CLK of SERDES(TLK2501)

CLK40

CLK80

Princeton FanoutBoard

TTCrx

QPLL

HTRHTR

Cat6Eor Cat7Cable


HTR Firmware


TPG Path

• Still under development– The following is already coded/simulated but not tested in HTR

• TPG path tasks (not necessarily in order)– Linearize QIE data to 10 bits

• With .5GeV resolution gives 512 GeV max

– Apply BCID filter• Probably will sum over 2 buckets and assign based on high/low patter

around the bucket that has the max energy

– Sum or divide depending on HB, HE, or HF– Extract a muon window for the “feature bit”– Apply logic to eliminate false muons from shower leakage, etc.– Compress and send to SLB


TPG Path Schematic


Firmware/TPG Plans

• Firmware– Focus on TPG Path

• Latency– Measure and scrub, scheme for random latencies, meeting latency budget, etc.

• SLB checkout– Connectivity, localbus access, timing, data integrity, etc.

• This will be the main activity from now until we go into production.

– Bells and whistles for error reporting/recover• Meetings between Maryland, Princeton, and Boston groups• More to be learned in 2004

• HTR boards– Make Rev4 this fall, go into production early 04

• ASAP given schedule• Depends on results from above


Latency

• Definition: from BX to input to RCT

HCAL O-E QIE CCA HTR SLB RCTBX TOF To RBX Data To RCT

RBXHPD or PMT (HF)

46 clocks = 1,147.7ns

GOL


TOF+Y11/fiber to RBX

10.855

4.332


RBXHPD or PMT (HF)

46 clocks = 1,147.7ns

GOL

TOF Tx to RBX (7ns/m) TOTAL

HB =1 layer 1 (1.8m) 6.1ns (4.9m) 34.1ns 40.2ns

HB =15 layer 9 (4.6m) 15.4ns (0.6m) 4.2ns 19.6ns

HE =34 layer 1 (4.0m) 13.3ns (3.3m) 22.9ns 36.2ns

HE =17 layer 14 (5.7m) 19.1ns (0.3m) 2.0ns 21.1ns

HF (10.9m) 36.3ns (2m quartz) 14ns 50.3ns


Inside RBX

• CCA delays will be set using LED system

TOF+Tx O-E Total to QIE Clocks QIE CCA+GOL Total

HB 40.2 (HPD) 0ns 40.2ns 1.6 (2) 4 3+2 11

HE 36.2 (HPD) 0ns 36.2ns 1.5 (2) 4 3+2 11

HF 50.3 (1.5kv PMT + 6m coax)

14ns + 30ns = 44ns

94.3ns 3.8 (3) 4 2+2 12

CCA operation needs more experience...


RBXHPD or PMT (HF)

46 clocks = 1,147.7ns

GOL


Digital Data Fiber

• Measured = 2/3 (51m adds 250ns delay)– Gives 4.99 meter/clock tick (@40.08MHz LHC RF frequency)

• Default fiber length 90m gives 18.036 clocks– HE is longest due to routing around ME outer radius

• Change HE fiber routing to go via EE inner radius

– We believe we can shave off 15-20m• 3 clocks – 15 total

• maybe more?Thru RBX Data Fiber Total

HB 11 15 26

HE 11 15 26

HF 12 15 27


RBXHPD or PMT (HF)

46 clocks = 1,147.7ns

GOL


Random Latencies in HTR

• Latency due to – TI deserializer (TLK2501)

• Advertised random latency with respect to reset 76 to 106 bit times

– Remember, TLK2501 has 20-bit frames (80MHz frame clock)

– Latency will be 47.5 to 66.9 ns random

• We measured the random latency at UMD

– Latency difference is randomly distributed between 0 and 6ns.

– Factor of ~1/3 of TI spec. Will investigate if this is “typical”

– Asynchronous fifo• Relative phase between recovered

clock and refclk will introduce a random latency of 0 to 1 clock tick

TLK2501

TLK2501

Common 80MHz system clock = 40MHz clock/2 from TTC

Recovered clocks80 MHz system clock


Latency in HTR

• Measured: – From input to GOL, through 7m fiber,

through HTR firmware to SLB input connectors

– Total: 395ns = 16 clock ticks– GOL takes 2 clocks, 7m fiber takes 1.4 (call

it 2), remainder is ~12 clock ticks• Relative phase between GOL and HTR

system clock and TLK refclk varied– Latency seen is some combination of TLK

and asynchronous fifo– Measured for a single Temp and VCC

• Need to repeat with variations

Thru RBX Data Fiber HTR Total

HB 11 15 12 38

HE 11 15 12 38

HF 12 15 12 39


RBXHPD or PMT (HF)

46 clocks = 1,147.7ns

GOL


SLB + TPG Cables

• SLB nominal latency 3 clock ticks– SLB job is to make sure all HCAL/ECAL towers from same BX arrive at RCT

in synch by…

• Histogram TPGs to find the LHC beam structure– Find “BC0_DATA”

• Adjust with BC0 broadcast timing (TTCvi) so all partitions are in synch– “BC0_TTC”

• Delay accordingly knowing |BC0_DATA BC0_TTC|• We might have to suffer 1 or 2 clock ticks here – we don’t know yet

• TPG Cables– 20m nominal 2xdual skew-clear: 5m/clock tick– Discussions so far indicate reducction to 15m cables quite possible.

• Results in 3 clock ticks for TPG cables

• Total – 3(SLB nominal) + 1 (SLB contingency) + 3 (TPG cables) = 7 clock ticks


Grand Total

• Caveats and assumptions– Numbers for TOF+prog delay are correct

(checked)– 75m max for all fiber cables (saves 3 clocks)– CCA can be run with a latency of 3 and still do its

job (5 max for the chip)– 3m TPG cables– Summing in HTR firmware does not add to

latency– Total delay through SLB < 4 (1 for contingency)

• Need experience from this fall’s tests

Thru RBX

Data Fiber

HTR SLB/TPG

Total

HB 11 15 12 7 45

HE 11 15 12 7 45

HF 12 15 12 7 46

• Possible further scrubbing– CCA can perhaps take up some of the

slack for some of the non-integer TOF+propogation delays (1clock)

– Digital fiber cabling (every 5meters = 1 clock!)

– “Tricks” in HTR firmware• Some parallelism, routing constraints,

faster LUT access…– Other clever ideas

• Possible further additions to total latency

– TOF+propogation (HF delays thru PMT and coax, need measurements)

– Digital fiber cabling could go back to 90?

• HCAL policy: 75m cables or less– Summing in HTR firmware

• Need a MC study to tell us• Default should be no summing unless

MC study tells us otherwise– Difficulties with HCAL/SLB timing

• Need experience


Project Timeline

2003 2004

Firmware TB 03 firmware should be adequateBoards Can use current crop of Rev3

To do: Commission QPLL, global clocking. Level 1 Trigger: SLB, TPG, latency…

“Vertical Slice” (~March)

2005

HB, HE, and ME plus Level 1/TPGFiber synchronization Check in alcove…compare environmentsFirmware TPG firmware has to be ready. Fall 03 task.Boards Must have Rev4. How many TBD.

FE Commissioning (~Feb)

“Magnet” Tests (no HCAL)

CMS Magnet Integration System TestBoards Will have Rev4.

H2 Testbeam

Fiber synchronization Check in H2…compare environmentsBoards Will have Rev4. How many TBD.

hcal/tridas. november, 2003 hcal tridas 1 tridas status drew baden university of maryland november...

Documents

hcal tridas

vme use

production slide

htr testbeam

todo tpg htr production

ttc receiver ttc

ttc mezzanine rev

tpg data validation