NA62 TDAQ WG MeetingNews, planning, issues

M. SozziCERN – 28/10/2009

• Technical Document

• Clock infrastructure

• TDC/TELL1 statusTDC boardTELL1 productionTELL1 Crates

• Components/productions

• L0 processorspecifications, back-control

• Urgent issues

Technical DocumentThe TDAQ TD chapter must be a real collaborative effort

We should exploit it to help defining the system

If we have to meet the TD schedule:

(1) the specifications for the readout and trigger interfaces of each sub-detector(2) the kind of expected processing to be performed by the readout systems(3) the processing and data reduction expected by the L1 PCs

should be clearly spelled out by the September meeting.

A report from each sub-system addressing the above issues was requested for the TDAQ WG session in Anacapri.This should become the corresponding section of the TD.

Can we make the deadline?

FROM JULY MEETING

General overview (Marco) CHOD (Mauro P.)

Trig. logic & simulat. (Marco, Giuseppe, Phil) RICH (Mauro P.)

Requirements and specifications (Marco) Straws (Hans)

Clock distribution (Marian, Marco) MUV (Rainer)

L0 distribution (Marco) CHANTI (Fabio)

SPS interface (Riccardo, Marco) Anti halo (Riccardo)

Data flow chocking/errors (Marco) SAC (Leandar)

Crates (Valeri) IRC (Leandar)

Local networking (Alberto) LKr (Bjorn, Riccardo)

Common TELL1-based TDAQ (Marco) LKr trigger (Andrea)

L0 processor (Marco) Online farm (Paolo)

CEDAR (Cristina) L2 processing (Phil + subdetectors)

Gigatracker (Angelo) Event building

LAV (Mauro R., Riccardo) Data logging (Paolo)

Data monitoring (Paolo)

= Written, to be improved = Something written = Completely missing

As of 28/10/2009: Draft version 3 uploadedTD

Clock/L0 infrastructureTTC: Birmingham proposal in Anacapri:

use ALICE LTU+TTCex: one (+spares/lab?) for each sub-detector who wants to be able to run standalone (3KCHF + 3.4 KCHF, about 6 weeks) → Marian’s talkTake decision now, then start securing components and make productionCentral oscillator: buy/installFibres: buy/installLocation: Technical Gallery?

Old NA48 clock (for LKr): Responsibility (CERN?)Make inventory, build more modules (help from Vienna?)

TELL1/TDC “Customers”• LAV, RICH, CHOD, MUV, CHANTI • CEDAR:

Use of HPTDCs at very high rates?Reduce number of channels/TDC→ Angela’s and Giuseppe’s talks

• STRAWS:A less performant system would be enough Options:(1) Use “standard” TDC/TELL1(2) Use TELL1s with custom daughter cards(3) Design completely custom systemPossible solution: buy components for TDC/TELL1 system and decide later→ Presentation & discussion needed at next TDAQ WG !

Work Cost

+ +++ ++ ++ +++ +

TELL1/TDC: TDC board• Finalize design of final TDC board, start pre-

production by end 2009Current prototype cost estimate: 1.5K€/board (driven by FPGA)

Changes:(a) Cables/connectors(b) FPGAs

• Need to secure components now• Re-engineering of NINO board (Firenze)

No interest from other sub-detectors: so for RICH only → Massimo’s talk

TELL1/TDC: TDC board - cablesThe cables saga continues (now entering its second year…)Brief reminder:

- Large cost differences- Mechanical issues- Performance differences not seen with RICH PMs

July meeting: - cheap twisted-pair solution seems good enough- 34 pairs (32 channels + ground + 1 user-defined pair to front-end)

Since then:- cheap plastic connectors solution? (no ground on connector)- Samtec cable: no halogen-free solution- Nice Samtec connector & flat cable (to leave possibility for Samtec high-performance cable?): not possible

TELL1/TDC: TDC board - cables

Present “black” cable About 140€/16 channels

Samtec cable 570€/64 channels

“Blue” (ALICE) Amphenol cable 360€/32 channels (>> ALICE)

Cheap twisted flat solution About 80€/32 channels

As of last week:

“Blue” (ALICE) Amphenol cableAbout 150€/32 channels

4 connectors/board (2 on each side)Difficult to get SMD connector for PCB:some routing issues

TELL1/TDC: TDC board - FPGA

FPGAs:Present design has quite powerful Stratix II FPGA (underutilized)

Want to keep power for possible on-board processing

Change to Cyclone III

Stratix II Cyclone III

Logic elements 60K 120K

Total memory 2.5M 8.2M

User I/O pins 718 413

Cost 1325 $ 418 $

Cost reduction with no expected performance degradation

TELL1/TDC: TELL1 board• Old board cost: 4550 CHF• Technical meeting on TELL1 board upgrade todayChanges:

(1) FPGAs(2) Memories

• Decision on upgrade• Finalize design of new TELL1 board, start pre-

production by spring 2010?• Need to secure components now• Involvement in the upgrade/testing?

28/10/2009: Decided to make upgrade!

PP-FPGA

TELL1 board upgrade

FPGA usage with present “naive” firmware (no DDR memory controller, no trigger processing, no inter-board communication)

Stratix I (2001) considered “mature” product, not recommended by ALTERA

SL-FPGA

PP-FPGA

SL-FPGA

TELL1 board upgradeWith present “naive” firmware (no DDR memory controller, no trigger processing, no inter-board communication)

Replacement of (soldered) DDR memory with SODIMM socket version

Change in PCB layout required (to start at CERN before end 2009) if upgrade decided

Implications on production times, testing

NOW

TELL1/TDC: Crates

Hytec hybrid VME/TELL1 crate (5 VME slots + 10 TELL1 slots = max 4-5 TELL1/TDC) with remote (ethernet/USB) control and monitoring by Wiener card (4260 €)

One to be bought in Pisa

Crates

Discussions with CERN experts (M. Dutour) concerning integration/control→ Presentation at December TDAQ WG

Before buying any crate you should check with Mathias for compatibility issues with CERN slow control infrastructure

TDC/TELL1 components: estimatesTo increase (upgrade)

To decrease

Here “first batch” means “as soon as possible”in case of upgrade this will be anyway after prototypes

Straw option 2 = TELL1s with custom TDCsStraw option 3 = No TELL1 at all

Getting componentsBlack numbers are confirmed, red numbers are guesses (= will not buy !)Prices are only indicative (see previous slide)

Getting components

Total (including straws)

Straws only

TTCrx 5000 CHF 2000 CHF Urgent

Quartzes 4400 CHF 300-1600 CHF Availability?

QPLL 2800 CHF 200-1000 CHF

HPTDC 43200 CHF 0-19200 CHF

Glue cards 11900 CHF 2380 CHF Urgent

TOTAL 67300 CHF 4880-26180 CHF

Development team

CEDAR R. Staley

CHANTI Needed ?

LAV ?

Straws ?

RICH M. Piccini

CHOD Needed ?

IRC/SAC ?

LKr ?

LKr L0 trigger A. Salamon

MUV ?

As stated, Pisa will deliver a “vanilla” firmware, which will need to be modified for most sub-detectors to implement specific processing.One person will have to write the firmware (VHDL) for each sub-detector:

L0 central processor: functions

- Receive (asynchronously) trigger primitives from sub-detectors:condition + time with sub-25ns resolutionwithin a maximum latency: O(100-200 μs)already merged between parts of sub-detectors

L0 central processor: functions- Time-match with programmable tables and time-windows- Re-synchronize and drive TTC with

L0 pulse (synchronous)6-bit trigger word (asynchronous)within a maximum latency: O(1 ms)

- Drive TTC with start/end burst messages- Collect choke/error signals

react to them by stopping L0 triggersmonitor and record occurrences

- Provide its own data packet to farm via GbE

Sub-detectors shall: add upper timestamp bits, add 24-bit event number, send these (and trigger word) with data

Is 25ns timestamp granularity sufficient for all sub-detectors (minimum 75ns readout window)?

L0 trigger back-control

Each sub-detector should drive two lines to the L0 processor for signaling problems in handling the L0 triggers

Only one pair of lines per sub-detector (OR of individual boards taken care by sub-detector)

In normal conditions both lines should never be set (not a flow control feature)

Choke signal: sub-detector about to face trouble in handling triggers, no trigger lost so far: L0TP will send “choke-on trigger” and then pause

Error signal: sub-detector lost triggers or data, entire burst is compromised:L0TP will send “error-on trigger” and then pause

L0TP will send “choke-off” or “error-off” triggers when resuming

More commands through L0

Each sub-detector MUST react to EVERY L0 trigger“Synchronization triggers” sent at specific (predefined, even

non-periodic) times by L0TP; sub-detector respond with “sync frames” which will form “sync events” to be kept by all trigger levels, allowing an offline check of data integrity

“Calibration triggers” sent by L0TP in response to a sub-detector calibration primitive

“End-of-burst triggers” sent by L0TP after end of burst to require special frames with summary data (counters, etc.)

L0 central processor: implementationPossible implementations:1. Birmingham proposal (custom) → Marian’s talk2. FPGA devel. board + custom interface (Roma1)?3. PC+interface (Pisa)?4. TELL1+interface?→ dedicated discussion @ next TDAQ WG @ CERN

Come with proposalsDefine responsibilities and implementation

The (usual) urgent issues

• (Fast) MC simulation of (L0 and) L1/L2P. Rubin to coordinate the simulation effort Need information from sub-detectors on cuts, rejection, algorithms, etc.

• Farm infrastructure and dimensioningSome discussion started, need numbers Which are the hard constraints?Links: how many? (1GbE for data, other ethernet for slow control)

Conclusions

• Write/update your TD sub-section• Read TD and comment• Decision on clock system: procurement• Decision on TELL1 upgrade • Procurement of components for TDC/TELL1:

sub-detector commitments• Crate control compatibility (next WG)• L0 processor: @ next TDAQ WG• Simulation and farm discussion