instrumentation department john coughlan rutherford appleton laboratory14 november 2002 cms tracker...

Instrumentation Department John CoughlanRutherford Appleton Laboratory 14 November 2002

CMS Tracker Readout

Effort Requirements inInstrumentation Department

John Coughlan


CMS Tracker Readout Summary

•Overview.

•Scope of project in INS

•Team members

•Project Status

•List of tasks to completion

•Effort required

•Summary & Comments


CMS Tracker ReadoutOverview

Task: To readout a very large Silicon Tracking detector.

~ 9 million Silicon Strip channels

ON Detector: 73K APV25 pipeline chips

@ L1 Trigger: MUX APV Frame output

Analogue Data readout via Optical links

(APV Frame: Header + Strip Data)

=> INS Effort : essentially complete

OFF Detector: Front-End Drivers (FED)

Digitise / Zero Suppress / DAQ readout

~ 500 x 9U VME64x boards (incl spares)

96 ADC channel boards

=> INS Effort : major activity

DAQ

CountingRoom

On Detector (Hard Radiation)

FPGA

25

VME 9U FEDs

HybridFront-End HybridSilicon Strips 70m


CMS Tracker FED Block Diagram

VME-FPGA

TTCrx

BE-FPGAEvent Builder

Buffers

FPGAConfiguration

PowerDC-DC

DAQInterface

12

12

12

12

12

12

12

12

Front-End Modules x 8Double-sided board

CERNOpto-

Rx Analogue/Digital

96 TrackerOpto Fibres

VMEInterface

XilinxVirtex-IIFPGA

Modularity

9U VME64x Form Factor

Modularity matches Opto Links

Data suppression board.

8 x Front-End “modules”

OptoRx/Digitisation/Cluster Finding

Back-End module / Event Builder

VME module / Configuration

Power module

Other Interfaces:

TTC : Clk / L1 / BX

DAQ : Fast Readout Link

VME : Control & Monitoring

JTAG : Test & Configuration

FE-FPGAClusterFinder

TTC

TempMonitor

JTAG

9U VME64x


CMS Tracker FED Data Processing

VME-FPGA

TTCrx


Buffers

FPGAConfiguration

PowerDC-DC

DAQInterface

12

12

12

12

12

12

12

12


CERNOpto-

Rx Analogue/Digital


VMEInterface

XilinxVirtex-IIFPGA

Data Suppression

9U VME64x Form Factor

Analogue: 96 ADC channels (10-bit @ 40 MHz )

@ L1 Trigger : processes 25K MUXed silicon strips / FED

Raw Input: 3 Gbytes/sec*

after Zero Suppression...

DAQ Output: ~ 200 MBytes/sec*(@ L1 max rate = 100 kHz)

Digital Processing

Flexible Digital Logic:

Xilinx Virtex-II FPGAs 40K->3M gates*FPGAs programmed in VHDL & VERILOG


TTC

TempMonitor

JTAG

9U VME64x


CMS Tracker FED Board Layout Primary Side

•Complex board

•Pushing density limits on large board for cost

•Opto & Analogue & Digital issues

•Brings together considerable INS Dept expertise and experience from several areas into one project


CMS Tracker FED Board Layout Primary Side

View thru the board

Double sided


FEDv1 Front-End module Primary Routed

Note: FPGA de-coupling

OptoRx ADCs 40K FPGAs 1500K FPGAOpAmps

12 optical channels Dense circuitry


DAQFront-endReadout

LinkFRL

DAQ MezzanineCard

Transition Card

S-LINK64

CMS Tracker FED Transition card option

TTCrx


Buffers

12

12

12

12

12

12

12

12

TTC

Due to mechanical constraints place DAQ link card on Transition card

Very simple card


CMS Tracker FED Crate Layout (illustration)

• 440 Boards 96 ADC/Board

• 24 Crates

• 8 Racks

• 440 Boards 96 ADC/Board

• 24 Crates

• 8 Racks

• 40 K ADC Channels 10 Bit@40MHz• Max Trigger Rate 100 kHz• Input Rate 1.5 T Byte/s• Output rate 25 Gbyte/s/%

• 40 K ADC Channels 10 Bit@40MHz• Max Trigger Rate 100 kHz• Input Rate 1.5 T Byte/s• Output rate 25 Gbyte/s/% 4 TTC Partitions

DAQFED


CMS Tracker FED INS Main Deliverables (Project Spec)

Includes

•Design of 9U FEDs. Electrical tests of design.

•Design & Testing Baseline FPGA Firmware.

•Production of prototypes for CMS silicon tests.

•Production and Testing of ~ 500 FEDs (incl spares.)

•Transition cards for DAQ links (tbd)

•Low level driver software.

•Full Documentation.

•Communication with CMS groups

•Assistance during Tracker installation & commissioning.

Excludes

•OptoRx design & testing.

•Opto Test equipment for FEDs.

•DAQ link mezzanine card.

•High level software.

•Provision of VME crates and cabling.

•Other modules in FED VME crates e.g. CPUs.

FED project institutes: •RAL INS

•RAL PPD

•Imperial College

•Brunel


CMS Tracker FED Future Team Members in INS

Core Team

•John Coughlan

Project Manager ; Stakeholders contact

Software

•Saeed Taghavirad

Hardware Design ; Firmware

•Ivan Church

Test Engineer ; Firmware

Plus

•Ed Freeman

Firmware

James Salisbury

Analogue Design

•Rob Halsall

Technical Advisor

•Chris Day

PCB Design

Others

Procurement

Test Section

Moving from Design phase to Test phase…

Principal responsibilities are shown but

most personnel have flexible/multiple skills.

Several with past experience on similar projects.


CMS Tracker FED Project Status

1998->2001•FED-PMCs x 60 produced for CMS Silicon module testing.•Small nr channels with limited functionality.

•Cheap. Very flexible. Plug in PC or use in VME crate.

•Very popular with CMS! (and others)

•2001•Final 9U FED design started in earnest.

•Requirements document produced.

•Algorithms studied. Board interfaces investigated.

•Choice of (NEW) FPGA family made. (Cost critical)

•Cluster finding algorithms implemented in Firmware.

2002•Implementation of design; 96 chan ADC on 9U.

•Analogue design complete (OptoRx finalised by CERN).

•DAQ interface baseline decided.

•Basic simulations and test bench measurements made.

•Board Design complete. Components chosen. Board layout & Routing.

•Final Review carried out. Manufacture of first 2 x FEDv1s imminent.

•Firmware for Event Building and VME continuing in parallel. Software design started.

FED-PMC (FED prototype)


CMS Tracker FED FEDv1 Status

VME-FPGA

TTCrx


Buffers

FPGAConfiguration

PowerDC-DC

DAQInterface

12

12

12

12

12

12

12

12


CERNOpto-

Rx Analogue/Digital


VMEInterface

XilinxVirtex-IIFPGA


TTC

TempMonitor

JTAG

9U VME64x

Board StatusHardware:Analogue Design CompleteDigital Design CompletePower Design CompleteFirmware:Cluster Finding CompleteEvent Building 60% DoneVME Interface 25% DoneBoard:Layout & Routing Front-End Module

CompleteBack-End Layout doneRouting 90% done

Manufacture Tape out to PCB week 47Testing Start at RAL week 51


CMS Tracker FED INS Dept Outline Tasks

02/03•Manufacture 2 x FEDv1.

•Complete VME Firmware (EF) [4 months]

•Complete BE Firmware (ST) [4 months]

•Basic board tests Power/JTAG/Analogue (JS, IC) [3 months]

•Primitive Driver Software (JC) [4 months]

03/04•Test of FEDv1 Design: (by Designers)

•Advanced board tests (Opto tests @ IC) (IC, ST) [6 months]

•Firmware (Front End & Event readout) (EF, ST) [6 months]

•VME Interface (EF) [6 months]

•DAQ interfaces (with Imperial) (ST, JC) [6 months]

•Transition board (ST) [3 months]

•Driver Software (JC) [6 months]

•Production of 20 FEDv1s for CMS tracker testing. (IC, ST, JC) [6 months]

04/05

•Pre-Production and Test of 20 x FEDv2. (ST,IC,JC) [12 months]

•Procurement for 500 FEDs. (JC) [9 months]

•Set up for “automated” Production Tests. (IC,ST,JC) [12 months]

05/06

•Manufacture of 500 x FEDv3. (ST,JC) [12 months]

•Production Testing @ RAL (IC +) [12 + months]

•Assist Installation & Commissioning @ CMS. (JC,ST) [6 months]

06/07

•Complete commissioning @ CMS. (JC,ST) [9 months]

•Operation


CMS Tracker FED Schedule

•Tracker/FED installation schedule delayed by ~ 9 months.•FED production & installation now expected in 2005

Design TestProduction& InstallationPre-ProNOW->

FEDv3FEDv2FEDv1


CMS Tracker FED INS Dept Effort

•Approx. 6 months behind original FED schedule. (but Tracker installation is also delayed).•Spent more time and effort getting design of FEDv1 close as possible to Production requirements.•(OptoRx and DAQ link specifications came later than expected).•Gross manpower figures are still reasonable. But no slack.•Profile needs adjustment.

Design Test Production& Installation

Includes APV effort~ 2 SY integrated


CMS Tracker FED INS Dept Effort Year by Year (FYs)

Surname First Name Band Group Section 01/02. 02/03. 03/04. 04/05. 05/06. 06/07. Totals Main ResponsibiliesChurch Ivan 5 ESSG TEST 0.00 0.20 0.70 1.00 1.00 0.25 3.15 Test Engineer Final FED ; FirmwareCoughlan John 4 RTSG DAQ 0.90 0.90 0.90 0.90 0.90 0.50 5.00 Project Manager ; Software ; FirmwareDay Chris 5 ESSG DO 0.50 0.70 0.25 0.30 0.15 0.00 1.90 Drawing OfficeFreeman Edward 6 ESDG DAQ 0.50 0.70 0.60 0.00 0.00 0.00 1.80 FEnd & VME FPGA FirmwareGannon Bill 4 MEDG DAQ 0.50 0.00 0.00 0.00 0.00 0.00 0.50 FEnd FPGA FirmwareHalsall Rob 3 ESDG DAQ 0.60 0.50 0.25 0.25 0.10 0.00 1.70 Technical AdvisorSalisbury James 5 RTSG DAQ 0.60 0.70 0.00 0.00 0.00 0.00 1.30 Analogue & Power designShah Azmat 5 ESDG DAQ 0.50 0.00 0.00 0.00 0.00 0.00 0.50 BEnd FPGA designTaghavirad Saeed 5 ESDG DAQ 0.50 1.00 1.00 1.00 0.70 0.25 4.45 Board design & BEnd FPGA FirmwareOthers 1.00 0.25 0.25 0.25 0.25 0.00 2.00 Other effort

Total 5.60 4.95 3.95 3.70 3.10 1.00 22.30


CMS Tracker FED Conclusion & Comments

INS has an excellent track record in delivering fully tested large scale electronic systems which are maintainable for the long operational lifetimes of Particle Physics experiments such as CMS.

Future APV effort minimal (consultancy)

Gross INS effort projections for FED stand given following assumptions...

•Effort profile is adjusted to reflect schedule changes (incl carry overs).•No major re-design of FEDv1 to FEDv3 for production.•High quality of Board manufacture.•No extra deliverables from CMS (e.g. Firmware algorithms, OptoRx spec changes).•No further schedule slippage (installation in 2005) •Personnel with key skills (eg. FPGA firmware) stay with project.•Assistance in automated FED production testing.•Absolute minimial support for FEDs for CMS tracker module testing.•Level of Installation & commissioning support from INS to be agreed.

NB No contingency is included (e.g. for staff turnover).

•INS has an excellent working relationship with Imperial College and RAL PPD•Many tasks are shared.

instrumentation department john coughlan rutherford appleton laboratory14 november 2002 cms tracker...

Documents