daq issues for the 12 gev upgrade coda 3. a modest proposal… replace aging technologies run...

DAQ Issues for the 12 GeV UpgradeDAQ Issues for the 12 GeV Upgrade

CODA 3CODA 3

A Modest Proposal…A Modest Proposal…

Replace aging technologiesReplace aging technologies Run ControlRun Control Tcl-Based DAQ componentsTcl-Based DAQ components mSQLmSQL

Hall D Requirements drive developmentHall D Requirements drive development Don’t forget the Don’t forget the littlelittle guy guy Maintain cross-platform compatibilityMaintain cross-platform compatibility Support new commercial hardware Support new commercial hardware

advancesadvances

HALL DHALL D

ExistingHalls

HALL D - RequirementsHALL D - Requirements Pipelined Electronics (ADC, TDC)Pipelined Electronics (ADC, TDC)

Dead-timeless systemDead-timeless system 200 KHz L1 Trigger200 KHz L1 Trigger

Parallel/Staged Event BuildingParallel/Staged Event Building 1 GByte/s aggregate data throughput1 GByte/s aggregate data throughput

L3 Online FarmL3 Online Farm x10 reduction in data to diskx10 reduction in data to disk

Storage Management Storage Management Time ordering?? of built events (at 15-20 kHz)Time ordering?? of built events (at 15-20 kHz) 100 MB/s --> 8 TB/day --> 2.5 PB/year100 MB/s --> 8 TB/day --> 2.5 PB/year

Front-EndFront-End Trigger rate - 200 KHzTrigger rate - 200 KHz

Block up Events (200 event block -> 2kHz)Block up Events (200 event block -> 2kHz) Move some ROL runtime code to modules (FPGAs)Move some ROL runtime code to modules (FPGAs) ADCs provide L1 trigger data (need a distributed high speed ADCs provide L1 trigger data (need a distributed high speed

clock - 62.5 MHz)clock - 62.5 MHz) High speed switched serial interconnect (4Gbit/s links) between High speed switched serial interconnect (4Gbit/s links) between

ModulesModules & & CratesCrates needed for L1 trigger needed for L1 trigger Commercial solution - VXS ??Commercial solution - VXS ??

Form-factor (VME64X Crates)Form-factor (VME64X Crates) New commercial bridge (TSi148) supports 300MB/s on existing New commercial bridge (TSi148) supports 300MB/s on existing

backplanesbackplanes Support other Hall DAQ applicationsSupport other Hall DAQ applications

New Trigger SupervisorNew Trigger Supervisor Perhaps 100+ cratesPerhaps 100+ crates Support pipeline, event blockingSupport pipeline, event blocking

Level 1 TriggerLevel 1 Trigger

• Distributed high speed clock (62.5 MHz ?)• Subset of ROCs collect sampled ADC data and send it to L1 Trigger in sync• 12 bit sums/crate x 250MHz --> 3 Gbit/s links• Trigger decision goes to Trigger Supervisor

Front-EndFront-End 200 KHz200 KHz

Block up Events (200 event block -> 2kHz)Block up Events (200 event block -> 2kHz) Move some ROL runtime code to modules (FPGAs)Move some ROL runtime code to modules (FPGAs) ADCs provide L1 trigger data (need distributed high speed clock - ADCs provide L1 trigger data (need distributed high speed clock -

62.5 MHz)62.5 MHz) High speed switched serial interconnect (4Gbit/s links) needed for L1 High speed switched serial interconnect (4Gbit/s links) needed for L1

triggertrigger Commercial solution - VXS ??Commercial solution - VXS ??

Form-factor (VME64X Crates)Form-factor (VME64X Crates) New commercial bridge (TSi148) supports 300MB/s on existing New commercial bridge (TSi148) supports 300MB/s on existing

backplanesbackplanes Support other Hall DAQ applicationsSupport other Hall DAQ applications

New Trigger SupervisorNew Trigger Supervisor Perhaps 100+ cratesPerhaps 100+ crates Support pipeline, event blockingSupport pipeline, event blocking

VME64X - VXS InterconnectVME64X - VXS Interconnect

J0 - 200 total pins 45 differential pairs 6 GHz Bandwidth

18 VME Payload Slots

2 Switching slots

Staged/Parallel Event BuildingStaged/Parallel Event Building

• EMU built around the ET system for customizable processing

• Data Concentrator for ROCs• Sub-Event builder• Farm distribution point• Event Recorder• User Processes can attach to any EMU in the system

L3 FarmL3 Farm

Can be used for analysis Can be used for analysis or filteringor filtering

Support 100s of nodesSupport 100s of nodes Nodes can come and go Nodes can come and go

during event takingduring event taking Do other experimental Do other experimental

halls need this?halls need this? Do filtered events need Do filtered events need

to be time ordered? to be time ordered? 1 GB/s

100 MB/s

RunControl / Monitoring / Slow ControlsRunControl / Monitoring / Slow Controls

First generation java agent RunControl is hereFirst generation java agent RunControl is here Robust fault toleranceRobust fault tolerance Process abstraction through COOL languageProcess abstraction through COOL language

Integration of foreign processesIntegration of foreign processes DP, vxServer, shellsDP, vxServer, shells EPICS, CAEN OPC comingEPICS, CAEN OPC coming Move toward full integration of Slow ControlsMove toward full integration of Slow Controls

Web Interface for remote monitoringWeb Interface for remote monitoring Extended and customizable graphing and DAQ Extended and customizable graphing and DAQ

system monitoring capabilitiessystem monitoring capabilities Backwards compatible - Start using it!!Backwards compatible - Start using it!!

Other IssuesOther Issues Remember the Little guy (1 CPU systems)Remember the Little guy (1 CPU systems)

PCI Based DAQ on LinuxPCI Based DAQ on Linux Support PCI interfaces to CAMAC/VMESupport PCI interfaces to CAMAC/VME Support other commercial hardware Support other commercial hardware

Maintain cross platform compatibilityMaintain cross platform compatibility SUN, LINUX, VxWorksSUN, LINUX, VxWorks 64bit Arch - Athelon, Opteron, G5 (Mac X)64bit Arch - Athelon, Opteron, G5 (Mac X) Embedded Linux (on SBCs)Embedded Linux (on SBCs)

Move to database independenceMove to database independence Proxy Server (JDBC) to support User’s database choiceProxy Server (JDBC) to support User’s database choice

User Hooks into the DAQ systemUser Hooks into the DAQ system JAVAJAVA Updated Tcl support ??Updated Tcl support ??

SummarySummary

CODA version 3 is now being molded - CODA version 3 is now being molded - nothing is irreversible, Yet! nothing is irreversible, Yet!

Our hope is to “phase in” new tools to Our hope is to “phase in” new tools to provide a smooth transition from provide a smooth transition from CODA2 -> CODA 3CODA2 -> CODA 3

The DAQ Group is small - we welcome User The DAQ Group is small - we welcome User input and contributions…input and contributions…

Extra slidesExtra slides

Pipelines (Dead-timeless DAQ)Pipelines (Dead-timeless DAQ)

250MHz

10µs “snapshot” can be stored in memory (5KB/FADC)

A Trigger generates a lookback and extraction of the sampled ADC data