the alice data-acquisition software framework date v5 f. carena, w. carena, s. chapeland, r. divià,...
TRANSCRIPT
The ALICE Data-Acquisition Software Framework DATE V5
F. Carena, W. Carena, S. Chapeland, R. Divià, I. Makhlyueva, J-C. Marin, K. Schossmaier, C. Soós, P. Vande Vyvre, A. Vascotto
for the ALICE collaboration
CHEP0613 – 17 February 2006T.I.F.R. Mumbai, India
CHEP06, 13 - 17 February 2006
2ALICE DAQ Software Framework DATE V5
ALICE Experiment
Counting
rooms
Cavern
Computing Center
DDLs
10Gb Ethernet
Machine: LHC @ CERN, Point 2ALICE detectors: 17Online Systems: ECS, DAQ, TRG, HLT, DCS Readout: ~400 DDLs @ 2125 MbpsStorage rate: max. 1.25 GB/sEvents: 2.5 MB (p-p), 87 MB (Pb-Pb)DAQ Software: DATE V5
DAQ
CHEP06, 13 - 17 February 2006
3ALICE DAQ Software Framework DATE V5
DATE V5 Software
ALICE Data Acquisition and Test Environment (DATE): software framework for the ALICE DAQ system
Latest DATE version V5: for the test and commissioning phase (Hall SXL2)
DATE V5.x kits:RPM package (~20 MB), User’s Guide (~470 pages)
Platforms:IA32 family, SLC3 using a 2.4 kernel
Development:~20 packages, mostly in C, using CVS
Auxiliary Software:DIM, SMI, MySQL, Tcl/Tk, libshift, ROOT, VMEbus driver
Testing:reference system, on-line data challenges, test beams
DATE V5
Packages
CHEP06, 13 - 17 February 2006
4ALICE DAQ Software Framework DATE V5
Architecture
CTPFEEs
HLT Farm
DDLs
TTC
CTP = Central Trigger Processor
TTC = Trigger, Timing, and ControlFEE = Front-End Electronics
HLT = High-Level Trigger
DDL = Detector Data Link
StorageNetwork
PDS
TDS
10Gb Ethern
et
TDS = Transient Data Storage
PDS = Permanent Data Storage
SoftwareFramework
DATE V5
LDCs
GDCs
Event-BuildingNetwork
Servers
LDC = Local Data Concentrator
GDC = Global Data Collector
~200
~50
~10
CHEP06, 13 - 17 February 2006
5ALICE DAQ Software Framework DATE V5
LDC Dataflow
equ. header
LDC header
equ. header
sub-event
payload
CDH
fragment
fragment
fragment
LDCs
DDLs
payload
CDH
fragment
payload
CDH
fragment
readout
recorder
edmAgent
htlAgent
readout process:- an “equipment” handles one data source- equipment library for DDL, VMEbus, etc.- merges fragments into sub-events - checks the Common Data Header (CDH)
recorder process: - streamlines sub-events- writes sub-events onto local disks - sends sub-events to GDCs
edmAgent process:- dispatches the sub-event to the GDCs- guided by the Event Distribution Manager (EDM)
htlAgent process:- handles the HLT decision and data- instructs to drop/keep the sub-event
32 bytes: - orbit number- bc number- trigger class- FEE status/error
68 bytes:- size- event ID- event type/id- timestamp- LDCGDC
24 bytes:- size- equipment type/id
CHEP06, 13 - 17 February 2006
6ALICE DAQ Software Framework DATE V5
DDL Interfaces of an LDC
LDCs
CTPFEEs
HLT Farm
DDLs
TTC
CTP LDC: - CTP readout (L2a information)- Interaction Record (bunch crosssings orbit)
FEE LDC: - LDC dataflow
LDC FEE:- data blocks and commands to the FEE- Front-end Control and Configuration (FeC2) scripts- C API by the rorc library
FEE HLT Farm:- send a copy of the incoming fragments- performed by the DDL hardware
HLT Farm LDC:- trigger decisions- compressed data- results of HLT processing
CHEP06, 13 - 17 February 2006
7ALICE DAQ Software Framework DATE V5
GDC Dataflow
LDC header
sub-event
full event
GDCs
LDC header
sub-event
LDC header
sub-event
GDC header
sub-event
sub-event
sub-event
68 bytes:- size- event ID- event type/id- timestamp- GDC
eventBuilder
mStreamRecorder
eventBuilder process:- handles incoming sub-events- assembles the full events- guided by event-building rules + LDC headers- tells its load to the EDM
mStreamRecorder process: - get full events via a memory mapped scheme- records with multiple stream- data formats: raw binary, ROOT tree- transmission protocol: standard, RFIO, ROOTd
CHEP06, 13 - 17 February 2006
8ALICE DAQ Software Framework DATE V5
Control
LDCs
GDCs
Servers
rcServer
rcServer
runControl
rcServer process:- launched by the xinetd daemon- starts/stops DATE processes- gets commands via DIM- shared memory control region
runControl process:- one per data acquisition; several are allowed- interface to get commands from operator or ECS
logic engine process:- hosts finite state machines (SMI)- communicates with the rcServer processes (DIM)
runControl, Status display, SMI
display Experiment
Control System
(ECS)
logicEngine
CHEP06, 13 - 17 February 2006
9ALICE DAQ Software Framework DATE V5
Static Database
Servers
Roles: LDC, GDC, EDM, detectors, trigger mask Trigger: relationship detector and trigger mask Detectors: composition of detectors/sub-
detectors Memory banks: type, size, name per role Event building: policies for build, no-build, partial Readout equipments: type, parameter per LDC
MySQLor
text files
graphical tools for
configuration configuration
files >EQTYPES
>Rand 1 GENDATA
EvMinSize %ld
EvMaxSize %ld
EqId %hd
triggerPattern %hd
>Timer 4 TRIGGER
EvInterval %ld
>GLOBALPARAMS
>LDCS
>pcald10
+ Timer (TimerAsTrigger)
1000
+ Rand (1stRand)
10000 10000000 13 4
>pcald12
...
CHEP06, 13 - 17 February 2006
10ALICE DAQ Software Framework DATE V5
Monitoring statsBrowser + infoLogger:
- collect and display statistics of the runs- Web access
infoBrowser + infoLogger:- collect log messages from LDCs, GDCs, servers - display, search, export log messages
AFFAIR:- performance plots of DATE specific values - utilization of system resources- Web access
MOOD:- monitoring data quality- online/offline monitoring- local/remote monitoring- plug-in ROOT modules- based on the DATE monitoring library
CHEP06, 13 - 17 February 2006
11ALICE DAQ Software Framework DATE V5
Performance
LDC with 4 DDLsSLC3 + DATE V5.13ALICE Silicon Pixel Detector Max. event rate: 3500 Hz
Example: Setup for integrating detector electronics
with the DAQ
0
100
200
300
400
500
600
700
0 50000 100000 150000 200000 250000 300000 350000 400000
event size [Bytes]
throughput [MBytes/s]
4x DDL
3x DDL
2x DDL
1x DDL
CHEP06, 13 - 17 February 2006
12ALICE DAQ Software Framework DATE V5
Conclusions
DATE V5: - covers the whole range of DAQ tasks for ALICE- satisfies the functional + performance requirements- runs on single machine and large-scale setups- is available as kits (RPM package) - user’s guide: ALICE-INT-2005-015- regular training courses - is tested at reference system, data challenges, test beams - is ready for the test and commissioning phase- homepage: http://cern.ch/alice-daq
Future work (DATE V6):- port to 64-bit platforms- port to SLC4 using a 2.6 kernel- electronic logbook- Transient Data Storage Manager (TDSM)- new upcoming requests
Thank you!