Download - DAQ Overview
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 1
DAQ OverviewDAQ Overview
o Terminology
o Requirements
o Front End Electronics
o Trigger System
o DAQ Hardware and Software
o Event Building and Data Flow
o Interfaces: Monitoring, Controls, G4MICE, and Configuration DB
o Project Plan: Human resources and Time Line
Jean-Sebastien Graulich, Geneva
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 2
DAQ Terminology
ISIS / MICE / DATE Jargon Isis Cycle:
The injection and acceleration cycle of ISIS. It is 20 ms long (50 Hz). Machine Start or MS:
This is the pulse used for the ISIS synchronization Spill or Spill-Gate:
The time window during which the MICE Target is crossing the ISIS beam. The Spill cycle is driven by the cycle of the MICE target
Burst: The ~100 ns time window during which muons can be expected in the MICE
detectors. It corresponds to the time it takes for a proton bunch in ISIS to cross through the MICE target
DAQ-Trigger: Signal triggering the readout of the FE-electronics modules of the MICE detectors.
By definition, one DAQ-Trigger corresponds to one DAQ-Event Particle-Trigger:
Signal generated when the desired Trigger Condition is met. It is distributed to the sub-detectors Front End Electronics and initiates the digitization of the data therein.By definition, a Particle-Trigger corresponds to one Particle-Event
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 3
RF Cycle
1 secPULSE !
RF Pulse1 ms
20 ms
Isis Beam Radius
MS
RF Trigger
Target Position
SPILL !
SoS
DAQ gate
EoS~1 ms
DAQ Trigger
DAQ Event !
Particle Events !
“GoodEvent”
(TOF0xTOF1xTOF2)
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 4
100 ns
Digitisation Dead time
Particle Trigger
Bursts ! IsisMicrostructure
224 nsShould be ~500 ns
1700 ns on average
Next Particle Trigger
SoS
DAQ gate
EoS~1 ms
DAQ Trigger
DAQ Event !
Particle Events !
“GoodEvent”
(TOF0xTOF1xTOF2)
RF Pulse1 ms
20 ms
Isis Beam Radius
MS
RF Trigger
Target Position
SPILL !
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 5
Requirements
See the full list of system requirement in MICE Note 222.
The main requirement is on the rate:
3.2.1 The DAQ system should allow acquiring data for up to 600 Particles in a 1 ms spill repeating at 1 Hz.
Important consequencesDetector data Readout must be performed at the end of the spill
Data has to be buffered in the FEE
This is because the readout of 1 particle event takes several 100 This is because the readout of 1 particle event takes several 100 µµs…s…
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 6
Digitization dead time4.3.4 The DAQ dead time after a Particle
Trigger should not exceed one burst after the one that generated it.
Average Time between 2 muons is 1.7 µs Conversion time for conventional ADC > ~3 µs
Critical for EMCal and TOF Solution: Flash ADC after Signal stretching
Even if the event buffer was large enough, conventional ADC
can not collect 600 muons/ms
Requirements (2)
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 7
Charge Measurement in MICE (except Tracker)
100 MHz Flash ADCs (CAEN 1724)
Advantages No Splitter, no delay cable, no discri, no TDC ! Cabling very easy
Need custom made shaper Strong impact on data size
2 ns rise time
ShaperVthr
tthr >30 ns rise time
By fitting the rising edge, time resolution is much better than the 10 ns of the sampling rate (~1 ns obtained).
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 8
Requirements (3)
Event Size DAQ Event Size is quite large because it
corresponds to 600 muons
3.2.3 The DAQ system should be able to deal with DAQ event size up to 60 MB (up to 10 MB per sub-event)
Means ~ 60 MB/s data transfer rate (readout between spills)
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 9
Requirements (4)
Stability and Reliability Flexibility Partitioning
Following MICE Installation plan
Allow calibration events between spills
AND maintainable by a single person…
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 10
Front End ElectronicsFront End Electronics
o TOF
o KL
o CKOV
o Tracker
o EMR
o Target Position and Beam Loss
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 11
TOF FEE Sensor: PMT (Hamamatsu R4998)
Signal transmission: Single ended, 50 Ohm, coax cable (RG213)
Number of Channels 40 (TOF0) + 28 (TOF1) + 40 (TOF2) = 108 ch
Main Constrain Time Resolution -> Time-walk correction
TDC CAEN V1290, 32 ch Large Event Buffer ECL input
Discriminator Lecroy 4415 16 Channels ECL output Twisted Pair Input (110 Ohm)
Need Signal splitting for charge measurement
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 12
KL FEE
Sensor: PMT (Hamamatsu R1355) Signal transmission: Differential, 120 Ohm, Twisted
pair cable
Number of Channels 42 ch
Main Constrain Charge measurement Time ~ 1 ns
Flash ADC (WFD) CAEN V1724 100 MS/s, 14 bits Best commercial deal Single Ended, 50 Ohm
Input
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 13
Shaper development
Splitter designed in Geneva (P. Bene) Passive splitter, shielded, impedance matched (minimal signal loss) 16 x Lemo, 50 Ohm Input 2 twisted pair outputs, 34 contacts connectors,
110 Ohm to Discriminator120 Ohm to Shaper
Shaper designed in Sofia (I. Rusinov) 120 Ohm Twisted Pair Input, 34 contacts connectors 16 Lemo, 50 Ohm, single-ended outputs to fADC Single-Ended or Differential mode jumper selectable
P-Trg Signal
TOF PMT
DISCRI
ShaperSE Mode
TDC
Trigger LogicSplitter
fADC
BNC Cable50 Ohm
KL PMTTwisted pair Cable
ShaperDiff Mode
fADC
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 14
Shaper and Splitter
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 15
F(t) = offset – norm (0/(0-)) [ (e-(t-t0)/-e-(t-t0)/0) + (t-t0)/ e-(t-t0)/]
Shaper Output
V1724 - Seq
norm 264.6 2236 0.839 6.177 0.321 1.666t 0.33 27.48offset 7.1 8194
0 20 40 60 80 100
8200
8400
8600
8800
9000
9200
9400
Need for better tuning of baseline restorerUsed for individual
baseline evaluation
- Signal shape well understood- Signal shape well understood- Time resolution better than 1 ns- Time resolution better than 1 ns
Time (sample)
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 16
CKOV FEE
Sensor: PMT (8”) Signal transmission: Single ended, 50 Ohm,
coax cable (RG58) Number of Channels
4 + 4 = 8 ch Main Constrain
Rate (no segmentation) Small charge
Flash ADC (WFD) CAEN V1731 500 MS/s, 8 bits Single Ended, 50 Ohm
Input No Shaper !
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 17
Tracker FEE
4 VLSB Boards in VME Crate
= Data Buffer
MICE HALL
1 Cryo-Cooler
= ½ Tracker
4 AFE2-t
Boards
Fibers from VLPCs
Digital Signal
MIL1553(Control)
Read out PC
In Control Room
Optical Fiber
Custom Made Digital Data Buffer VLSB = VME LVDS CERDES Buffer MICE defined Data format Measure Both discriminated time and Zero
suppressed Charge 4096 + 4096 = 8198 channels
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 18
EMR FEE
Still in prototyping phase 50 layers x 59 bars = 2950 channels Digital information only
only -> 2950 bits (100 words = 400 bytes) In custom made VME buffer (similar to
tracker)
Charge measurement per layer only 50 fADC channel Direct connection, similar to CKOV
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 19
Target Monitoring FEE
The Target system records Target dip trajectory ISIS Beam loss monitors analog signals
Duplicate data flow CAM DAQ
Max 8 channels Read out once per spill
Nuclear Instrument PCI I/O card PCi6254
in target PC Marginal in term of data size
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 20
Data Size TOF TDC: Maximum 108 hits, 4 bytes/hit
432 bytes/pt TOF fADC: 60 samples per channel
13 kBytes/pt KL fADC: 60 samples per channel
6 kBytes/pt CKOV fADC 300 samples per channel, 1
byte/sample 2.4 kB/pt
Tracker: (w/o zero suppression) 5536 bytes per tracker/pt
10.8 kB/pt TOTAL w/o EMR:
~33 kB/pt 16.5 MB/Spill Maximum 16.5 MB/s
Electron Muon Ranger (Coming up after spring 2010) Digital: 0.4 kB/pt or 2 kB/pt if TDC information fADC: 300 samples/ch, 1 byte/sample: 15kB/pt TOTAL for EMR: 21 kB/pt -> 10.5 MB/Spill -> 10.5 MB/s
TOTAL at completion ~ 30 MB/s
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 21
Trigger SystemTrigger System
o Particle Trigger
o Timing
o Logic
o DAQ Trigger
o Spill Gate
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 22
Particle Trigger Triggers the digitization of the signals arriving at
the Front End Electronics Should arrive a few s around the signal Distributed to all FEE boards We expect about 500 Particle triggers for 1 DAQ trigger
The timing of the trigger should be given by the burst
Delay TOF0, TOF1 TOF2 such that they arrive approximately at the same time in the trigger logic
Make the TOF logic pulses ~200 ns long Make the Burst Gate narrow and Delay it such that it
arrives more than 100 ns after the TOF signals All single raw time distribution will be ~ 100 ns
wide Problem for beam commissioning !
Protons have very different timing Will use TOF0 timing until stable muon running is
established
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 23
Particle Trigger timing
TOF1
Burst Gate
TOF0
TOF0 TOF1 Burst Gate
200 ns
GVA1 hit
Burst Gate (made narrow)
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 24
TOF0 LogicTOF0 Logic
TOF0 has 40 TOF0 has 40 channelschannels
10 horizontal slabs 10 vertical slabs 2 PMts per slab
Because of hardware constraints,We also defineTOF0_H: OR over 8 central H slabsTOF0_V : OR over 8 central V slabsTOF0_EXT: OR over the 4 frame slabs
TOF0 signal is the OR over all the slabs of the AND of the two pmts of each slab
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 25
TOF0 Logic TOF0 Logic implementationimplementation
TOF1 will be included in the trigger in a similar way
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 26
Trigger ElectronicsTrigger Electronics
Crate LayoutCrate Layout DAQ Trigger distribution
Scalars and particle trigger NIM Logic
TOF discriminators and trigger CAMAC Logic
Interfacew/Target &Spill Gate
ShapersGVA Discri,KL cosmics trgTOF & CKOV
KL
MICE Ready
Target Ready
RF Ready
DAQ Ready
Gated Machine Start
Spill Request
Target Trigger
Protons on target
RF Trigger
RF Power
DT Gate
DAQ Trigger
Target Delay
RF Delay
DT Delay
20 ms Extraction
Validated Machine Start
MICE Systems synchronization
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 28
DAQ Trigger
Four types of DAQ Four types of DAQ triggerstriggers
- PHYSICS- PHYSICS
- CALIBRATION- CALIBRATION
- Start of Spill- Start of Spill
- End of Spill- End of Spill
Event Type is tagged in Event Type is tagged in the data headerthe data header
Used to impose a well-Used to impose a well-defined sequence of defined sequence of events and online events and online check of check of synchronisationsynchronisation
Fixed delay ~ few ms
Fixed delay
Software Check: No overlap with SOS busy (otherwise stop with error)
Depends on Data Size ~ 1 s
VMS
Particle Triggers
Fixed width ~ 1 ms
Physics Event DAQ Trigger
Fan out to LDCs
Fan out to LDCs
Fan out to LDCs
EOS Trigger
SOS Trigger
DAQ Idle = DAQ Ready
DAQ trigger distribution
Target Trg
Trigger ReceiverConnections
V977
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Common toall LDCs
Central trigger only
All PE Busy
All CALIB Busy
All SOS Busy
All EOS Busy
DT Gate
PE DAQ Trigger
CALIB Trigger
SOS trigger
EOS Trigger
PE Local Busy
CALIB Local Busy
SOS Local Busy
EOS Local Busy
DAQ Idle
2 spares
In Out
Ireg0
Oreg0 = Ireg1
Oreg1
Reset by software as soon as the trigger is seen
Reset by software as soon as the readout is done
CAEN V977
Channels 0-7 configured as Flip-Flop, reset by software
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 31
DAQ Spill GateDAQ Spill Gate
Detailed documentation of the procedure to adjust Detailed documentation of the procedure to adjust the DAQ Spill Gate is availablethe DAQ Spill Gate is available
MSD
R1-N1-M3-P1 End Mkr
Particle TriggerRequests
Acceleration
DAQ Spill Gate
Target DepthInjection
Extraction
R1-N1-M4-P1 End Mkr
Adjust to tune the DAQ Spill Gate width
Normaly constant delay (DAQ Spill timing)
Machine start after delay in the target system
Spill Gate is always aligned to extraction
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 32
Machine
StartDAQ Gate ~ 3.5ms
Hits in GVA1
DAQ Spill Gate SyncDAQ Spill Gate Sync
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 33
DAQ Hardware and Software
o Hardware overview
o Software Framework
o Event Building
o Data flow
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 34
D-DAQ Hardware Overview
RAL Net
MICE Main 1GB Switch
Control & Monitoring
100 MB Switch
MICE DAQ 1GB Switch
User PCs and Wi-Fiin Local Control Room
EPICS network for MICEControl and Monitoring (MCM)
Trigger distribution
Trackers EMCal TOF Trigger + Ckovs
Event Builder(GDC)
Run Control
Dedicated Link to Remote Mass Storage
VME Crates
Optical links
Linux PCs(LDCs)
MCMReadout
Data Saver
Online Monitoring
MICE Private Subnet
RAL Net
MICE Main 1GB Switch
Control & Monitoring
100 MB Switch
MICE DAQ 1GB Switch
User PCs and Wi-Fiin Local Control Room
EPICS network for MICEControl and Monitoring (MCM)
Trigger distribution
Trackers EMCal TOF Trigger + Ckovs
Event Builder(GDC)
Run Control
Dedicated Link to Remote Mass Storage
VME Crates
Optical links
Linux PCs(LDCs)
MCMReadout
Data Saver
Online Monitoring
MICE Private Subnet
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 35
Software Framework MICE has decided to use the ALICE DAQ
software framework The ALICE DAQ software framework, DATE, will
provide us with the necessary EventBuilder tool:Subevents collected by different processors have to be synchronized and put together before storage
DATE has much more functionalities than what MICE needs
Previous Experience (from HARP) Experiment could be used
Agreement with ALICE collaboration included 6 months hands on training with real contribution to the software (2006)
Currently using DATE version 6.05 (20/06/2007)
Historical reasons Upgrade in July and then froze it until the end of the
experiment
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 36
DATE Vocabulary
LDC : Local Data Concentrator The PC connected to the VME crate via the PC-
VME Interface
GDC : Global Data Collector Event Builder
Event DATE Event = DAQ Event !!!
It contains data for several Particle Events (~500)
Event Type Tag attached to the event depending on which
trigger receiver‘s input has been used
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 37
DATE Readout Process
Two processes running in each LDC The readout process waits for a trigger, reads out the
front-end electronics, and fills a FIFO buffer with the sub-event data
The recorder process off-loads the FIFO and sends the sub-event data to one (or several) GDC over the network
Each LDC contains a set of Equipments Equipment =~ 1 Vme board (in MICE) Each equipment has its own set of routines for its
initialization and readout. Adding an equipment is done without recompiling all
DATE Equipment configuration data is saved in MYSQL
database (but not archived)
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 38
DATE Readout Algorithm
5 user routines have to be implemented
(XXX is the name of the equipment) ArmHwXXX
Executed at the beginning of the Run
Allows initialization of the board AsynchReadXXX
Executed constantly even when there is no trigger
Don’t use ! EventArrivedXXX
Used only if the equipment needs to trigger the readout ( Trigger Receiver)
ReadEventXXX That is the readout itself
DisArmHwXXX Executed at the end of the Run
General algorithm for equipment readout:
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 39
Readout Code
Available and tested for TDC V1290A (TOF) FADC V1724 (TOF and KL) FADC V1731 (CKOV and EMR) Scaler V830 Trigger Receiver I/O V977 NI I/O PCi 6254 (Target) VLSB (Tracker) Trailer (special equipment handling the
release of the busy)
Todo EMR Front End Board
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 40
DATE Data Format The data sent by the equipment is just wrapped with a
LDC header (+ a GDC header if used)
The data format in the payload is defined by the manufacturer of the equipment ! (we will stick to 32 bits words)
DATE Header format defined in a header file event.hThis file contains all the information the offline codes needs to know
about DATE
Data from the equipment ->
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 41
Interfaces
o Online Monitoring
o Reconstruction
o Controls
o Configuration Data Base
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 42
Online Monitoring
Interface is based on DATE monitoring facility
A simplified version independent from DATE (running only from files) exists
Online monitoring process produces histograms and makes then available on demand on a ROOT socket
Socket handling implemented has a thread to avoid clashes with DATE monitoring semaphore system
Online Monitoring GUI is just a ROOT macro allowing the user to request histograms
See Linda’s talk
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 43
Reconstruction
Unpacking has to take into account the DATE format
Common library for online monitoring and Reconstruction
Dependency limited to the event.h file
All implemented in C++
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 44
Control And Monitoring
Two ways Interface CAM should know the DATE status A summary of CAM data should be inserted in the
online data stream The run should stop automatically when the CAM
goes in severe alarm state An EPICS Client uses the DIM layer of DATE
to make the status available A special DATE equipment will be
implemented to readout some CAM data identified as relevant for Offline Analysis
Currently under development The same equipment will stop the run if a
dedicated CAM variable is set More details in James’ and Pierrick’s talks
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 45
Configuration Database
The Equipment Parameters (configuration of the VME boards) will be stored in the configuration database
A run configuration file (in xml format) will be saved in the CDB each time a run is started
The CAM monitors DAQ status When the run starts, the CAM calls an API
function saving the configuration file automatically
More details in David’s talk
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 46
Project Plan
o Organization and Resources
o Time Line
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 47
OrganizationOrganization
Online GroupOnline Group ResponsibilitiesResponsibilities
Detector DAQ : J.S Graulich (80%) (Group Leader) Control and Monitoring: J. Leaver (50%)
P. Hanlet (20%) Online Reconstruction: L. Coney (20%) MLCR Manager: C. MacWaters (20%)
Other ContributorsOther Contributors Vassil Verguilov (DAQ) David Forrest (Config DB) Henry Nebrensky (GRID – Data Flow) Mike Courthold (Network Supervisor)
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 48
Time LineTime Line Project started from scratch in July 2005 First data taken acquired in March 2008 The DAQ is running stable with some limitations
Trigger selection by hardware Limit on DAQ Event Size is currently 30 MB
July 2009: Integration of Tracker in DAQ Include CAM Data in Online data stream (at the Spill level)
Current Priority: Problem with DAQ trigger distribution system
Cause Event building desync Need to include that in the Online Monitoring
Include EMR Include RF System !
MICE DAQ and Controls Review Jean-Sebastien Graulich
Slide 49
The Cat only grinned when it saw Alice. It looked good- natured, she thought: still it had VERY long claws and a great many teeth, so she felt that it ought to be treated with respect.
`Cheshire Puss,' she began, rather timidly, as she did not at all know whether it would like the name: however, it only grinned a little wider. `Come, it's pleased so far,' thought Alice, and she went on. `Would you tell me, please, which way I ought to go from here?'
`That depends a good deal on where you want to get to,' said the Cat.
`I don't much care where--' said Alice.
`Then it doesn't matter which way you go,' said the Cat. `--so long as I get SOMEWHERE,' Alice added as an explanation.
`Oh, you're sure to do that,' said the Cat, `if you only walk long enough.
' Alice felt that this could not be denied, so she tried another question.
`What sort of people live about here?' `In THAT direction,‘ the Cat said, waving its right paw round, `lives a Hatter: and in THAT direction,' waving the other paw, `lives a March Hare. Visit either you like: they're both mad.'
`But I don't want to go among mad people,' Alice remarked.
`Oh, you can't help that,' said the Cat: `we're all mad here. I'm mad. You're mad.‘
`How do you know I'm mad?' said Alice.
`You must be,' said the Cat, `or you wouldn't have come here.'