atlas trigger "sur le terrain" physique atlas france evian 19-21 octobre 2009 c. clément...

ATLAS Trigger "sur le terrain"

Physique ATLAS FranceEvian 19-21 Octobre

2009

C. Clément (Stockholm University)

Evian, 19-21 Octobre 2009 ATLAS Trigger 1

ATLAS Trigger architecture

Rate budget at L1/L2/EF/record rate, dead time

What can we do to prepare to LHC operation? How to test L1, HLT Trigger timing

Use cases for detector commissioning

Evian, 19-21 Octobre 2009 ATLAS Trigger

Overview

Level 1 TriggerL1Calo L1Mu


L1 Muon

AB

CD

Signals to the Front end to send data to Read Out System

L1 trigger rate <100kHz

40M

Hz

L1 Calo

7200 projective trigger towersx =0.1 x 0.1e/, /hadron up to 2.5jets up to total energy, ETmiss up to

EM

HAD

HAD

Acc

ess

shaft

s

Ele

vato

rs

SDX1

USA15

Read-Out

Drivers(RODs) First-

leveltrigger

Read-OutSubsystems

(ROSs)

UX15

USA15

Dedicated links

Timing Trigger Control (TTC)

1600Read-OutLinks

Gig

ab

it E

thern

et

RoIBuilder

pROSR

eg

ion

s O

f In

tere

st

VME~150PCs

Data of events acceptedby first-level trigger

Even

t d

ata

req

uest

sD

ele

te c

om

man

ds

Req

uest

ed

even

t d

ata

stores LVL2output

Event data pushed @ ≤ 100 kHz, 1600 fragments of ~ 1 kByte each

Second-leveltrigger

LVL2Super-visor

SDX1CERN computer centre

DataFlowManager

EventFilter(EF)

pROS

~ 500 ~1600

dual-CPU nodes

~100 ~30

Network switches

Event data pulled:partial events @ ≤ 100 kHz, full events @ ~ 3 kHz

Event rate ~ 200 HzData

storage

LocalStorage

SubFarmOutputs

(SFOs)

LVL2 farm

Network switches

EventBuilder

SubFarmInputs

(SFIs)


HZZ'e e

Evian, 19-21 Octobre 2009 ATLAS Trigger ... 6

L2 / Region of Interest

High-Level-Trigger (HLT) farm:

850 PC (CPU: 2 x quad-core) installed = 35% of final system Final size: ~ 500 PC for L2 + 1800 PC for

EF (multi-core technology): processor purchase deferred until Cost to Completion funds become available

Test realistic physics load on readout and HLT by preloading fake high pT physics signals in the readout system.

High Level Trigger Farm


Maximum L1 rate 75kHz - 100kHz (high luminosity)

Recording rate at the SFOs ~ 400MB/s Safe operating point with some margin, stream overlap

Bandwidth Point 1 – Tier 0 < 1GB/s

Tier 0 processing: steady state of 200 events/s and 300MB/s for an operational efficiency of 60%

=> ATLAS event size is a crucial parameter... Design ~ 1-1.5MB


Rate budgets ...


Commissioning of the Trigger System

Data flow at L1 rateup to 100kHz

Stress test in cosmics with physics triggers

Up to 500Hz of 'physics' muon triggers Up to 100kHz of random triggers

Remove the random triggers with the HLT

Also used high rate of calorimeter noise triggers


Commissioning of the Trigger System

Run by default run at L1 rates >30kHz.

Limitation from calibration and commissioning needs.

Also stress test of the trigger shifters and experts.


Preload simulated events: multijets, ttbar, black holes... into the Read out System.

Stress Test High Level Trigger


Preload simulated events: multijets, ttbar, baclk holes... into the Read out System.

Stress Test High Level Trigger

Check the performance for very 'busy' eventsTest the limits of the HLT

0.8 Mbyte event size

Performance on Monte Carlo Data

Dedicated 24 hour period almost every week

Run the 1031 Hz/cm2 menu (900 trigger chains)

Sustained L2 input rate of 90kHz

L2 output 18kHz and EF output 200Hz.

Event Building rate in 2008

1031 menu / 14 TeV(out of date)

1031 menu builds on expected L1 rate ~ 10kHz

HLT can cope with ~double the amount

L1 trigger and subsystems run daily at >30kHz


Example of High Level Trigger menu in cosmic runs

High Level Trigger chains


Running L1 and HLT in Cosmics

Trigger readout

Pre

cisi

on r

eadout

ATLAS Preliminary2008 cosmic-ray data

L2 MuonID with calorimeter

Shower shape at L2 trigger

Tracking efficiency at L2 triggerBarrel track Sagitta

L2 Muon tracking

Thre

e M

DT fi

t se

gm

ents


Running L1 and HLT in Cosmics


Data Streams

- Stream overlaps waste bandwidth

- Non trivial to understand overlaps, not always easy to fix

- Generates additional sources of errors, difficult overview

- Large differences in size b/w streams

=> problem for Tier 0 with small and large files

+ ATLAS convenienty writes out different streams

+ Convenient for physics analysis

+ Monitoring quantities on certain streams or

triggers are much easier to interprete

+ Express streams strengthen monitoring capability

L1 Latency / Timing


t

cosmic

CTP Can apply delays to equalize eg. L1Calo with L1mu

Can apply delays to equalize eg. L1Calo with L1mu

Before timing calibrationDelay not independentof detector region, ToF

Trigger Accept to the front end

Ensure L1 trigger latency uniform within each muon trigger sector,

easier with collisions, but requires decent lumi

tedious with lots of cosmics...<= but worth it

each calorimeter TTC partition, pulser runs, charge and laser injection

L1 latency uniform over partitions, MU/CAL cosmics, collisions

t

cosmic

Timing and Synchronisation

Tile (HCAL) timing / single beam

Pretimed using the TileCal laser system

ATLAS Preliminary 2008

Pixel module timing SCT timing


L1 Mu Timing


Challenging in cosmics due to Very large system Large time of flights Need an efficient time reference Even in beam would take up O(weeks)

TRT L1 Trigger: only for commissioningprovides a pure cosmic trigger.TRT is small in size ~ 3ns

For events triggered by TRT, monitor the timing of the muon triggers.

Time of the TRT trigger – Time of the RPC trigger

before calibration

after calibration

Actual plot stillin review...

Typically trigger only on TRT ~10Hz (if barrel only) Good match with LAr Calorimeter in 32 samples High rate of ID tracks with small variation in latency

=> ID alignment

L1 Mu Timing


TRT L1 Trigger: only for commissioningprovides a pure cosmic trigger.TRT is small in size ~ 3ns

For events triggered by TRT, monitor the timing of the muon triggers.

Time of the TRT trigger – Time of the RPC trigger

before calibration

after calibration

Actual plot stillin review and looks great

Typically trigger only on TRT ~10Hz (if barrel only) Good match with LAr Calorimeter in 32 samples Large monitoring window at the central trigger processor, large dead time. High rate of ID tracks with small variation in latency

=> ID alignment

Challenging in cosmics due to Very large system Large time of flights Need an efficient time reference Even in beam would take up O(weeks)

Full Lar pulse shape provides Tdrift

And check of mechanical uniformity

Requires read out of 32 LAr samples Event size of ~12MB

Requires very long dead time limiting L1 rate

Need to reconfigure dead time, prescales to limit L1 rate

Rate budget<400MB/s Maximum record rate of 33Hz, Tier 0 limit 300MB/s => 25Hz for

periods more than O(12 hours).

Limit the overall ATLAS recording rate to 15% of nominal.

Defined a calo-centric running cosmic running mode, both calorimeters in multiple sample read-out, Tier0 processing accordingly

It is worth it, but scheduled in priority when muon system cannot provide trigger, or low trigger rate is required for some other reason (eg. TRT-RPC timing)


ECAL Expected vs measured drift Time

LAr Calorimeter in 32 sample read out

Requires large samples of cosmic muons, not necessariy pointing.

Toroid OFF, solenoid ON, pT measurement in ID, straight tracks in muon system


Muon Alignment with Cosmics

October 16th – October 19th

L1 rate RND (>30kHz, phys 500Hz), Record rate > 300Hz

Small calorimter data size (physics mode)

Small dead time protection

Sagitta in the precision plane

x

x

x

BPTX

Beam Condition Monitors

Forward detectors

Calibration triggers

Transition radiation tracker (commissioning)

Evian, 19-21 Octobre 2009 ATLAS Trigger. 23

Other triggers for cosmics or beam

MinBias trigger scintillatorsMinBias trigger scintillators

LUCID at 17 m

Zero Degree Calorimeter at 140 m

ALFA at 240 m

Beam intensity - BPTX

H. Pernegger / CERN

BCM : Timing of background vs. interactions

Distinguish collisions from background through time-of-flight measurement Is measured for every bunch crossing (25ns) Requires fast and radiation hard detector+electronics Crucial to establish z-position of collisions

BCM timing

BCM Side A - Side CTiming in ATLAS smallest detector!Beam Condition Monitor

Evian, 19-21 Octobre 2009 ATLAS Trigger.

Operation: Bright and Dark Side

Daily changes among configurations Calos in mutisample, low rate, high dead time protection L1Mu timing or ID alignment, medium rate, medium dead time Muon alignmnent, high rate, low dead time, also as 'high rate test' Everything can change when you put a new menu, or push the system Becoming more and more efficient at that, more quiet with beam?

Change of L1 prescales without stopping the run powerful tool, crucial to deal with variations in rate due to beam conditions

Change of HLT prescales Dangerous for analyzers Extremely useful for commissioning Enable some chains at end of fill to testthem on real data/ no stop During the commissiong period change for instance from high rate muon

centric ATLAS run to low rate calorimeter centric run.

Evian, 19-21 Octobre 2009 ATLAS Trigger.

Start data taking right after injection with muon, pixel, SCT, (FCAL) in standby, configure with menu for the LHC fill. Pixel preamps disabled.

Special trigger prescale to limit the trigger rate during injection

When beams become stable (flag from the machine): Pause triggers

Ramp HV on pixel, sct, muon, (fcal)

Reneable pixel preamps

Change prescale set

Mark lumi block in cool (data quality)

Resume triggers

In case beam becomes unstable pause triggers


Operation of ATLAS

simulated RF ramps

In case of detector ROD or FE busy, ideally pause the run, fix the problem, resynchronise, resume triggers.

Step 1: Pause and remove faulty element (working for all but 1 subsystem of ATLAS)

Step 2: Reconfigure faulty element, resynchronise and put back into ATLAS data taking (in place for 2 subsystems so far)

Conclusions

After two years of running ATLAS with cosmics, subtantial improvements have been made to increase the data taking efficiency.

ATLAS is ready and waiting for beam, week 48...



Backup Slides

Consider a bucket with a hole in the bottom = Central Trigger Processor If trigger arrives, it is placed in the bucket. If the bucket is full, triggers are discarded. Triggers in the bucket are at a constant rate, equivalent to the size of the hole in the bucket.

Bunch crossing

Alive and waiting for trigger

No triggers are issuedto ATLAS, simple dead

time

Trigger receivedand issued to ATLAS

Alive and waitingfor trigger

Simple Dead Time

Complex Dead Time

Trigger receivedissued to ATLAS


Dead Time

To protect readout electronic from bursts of triggers...

L1 LatencyATLAS takes 40 million 'pictures per second'

Upon a positive trigger decision, need to select & send out the right 'picture'

99th LHCC –23 September 2009 Status of ATLAS

Timing in TGC, TGC triggerBCM timing

Timing in ATLAS smallest detector!Beam Condition Monitor

Tile (HCAL) timing / single beam

ECAL timing / single beam

ATLAS Preliminary2008 single beam data

ATLAS Preliminary2008 single beam data

L1Calo timing / beam on collimators


Pixel module timing SCT timing

atlas trigger "sur le terrain" physique atlas france evian 19-21 octobre 2009 c. clément...

Documents

atlas trigger sur

trigger shifters

trigger systemevian

high level trigger farmevian

trigger hlt farm

atlas triggeroverviewevian

atlas event size

khz stress test