one scenario for the cbm trigger architecture

One scenario forOne scenario forthe CBM Trigger the CBM Trigger

ArchitectureArchitecture

Ivan KiselIvan Kisel

Kirchhoff-Institut für PhysikKirchhoff-Institut für Physik, Uni-Heidelberg, Uni-Heidelberg

FutureDAQ Workshop, MünchenMarch 25-26, 2004

KIPKIP

25-26 March 2004, München25-26 March 2004, München Ivan Kisel, KIP, Uni-HeidelbergIvan Kisel, KIP, Uni-Heidelberg 22

Particle Multiplicities and Data RatesParticle Multiplicities and Data Ratesnchar

nneut

980

1080

680

700

1000

700

STS

60

Sub-System

GByte/secRICH

90

TRD

130

RPC

10

ECAL

50Total

350

Total

350

Multiplicities

for central

collision

Data rate for

107 int / sec

for Au+Au

25 A GeV

UrQMD

not including:

background

link overhead

W.Müller


DAQ-Trigger ScenarioDAQ-Trigger Scenario

Bandwidth 1 TByte/sec

Hit Level Processing

Local Level Processing

Buffering; Event association;

Regional Level Pre-Processing

3-10 MHz Event Rate

First Level Trigger:

FGPA; DSP; PC

300 kHz Event Rate

Second Level Trigger:

PC Farm

From

LoI

20 kHz Event Rate

1 GByte/sec Bandwidth

W.Müller


Modular Structure of DAQModular Structure of DAQ

MAPS, STS RICH ECALDetectorDetector

Trigger/Offline PC FarmTrigger/Offline PC Farm

Time-Slice Builder NetworkTime-Slice Builder Network

101077 ev/s ev/s

101055 slsl/s/s

50 50 kBkB//evev

100 100 evev//sliceslice

5 M5 MBB//sliceslice

N x MN x MN x MN x MSchedulerSchedulerSchedulerScheduler

TRD

Sub-FarmSub-Farm

RURURURURURURURURURURURURURURURU

RURURURURURURURURURURURURURURURU

Sub-FarmSub-Farm Sub-FarmSub-Farm Sub-FarmSub-Farm Sub-FarmSub-FarmFarm

Control SystemSub-FarmSub-FarmSub-FarmSub-Farm Sub-FarmSub-Farm Sub-FarmSub-Farm Sub-FarmSub-Farm

Sub-FarmSub-FarmSub-FarmSub-Farm Sub-FarmSub-Farm Sub-FarmSub-Farm Sub-FarmSub-Farm



SF

n

availab

le

SFnt MAPS STS RICH TRD ECAL

SFnt MAPS STS RICH TRD ECAL

SFntSFnt SFnt SFnt


Scheduled Data TransferScheduled Data Transfer

Deyan Atanasov, Scheduled Data Transfer, CBM DAQ, HD 18.03.04

The Scheduler assigns The Scheduler assigns Time-Slices to Sub-Time-Slices to Sub-Farms.Farms.

TXTX

EventsEvents

SourceSource

DestinationsDestinationsIN

Scheduler Scheduler CoreCore

SchedulingSchedulingDisciplineDiscipline

IN

OUT

1 2 3New

EventEntry

1. Idle.2. Obtain Destination.3. Produce a Tag.


Control

Experiment Control System

Monitor FCS Network

HostPCI bus

Farm Control System

FCS - Farm Control SystemFCS - Farm Control System

Cluster Interface Agent (CIA)• every node can be (re)configured, turned on/off remotely via Ethernet• it can save hardware components like display cards, Floppy Disk Drives• monitor the whole cluster, every host regardless of its current state with additional monitoring possibilities independent from operating system

•The farm implements its own The farm implements its own control networkcontrol network•Control over every node is Control over every node is done via a redundant done via a redundant hardware unit attached to the hardware unit attached to the nodenode•The interface between FCS The interface between FCS and ECS is done via a and ECS is done via a dedicated nodededicated node

Ralf Panse, Farm Control System, CBM DAQ, HD 18.03.04


Distributed Local Mass StorageDistributed Local Mass Storage

10 TB/PC10 TB/PC = 10 000 000 MB/PC = 10 000 000 sec = 120 = 10 000 000 MB/PC = 10 000 000 sec = 120 daysdays = =

4 4 months of data takingmonths of data taking

1 MB/sec/PC1 MB/sec/PC

Data TakingData Taking3 Months3 Months

Data AnalysisData Analysis9 Months9 Months

Data ReductionData Reduction1000:11000:1

Data ReductionData Reduction100:1100:1

Lord Hess, ClusterRAID1 Prototype, CBM DAQ, HD 18.03.04

Arne Wiebalck, ClusterRAID, CBM DAQ, HD 18.03.04


PC Sub-FarmPC Sub-Farm

Input Input DataData

Farm Control System

PCPCPCPC PCPCPCPCPCPCPCPCPCPCPCPC PCPCPCPC PCPCPCPC PCPCPCPCPCPCPCPCPCPCPCPC PCPCPCPC

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

PCPCPCPC PCPCPCPCPCPCPCPCPCPCPCPC PCPCPCPCPCPCPCPC PCPCPCPC

Sub-FarmSub-Farm Sub-FarmSub-FarmSub-FarmSub-Farm-Farm-Farm

•Various CPU powerVarious CPU power•Shared PCsShared PCs•ReconfigurableReconfigurable•Fault tolerantFault tolerant•Offline on backgroundOffline on background

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

FP

GA

SchedulerSchedulerSchedulerScheduler


MA

PS

MA

PS

STSSTS RICHRICH TRDTRDECALECAL

Local time slice 100Local time slice 100Local time slice 99Local time slice 99

Local time slice 1Local time slice 1Local time slice 2Local time slice 2Local time slice 3Local time slice 3

Local time slice …Local time slice …

Local time slice iLocal time slice iLocal time slice iLocal time slice i

MA

PS

MA

PS

STSSTS RICHRICH TRDTRD ECALECAL

LM/LM/DSPDSP LM/DSPLM/DSP LM/DSPLM/DSPLM/DSPLM/DSP

FPGA: Pre-process/L1FPGA: Pre-process/L1

NICNICNICNIC

InIn

NetworkNetworkNetworkNetwork

50

0 M

B/s

/FP

GA

50

0 M

B/s

/FP

GA

Fre

eFre

e// B

usy

Bu

sy

Offl

ine

Offl

ine

Local time slice i+1Local time slice i+1Local time slice i+1Local time slice i+1LM/LM/DSPDSP LM/DSPLM/DSP LM/DSPLM/DSPLM/DSPLM/DSP

FPGAFPGAPre-processPre-process/L1/L1 10000 ev/s10000 ev/s

100 100 s/evs/ev

Ou

tO

ut

PCsPCsPCsPCs

50

0 M

B/s

/FP

GA

50

0 M

B/s

/FP

GA

Memory BuffersMemory Buffers


MA

PS

MA

PS

STSSTS RICHRICH TRDTRDECALECAL

Updated local time slice 100Updated local time slice 100

Updated local time slice 99Updated local time slice 99

Updated local time slice 1Updated local time slice 1Updated local time slice 2Updated local time slice 2Updated local time slice 3Updated local time slice 3

Updated local time slice …Updated local time slice …

CPU: L2/OfflineCPU: L2/Offline

NICNICNICNIC

InIn

FPGAsFPGAsFPGAsFPGAs

50

0 M

B/s

/CP

U5

00

MB

/s/C

PU

Fre

eFre

e// B

usy

Bu

sy

Offl

ine

Offl

ine

CPUCPUL2/OfflineL2/Offline

300 ev/s300 ev/s3 ms/ev3 ms/ev

StorageStorageStorageStorage

1 M

B/s

/CP

U1

MB

/s/C

PU

Memory BuffersMemory Buffers


ECALECALECALECAL

FPGA 4D Pre-processor/L1 TriggerFPGA 4D Pre-processor/L1 Trigger

Reconstruct primary vertexSelect detached secondary D tracksSelect RoIs of secondary J/ tracksFit secondary tracksFit secondary verticesPre-process/Trigger

TRDTRDTRDTRDRICHRICHRICHRICH

STSSTSSTSSTS

STSSTSSTSSTS

STSSTSSTSSTS

STSSTSSTSSTS

STSSTSSTSSTS


ECALECALECALECAL

ECALECALECALECALTRDTRDTRDTRD

RICHRICHRICHRICH

CPU 4D L2 TriggerCPU 4D L2 Trigger

Reconstruct primary vertexSelect detached secondary D tracksSelect RoIs of secondary J/ tracksFit secondary tracksFit secondary verticesTrigger


STSSTSSTSSTS

STSSTSSTSSTS

STSSTSSTSSTS

STSSTSSTSSTS

STSSTSSTSSTS

Improve (m)any of the L1 results:Improve (m)any of the L1 results:


ECALECALECALECAL


RICHRICHRICHRICH


RICHRICHRICHRICH

Offline AnalysisOffline Analysis

Reconstruct primary vertexSelect detached secondary D tracksSelect RoIs of secondary J/ tracksFit secondary tracksFit secondary verticesAnalysis


STSSTSSTSSTS

STSSTSSTSSTS

STSSTSSTSSTS

Improve (m)any of the L2 results:Improve (m)any of the L2 results:


RICHRICHRICHRICH


RICHRICHRICHRICHSTSSTSSTSSTS

STSSTSSTSSTS


AlgorithmsAlgorithms

SimpleLocalParallelFast

Hough TransformCellular AutomatonElastic NetKalman Filter


Trigger SimulationTrigger Simulation

L1 FPGA C++, SystemC, VHDL, Synopsys, Quartus

L2 CPU Framework, C++, GEANT

Offline Framework, C++, GEANT

Overall behavior SystemC, Ptolemy


SummarySummary

Modular structure of DAQ Online reconfigurable farm Commercial hardware Modular software „All-in-one“ sub-farm No canonical event building Access to all data at any stage 4D event reconstruction Flexible L1/L2/Offline definition No need to re-process L1 at L2 Offline can run/test L1 and L2 All data on local mass storage Online alignment Online database update

one scenario for the cbm trigger architecture

Documents

farmsubfarmsu kisel

mllerivan kisel

backgroundschedulerivan

kipivan kisel

cbm daq

control networkcontrol

scheduler assigns timeslices

event ratefirst level