scalable tridas for the nemo projectricap09.roma2.infn.it/slides/chiarusi_ricap2009r.pdf · n. of...
TRANSCRIPT
![Page 1: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/1.jpg)
Scalable TriDAS for the NEMO Project
Tommaso ChiarusiINFN Sez. Bologna & Phys. Dep. Univ. Bologna
for the NEMO Collaboration
Monte Porzio Catone, Villa Mondragone 14/05/2009
![Page 2: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/2.jpg)
Talk overview
Neutrino Telescope: just add water...all data to shore: challenging throughput TriDAS base conceptFrom NEMO Ph.1 to the scalable km3 TriDAS architecture
Tommaso Chiarusi RICAP 2009
![Page 3: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/3.jpg)
Cherenkov light
! "
Upgoing "
seabed depth > 3000 m
Atmospheric downgoing "
Less atm. bkg. when deeper ( below 3000 m usl)Expected signal > atm. bkg. when E!>10 TeV !Telescope Vol > 1 km3 for some evts/year ! Many PMTs (> 5000) ! complex experimental setup!NEMO location: CAPO PASSERO @ 3400 m usl
See Paolo Piatteli’s talk in plenary session todayNEMO
Capo Passero Site
100 km
Tommaso Chiarusi RICAP 2009
![Page 4: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/4.jpg)
some km3 detector possible layouts:
Det.Unit
cable PJB-SJB
cable SJB-DU
Square Grid
Hexagonal Grid90 Detection Units80 PMTs / D.U.7200 PMTs
81 Detection Units80 PMTs / D.U.6480 PMTs total
Tommaso Chiarusi RICAP 2009
![Page 5: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/5.jpg)
One muon event (only muon hits shown, no background)theTelescope is made of
DETECTION UNITS
20 floors, with 4 10” PMTs each
Tommaso Chiarusi RICAP 2009
![Page 6: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/6.jpg)
Expected Optical Data sources bioluminescence (neglig. @ depth !2500 m) upgoing neutrinos (atm. + signal) (50 day-1 / km3) atmospheric muons (10÷100 Hz / km3) 40K decays (30÷40 kHz / 10” PMT @ 0.5 p.e )
Hit samples Vs. pulse time on a 10” PMT with 0.5 p.e. thr. (NEMO Ph.1 data)
S.P.E. wave form(~16 samples = 16 Bytes)
S.P.E. HIT SIZE:Hit PMT Info
+ Hit Time
+Hit Charge
+Hit Wave Form(samples)
28 BytesTommaso Chiarusi RICAP 2009
![Page 7: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/7.jpg)
Expected Data Rates10” PMT
single rate(kHz)
Data Rateper PMT(Mbps)
Data Rate per Floor*
(Mbps)
Data Rateper D.U.**
(Gbps)
Data Rate per km3 ***
(Gbps)
40 8.8 35.2 0.7 70
80 16.8 67.2 1.3 130
150 32 128 2.5 250
300 64 256 5.0 500
** 20 Floor/D.U. * 4 PMT/ Floor ** 100 D.U.
Tommaso Chiarusi RICAP 2009
(bare 40K)
(NEMO Ph. 1)
(present DAQ)
(expanded DAQ )
![Page 8: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/8.jpg)
The TriDAS reduces the data rate by filtering the data stream bunched in Time Slice (TS).
A TS contains all data from all ( or part of) the detector occurred in a given time interval (~100 ÷200 ms).
WHAT IS SCALABLE?
The TriDAS principal elements:
- Trigger System Controller (TSC): monitors and serves the TriDAS - (1 GbE I/O)
- Hit Managers (HM): receive optical data; prepare and distribute the TS to the TCPU
- TriggerCPUs (TCPU): apply the trigger logics to the assigned TS and transfer the selected data to the EM;
- Event Manager (EM): receives the triggered data from TCPUs and build the Post-Trigger events data file
Atmospheric Muon Signal (@ 3000 m depth) assuming! - Rate µ atm:! ! 100 Hz
! ! ! - RateK40 : ! ! 300 kHz
! ! ! - N. PMTs:! ! 8000
! - Rec. time window:! 6 µs
Post-Trigger data rate: ~ 40 MBps
Stored data /day ~1 TB !!!!
Upgoing Neutrino Signal assuming! - Rate ! :! ! < 4 10-3 Hz
! ! ! - RateK40 : ! ! 300 kHz
! ! ! - N. PMTs:! ! 8000
! ! - Rec. time window:! 6 µs
Post-Trigger data rate: ! 2 kBps
Stored data /day ! 150 MB !!!!
Tommaso Chiarusi RICAP 2009
![Page 9: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/9.jpg)
HM 1 HM 1
HM 1 HM 1
HM 2 HM 2
HM 2 HM 2
HM 3 HM 3
HM 3 HM 3
TCPU 1 TCPU 1
TCPU 1 TCPU 1
TCPU 2 TCPU 2
TCPU 2 TCPU 2
TCPU 3 TCPU 3
TCPU 3 TCPU 3
DU-TS i, 1 DU-TS i, 2 DU-TS i, 3
Full TS i
Time slice and Barrel Shift Paradigm
If a TCPU needs more time, just add one more!
[refer to: M.F. Letheren, 1995 CERN School of Computing]
Tommaso Chiarusi RICAP 2009
![Page 10: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/10.jpg)
TriDAS for NEMO Ph.1
Gbit switch
Master CPU <70 MB/s
Hard
Disk
Hard
Disk
CPU
PC Floor 1
PC Floor 2
< 70 MB/s
Windows
Linux
PC Monitor
DM - RC
PC Floor 3
PC Floor 4
The maximum throughput of the MiniTower was " 512 Mbps
One Machine, the Master CPU, played all the rules: Hit Manager, Trigger, Event Manager
CABLE
Floor 1
Floor 2
Floor 3
Floor 4
Tommaso Chiarusi RICAP 2009
![Page 11: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/11.jpg)
TriDAS for a first prototype Detection Unit with 16 Floors; expected through put " 2 Gbps standard 1 GbEthernet Networking
Floor 1
Floor 2
Floor 3
Floor 4
Floor 5
Floor 6
Floor 7
Floor 8
Floor 9
Floor 10
Floor 11
Floor 12
Floor 13
Floor 14
Floor 15
Floor 16
Shore
interfaces to:
Tommaso Chiarusi RICAP 2009
![Page 12: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/12.jpg)
Possible Network infrastructure for km3 TriDAS (90 D.U.)
Tommaso Chiarusi RICAP 2009
![Page 13: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/13.jpg)
0
10
20
30
40
50
60
0 50 100 150 200 250 300 350
Single Rate on a PMT (kHz)
N.
of
TC
PU
s
!KNCNPMTNTCPU = RCPU
RCPU SHitNACC = Drate HM#TCPU
Drate HM#TCPU = 10 GbpsSHit= 224 b
RCPU = 3 GHzNACC= 70
NPMT=8000NC=NACC=70
Number of Available Clock Cycles per CPU per TS:
Estimed number of necessary TCPU:
If the req. NC>NACC(i.e. trigger algo. is slow)
ADD TCPU!Tommaso Chiarusi RICAP 2009
![Page 14: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/14.jpg)
Monitoring
Dispatcher for
TriDAS status
Dispatcher for
Physics Data
Trigger Time Window
transferred to EM
events written and
stored
Time Slices
tranfser to TCPU
TCPU
Via ControlHost: a Tag Controlled Data Dispatching [V. Maslenikov et al. CASPUR, http://afs.caspur.it/temp/ControlHost.pdf]
Tommaso Chiarusi RICAP 2009
![Page 15: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/15.jpg)
Multi purpose on-line Visualizer ( with D. Bonfigli - UniBo)
![Page 16: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/16.jpg)
On(Off)-line Event Display ( with A. Riccardo - UniBo)
Tommaso Chiarusi RICAP 2009
![Page 17: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/17.jpg)
Conclusions“all data to shore” is a challenging BUT feasible strategy for km3 underwater !-telescope;scalable TriDAS architecture suppliyng high data-stream (up to 500 Gbps) is possible and affordable with the present technology;The NEMO Collaboration is completing a scaled TriDAS for the prototype Detection Unit.
Tommaso Chiarusi RICAP 2009
![Page 18: Scalable TriDAS for the NEMO Projectricap09.roma2.infn.it/slides/Chiarusi_RICAP2009r.pdf · N. of TCPUs N ! KN CN PMT TCPU = R CPU N R CPU S Hit ACC = D rate HM#TCPU D rate HM#TCPU](https://reader035.vdocuments.us/reader035/viewer/2022070901/5f4b3c2c6a1f300cfc51ac14/html5/thumbnails/18.jpg)
G. Priede et al., Deep-Sea Research I 55 (2008) 1474–1483