![Page 1: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/1.jpg)
www.consorzio-cometa.it
Consorzio COMETA - Progetto PI2S2UNIONE EUROPEA
NEMO Monte Carlo Application on the Grid
NEMO Monte Carlo NEMO Monte Carlo Application on the GridApplication on the Grid
R. CalcagnoR. Calcagno for the NEMO Collaborationfor the NEMO Collaboration
Grid Open Day at LnsGrid Open Day at LnsCatania 2008 January 23Catania 2008 January 23
![Page 2: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/2.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 2
The NEMO project
Shore stationShore stationShore stationShore station
Electro-optical cableElectro-optical cableElectro-optical cableElectro-optical cable
KmKm33 detector detectorKmKm33 detector detector
R&D for the design and construction of a km3 high energy neutrino detector
![Page 3: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/3.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 3
Neutrino astronomy: detector concept
neutrino
muon
Cherenkov light
~5000 PMTatmospheric muon
depth>3000m
Detection of Cherenkov light in transparent natural media (water or ice)
neutrino
![Page 4: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/4.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 4
Neutrino detection: signal and background
Search for muon tracks from neutrino interactions
Earth is used as shield to cut off the downgoing atmospheric µ
background
Upgoing tracks can unambigously be attributed to
neutrinos
The predicted neutrino fluxes are weak and it is necessary to take
into account the expected background noise.
One of the main sources of background are the atmospheric
muon generated by the interaction of cosmic rays with
the atmosphere.
Simulation of this background needs a huge number of
MonteCarlo events.
![Page 5: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/5.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 5
NEMO proposed km3 telescope
main EO cablemain Junction Box
secondary JB
“tower”
km3 size square array of detection structures ( “towers“ )
![Page 6: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/6.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 6
Code Chain
Corsika Music Km3 Modk40 Femsim Reco
Code chain for atmospheric muon simulation are:
![Page 7: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/7.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 7
Simulation
• Corsika code simulates the interaction of cosmic rays with the atmosphere and the propagation of the generated showers.Muons are propagated up to the sea level.
• Km3 simulates the light emission, the propagation in water,and the detection by the PMTs. In order to simulate the light radiation is necessary a parameterization which depends on the muon energy lost in the trace segment and on the optical water properties. PMT hits are simulated.
• Music simulates the muon propagation in the sea water.
![Page 8: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/8.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 8
Simulation
• FemSim simulates the working operation of the electronic devices
• Reco performs the reconstruction of the muon track direction
• Modk40 adds hits due to optical background that in salt water is due 40K
![Page 9: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/9.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 9
Simulation Codes Structure
Simulation codes are performed in sequential way: the output of the first code is the input of the second one, the output for the second one is the input of the
third one and so on
Moreover, many files are necessary to the code chain to
run (detector geometry file, input parameter files, tables …)
Simulation codesI n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
CORSIKA
MUSIC
KM3
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t RECO
MOD40
FEMSIM
![Page 10: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/10.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 10
Grid Application
Background Monte Carlosimulation needs a wide
statistic and big processing time, therefore GRID is the
ideal tool to simulate this huge amount of events.
I n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
I n p u t
O u t p u t
CORSIKA
MUSIC
RECO
KM3
MOD40
FEMSIM
![Page 11: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/11.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 11
Grid Application
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t CORSIKA
MUSIC
KM3
MOD40
FEMSIM
RECO
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t CORSIKA
MUSIC
KM3
MOD40
FEMSIM
RECO
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
O u t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t
I n t p u t CORSIKA
MUSIC
KM3
MOD40
FEMSIM
RECO
The developed application is a trivial parallel execution of simulation codes on the Grid.
...
![Page 12: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/12.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 12
Implementation
• The first of Code Chain is Corsika
O u t p u t
CORSIKA
Input file size < 10MB...
Inputs
EGSDAT4_.15 esegui_100.sh
corsika_flat.exe EGSDAT4_3.
EGSDAT4_1.QGSDAT01
EGSDAT4_.05 EGSDAT4_3.
EGSDAT4_.4 EGSDAT4_25.
NUCNUCCSinputs_v2_100
![Page 13: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/13.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 13
Upload on SE
A virtual directory is created on the storage element (SE)
The output file is downloaded
The upload of Corsika file is performed on a SE and is recorded on the LFC Catalog
CORSIKA
CORSIKA
SE
O u t p u t
WN
CORSIKA
The submitted job transfers the input files from the SE to the WN
![Page 14: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/14.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 14
Upload on SE: User
SE
SE
Catalog File
SEU I
![Page 15: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/15.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 15
Input for Music
( ~200 MB zipped file )
TestSite.dat
Input file size > 10MB
O u t p u t
MUSIC trasf_out_wn_100.sh
music-eloss-seawater.dat
music-cross-sections-seawater.datmusic-double-diff-water.dat
music-double-diff-seawater.dat
![Page 16: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/16.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 16
I/O Upload on the SE
The output Corsika and the Input file are uploaded on a SE and are recorded in the LFC Catalog
The job is submitted
The output file is downloaded
OUTPUT CORSIKA
WN
INPUT >10M
SE
WN
...MUSIC
...
CORSIKA ...
O u t p u t
![Page 17: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/17.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 17
Replicate file
• A file can be stored on multiple SE.
lcg-rep –-vo VO –d addressSE pathLFN
SE
SE
SE
SE
INPUT >10M
INPUT
(OUTPUT_
_CORSIKA)
![Page 18: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/18.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 18
Job Description Language (JDL)The jdl is a high-level, user-oriented language for describing jobs and
aggregates of jobs such as Direct Acyclic Graphs (DAG)
A jdl file describes a job that can be run on the Grid
UI
CE
SE
Mass
Storage
Input
Sandbox
Output
Sandbox
WN WN WN WN WN WN
RB
Job execution
![Page 19: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/19.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 19
InputSandbox
The input file is transferred
from the local UI to the WN
![Page 20: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/20.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 20
OutputSandbox
The output for downloading
from the WN to the UI
![Page 21: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/21.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 21
Job requirements on resourse (CE)
Requirements
![Page 22: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/22.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 22
Job List Match
• This list shows the Computing Element that matching our job requirements.
![Page 23: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/23.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 23
no success
Polling
sleep 90 s
status
submit
![Page 24: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/24.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 24
no success
Polling
sleep 900 s
status
submit
outputsuccess
![Page 25: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/25.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 25
Input File
• The input for Corsika will be different for every chain performed in parallel way, changing the SEED parameter.
![Page 26: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/26.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 26
Step
User chooses:• the number of job to execute • a seed value as reference
To set up: • n files .jdl to submit on Corsika• n files .jdl to submit on Music• n files .jdl to submit on Km3• … … …• n files .jdl to submit on Reco
To observe
the sequence
![Page 27: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/27.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 27
Trivial Parallel Execution
CE
WN WN WN WN WN WN
CE
WN WN WN WN WN WN
RB
Every jdl file
works in independent
way the one from the
other on the destination
WN chosen by the CE
infnlns-ce-01.ct.pi2s2.it:2119
unipa-ce-01.pa.pi2s2.it:2119
unime-ce-01.me.pi2s2.it:2119
CE
WN WN WN WN WN WN
CORSIKA-1 CORSIKA-2
MUSIC-1
![Page 28: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/28.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 28
GRID implementation
Expert
jdl 1
jdl 2
jdl N
jdl 1
jdl 2
jdl N
jdl 1
jdl 2
jdl N
WN
WN
WN
WN
WN
WN
IN File 1
EXE File
IN File 1EXE File
IN File 1
EXE File
IN File 1
EXE File
IN File 1EXE File
IN File 1
EXE File
Output
Output
Output
Output
Output
Output
submit status get-output execute
CONTROLLER
submit status get-output execute
CONTROLLER
CORSIKA MUSIC KM3
...
...
...
...
......... .........
The codes need the use of libraries (CERN libraries, NAG libraries…)
![Page 29: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/29.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 29
Why Grid ?
• Many CPU
• To ask specific requirements
• Redundance of the systems
• Many SE
![Page 30: Www.consorzio-cometa.it Consorzio COMETA - Progetto PI2S2 UNIONE EUROPEA NEMO Monte Carlo Application on the Grid R. Calcagno for the NEMO Collaboration](https://reader035.vdocuments.us/reader035/viewer/2022062803/56649c8b5503460f94945a0d/html5/thumbnails/30.jpg)
Grid Open Day at Laboratori Nazionali del Sud, 2008 January 23 30
Any Questions ?
G R
I D