EGEE-II INFSO-RI Enabling Grids for E-sciencEEGEE and gLite are registered trademarks Simulations and Offline Data Processing for the Auger Experiment Ji Chudoba, Institute of Physics, ASCR and CESNET Jaroslava Schovancov CESNET Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 2 Outline Cosmic rays Pierre Auger Observatory Computing Model VO Auger Data Management Policy Auger Grid CPU Challenge Simulation Plans Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 3 Cosmic rays Paul Sommers, 2003 Sources of particles with E > eV have to be within GZK-sphere (100 Mpc) Simon Swordy, 1996 Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 4 Showers Primary particle interacts with atmosphere Number of secondary particles is created Secondaries interact again, and again,... Typical shower eV: particles at ground Animation color code: blue: electrons/positrons cyan: photons orange: protons red: neutrons gray: mesons green: muons (10 -6 thinning) Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 5 Pierre Auger Collaboration More than 250 PhD scientists from more than 60 institutions from 16 (+2) countries > 400 authors Participating countries: Argentina, Australia, Bolivia *, Brazil, Czech Republic, France, Germany, Italy, Mexico, Netherlands, Poland, Portugal, Slovenia, Spain, United Kingdom, USA and Vietnam * * - associated countries Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 6 Pierre Auger Observatory Southern observatory almost completed Some results already presented Southern hemisphere: Malarge, Mendoza province, Argentina Northern hemisphere (planned): Lamar, Colorado, USA Lifetime of the observatory: yrs Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 7 Detectors Ground detectors: Covered surface: 3000 km 2 Number of detectors: 1600 Type of detector: Detector of Cherenkov radiation, each consisting of litres of ultrapure water and equipped with 3 photomultipliers. Spacing between detectors: 1.5 km. Fluorescence telescopes: Number of telescopes: 24 Mirrors: 3.6 m x 3.6 m with field of view 30 x 30, each telescope is equipped with 440 photomultipliers. Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 8 Event Reconstruction Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 9 Auger Computing Model Data recorded at the experiment site are copied to Lyon - CCIN2P3 slow line from Malargue, transfer via recorded DVD EU funded project AUGERACCESS will improve connectivity All Auger members have accounts at CCIN2P3 Data is analyzed remotely at CCIN2P3 or copied and processed on PCs or small computing farms users must install new versions of the offline sw users check for availability of new data CPU intensive simulations done at various sites and results copied to CCIN2P3 some sets are not at CCIN2P3, available to only some users Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 10 Expectations from Grid more computing resources available to more collaboration members uniform way for simulation productions easier access to data tests with simulations were done using regional VOs (central Europe - VOCE, SouthWest) Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 11 VO AUGER created in 2006 by the Czech group in cooperation with CESNET CESNET provides and maintains central resources LCG RB, gLite WMS, LB, UI, LFC, registration portal, VOMS server VOMS server replica at FNAL 5 sites from 4 countries support VO AUGER we expect more sites after discussions during collaboration meeting at the end of April EELA project considers AUGER as supported application E-infrastructure shared between Europe and Latin America www.eu-eela.org, IST www.eu-eela.org UNAM (National Autonomous University of Mexico) is setting up support for VO AUGER Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 12 Data Management Data Management Policy document 1 LFC server All files must be registered there orphan files can be deleted by administrators LFC: Users can use only directories /grid/auger/user/ /... SE: Files stored in a generated area are not allowed -P option must be always used with lcg-cr and lcg-rep commands Regular checks Middleware tools for policy enforcement would be useful Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 13 Small Productions Set of scripts for CORSIKA simulations user defines number of jobs, lower and upper values of parameters scripts do job submission, job monitoring, output retrieval and optional job resubmission tested by users, not used for bigger productions so-far Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 14 Auger Offline on Grid Auger offline code was tested on Grid CESNET in collaboration with partners from IJS, Slovenia installation done locally in directory accessible from WN binary relocatable tarball is being prepared separate MySQL server with data needed by jobs set up for grid jobs Atmosphere monitoring jobs modified version of offline code golden hybrid events reconstructed with varied atmospheric conditions Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 15 Auger Grid CPU Challenge Purpose: get realistic numbers for number of CPUs available check stability of sites and infrastructure Jobs about 1 CPU hour long Submit enough jobs to keep queues full All jobs submitted by 1 common user Phase 1: no warning to sites Phase 2: inform sites Timescale: 58 hours per phase (2 weekends) Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 16 CPU Challenge - Phase 1 Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 17 CPU Challenge - Phase 1 Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 18 CPU Challenge - Phase 2 Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 19 CPU Challenge - Phase 2 Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 20 Auger Grid CPU Challenge - Results Phase 1 3 sites running jobs + 2 sites aborting all jobs 160 (maximum), 90 (average) concurrent jobs running 2152 (80 %) jobs done Phase 2 4 sites 220 (maximum), 100 (average) concurrent jobs running 2250 (51 %) jobs done Some problems were already solved Observed issues stability - WMS, sites Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 21 Simulation plans for Grid Library of CORSIKA showers with EPOS model 8 energy bins, 7 angle bins, 100 events per bin 5600 proton showers 5600 iron showers 5 TB of simulated data Timescale: 50% in November 2007 similar production with faster model took 5 months on dedicated resources at CCIN2P3 Collaboration will consider to include more resources in the Grid if we demonstrate effective usage Easy interoperability with OSG would be welcome FNAL accessible through gLite CE Enabling Grids for E-sciencE EGEE-II INFSO-RI VO AUGER 22 The End Many thanks to members of Auger Collaboration for pictures and animations used in this presentations