interoperability demos and atlas applications luca vaccarossa (infn - milano)
Post on 01-Jan-2016
26 Views
Preview:
DESCRIPTION
TRANSCRIPT
November 21, 2002 Luca Vaccarossa INFN - Milano 1
Interoperability Demos and Atlas applications
Luca Vaccarossa (INFN - Milano)ATLAS Software Workshop21/11/2002 CERN - Genève
November 21, 2002 Luca Vaccarossa INFN - Milano 2
Layout of Talk
• Goals of EU-US joint Testbed• Testbed Infrastructure• Screenshots of Monitoring Tools• Distribution of Atlas Software• Description of the Demo Submission• Conclusions
November 21, 2002 Luca Vaccarossa INFN - Milano 3
Joint EU-US grid demosIST2002 4-6 November, Copenaghen
SC2002 16-22 November, Baltimore
Goals of WorldGrid Testbed :
• Interoperability between grid domains for applications submitted by users from different virtual organizations
• The testbed should provide a grid-integrated infrastructure of middleware software transparent to the higher layers so that applications on EU and US could run as they did before the integration, without major modifications.
November 21, 2002 Luca Vaccarossa INFN - Milano 4
Joint EU-US grid demos
• The testbed should be heterogeneous supporting various flavors of RH and various flavors of the basic Globus services declared to be interoperable.
• Controlled use of shared resources subject to agreed policy
• Integrated use of heterogeneous resources from iVDGL, EDG and CrossGrid testbed domains
November 21, 2002 Luca Vaccarossa INFN - Milano 5
WorldGridShared Global Resources, Interoperability, Policy
November 21, 2002 Luca Vaccarossa INFN - Milano 6
DataTAG WP4 WorldGrid Team• DataTAG WP4
• Main Subject:• Interoperability between EU and US Grids services from DataGrid, GriPhyN, PPDG and in collaboration with iVDGL, for the HEP applications.
• WP Manager• A. Ghiselli INFN -CNAF
• DataTAG WP4 Task Force for Demos Testbed Setup
• Coordinated by Flavia Donno (CERN)• David Rebatto INFN - Milano• Sergio Fantinel INFN - LNL• Gennaro Tortone INFN - Napoli• Federica Fanzago INFN - LNL• Marco Verlato INFN - Padova• Vincenzo Ciaschini INFN - CNAF• Luca Vaccarossa INFN - Milano
November 21, 2002 Luca Vaccarossa INFN - Milano 7
WorldGrid Testbed Infrastructure
• Two EDG 1.2.3 Resource Broker available (Milano
and Pisa), each one with a related EDG 1.2.3
Information Index (Milano and Pisa) running MDS 2.2
• One central EDG Resource Broker “Glue-aware”
(CNAF-Bologna)
• EDG and CG Computing and Storage Elements
with the EDG and GLUE schema and Information
Providers
• VDT servers serving as Computing and Storage
Elements with the EDG and GLUE schema and
Information Providers
November 21, 2002 Luca Vaccarossa INFN - Milano 8
WorldGrid Testbed Infrastructure
• PBS, LSF and Condor-G as LRM but only PBS and
LSF used with RB.
• Multiple UI running EU and US portals such as
Genius and Grappa (Pisa, ANL, Padova, CNAF-
Bologna)
November 21, 2002 Luca Vaccarossa INFN - Milano 9
UIUIGenius/GrappaGenius/Grappa
RB/JSS
II/MDS2.2
Replica Catalogue
VDT Servers
CE
SE
World-Grid InfrastructureWorld-Grid Infrastructure
November 21, 2002 Luca Vaccarossa INFN - Milano 11
Monitoring Tools
• Nagios – Sergio Fantinel, Gennaro Tortone
(DataTAG)
• Ganglia - Catalin Dumitrescu (University
of Chicago)
• VO Centric
November 21, 2002 Luca Vaccarossa INFN - Milano 15
Distribution of Atlas SW• Starting from RPM distribution for Atlas software,
Version 3.2.1. Release 2 from EDG repository● Atlas kit 3.1.2 rpm pacman-ized (Atlas-
kit.pacman) for US sites
● Included additional Components needed at run-time missing in many US sites (LessTif, zsh)
● Distributed as part of ScienceGrid Cache
● Installation check package Atlas-kit-verify
November 21, 2002 Luca Vaccarossa INFN - Milano 16
US Sites EU Sites
Gainesville/FL (VDT) Milano/Italy (EDG)
Milwaukee/WI (VDT) Padova/Italy (EDG)
Batavia/IL (VDT) Geneve/Switzerland (EDG)
Bloomington/IN (VDT) Valencia/Spain (CG)
Karlsruhe/Germany (CG)
Lisbone/Portugal (CG)
Sites where Atlas- kit has been installed
Distribution of Atlas SW
November 21, 2002 Luca Vaccarossa INFN - Milano 17
Demo Submission ScriptExecutable = "/usr/bin/env";
Arguments = "zsh prod.dc1 00001";
StdOutput = " dc1.002000.test.00001.hlt.pythia_jet_17.log";
StdError = "dc1.002000.test.00001.hlt.pythia_jet_17.err";
InputSandbox={"prod.dc1","dc1.002000.evgen.0001.hlt.pythi
a_jet_17.root","plot.kumac"};
OutputSandbox =
{"dc1.002000.test.00001.hlt.pythia_jet_17.log",
"dc1.002000.test.00001.hlt.pythia_jet_17.his",
"dc1.002000.test.00001.hlt.pythia_jet_17.err","plot.kumac"};
Rank = other.FreeCPUs;
Requirements =
IsMember(other.RunTimeEnvironment,"ATLAS-3.2.1");
November 21, 2002 Luca Vaccarossa INFN - Milano 18
Demo Submission Script
• Simulation of 10 events
• Based on DC1 Grid script
• Datafile of 1.7 MB sent trough RB InputSandbox
• Requirements = (RunTimeEnvironment, "ATLAS-
3.2.1”)
• Rank FreeCPUs
• Trough Genius Portal
November 21, 2002 Luca Vaccarossa INFN - Milano 19
•Matchmaking by EDG RB
# dg-job-list-match dc1_part1.jdl
The following CE(s) matching your job requirements have been
found:
- grid001.mi.infn.it:2119/jobmanager-pbs-workq
- grid011.pd.infn.it:2119/jobmanager-lsf-datatag
- ufdemo.phys.ufl.edu:2119/jobmanager-pbs-Long
- lxshare0229.cern.ch:2119/jobmanager-pbs-short
- lxshare0229.cern.ch:2119/jobmanager-pbs-medium
- lxshare0229.cern.ch:2119/jobmanager-pbs-long
- - worldgrid.iu.edu:2119/jobmanager-pbs-Long20
- worldgrid.iu.edu:2119/jobmanager-pbs-Long
- ricci.phys.uwm.edu:2119/jobmanager-pbs-Long
- nest.phys.uwm.edu:2119/jobmanager-pbs-Long
- lngrid02.lip.pt:2119/jobmanager-pbs-qgrid
- handsome.dhcp.fnal.gov:2119/jobmanager-pbs-Long
- bee001.ific.uv.es:2119/jobmanager-pbs-qgrid
November 21, 2002 Luca Vaccarossa INFN - Milano 20
GENIUS (Grid Enabled web eNvironment for
site Independent User job Submission)
[https://genius.ct.infn.it]
OS & Net services
Basic Services
High level GRID middleware
ALICE ATLAS CMS LHCbApplications’specific layer
Other apps
GLOBUS
toolkit
DataGRID architectur
e
GENIUS web portal
Demo Submission
November 21, 2002 Luca Vaccarossa INFN - Milano 29
Demo Submission
• Job executed at remote Cluster
• dg-job-status retrieves job status (Ready,
Scheduled, etc.)
• when Status = OutputReady
dg-job-get-output JobID
• Output sandbox files for the job is retrieved and
stored in a local directory
November 21, 2002 Luca Vaccarossa INFN - Milano 31
Demo Submission
Job Output:
• Standard output & error
• Histogram file of 190 KB
Interactive Session
• Genius Interactive Services
• PAW
November 21, 2002 Luca Vaccarossa INFN - Milano 36
Conclusions
• The demo has been successful and has demonstrated smooth execution of ATLAS applications on the integrated testbed• Many interested users and many sites available to join• DataManagement not fully exploited but possible (yet tested)• Possibility to enlarge pool of resources for next phases of DC1
top related