jinr participation in the egee/lcg projects

http://www.egee-rdig.ru РАН 06.12.2006

JINR participation in the JINR participation in the EGEE/LCG projectsEGEE/LCG projects

Vladimir KorenkovVladimir Korenkov (JINR, Dubna) (JINR, Dubna)

NEC’2007, Varna15 September, 2007


A necessary level of all the elements of the JINR telecommunication, network and information infrastructure should be provided:

• High-throughput telecommunication data links,

• JINR local area network (LAN) backbone,

• Central computer complex and Grid segment,

• Software support of the LHC experiments.


Upgrade of Dubna-Moscow data link up to 10 Gbps in 2007 and up to 40 Gbps in 2010.

Telecommunication channels 10 Gbps construction


LHC experiments support

– Networking– Computer

power– Data storage– Software

installation and maintenance

– Mathematical support

Grid – solution for LHC experiments

Tier-2s and Tier-1s are inter-connected by the general

purpose research networks

Any Tier-2 mayaccess data at

any Tier-1

Tier-2 IN2P3

TRIUMF

ASCC

FNAL

BNL

Nordic

CNAF

SARAPIC

RAL

GridKa

Tier-2

Tier-2

Tier-2

Tier-2

Tier-2

Tier-2

Tier-2Tier-2

Tier-2

Tiers-1 for JINR

ATLAS, CMS,ALICE


JINR Central Information and Computing Complex (CICC)

June 2007

650 kSi2K

57 TB Disk


CICC comprise45 servers7 interactive nodes60 4-core computing nodes, Xeon 5150, 8GB RAM.6 2-core computing nodes, Athlon , 2GB RAM, Mirynet.

Site name: JINR-LCG2Internal CICC network – 1Gbit/secOperating system - Scientific Linux 4.4,

Scientific Linux CERN 4.5; Scientific Linux CERN Release 3.0.8.

Middleware version gLite-3_0_0 / gLite 3.1File Systems – AFS (the Andrew File System) for user Software and home directories is a world-wide distributed file system. AFS permits to share easily files in an heterogeneous distributed environment (UNIXes, NT) with a unique authentication scheme (Kerberos).

dCache- for dataUser registration System – Kerberos 5 ( AFS use Kerberos 5 for

authentication )


• Current status of GRID infrastructure at JINR• The JINR Central Information and Computing Centre

(CICC) is the element of Russian GRID Segment used for LHC computing and for the other applications. The CICC provides the necessary resources for different tasks, implied by many projects JINR researchers take part in: E391A (KEK), KLOD, COMPASS, D0, DIRAC, HARP, CMS, ALICE, ATLAS, HERAb, H1, NEMO, OPERA, HERMES, CBM, PANDA, etc. This year three cabinets (19”) were installed in the CICC with total amount of 60 dual-CPU dual-core computing nodes. Now 252 CPUs are used as working nodes. The CICC software includes a number of program packages which form GRID environment. A current version of the LCG software is gLite 3.0/3.1.

JINR LCG infrastructure


• The CICC provides the following services in the LCG environment:

• Basic services - Berkley DB Information Index (top level BDII); site BDII; Computing Element (CE); Proxy Server (PX); Resource Broker (RB); Workload Management System + Logging&Bookkeeping Service (WMS+LB); RGMA-based monitoring system collector server (MON-box); LCG File Catalog (LFC); Storage Element (SE), dCache 42TB;

• Special Services - VO boxes for ALICE and for CMS; ROCMON;

• PPS and testing infrastructure - Pre-production gLite version; xrootd door in dCache for ALICE;

• Software for VOs: dCache xrootd door, AliROOT, ROOT, GEANT packages for ALICE; ATLAS packages; CMKIN, CMSSW, OSCAR, ORCA, CORBA packages for CMS and DaVinchi, Gauss packages for LHCb.

JINR LCG infrastructure


The protocol between CERN, Russia and JINR on a participation in LCG Project has been

approved in 2003.

The tasks of the Russian institutes in the LCG have been defined as:

LCG software testing;

evaluation of new Grid technologies (e.g. Globus toolkit 3) in a context of using in the LCG;

event generators repository, data base of physical events: support and development.

LHC Computing Grid Project LHC Computing Grid Project (LCG)(LCG)


The tasks of the Russian institutes & JINR in the LCG (2004 - 2007 years):

• Task 1. MW (gLite) Testsuit (supervisor Z. Sekera) • Task 2. Castor (supervisor Sebastien Ponce)• Task 3. Genser (PH/SFT, supervisor V. Pokorski)• Task 4. MCDB (PH/SFT, supervisor V. Pokorski)• Task 5. ARDA (supervisor M. Lamanna)• Task 6. MW development (supervisor J.Knobloch)

LHC Computing Grid Project LHC Computing Grid Project (LCG)(LCG)


- support and development LCG/EGEE infrastructure;- participation in LCG middleware testing/evaluation, - participation in Data and Service Challenges,- CASTOR and dCache usage/development; - grid monitoring and accounting system development;- EGEE-RDIG portal support and development, - MCDB development; - HEP application; - User & Administrator Training and Education - support of JINR member states in the LCG/EGEE

activities.

•

JINRJINR in the LCG/EGEE in the LCG/EGEE


• Management and support of RDIG accounting. Interoperability with other national and international Grids.

• Application-resource provider coordination.• Operation monitoring of the RDIG sites. • 1st line support for operations problems of HEP VOs CMS, ALICE,

ATLAS, BIOMED and FUSION VOs.• Run grid services for computing and storage (dCache)

production services at JINR. • Coordination of grid middleware deployment and support on grid

sites of JINR member states. • Run ALICE VO-specific services for production on RDIG sites.• The further support for HEP VOs, for BIOMED and FUSION VOs,

support for new VOs. User support effort. • SA1 participation in site and user training.• Testing of gLite components. Development of certification, stress

and performance tests for gLite software and participation in gLite PPS. Debugging of gLite software. Analysis of performance and stress tests results.

• GILDA middleware deployment in RDIG Grid operation support for GILDA at JINR.

•

JINRJINR in the EGEE 3 in the EGEE 3


• Coordination of RDIG outreach and dissemination activity. Contacts with local press offices, reporting, metrics (in Russia and CIS countries).

• Support of bilingual RDIG web-site www.egee-rdig.ru. Local material and publication preparation, local info sheets, tailoring common EGEE information and content to local area: translation of key EGEE material.

• Development and preparation of the training materials (presentations, lectures, web-pages, printed materials) for training of beginners, experienced users and system administrators (mostly in Russian).

• RDIG web-portal for training: creation and support. Publication of user and system administrators guides. Preparation of the practical training courses based on the user and application requirements. Translation into Russian of the training materials, user's guides. Preparation, printing and distribution of the translated materials. Support Web page for training activity in RDIG Moscow region. Training activity management and partner coordination in RDIG Moscow region.

JINRJINR in the EGEE 3 in the EGEE 3


JINR in the LCG

• support and development of the JINR LCG-segment in frames of the global LCG infrastructure;

• grid-monitoring and accounting system of Russian LCG/EGEE infrastructure;

• evaluation of new Grid technologies (OMII Open Middleware Infrastructure Institute, Globus Toolkit 3 & 4);

• participation in testing gLite middleware (testing of VOMS)• Participation in ARDA activities in coordination with experiments

(AMGA. Dashboard, condor-G monitoring);• Monitoring and testing FTS• development of Russian EGEE-RDIG and JINR LCG web-portals; • development of the MCDB system;• participation in development CASTOR2 system;• participation in Service Challenges for CMS, ATLAS and ALICE• Support ALICE software on 12 Russian LCG-sites • courses for Grid administrators and users • international conference “Distributed computing and Grid

technologies in science and education”(GRID-2004, GRID-2006);• close cooperation and support with partners in Ukraine, Belarus,

Czech, Romania, Germany, South Africa, Bulgaria.


www.egee-rdig.ru


RDIG monitoring & accounting http://rocmon.jinr.ru:8080


Russian Grid Sites monitoring & accounting http://rocmon.jinr.ru:8080

Monitored valuesCPUs - total /working / down/ free / busyJobs - running / waiting Storage space - used / availableNetwork - Available bandwidth Accounting valuesNumber of submitted jobsUsed CPU time

Totally sum in secondsNormalized (with WNs productivity)Average time per job

Waiting timeTotally sum in secondsAverage ratio waiting/used CPU time per job

Physical memoryAverage per job


Total number of records in the Database of RDIG accounting System – 1 420 000


RDIG/LCG Farm usage

Normalised CPU time June-Sept 2007


Integration, testing, certification of new MW

Testing new MW components (ARDA):

- Metadata catalog, Fireman catalog, gridFTP,

- testing of gLite-AMGA metadata service,

-testing gLite for ATLAS and CMS,

-- Collaboration with the condor-g team and development of monitoring for CONDOR

-- development dashboard

-monitoring of CMS productionEvaluation of new MW: JINR, KIAM, SINP MSU

-evaluation of new Grid technologies (OMII Open Middleware Infrastructure Institute, Globus Toolkit 3 & 4);


Participation in the development of certification tests for VOMS

1) Creation of tests for the operations of the administrationof virtual organization (operation voms-admin). There are 2 versions of this procedure: with the use of a WEB interface; with the use command mode CLI, Command Language Interface As a result of this task 30 programs were developed,

its were included in SAM and CVS.

2) Development of tests for the user command voms-proxy-init - generation of proxy certificate.

As a result of this task 20 programs were developed, its were included in SAM and CVS.


The Russian testbed (PNPI, INR, IHEP, JINR)

was installed which provide the following gLite

3.X services:

CE

The Russian testbed

WMS+LB BDII

SE(DPM) LFC UI

WNVOMS


Monitoring of the CMS production system

ProdAgent 1ProdAgent N

ProdAgent 2

PHP based UI PHP

based UI

PHP based UI

CMS dashboard global repository

ImportHigh scalability & performance

Business IntelligenceDashboard

Python based global repository production UI

http://www.egee-rdig.ru РАН 06.12.2006Full information at https://twiki.cern.ch/twiki/bin/view/LCG/TransferOperations

1. Daily log – tracking of current problems and open issues2. Daily log Archive – history of situation on channels from 23.02.2007 till present time

3. Weekly Report – summary report for the Joint Operation Meeting4. Weekly Castor – summary of issues noticed on the Castor Tier-0 service

CERN FTS channels monitoring


User Training and Induction

User’s support to stimulate their active usage of LCG resources (courses, lectures, trainings, publication of user guides in Russian):

• courses for LCG administrators and ALICE users from Russian and Ukraine institutes (February 28- March 3, 2006);

• LCG training for ATLAS users at JINR (May 15-19, 2006);• courses on gLite for participants of the international

conference "Distributed computing and Grid technologies in science and education" (June 25-30, 2006).


GRID User Training at JINR in 2007

Courses “CMS user analysis using EGEE/LCG infrastructure” - January 19, 2007

Lectures: CMS computing support at JINR: current status and plans; Short introduction to LCG/EGEE; CMS user jobs submission with the usage of ASAP

Practical part: Usage of ASAP ("private" user jobs and jobs needed an access to CMS data bases of simulated events)

http://rdms-cms.jinr.ru/docs/rdms_1/cours1.htm

Tutorial on distributed analysis of ATLAS data - April, 2007 commands and operations with files; Data analysis with GANGA and practical part http://atlasinfo.jinr.ru/computing/tutorial_190407.html


The two major Grid conference in Russia (Grid2004 and Grid2006), hosted by Joint Institute for Nuclear Research in Dubna, has been hailed a success: - more than 200 participants; - about 100 reports.

http://lit.jinr.ru/grid2004/http://grid2006.jinr.ru

jinr participation in the egee/lcg projects

Documents