Development of Russian Grid Segment in the frames of EU

DataGRID, LCG and EGEE projects

V.A.Ilyin (SINP MSU),

V.V.Korenkov (JINR, Dubna)

NEC’2003, Varna

19 September, 2003

total processor performance 2106 MIPSdisk space for data storage50 TBrobotic mass storage 5104 TBcommunication channel CERN - regional centre 622 Мbps

Creation of a regional centre for experimental data processing of the Large Hadron Collider (LHC) in Russia

The project is intended for the years 1999 - 2007, first stage is the creation of a prototype of the centre - 1999-2001.

By the year 2005 the computing resources of the regional centre and the throughput of the links to CERN will provide:

622 Мbps

Institute of Institute of Nuclear Physics Nuclear Physics SB RSB RASAS

The distributed regional centre is expected to be created on the basis of the infrastructure of 4 centers: SINP MSU, ITEP, IHEP, and JINR. The unified computer network will be constructed for all Russian institutes participating in the LHC project

MONARC project

regional group

LHC Computing Model2001 - evolving

CERNTier3physics

department

Desktop

Germany

UK

France

Italy

CERN Tier1

USA

Tier1

The opportunity ofGrid technology

Tier2

Uni a

Lab c

Uni n

Lab m

Lab b

Uni bUni y

Uni x

Russia

DataGrid Architecture

Collective ServicesCollective Services

Information &

Monitoring

Information &

Monitoring

Replica ManagerReplica

ManagerGrid

SchedulerGrid

Scheduler

Local Application

Local Application

Local DatabaseLocal Database

Underlying Grid ServicesUnderlying Grid Services

Computing Element Services

Computing Element Services

Authorization Authentication & Accounting

Authorization Authentication & Accounting

Replica CatalogReplica Catalog

Storage Element Services

Storage Element Services

Database Services

Database Services

Fabric servicesFabric services

ConfigurationManagement

ConfigurationManagement

Node Installation &Management

Node Installation &Management

Monitoringand Fault Tolerance

Monitoringand Fault Tolerance

Resource Management

Resource Management

Fabric Storage

Management

Fabric Storage

Management

Grid

Fabric

Local Computing

Grid Grid Application LayerGrid Application Layer

Data Management

Data Management

Job Management

Job Management

Metadata Management

Metadata Management

Object to File Mapping

Object to File Mapping

Logging & Book-

keeping

Logging & Book-

keeping

EDG overview : structure , work packages

The EDG collaboration is structured in 12 Work Packages WP1: Work Load Management System WP2: Data Management WP3: Grid Monitoring / Grid Information Systems WP4: Fabric Management WP5: Storage Element WP6: Testbed and demonstrators WP7: Network Monitoring WP8: High Energy Physics Applications WP9: Earth Observation WP10: Biology WP11: Dissemination WP12: Management

}} ApplicationsApplications

Russian HEP institutes:

IHEP (Protvino), ITEP (Moscow), JINR (Dubna), SINP MSU, TC “Science and Society”(Moscow), Keldysh IAM (Moscow), RCC MSU, PNPI (St.Petersburg)

participated in the first European GRID project, EU DataGRID (WP6, WP8, WP10), with success deployment of EDG middleware and participation in EDG testbeds.

These activities led to accumulation of an experience in a work with modern Grid environment and integration of Russian Grid segment into European Grid infrastructure.

Activities of Russian institutes in EDG Project:

information service (GIIS)

certification service (Certification Authority)

data management (GDMP; OmniBack&OmniStorage)

monitoring

Metadispetcher

mass events production for CMS&ATLAS experiments

DOLLY – a solution proposed to integrate mass events production for CMS into Grid infrastructure

The technology of creation of GIIS information servers [which collect the information on local computing resources and resources of data storage (this information is created by GRIS Globus service at an each node of a distributed system) and transmit this information in a dynamical way to the higher GIIS server] has been put into practice. So way, a hierarchical structure of GRIS-GIIS information service building has been applied and tested. A common GIIS information server (ldap://lhc-fs.sinp.msu.ru:2137) has been organized. It transfers the information on local resources of Russian centers to information server (ldap://testbed1.cern.ch:2137) of European EU DataGrid project.

G I I S

Russian National GIIS

SRCC MSU, KIAM and TCSS participate only in Russian DataGrid project and are not involved in CERN projects.

dc=ru, o=gridCountry-level GIISlhc-fs.sinp.msu.ru:2137

dc=ru, o=gridCountry-level GIISlhc-fs.sinp.msu.ru:2137

dc=sinp, dc=ru, o=gridSINP MSU, Moscowdc=sinp, dc=ru, o=gridSINP MSU, Moscow

dc=srcc, dc=ru, o=gridSRCC MSU, Moscowdc=srcc, dc=ru, o=gridSRCC MSU, Moscow

dc=itep, dc=ru, o=gridITEP, Moscowdc=itep, dc=ru, o=gridITEP, Moscow

dc=jinr, dc=ru, o=gridJINR, Dubnadc=jinr, dc=ru, o=gridJINR, Dubna

dc=kiam, dc=ru, o=gridKIAM, Moscowdc=kiam, dc=ru, o=gridKIAM, Moscow

CERN Top-levelWP6 GIIStestbed001.cern.ch:2137

CERN Top-levelWP6 GIIStestbed001.cern.ch:2137

dc=ihep, dc=ru, o=gridIHEP, Protvinodc=ihep, dc=ru, o=gridIHEP, Protvino

dc=tcss, dc=ru, o=gridTCSS, Moscowdc=tcss, dc=ru, o=gridTCSS, Moscow

dc=?, dc=ru, o=gridSt. Petersburgdc=?, dc=ru, o=gridSt. Petersburg

.

Certification authority (СА) center for Russian grid segment has been created at SINP MSU. The certificates of this center are accepted by all the participants of EU DataGRID project.

A scheme of confirming of requirements to certificates by an electronic signature has been created with an assistance of Registration authority (RC) centers which are located in another institutes.

The programs on installing and checking an electronic signature and a package of automated operation of certification center have been developed.

The scheme CA+RA proposed and a program package have been accepted at CERN and other participants of EU DataGrid project.

C A

.GDMP (GRID Data Mirroring Package) – a program for replication of files and data bases - has been installed and tested. GDMP had been created for remote actions with distributed data bases. GDMP uses GRID certificates and works in accordance with a client-server scheme i.e. a replication of changes in a data base is accomplished dynamically. Periodically the server notifies the clients on the changes in a data base and the clients send the updated files with a use of GSI-ftp command. GDMP is actively user for replication purposes and is considered to become a Grid standard for replication of changes in distributed data bases.

GDMP

OMNIBACK Usage

Some tests on transfer of data from Protvino (sirius-b.ihep.su; OS Digital UNIX Alpha Systems 4.0) to ATL-2640 mass storage system in Dubna (dtmain.jinr.ru; OS HP-UX 11.0) to define a transmission capacity and a stability af a system including communication channels and a mass storage (OmniBack disk agent in Protvino and OmniBack tape agent in Dubna). No abnormal terminations have been fixed. The average speed of a transmission by all the attempts – 480 Kb/s or 1.68 Gb/h. A maximal speed – 623 Kb/s. A minimal speed – 301 Kb/s.(A distance between Dubna and Protvino is about 250 km; communication between Protvino and Moscow – 8 Mbps).

OMNISTORAGE Usage

Data storage of data obtained during CMS M.-C. Mass Production runs is provided with the usage of Omnistorage : the volumes of data from SINP MSU have been transferred to Dubna (~1 TB) tp ATL-2640; an access to data via scp.

Some first experience with a common usage of mass storage system in Dubna (ATL-2640)

Complex of works on monitoring of network resources, computing nodes, services and applications had been fulfilled.

The JINR members of staff take part in a development of monitoring facilities for computing clusters with a large number of nodes (10 000 and more) which are used in the EU Data Grid infrastructure created.

In the frames of a task of Monitoring and Fault Tolerance they take part in a creation of a Correlation Engine system. This system serves for an operative discovering of abnormal states at cluster nodes and taking measures on preventing of abnormal states.

A Correlation Engine Prototype is installed at CERN and in JINR for accounting of abnormal states of nodes.

MONITORING

The Metadispetcher program had being installed in Russian EU Data Grid segment in a cooperation with the Keldysh institute of applied mathematics.

The Metadispetcher program is served for a jobs start planning in a distributed computing grid-environment.

The program had being tested; after that the program had being modified to provide an effective data transfer by means of Globus toolkit.

Metadispetcher

A Task of Mass Event Generation for CMS Experiment at LHC

(the solution proposed)

GRID

Environment

DOLLY

BOSS

jobsmySQL DB

RefDB at CERN

CEbatch manager

NFS

WN1 WN2CMKIN

IMPALAWNn

UI

EDG-RB

UI

job executerjob

Fundamental Goal of the LCGTo help the experiments’ computing projects get the

best, most reliable and accurate physics results from the data coming from the detectors

Phase 1 – 2002-05prepare and deploy the environment for LHC computing

Phase 2 – 2006-08acquire, build and operate the LHC computing service

The protocol between CERN, Russia and JINR on a participation in LCG Project has been approved in 2003.

The tasks of the Russian institutes in the LCG:

LCG software testing;

evaluation of new Grid technologies (e.g. Globus toolkit 3) in a context of using in the LCG;

event generators repository, data base of physical events: support and development;

LCG infrastructure creation in Russia.

Since April, 2003 the groups on the directions mentioned above are created and began their work.

CollaboratingComputer Centres

Building a GridThe virtual LHC Computing Centre

Grid

Alice VO

CMS VO

Russian LCG Portal

Russian LCG Portal

Russian LCG Portal

Monitoring FacilitiesMonitoring Facilities

Monitoring FacilitiesMonitoring Facilities

Monitoring FacilitiesMonitoring Facilities

The EGEE (Enabling Grids for E-science in Europe) project is accepted by the European Commission (6th Framework program). The aim of the project is to create a global Pan-European computing infrastructure of a Grid type.

Main goal is the integration of Russian GRID segments, created during past two years, in the European GRID infrastructure to be developed in the framework of EGEE project.

EGEE

Russian Data Intensive GRID (RDIG) Consortium EGEE

FederationEight Russian Institutes made up the consortium RDIG (Russian Data IntensiveGRID) as a national federation in the EGEE project. They are: IHEP - Institute of High Energy Physics (Protvino), IMPB RAS - Institute of Mathematical Problems in Biology (Russian Academy of Science, Pushchino, ITEP - Institute of Theoretical and Experimental Physics (Moscow), JINR - Joint Institute of Nuclear Physics (Dubna), KIAM RAS - Keldysh Institute of Applied Mathematics (Russian Academy ofScience, Moscow), PNPI - Petersburg Nuclear Physics Institute (Russian Academy of Science,Gatchina), RRC KI - Russian Research Center 'Kurchatov Institute' (Moscow), SINP-MSU - Skobeltsyn Institute of Nuclear Physics (Moscow State University,Moscow).

The Russian memorandum on a creation of a Grid type computing infrastructure on distributed processing of huge data volumes has being signed in September, 2003 by the Directors of the eight institutes.

Russian Contribution to EGEE RDIG as an operational and functional part of EGEE

infrastructure (CIC, ROC, RC; integration with EGEE).

Specific Service activities:

SA1 - Creation of Infrastructure

SA2 – Network Activities

NA2 – Dissemination and Outreach

NA3 – User Training and Induction

NA4 - Application Identification and Support