eu datagrid segment in russia. testbed wp6

EU DataGrid segment in Russia. Testbed WP6.

V.Ilyin1, N. Kruglov1, A. Kryukov1, V. Korenkov2, V. Kolosov3, V. Mitsyn2, L. Shamardin1

1SINP MSU Moscow, Russia2JINR Dubna, Russia3IHEP Moscow, Russia

Sep 17, 2001 Lev Shamardin 2

Russia participates in the following work packages of the EU-DataGrid project:

WP6 Testbed WP8 HEP Application

Building of DataGrid infrastructure in Russia started in Moscow region on the base of several institutes (SINP MSU, ITEP, JINP, IHEP and some others). Further development for other regions (St.Petersburg, Novosibirsk) and institutes is under discussion.Main goal for Russian EU-DataGrid segment is to support Tier2 center for LHC computing. Secondary goals include chemical calculations and others.Research is sponsored by CERN-INTAS-0440 grant.

Russian EU-DataGrid Segment


Research directions

Data transfer and traffic investigation Investigation of local area links traffic speed and

performance with different size of buffering disk server. Practical implementation and testing and testing of high-

speed site-to-site data replication through Moscow region backbone.

Practical implementation and testing of site-to-site data replication Moscow-CERN.

Investigation of distributive and local data storage The study of feasibility and robustness of disk-based, tape

less data repositories of data, assembled from low cost industrial mass production components; investigation of novel storage technologies.

Investigation of the high capacity storage system implementation (tape robots) at the conditions of medium/long range domestic communications structure.


Research directions (cont.)

GRID tools (middleware) in HEP applications Elaboration of elements of cooperative network

management for communications, connecting participating institutes.

GRID system software tools test in essentially heterogeneous local and wide-area network facilities.

Research of effectiveness of CPU utilization of production farms (PC clusters)

Investigation of effectiveness of CPU utilization and other computer resources during test data production using distributed OO databases.

The study of efficiency of local schedulers of batch tasks and their scalabilities and robustness.


Testbed 0

Globus installationGRIS-GIIS configurationCertification AuthorityMan power 8 FTE

Testbed 1

Coordinated at the national level4+ participating institutesSignificant computational resources (160 CPUs, 7.5 TB)Wide range of used software“Complicated” network configuration

Testbed Activities


Hardware for the testbed 1

Software environment

Network configuration

Participating institutes: SINP, IHEP, JINR, ITEP and others. Total: 160 CPUs Pentium III 500 MHz – 1 GHz, 256 MB RAM per CPU, 6 IDE fileservers 1.2 TB each.

Linux RedHat 6.2 (kernel 2.2.14), PBS, NFS & AFS, ObjectivityDB (GDMP 1.1), Globus toolkit 1.1.3 (with GRIS/GIIS patches) or newer version if available, CONDOR on several farms.

Internal FastEthernet network, 400 Mbit interfaces on fileservers.Regional and international links up to 1Gbit

Testbed 1


Russian National GIIS

• SRCC MSU, KIAM and TCSS participate only in Russian DataGrid project and are not involved in CERN projects.

dc=ru, o=gridCountry-level GIISlhc-fs.sinp.msu.ru:2137

dc=ru, o=gridCountry-level GIISlhc-fs.sinp.msu.ru:2137

dc=sinp, dc=ru, o=gridSINP MSU, Moscowdc=sinp, dc=ru, o=gridSINP MSU, Moscow

dc=srcc, dc=ru, o=gridSRCC MSU, Moscowdc=srcc, dc=ru, o=gridSRCC MSU, Moscow

dc=itep, dc=ru, o=gridITEP, Moscowdc=itep, dc=ru, o=gridITEP, Moscow

dc=jinr, dc=ru, o=gridJINR, Dubnadc=jinr, dc=ru, o=gridJINR, Dubna

dc=kiam, dc=ru, o=gridKIAM, Moscowdc=kiam, dc=ru, o=gridKIAM, Moscow

CERN Top-levelWP6 GIIStestbed001.cern.ch:2137

CERN Top-levelWP6 GIIStestbed001.cern.ch:2137

dc=ihep, dc=ru, o=gridIHEP, Protvinodc=ihep, dc=ru, o=gridIHEP, Protvino

dc=tcss, dc=ru, o=gridTCSS, Moscowdc=tcss, dc=ru, o=gridTCSS, Moscow

dc=?, dc=ru, o=gridSt. Petersburgdc=?, dc=ru, o=gridSt. Petersburg


SINP MSU GIIS Configuration

• There are both batch and interactive machines• Batch cluster uses PBS as a batch system• Job submission via globus can be done both to PBS cluster and to separate

machines• There are some preliminary solutions for providing cluster status in GRIS (there is

no native support for this in globus)

dc=sinp, dc=ru, o=gridInstitute-level GIISlhc.sinp.msu.ru:2137

dc=sinp, dc=ru, o=gridInstitute-level GIISlhc.sinp.msu.ru:2137

lhc.sinp.msu.ru:2135 GRISlhc.sinp.msu.ru:2135 GRIS

lhc10.sinp.msu.ru:2135 GRISlhc10.sinp.msu.ru:2135 GRIS

PBSPBSlhc-fs.sinp.msu.ru:2135lhc-fs.sinp.msu.ru:2135


http://lhc.sinp.msu.ru/CA/

Russian DataGrid Certification Authority


Russian DataGrid CA issues certificates for russian institutes, labarotories and individuals participating in EU-DataGrid project.

A certificate identifies authenticy of the owner’s name, but it does not guarantee any rights.

Only the local administrator of the resource gives or cancels the permission to use the resource.

The Russian DataGrid Certification Authority is not the only CA for Russia in the EU-DataGrid project. There can be more certification authorities for different regions.

Russian DataGrid CA (cont.)


Russian DataGrid CA (cont.)

• Russian DataGrid CA issues certificates for Russian institutes, laboratories and individuals participating in EU-DataGrid project.

• A certificate identifies authenticy of the owner’s name but does not guarantee any rights.

• Only the administrator of the local resource gives or cancels the permission to use the resource.

• The Russian DataGrid Certification Authority is not the only CA for Russia in the EU-DataGrid project. There can be more certification authorities for different regions

• Russian DataGrid CA uses modified registration authorities scheme for issuing certificates.


Why do we need Registration Authorities?

• Certification Authorities are trusted by lots of people, so we need to grant “correct” issuing of certificates

• Certification Authorities must be hack proof, so “correct” issuing of the certificates means: Only valid persons can get the certificates. To check that the person is valid we must

contact him in hack proof way (by phone or direct contact for example).


Certification Authority without RA*

*RA stands for Registration Authority

A user submits a request to certification authority.

Certification Authority administrator reads the request and phones theuser to validate that this is a normal request, not a result of mail hack oran error.If the phone validation is ok, Certification Authority administrator issues a certificate and emails it back to user


Why this is not the best solution

• Performance problems with large amount of requests per day in future

• Possible errors in user identity validation by phone, direct contact is much better

• Low requests processing speed• Possible security problems with phone

contacts?


Certification Authority with a RA

• One of the ways to solve the mentioned problems is to assign “trusted” persons – Registration Authorities – in different divisions and allow them to validate the users identity.

• Certification Authority administrator in this case trusts to the RA’s and does not make any validation for requests signed by RA’s.

• We can also implement revocation mechanisms for RA’s so they can send trusted (no further checks by CA) certificate revocation requests to the CA.


CA with RA work scheme

This scheme differs from the classical scheme on the second stage when we use globus certificates for signing the requests. This scheme requires some new software, but allows to use standard globus certificates for the whole process instead of using separate certificates for RA’s and RA’s owners globus certificates.

A user generates a certificate request with grid-cert-request and sends it to his division administrator

Division administrator validates the request and signs it with his globus certificate (digital signing software is a part of the package). After this the signed request is emailed to the Certification Authority

The Certification Authority software checks the signature and trust settings for the new emailed request, issues a new certificate and emails it back to the division administrator


Implementation details

• Globus certificates are used for digital signing email messages

• The person who signed the issue request can also revoke the certificate sending a special revoke request to CA email robot. CRL is updated automatically

• User can renew his certificate sending a special renew request generated with grid-cert-renew program to the CA email robot

• Email robot can perform several supplementary functions except issuing and revoking certificates like checking the certificate validity, obtaining an issued certificate with a specified DN or serial number

• In future the CA software will keep track of certificates expiration dates and send renew challenge text to certificate owner for challenge text renew request authorization used in globus.


Certificate Revocation Lists (CRLs)

• CRLs are the only way to check the validity of the certificate in GSI. This means that for the best security we must update CRLs on the hosts running globus as fast as possible.

• All current solutions of this problem are based on scheduled web update of the CRLs from CA sites. These solutions have some big disadvantages:

CRLs update frequency often does not match the CRLs generation frequency at CAs. This causes too frequent updates or CRL expiration problems.

Signing policies are not updated automatically now. Frequent CRL updates can cause big load on the web

servers of the CAs.


• Perhaps one of the best solution for updating the CRLs is that the update is done on all sites when the CA issues a new CRL.

• There should be a way for centralized update of signing policy and CA certificates

Requirements for “ideal” update scheme


Update scheme

CA Testbed1 CA update rootObjectclass=GlobusTestbed1CAupdate

o=Grid

National CA update center 1Objectclass=GlobusTestbed1CAupdate

dc=ch, o=Grid

Institute CA update center 1Objectclass=GlobusTestbed1CAupdate

dc=cern, dc=ch, o=Grid

National CA update center 2Objectclass=GlobusTestbed1CAupdate

dc=fr, o=Grid


dc=urec.cnrs, dc=fr, o=Grid


dc=in2p3, dc=fr, o=Grid

“new CRL” request

…


Benefits of this scheme

• CRLs on all hosts will be always up to date. There will be no “CRL expired” problems even with short CRL expiration times (like several hours for example).

• There is a possibility to add new CAs into testbed1 without any headache for site administrators.

eu datagrid segment in russia. testbed wp6

Documents