lcg / arc interoperability status michael grønager, phd (uni-c / nbi) january 19, 2006, uppsala

LCG / ARC Interoperability LCG / ARC Interoperability Status Status

Michael Grønager, PhD(UNI-C / NBI)

January 19, 2006, Uppsala

NGN5 Workshop, Uppsala, January 2006 2

OverviewOverview

Interoperability background

Workpackage status– LCG->ARC– ARC->LCG– Standards

Summery and Actions

NGN5 Workshop, Uppsala, January 2006 3

Interoperability backgroundInteroperability background

Many sites have deployed ARC(~50sites ~5000CPUs)

Sites wish to provide CPU power via ARC Experiment support are keyed towards

LCG

Previous Meetings:– Rome, February 2005– CERN, August 2005– CERN, October 2005– Uppsala, January 2006 - Yesterday

NGN5 Workshop, Uppsala, January 2006 4

Interoperability backgroundInteroperability background

Proposed goals from the first CERN meeting:

– Short term: Multiple Middlewares at large sites

– Medium term: Gateways between grids

– Long term: Common Interfaces

NGN5 Workshop, Uppsala, January 2006 5

Multi middleware clustersMulti middleware clusters

Simplest form of interoperability is co-existence

Clusters with ARC/LCG co-existing– Germany: FZK, LRZ (1200CPUs)– Sweden: Ingrid, Ingvar, Bluesmoke (300CPUs)– Denmark: Morpheus(20CPUs),

Steno(scheduled) (400CPUs)

Not nice for site administrators though -need to install/maintain two middlewares…

NGN5 Workshop, Uppsala, January 2006 6

Common Interfaces...Common Interfaces...

VO management

Job submission

Job description

Information

Data management

Data transfer

Basis

Service/component

GridFTPCondor-G to GRAM

VOMSVOMS

RSLJDL (based on classAds)

LDAP, ARC schema, MDS-GIISLDAP, GLUE1.1, MDS+BDII, R-GMA

RC, RLS, FiremanEDG RLS, Fireman & Co, LFC

GridFTP, SRM v1.1 clientGridFTP, SRM

GT2 own patch, GT4GT2 from VDT

ARCLCG-2, gLite

Diverges mainly on: Job submission and description Information system

NGN5 Workshop, Uppsala, January 2006 7

GatewaysGateways

Work plan proposed at the CERN meeting:– WP1: Document LCG CE (David Smith)– WP2: LCG to ARC job submission (Laurence

Fields)– WP3: ARC to LCG job submission (Balazs

Konya)– WP4: Service discovery (Laurence Fields)– WP5: GLUE2 (long term…)

NGN5 Workshop, Uppsala, January 2006 8

GatewaysGateways

Work plan proposed at the CERN meeting:– WP1: Document LCG CE (David Smith)– WP2: LCG to ARC job submission (Laurence

Fields)– WP3: ARC to LCG job submission (Balazs

Konya)– WP4: Service discovery (Laurence Fields)– WP5: GLUE2 (long term…)

NGN5 Workshop, Uppsala, January 2006 9

WP2: LCG to ARC job WP2: LCG to ARC job submissionsubmission

LCG submits to resources using Condor-G Condor-G can submit to:

– Condor– GT2, GT3, GT4– ARC(v4.5) needs Condor v6.7.13 or newer

Plan is to change the gLite RB to enable ARC submission through Condor-G

(Alternative is to mask ARC as one LCG CE with several Condor queues)

NGN5 Workshop, Uppsala, January 2006 10

WP2: Condor-G statusWP2: Condor-G status

Condor-G can submit jobs to ARC– Tested with small hello-world kind of jobs– > condor_submit job.jdl

• Where the jdl file has gridtype=”nordugrid”• And e.g. grid_scheduler=”morpheus.dcgc.dk”

NGN5 Workshop, Uppsala, January 2006 11

WP2: Condor-G statusWP2: Condor-G status

Condor-G can submit jobs to ARC– Tested with small hello-world kind of jobs– > condor_submit job.jdl

• Where the jdl file has grid_type=”nordugrid”• And e.g. grid_scheduler=”morpheus.dcgc.dk”

Condor-G can do brokering between ARC resources– Script: ldapsearch -> condor_advertise

Condor-G can even do brokering between ARC and LCG resources...

NGN5 Workshop, Uppsala, January 2006 12

WP2: gLite RB...WP2: gLite RB...

The gLite RB uses Condor-G to submit jobs to:– Condor CEs– LCG CEs– gLite CEs

Well prepared for adding new CE types Discussions with gLite developers to add

this once tested

Planned action before next meeting

NGN5 Workshop, Uppsala, January 2006 13

WP3: ARC to LCG submissionWP3: ARC to LCG submission

ARC client does the brokering Direct submission from arc-client to

LCG/gLite Ces Options:

– GRAM2 (scaling problems...)– CondorG (still needs FQHN on client and adds

extra dependencies)– gLite to support web-service based submission– ... submit via resource broker

NGN5 Workshop, Uppsala, January 2006 14

WP4: Resource DiscoveryWP4: Resource Discovery

Translation between arc and glue schema Script that creates a glue mapping of arc

resources. Online at: arc-bdii.cern.ch

– Exercise:• Ldapsearch -x -H ldap://arc-bdii.cern.ch:2170 -b

o=grid

Meant as input for the modified gLite RB

NGN5 Workshop, Uppsala, January 2006 15

WP5: GLUE2WP5: GLUE2

Proposal on the way for splitting into:

– Transport-level-schema for info system

– Abstract-level-schema for clients

NGN5 Workshop, Uppsala, January 2006 16

Summery: New actionsSummery: New actions

Glite RB modification

Administrative / organizational integration of interoperable ARC sites

Retrieving accounting-info from the ARC sites