CPT Demo May 10 2004

Build on SC03 Demo and extend it.

Phase 1: Doing Root Analysis and add BOSS, Rendezvous, and Pool RLS catalog to analysis workflow

Phase 2: Add analysis web client to analysis front endPhase 3: Add MC run job service and ability to submit ORCA files

Phase 1 is certainly feasible, Phase 3 uncertain (Therefore Phase 1 and 2 as backup)

People Involved:

Rick Cavanaugh (UFL) Applications

Dimitri Bourilkov (UFL) CAVES

Mandar Kulkarni (UFL) CAVES/Sphinx

Craig Prescott (UFL) (consultant on CMS production tools)

Jang Uk (UFL) Sphinx

Laukik Chitnis (UFL) Monitoring

Conrad Steenberg (Caltech)

Michael Thomas (Caltech)

Frank v. Lingen (Caltech)

…….??

People involved:

•Root Clarens Client (100%)•Web based Client (70%)•Chimera Clarens Service (100%)•Sphinx Job Submission Client (90%)•BOSS Clarens Service (90%)•Clarens Rendezvous Service (85%)•Clarens File Service (100%)•Clarens POOL Service (90%)•MCRunJob Clarens Service (??%) (Contact Anzar from FNAL)•MonaLisa (100%)

•Web Interface for JClarens (80%)•BOSS WSDL (0%)•ACL Management GUI(70%)•Catalog Browser Interface (90%)

Components used and work to be done for Demo: (between brackets an estimate of how “finished” these components are)

•Which hosts will we use (one at least located at CERN?)•What analysis and data will we use?•Will we show a multi user analysis?

RLS (RLI)

Chimera Virtual Data Catalog

Clarens-Service

Service Flow for SC03 Demo (Rick’s mods)

Sphinx Scheduler

Clarens-Service

Execution Globus + local sched

File Service Clarens-Service

Analysis Client (ROOT+Web)

WorkflowManagement:

McRunJob/MOPSphinx-Client

Clarens-Service

Computing Element

Storage Element

MonALISAMonitoring

RLS (LRC)

Grid Site

MonALISACentral Repostiory

SphinxDatabase

3

44.1

4.3

4.3.1

4.4

4.4.1

4.4.2

4.4.1

4.4.2

6

4.4.1

4.4.2

RLS (RLI)

VO management Clarens-Service

Look up Clarens-Service

Chimera Virtual Data Catalog

Clarens-Service

Service Flow for CPT Demo (Rick’s mods)

Sphinx Scheduler

Clarens-Service

Execution Globus + local sched

File Service Clarens-Service

Analysis Client (ROOT+Web)

WorkflowManagement:

McRunJob/MOPSphinx-Client

Clarens-Service

POOL RLS - Meta Data

Catalog

Clarens-Service

Computing Element

Storage Element

MonALISAMonitoring

RLS (LRC)

Grid SiteBOSS

Job-monitoringClarens-Service

MonALISACentral Repostiory

SphinxDatabase

1

2

3

44.1

4.2

4.3

4.3.1

4.4

4.4.1

4.4.2

4.4.1

4.4.2

5

6

4.4.1

4.4.2

4.4.2

Job

= BOSS

+

Clare

ns-c

lient

Service Flow (Rick’s Mods)• 1 User authenticates• 2 User looks-up which services are available• 3 User queries either:

– VDC for input data and defines the application + output data– or, POOL for input data

• 4 User sends “job” request to Workflow Manager– 4.1 WM extracts/puts abstract “job” into VDC– 4.2 WM extracts input POOL information for abstract “job”– 4.3 WM sends abstract “job” to scheduler

• 4.3.1 Scheduler queries RLS, MonALISA ; sends concrete “job” back to WM– 4.4 WM submits “concrete” job to grid site

• 4.4.1 Job executes under a BOSS-Clarens-Client Wrapper;• 4.4.2 Job finishes ; RLS/POOL is updated ; data is available via Clarens FS

• 5 User checks on status of “job” by querying BOSS Job-monitor

• 6 User uses Clarens File Service to access ROOT files

ReplicaLocation & Selection

VO management

Look up

Virtual Data Catalog

Service Flow for CPT Demo

Scheduling

Execution

Data collection

Analysis Clientauthentication

WorkflowManagement

Meta Data Catalog

Computing Element

Storage Element

RegisterRegister

Cla

rens

BOSS

Sphi

nx J

ob S

ubm

issio

n

MC Run

Job

Pool

RLS

Chim

era

Clare

ns

Clare

ns

Sphi

nx S

ched

ulin

g

Clare

ns F

ile S

ervic

e

Clare

ns R

oot C

lient

Java

web

inte

rface

clie

nt

ROOT

FAMOS

ORCA

(CMS) Implementations

Monitoring

MonALIS

A

GUI

GUI

GUIGUI

GUI

GUI

GUI

GUI

ReplicaLocation & Selection

VO management

Look up

Look up

Virtual Data Catalog

Policy & Accounting

Our Original Architecture as Comparison with the demo setup

Scheduling

Monitoring

Execution

Steering

Data collection

Analysis Clientauthentication

discovery

feedback

1

2 3

4

5

78

10

9

10

12

13

WorkflowManagement

Meta Data Catalog

ReplicaCatalog

ReplicaManagement

6

13

11

12

14

feedback

Computing Element

Sp

ecifica

tion

Storage Element

Aut

ho

rize

dA

uth

ori

zed

Analysis Analysis versioningversioning

AuthorizedAuthorized

Aut

ho

rize

dA

uth

ori

zed

9

9

Aut

ho

rize

dA

uth

ori

zed

Aut

ho

rize

dA

uth

ori

zed

AuthorizedAuthorized

12

12

Release Release resourcesresources

Auth

orized

Auth

orized

Replication based on

trend analysis

SchedulePerformance

analysis

Supervisor

AuthorizedAuthorized

Autonomous decisions

on behalf of user

Look up

Auth

orized

Auth

orized

resourcesresources

allocateallocate

RegisterRegister

Register

Reservation ExperationCla

rens

Clare

ns

Chim

era

Sphi

nx VDT-

Client

MonALIS

A

VDT-Se

rver

Clare

ns

(CMS) Implementations

ROOT-

Clare

ns/

Cojac

/

IGUANA

Pool

RefDB

CAVES

EDG R

B

ROOT

FAMOS

ORCA

Sphi

nx

Shak

ar

Clare

ns

Condo

r

Condo

rG

Multiple applications/users will execute multiple service flows in a grid environment

BOSSBOSS