reconstruction in jim: status and plan

Reconstruction in JIM: status and plan

Gabriele GarzoglioSep , 13 2004

Sep 13, 2004 Gabriele Garzoglio 2

The Goal

Submit a reconstruction job to the SAM-Grid infrastructure from any JIM client installationExecute the reconstruction job to any SAM-Grid execution siteNo d0 software release is assumed to be pre-installed at the siteRefine monitoring using the JIM information management infrastructure

Sep 13, 2004 Gabriele Garzoglio 3

Status

Working on the infrastructure at the execution siteIain has provided a tar bal of p14.06.02 that can be used together with mc_runjob to do reconstruction on a “bare” nodeThe tar ball has been tested interactively on a single local file and needs a few additions (mainly the RCP database).

Sep 13, 2004 Gabriele Garzoglio 4

Short term plan

Store the tar ball (+ RCP db) into SAM Write a jim-job-manager plug in, in order to submit the reco job to a batch system

Submission tests can be done at samgfarmEventually the submission will be triggered via the grid

A wrapper around the job will prepare the execution environment:

It will ask SAM for the tar ball, mc_runjob and an input fileIt will generate the runjob macro

Sep 13, 2004 Gabriele Garzoglio 5

Longer term plan

Enhance the jim-job-manager plug in to submit parallel batch jobsHave mc_runjob talk to sam to get the input files (as opposed to doing it in the job preparation wrapper) and store the outputWrite a client plug in to pass the user information to the jim-job-managerStart testing submission over the grid

Sep 13, 2004 Gabriele Garzoglio 6

Open questions 1

What information does the client need to pass to the job?

input sam datasetd0reco versionbinary sam dataset???

Is this information going to be encapsulated into a request system?

Sep 13, 2004 Gabriele Garzoglio 7

Open questions 2

What is the model of grid job submission?

Deal with the reco of 100 or 1,000 or 10,000 files as a single grid job?

What policy of local job submission should we enforce?

Submit as many local jobs as files in the dataset, processing only a file per jobSubmit a fixed number of jobs that will process the whole dataset???