Working with AliEnWorking with AliEn

Kilian SchwarzKilian Schwarz

ALICE Group MeetingALICE Group Meeting

April 19 2006April 19 2006

Initialisation and loginInitialisation and login

. /u/aliprod/bin/.alienlogin. /u/aliprod/bin/.alienlogin alien proxy-initalien proxy-init alien-token-initalien-token-init alienshaliensh

Searching technique for simulated Searching technique for simulated data with certain propertiesdata with certain properties

Not possible with PDC01-05-data Not possible with PDC01-05-data wait for PDC06 wait for PDC06 Current method:Current method: alienshaliensh alien: find /alice/cern.ch/demo/data %.root > alien: find /alice/cern.ch/demo/data %.root >

/tmp/collection.xml/tmp/collection.xml The XML collection file can be read directly by The XML collection file can be read directly by

ROOTROOT DC data can be found, e.g. in DC data can be found, e.g. in

alien://alice/cern.ch/user/a/aliprod/production_200alien://alice/cern.ch/user/a/aliprod/production_2005/5/

Large scale data simulation using Large scale data simulation using GridGrid

Example at Example at alien://alice/cern.ch/demo/production/flow.jdlalien://alice/cern.ch/demo/production/flow.jdl

Needed configuration files can be found in Needed configuration files can be found in alien://alice/cern.ch/demo/production/configs (e.g. alien://alice/cern.ch/demo/production/configs (e.g. Config.C, rec.C, sim.C, simrun.C,…)Config.C, rec.C, sim.C, simrun.C,…)

Start production with alien> submit flow.jdl 001 Start production with alien> submit flow.jdl 001 (run number, is used as SEED for random (run number, is used as SEED for random generator to be able to repeat simulation with generator to be able to repeat simulation with same random generator initialisation)same random generator initialisation)

If you want to modify, create your own jdl and If you want to modify, create your own jdl and configuration files, following the given examples.configuration files, following the given examples.

The production JDLThe production JDL # our executable# our executable Executable = “aliroot”; (compulsary !!!)Executable = “aliroot”; (compulsary !!!) Has to be in alien://bin, /alice/bin or $HOME/bin. “aliroot” exists already Has to be in alien://bin, /alice/bin or $HOME/bin. “aliroot” exists already

in /alice/bin and is a shell script executing AliRoot.in /alice/bin and is a shell script executing AliRoot. # this are pseudo names to identify the production# this are pseudo names to identify the production Jobtag={“comment:PDC05 flow events”, “round::2005”, “tag:v4-03-04”, Jobtag={“comment:PDC05 flow events”, “round::2005”, “tag:v4-03-04”,

“type:Flow”};“type:Flow”}; # the packages we need# the packages we need Packages={“AliRoot”::v4-03-04”,”GEANT3::v1-3”};Packages={“AliRoot”::v4-03-04”,”GEANT3::v1-3”}; Jobs runs only at sites where those packages are installedJobs runs only at sites where those packages are installed # the estimated maximal runtime in seconds# the estimated maximal runtime in seconds TTL = “86400”; WN has to provide the requested run timeTTL = “86400”; WN has to provide the requested run time # this script runs when the job is finished. If it returns 0, the job is # this script runs when the job is finished. If it returns 0, the job is

validated (SAVEDvalidated (SAVEDDONE), if it returns !=0, the job ends in status DONE), if it returns !=0, the job ends in status ERROR_VERROR_V

Validationcommand = Validationcommand = “/alice/cern.ch/demo/production/scripts/validation.sh”;“/alice/cern.ch/demo/production/scripts/validation.sh”;

The production JDL 2The production JDL 2 # this uses only sites, which are member in this particular grid partition# this uses only sites, which are member in this particular grid partition Requirements = member(other.GridPartitions,”Production”);Requirements = member(other.GridPartitions,”Production”); # this are the input files/macros we need# this are the input files/macros we need InputFile={ InputFile={ “LF:/alice/cern.ch/demo/production/configs/Config.C”, “LF:/alice/cern.ch/demo/production/configs/Config.C”,

“LF:/alice/cern.ch/demo/production/configs/simrun.C”, “LF:/alice/cern.ch/demo/production/configs/simrun.C”, “LF:/alice/cern.ch/demo/production/configs/sim.C”, “LF:/alice/cern.ch/demo/production/configs/sim.C”, “LF:/alice/cern.ch/demo/production/configs/rec.C”, “LF:/alice/cern.ch/demo/production/configs/rec.C”, “LF:/alice/cern.ch/demo/production/configs/CheckESD.C”“LF:/alice/cern.ch/demo/production/configs/CheckESD.C”};};

These files will be transported to the node where the job will run and These files will be transported to the node where the job will run and can bei either lfn or pfncan bei either lfn or pfn

# we want to pack the output together into archive files. One with all # we want to pack the output together into archive files. One with all the root files, one with all the log files.the root files, one with all the log files.

OutputArchive={ OutputArchive={ “root_archive.zip:galice.root,Kinematics.root,AliESDs.root@Alice::C“root_archive.zip:galice.root,Kinematics.root,AliESDs.root@Alice::CERN::Castor2ERN::Castor2”,”,

” ”log_archive:*.log,stdout, stderr@Alice::CERN::se01log_archive:*.log,stdout, stderr@Alice::CERN::se01”};”};

The production JDL 3The production JDL 3 # we define the arguments for the simulation script as run number and # we define the arguments for the simulation script as run number and

event numberevent number Splitargument=“simrun.C –run $1 –event #alien_counter#”;Splitargument=“simrun.C –run $1 –event #alien_counter#”; Arguments field for each job. Here the subjobs counter produced by Arguments field for each job. Here the subjobs counter produced by

the Split tag below is given. If more than one value is defined, each the Split tag below is given. If more than one value is defined, each subjob will be submitted as many times as there are items in the array, subjob will be submitted as many times as there are items in the array, and the subjobs will have the element in the array as arguments.and the subjobs will have the element in the array as arguments.

# we produce event number 1-10, the runnumber is given as argument # we produce event number 1-10, the runnumber is given as argument $1 during the submission$1 during the submission

Split = “production:1-10”; Split = “production:1-10”; The job can be splitted in several sub jobs, according to file (1 job per The job can be splitted in several sub jobs, according to file (1 job per

input data file), directory (all input files of 1 directory will be analysed in input data file), directory (all input files of 1 directory will be analysed in 1 job), event (all files with the same name of the last subdirectory in 1 1 job), event (all files with the same name of the last subdirectory in 1 job), production #start-&lt#end (no InputData required, it will submit the job), production #start-&lt#end (no InputData required, it will submit the same JDL several times from #start to #finish. This counter can be same JDL several times from #start to #finish. This counter can be referenced in Splitarguments using #alien_counter#referenced in Splitarguments using #alien_counter#

Workdirectorysize={“1500 MB”}; The node has to provide the Workdirectorysize={“1500 MB”}; The node has to provide the requested sizerequested size

Job statusJob status

Can be investigated by usingCan be investigated by using alien:// psalien:// ps e.g.: ps –f DONE (lists all finished jobs)e.g.: ps –f DONE (lists all finished jobs) ps –u user (all jobs from the user)ps –u user (all jobs from the user) ps –s site (all jobs on one site)ps –s site (all jobs on one site) ps –trace <jobid> all (gives job trace information)ps –trace <jobid> all (gives job trace information) Output can be found in alien://proc/userOutput can be found in alien://proc/user Copy output back to local via e.g.Copy output back to local via e.g. alien://cp output.root file:/tmp/output.rootalien://cp output.root file:/tmp/output.root