ganga 4 basics - tutorial jakub t. moscicki arda/lhcb ganga tutorial, november 2005

Ganga 4 Basics - Tutorial

Jakub T. MoscickiARDA/LHCb

Ganga Tutorial, November 2005

Ganga Tutorial, April 2005 2

Contents

• What is Ganga

• Quick introduction to Python Statements, variables, functions Modules, classes, objects

• Basics of Ganga Scripting Interface simple executable jobs Job Registry and looping


What is Ganga?

AtlasPROD

DIAL

DIRAC

LCG2

gLite

localhost

LSF

submit, kill

get outputupdate status

store & retrieve job definition

prepare, configure

Ganga4

JobJobJobJob

scripts

Gaudi

Athena

AtlasPROD

DIAL

DIRAC

LCG2

gLite

localhost

LSF


• Ganga Web Site: http://cern.ch/ganga

• Ganga Tutorial• Bug reports

• Ganga is based on Python 2.2 Python Website:

• http://www.python.org Excellent Python Tutorial:

• http://docs.python.org/tut/tut.html

• Other resources: IPython page:

• http://ipython.scipy.org/

Documentation


Running Ganga

• LHCb account in lxplus

• % GangaEnv• The last public release is 4.0.1. It is a default choice. Press ENTER.

• % ganga*** Welcome to Ganga ***Version: Ganga-4-0-1Documentation and support: http:/cern.ch/gangaType help() or help('index') for online help.

• You will be asked few questions. Answer “yes” by pressing ENTER.• GRID certificate: type your password or press Control-D several times.

• To quit press Control-D at the interactive prompt

In[1]: ^DDo you really want to exit ([y]/n)?%


Ganga Text Prompt

• Ganga is based on python 2.2 and has an enhanced python prompt (IPython): python programming/scripting

• myvariable = 5• print myvariable*10

easy access to the shell commands• !less ~/.gangarc # personal config file• !pwd

history <arrow up> TAB completion

• it works on keywords, variables, objects, try:– my<TAB>

many more features


• x = 2print x*3 # you may also skip print

if x==2: print "yes, x==2" # NOTE INITIAL SPACES!

alist = [1,2,3]for y in alist: print y

print len(alist) # built-in function lenhelp(len)

Python: Statements, Variables


• print range(11)help(range)

def square(v): return v*v

print square(x)

alist = [1,2,3]for i in range(len(alist)): alist[i] = square(alist[i])

blist = [square(x) for x in alist]

Python: Functions


• import sysprint sys.argv

import osprint os.environ['HOME']

from os import environfrom os import *

import math, cmath

print math.cos(math.pi)

v =(1+1j)print v*v

math.log(-1) # exception

Python: Modules


• # user defined classes

a = MyClass(2,3)a.my_method(1)

#built-in classes: for example strings

s = "hello at the tutorial" # double quotess.split()s.count('a')s.upper()

dir(s)

if 'tut' in s: print 'tut is in ',s print '-'*20 # single quotes

Python: Classes and Objects


• # file objects

f = file('/etc/password')lines = f.readlines()print len(lines)

# writing files

f2 = file('testfile','w')print >> f2, “this is a test”,20*'-'f2.close()

dir(f)

Python: Files


• Job Objects and Job Registry

[In:] j = Job()[In:] print j[In:] print j.application[In:] print j.backend

# job registry: a list of all your jobs[In:] jobs[In:] jobs[1]

Ganga Scripting


Jobs

Job ( status = 'new' , inputdir = '/afs/cern.ch/user/k/kuba/gangadir/workspace/Local/2/input/' , name = '' , outputdir = '/afs/cern.ch/user/k/kuba/gangadir/workspace/Local/2/output/' , outputsandbox = [] , id = 2 , application = Executable ( exe = '/bin/hostname' , env = {} , args = [] ) , inputdata = None , inputsandbox = [ ] , backend = Local ( exitcode = None , id = None ) )


• [In:] j = Job()[In:] j.application.exe = '/bin/hostname'[In:] j.name = "MyTest"[In:] print j[In:] j.submit()

• # wait until job is completed and look# at the output directory

[In:] print j.status[In:] print j.outputdir[In:] !ls $j.outputdir[In:] !cat $j.outputdir/stdout

# the ! and $ syntax is IPython – you # cannot do it in normal python# it is useful for interactive work# instead of mouse copy-paste

Basic Job Submission


• # once a job is submitted you cannot modify# it. if you want to submit a new job you# should create a new job object

[In:] j2 = j.copy()[In:] j2.backend = LSF()[In:] j2.submit()

•# if you have GRID certificate you can try

[In:] j3 = j.copy()[In:] j3.backend = LCG()[In:] j3.submit()

# print jobs to see all your jobs

More submission


• # see predefined plugins• # (only some combinations supported)• applications()• backends() • help('index')

• # submission in a loop

for i in range(5): j = Job() j.application.exe="/bin/echo" j.application.args=["hello ",str(i)]) print j.id, j.inputdir print j.submit()

More ideas


• # For frequently used jobs – use templates!t = JobTemplate(j3)print templates

• # Other useful methodsj.kill()j.remove() # Removes the output as well!for j in jobs: j.kill()jobs.clean() # Faster! Deletes ALL jobs.

More Ideas (2)


• Ganga 4.0.1 limitation: only one session running at a timeGanga.Runtime : CRITICAL Another Ganga session is running (lock file exists: /afs/cern.ch/user/k/kuba/gangadir/repository/.session_lock)Ganga.Runtime : CRITICAL Details: Ganga started at Tue Nov 15 12:31:44 2005, pid 1911, host: lxplus012.cern.chGanga.Runtime : CRITICAL THIS VERSION DOES NOT SUPPORT MULTIPLE SESSIONS USING THE SAME LOCAL JOB REPOSITORYGanga.Runtime : CRITICAL TRY TO FIND OLD RUNNING SESSION AND QUIT IT FIRSTGanga.Runtime : CRITICAL If you cannot find the old session this may mean that Ganga terminated uncleanly (e.g. was killed).Ganga.Runtime : CRITICAL In this case, remove the lock file manually and restart Ganga

• If you remove the lock file and have two sessions running you may lose some jobs. Be warned!

• Try: ganga -h to see options

• Have a look also in the config file: ~/.gangarc

Hints and Pitfalls


• Questions?

• Practical exercises

Questions?

ganga 4 basics - tutorial jakub t. moscicki arda/lhcb ganga tutorial, november 2005

Documents