software tools using pbs. software tools portland compilers pgf77 pgf90 pghpf pgcc pgcc portland...

Software Tools

Using PBS

Software toolsPortland compilers pgf77 pgf90 pghpf pgcc pgCC Portland debugger

GNU compilers g77 gcc

Intel ifort icc

…software toolsMPICH (MPI-1, MPI-2MPI compilers mpicc mpiCC mpif77 Mpif90

mpirunYour favorite text editorPBS

Using PBS

PBS an open source job scheduler* and queue manager

PBS – Portable Batch System

Others Torque Sun Grid Engine Platform LSF IBM Load Leveler

PBS/Torque widely used

PBS

openPBS – open source versionPBSpro – commercial versionTorque

qsub [options] script

Submits jobs to a queue (execution)Job specifications in the file scriptMore on pbs scripts laterPbs will queue, schedule and launch job specified in the command/scriptqsub returns jobid -jobnumber.servername (9999.master)

qsub – common options

-q queuename -- submit to the queue queuename -q parallel@master

-N name -- give the job the name name -N mybigjob1

-I – run the job in interactive mode rather than batch

qsub – common options

-o path -- location of pbs output, defaults to ~owner -e path -- location of pbs error file, defaults to ~owner -a datetime --places job in a wait state until datetime datetime is [[[[CC]YY]MM]DD]hhmm[.SS] -- run a job on/after 5:00 pm on Feb. 14 -a 0214 1700

qsub – common options

-m [a,b,e]a --mail is sent when the job is

aborted by the batch system*.

b --mail is sent when the job begins execution.

e --mail is sent when the job terminates.

qsub – common options

-M useraddress - defines mail addresses for email (see –m) -M Don.McLaughlin@mail.wvu.edu

-S path define the location of the shell to interpret the script-S /bin/csh

qsub – common options

-l job resource list (important)nodes=nppn=m-l nodes=2:ppn=2

qstat - get status of jobs/queues

qstat –a – get status report in alternative format

qstat –a – statesE - Job is exiting after having run. H - Job is held. Q - job is queued, eligible to run or routed. R - job is running. T - job is being moved to new location. (not

here)W - job is waiting for its execution time

(qsub -a option) to be reached.

qstat – other options

qstat –an -- lists jobs in queue/running. For running jobs shows nodes/processors assigned to the job.

qstat – options

qstat –q -- list all queues and that status

qdel – deleting a job from the queue

qdel jobidjobid is sequence_number[.server_name]for example –

qdel 9999.master or

qdel 9999

other PBS commands

qhold jobid -- places job jobid on holdqrls jobid -- moves job jobid from hold to queued or executing statusqalter jobid – modifies attributes of the job jobid specified in qsub after it is submitted

** not all attributes may be modified

other PBS commands

qmove dest jobid -- moves job jobid from current queue to the queue dest for example –

qmove short 9999

PBS qsub scripts

Standard (almost) script in whatever shell is supported on the system.Must include the line –

mpirun myprogmay include other statements/directives – like…

PBS qsub scripts – cont.

a simple script – demtest.sh

#!/bin/sh#PBS -N dem#PBS -q parallel#PBS -l nodes=2:ppn=2mpirun -np 4 -machinefile $PBS_NODEFILE

hellocat $PBS_NODEFILE

PBS scripts cont…

then –

qsub demtest.sh

PBS scripts cont… and your get dem.o3975 ---

Hello world! I'm node 1 Hello world! I'm node 2 Hello world! I'm node 3 Hello world! I'm node 0node7node7node6node6

Xpbs

For more information on PBS commands-

System man pages “man qstat”http://www-wisconsin.cern.ch/pbs/commands.htmlhttp://rsusu1.rnd.runnet.ru/opbs/commands.htmlhttp://www.sao.nrc.ca/imsb/rcsg/hpc/sgi_pbs2000/pbs/sld001.htmhttp://www.ccr.buffalo.edu/documents/pbs.PDF

Logging in to the cluster

using a secure shell clientlogin in to-

energy.cluster.wvu.edu

Compiling a MPI program

mpicc myprog.c –o myprog

or

gcc myprog.c –o myprog –lmpi –L/usr/local/packages/mpich/lib –I/usr/local/packages/mpich/include