introduction to advanced unix march 9 2010 kevin keay

Introduction to Advanced Introduction to Advanced UNIXUNIX

March 9 2010March 9 2010

Kevin KeayKevin Keay

OutlineOutline

• OverviewOverview• ResourcesResources• The basic form of a typical C-shell scriptThe basic form of a typical C-shell script• Introduction to C-Shell ProgrammingIntroduction to C-Shell Programming• Advanced UNIX materials: The following are PDF documents that may be Advanced UNIX materials: The following are PDF documents that may be

used for reference but won’t be discussed in detail during this short used for reference but won’t be discussed in detail during this short course:course:

Part 1 : Part 1 : ScriptsScriptsPart 2 : Part 2 : A summary of useful softwareA summary of useful softwarePart 3 : Part 3 : Reanalysis productsReanalysis products

• Lab session (informal) 2-3 PM UNIX LabLab session (informal) 2-3 PM UNIX Lab See:See:

http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Advanced_UNIX_Lab_Session_2010.pdfhttp://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Advanced_UNIX_Lab_Session_2010.pdf

http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Intro_to_C-Shell_Programming_2008.pdf

http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Intro_to_Advanced_UNIX-Part_1_2008.pdf



OverviewOverview

• This is a practical course that is This is a practical course that is designed to give you the basic skills designed to give you the basic skills to write UNIX C-shell scriptsto write UNIX C-shell scripts

• You can only truly understand the You can only truly understand the concepts by putting them into concepts by putting them into practice!practice!

Overview (2)Overview (2)

• We will look at the basic form of a C-shell scriptWe will look at the basic form of a C-shell script• There will be a quick tour through There will be a quick tour through Introduction to C-Introduction to C-

Shell ProgrammingShell Programming• There are some extra advanced UNIX materials There are some extra advanced UNIX materials

(Parts 1-3) that you may need to reference. Some (Parts 1-3) that you may need to reference. Some aspects will be looked at during the lab session but aspects will be looked at during the lab session but they are intended for personal reference they are intended for personal reference

• Part 1 covers C-shell scripts and some additional Part 1 covers C-shell scripts and some additional conceptsconcepts

• Part 2 is a summary of some useful softwarePart 2 is a summary of some useful software• Part 3 focuses on downloading and decoding Part 3 focuses on downloading and decoding

reanalysis data (NetCDF and GRIB)reanalysis data (NetCDF and GRIB)• Finally, there is a short Lab sessionFinally, there is a short Lab session

ResourcesResources

• A very useful and concise reference book covering UNIX and the C A very useful and concise reference book covering UNIX and the C shell is:shell is:UNIX in a NutshellUNIX in a Nutshell, O’Reilly and Associates (Engineering Library: , O’Reilly and Associates (Engineering Library: 005.43 GILL). 005.43 GILL).

• A useful online resource from the University of Surrey, A useful online resource from the University of Surrey, UNIX UNIX Tutorial for Beginners,Tutorial for Beginners, is available at: is available at: http://www.ee.surrey.ac.uk/Teaching/Unix/http://www.ee.surrey.ac.uk/Teaching/Unix/

• For PDF files of the handouts see: For PDF files of the handouts see: http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/

• All UNIX commands should have a manual (man) page on the All UNIX commands should have a manual (man) page on the machine that you are using e.g. man awk. However sometimes the machine that you are using e.g. man awk. However sometimes the pages are not installed or are in an unexpected location. It may be pages are not installed or are in an unexpected location. It may be more convenient to use a search engine like more convenient to use a search engine like GoogleGoogle to find to find information on a command.information on a command.

http://www.ee.surrey.ac.uk/Teaching/Unix/

http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/

The basic form of a typical C-shell The basic form of a typical C-shell scriptscript

• The first line of a C-shell script (text file) is:The first line of a C-shell script (text file) is:#!/bin/csh –f#!/bin/csh –fThe –f option tells the script not to source The –f option tells the script not to source (read) the user’s .cshrc file; this is faster and (read) the user’s .cshrc file; this is faster and also makes the script more portable.also makes the script more portable.

• All other lines starting with All other lines starting with ## are comments are comments• Commands may be continued onto Commands may be continued onto

subsequent lines with subsequent lines with \\• Multiple commands can be placed on a Multiple commands can be placed on a

single line withsingle line with ; ;• SpacesSpaces around operators and commands are around operators and commands are

required but there are no other restrictionsrequired but there are no other restrictions

The basic form of a typical C-shell The basic form of a typical C-shell script (2)script (2)

• For neatness, end the script with For neatness, end the script with exitexit (not essential)(not essential)

• Shell variablesShell variables start with a start with a $$ (this is only (this is only omitted with using omitted with using setset or or @@ - see below) - see below)

• The shell variable The shell variable $#argv$#argv contains the contains the number of argumentsnumber of arguments (items) typed on (items) typed on the command line. The items are the command line. The items are referred to as shell variables $1, $2, $3, referred to as shell variables $1, $2, $3, …, $#argv…, $#argv


• For example:For example:

myscript jja -4.56 yesmyscript jja -4.56 yes

would give:would give:

$#argv = 3$#argv = 3

$1 = jja$1 = jja

$2 = -4.56$2 = -4.56

$3= yes$3= yes


• Shell Shell variablesvariables are either are either stringstring (text) or (text) or numericnumeric

• A string variable is set (defined) by:A string variable is set (defined) by:set var = valueset var = valuee.g. e.g. set x = Fredset x = FredNote: Variables and values are Note: Variables and values are case-case-sensitivesensitive i.e. x is not the same as X i.e. x is not the same as X

• Variable names can be several characters Variable names can be several characters e.g. e.g. set Case2 = 3set Case2 = 3

• To print (display) the value of a variable:To print (display) the value of a variable:echo $varecho $vare.g.e.g. echo $x echo $xFredFred

The basic form of a typical C-shell The basic form of a typical C-shell script (5)script (5)• A numeric variable is set (defined) by:A numeric variable is set (defined) by:

@ var = integer@ var = integere.g. e.g. @ k = -7@ k = -7

• Simple calculations may be performedSimple calculations may be performede.g.e.g. @ j = ($k + 1) @ j = ($k + 1) [j would be – 7 + 1 = -6][j would be – 7 + 1 = -6]@ k = ($k + 1) @ k = ($k + 1) would change k from -7 to -6would change k from -7 to -6Could also use C notation: Could also use C notation: @ k ++@ k ++@ j = (2 * $k - 5) @ j = (2 * $k - 5) [k= -7 would give j= -19][k= -7 would give j= -19]Note: Put spaces around operators like *, + etc.Note: Put spaces around operators like *, + etc.

• Floating point operations are not normally used in Floating point operations are not normally used in scripts but can be done with the command bc scripts but can be done with the command bc e.g. e.g. echo "2.1 + 6.3" | bcecho "2.1 + 6.3" | bcwould print 8.4 on the screenwould print 8.4 on the screenset y = `echo "2.1 + 6.3" | bc`set y = `echo "2.1 + 6.3" | bc`would save the answer in variable ywould save the answer in variable y


• A shell variable may be given the A shell variable may be given the outputoutput of a command (or user program) of a command (or user program) as in the bc example. In general:as in the bc example. In general:set var = `command`set var = `command`e.g. e.g. set ff = `ls *.dat`set ff = `ls *.dat`Remember: ls is used to list filesRemember: ls is used to list files..If the files are 5.dat 12.dat 13.dat thenIf the files are 5.dat 12.dat 13.dat thenecho $ffecho $ff would display: would display: 5.dat 12.dat 13.dat 5.dat 12.dat 13.dat


• To extract the To extract the individualindividual elements of elements of variable ff we can use variable ff we can use $#ff$#ff to find the to find the number of items (similar to $#argv)number of items (similar to $#argv)e.g. e.g. echo $#ffecho $#ff would display 3 would display 3echo $ff[1]echo $ff[1] would display 5.dat would display 5.datecho $ff[2]echo $ff[2] would display 12.dat would display 12.datecho $ff[3]echo $ff[3] would display 13.dat would display 13.dat


• A A whilewhile loop is used to set up an iterative or loop is used to set up an iterative or repetitive procedure:repetitive procedure:while (condition is true)while (condition is true)

commandscommands endende.g.e.g. @ k = 1@ k = 1 # Initialise k# Initialise k

while ($k <= 3)while ($k <= 3) # Process loop # Process loop while k <= 3while k <= 3

echo $kecho $k # Display k# Display k

@ k = ($k + 1)@ k = ($k + 1) # Increment k by 1# Increment k by 1

endend


• A A foreachforeach loop is used to set up an iterative loop is used to set up an iterative or repetitive procedure involving or repetitive procedure involving filesfiles::foreach var (files)foreach var (files)

commandscommands endende.g.e.g. foreach f ([A-Z]*) foreach f ([A-Z]*) # f is a file starting with a # f is a file starting with a

capital lettercapital letter

echo $fecho $f # Print filename # Print filename

mv $f $f.capmv $f $f.cap # Rename file to have # Rename file to have extension .capextension .cap

endend

The basic form of a typical C-shell script The basic form of a typical C-shell script (10)(10)

• An An if-else-endifif-else-endif structure is used to control structure is used to control script branching:script branching:if (condition 1 is true)if (condition 1 is true)

commandscommandselse if (condition 2 is true)else if (condition 2 is true)

commandscommandselseelse

commandscommands endifendif• Note: use ‘Note: use ‘else ifelse if` ` notnot elseifelseif

The basic form of a typical C-shell script (11)The basic form of a typical C-shell script (11)

• e.g.e.g.

if ($x == 6 || $s == sea) thenif ($x == 6 || $s == sea) then

do somethingdo something

else if ($x > 10) thenelse if ($x > 10) then

do something elsedo something else

elseelsego here if neither of the above conditions is go here if neither of the above conditions is

truetrue

endifendif

• Simpler or more complex forms are possibleSimpler or more complex forms are possible

e.g.e.g. if ($y == 2) ls –l *.dat if ($y == 2) ls –l *.dat


• The The whilewhile or or ifif (condition) uses C (condition) uses C notation for logical operations:notation for logical operations:|||| oror&& && andand== == equalequal>= >= greater than or equalgreater than or equal<= <= less than or equalless than or equal

!= != not equalnot equal>> greater thangreater than<< less thanless than


• An exampleAn example#!/bin/csh –f#!/bin/csh –fif ($#argv != 1) thenif ($#argv != 1) then

echo “Usage: myscript name”echo “Usage: myscript name”exit exit # If incorrect usage end # If incorrect usage end

the scriptthe script

elseelse # If only 1 argument, branch here# If only 1 argument, branch here

set n = ($1) set n = ($1) # Put n equal to $1# Put n equal to $1

endifendifecho “Your name is “$necho “Your name is “$n# You could do other things here# You could do other things here

exitexit


• Another exampleAnother example#!/bin/csh –f#!/bin/csh –fset ff = `ls *.dat`set ff = `ls *.dat` # ff contains filenames *.dat# ff contains filenames *.dat@ nf = $#ff@ nf = $#ff # nf equals the no. of files (items # nf equals the no. of files (items in ff)in ff)@ j = 1@ j = 1 # Set counter j to 1# Set counter j to 1

while ($j <= $nf)while ($j <= $nf) # Loop while j <= nf# Loop while j <= nf

set f = $ff[$j]set f = $ff[$j] # Set f to be the jth file# Set f to be the jth file ls –l $fls –l $f # List the details of f# List the details of f# We could use the file f as an argument to a command or # We could use the file f as an argument to a command or programprogram

myprog $fmyprog $f @j = ($j + 1)@j = ($j + 1) # Increment counter j by 1# Increment counter j by 1

endend # Go here when j > nf# Go here when j > nf

echo “We are finished!”echo “We are finished!”# A message to the screen# A message to the screen

exitexit

Introduction to C-shell Introduction to C-shell programmingprogramming

• Refer to the PDF guide:Refer to the PDF guide:

http://www.earthsci.unimelb.edu.au/http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Intro_to_C-~kevin/UNIX_Course/Intro_to_C-Shell_Programming_2008.pdfShell_Programming_2008.pdf

Advanced UNIX materialsAdvanced UNIX materials

• The following are PDF documents that The following are PDF documents that may be used for reference but won’t may be used for reference but won’t be discussed in detail during this short be discussed in detail during this short course:course:

Part 1 : ScriptsPart 1 : ScriptsPart 2 : A summary of useful softwarePart 2 : A summary of useful softwarePart 3 : Reanalysis productsPart 3 : Reanalysis products

Lab sessionLab session

•Lab session (informal) 2-3 PM Lab session (informal) 2-3 PM UNIX LabUNIX Lab

See:See: http://www.earthsci.unimelb.edu.au/~kevinhttp://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Advanced_UNIX_Lab_Session/UNIX_Course/Advanced_UNIX_Lab_Session_2010.pdf_2010.pdf

introduction to advanced unix march 9 2010 kevin keay

Documents