unix advanced

8/3/2019 Unix Advanced

1/22

Introduction to AdvancedIntroduction to Advanced

UNIXUNIX

March 9 2010March 9 2010


2/22

OutlineOutline

OverviewOverview ResourcesResources The basic form of a typical CThe basic form of a typical C--shell scriptshell script Introduction to CIntroduction to C--Shell ProgrammingShell Programming

Advanced UNIX materials: The following are PDF documents that may be used forAdvanced UNIX materials: The following are PDF documents that may be used forreference but wont be discussed in detail during this short course:reference but wont be discussed in detail during this short course:

Part 1 :Part 1 : ScriptsScriptsPart 2 :Part 2 :A summary of useful softwareA summary of useful softwarePart 3 :Part 3 : Reanalysis productsReanalysis products

Lab session (informal) 2Lab session (informal) 2--3 PM UNIX Lab3 PM UNIX LabSee:See: http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Advanced_UNIX_Lab_Session_2010.pdfhttp://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Advanced_UNIX_Lab_Session_2010.pdf


3/22

OverviewOverview

This is a practical course that is designedThis is a practical course that is designedto give you the basic skills to write UNIXto give you the basic skills to write UNIX

CC--shell scriptsshell scriptsYou can only truly understand theYou can only truly understand the

concepts by putting them into practice!concepts by putting them into practice!


4/22

Overview (2)Overview (2)

We will look at the basic form of a CWe will look at the basic form of a C--shell scriptshell script There will be a quick tour throughThere will be a quick tour through Introduction to CIntroduction to C--ShellShell

ProgrammingProgramming

There are some extra advanced UNIX materials (Parts 1There are some extra advanced UNIX materials (Parts 1--3)3)that you may need to reference. Some aspects will bethat you may need to reference. Some aspects will belooked at during the lab session but they are intended forlooked at during the lab session but they are intended forpersonal referencepersonal reference

Part 1 covers CPart 1 covers C--shell scripts and some additional conceptsshell scripts and some additional concepts

Part 2 is a summary of some useful softwarePart 2 is a summary of some useful software Part 3 focuses on downloading and decoding reanalysisPart 3 focuses on downloading and decoding reanalysisdata (NetCDF and GRIB)data (NetCDF and GRIB)

Finally, there is a short Lab sessionFinally, there is a short Lab session


5/22

ResourcesResources

A very useful and concise reference book covering UNIX and the C shell is:A very useful and concise reference book covering UNIX and the C shell is:UNIX in a NutshellUNIX in a Nutshell, OReilly and Associates (Engineering Library: 005.43, OReilly and Associates (Engineering Library: 005.43GILL).GILL).

A useful online resource from the University of Surrey,A useful online resource from the University of Surrey, UNIX Tutorial forUNIX Tutorial forBeginners,Beginners, is available at:is available at: http://www.ee.surrey.ac.uk/Teaching/Unix/http://www.ee.surrey.ac.uk/Teaching/Unix/

For PDF files of the handouts see:For PDF files of the handouts see:http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/http://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/

All UNIX commands should have a manual (man) page on the machine thatAll UNIX commands should have a manual (man) page on the machine thatyou are using e.g. man awk. However sometimes the pages are not installedyou are using e.g. man awk. However sometimes the pages are not installedor are in an unexpected location. It may be more convenient to use a searchor are in an unexpected location. It may be more convenient to use a searchengine likeengine like GoogleGoogle to find information on a command.to find information on a command.


6/22

The basic form of a typical CThe basic form of a typical C--shell scriptshell script

The first line of a CThe first line of a C--shell script (text file) is:shell script (text file) is:#!/bin/csh#!/bin/csh ffTheThe f option tells the script not to source (read)f option tells the script not to source (read)

the users .cshrc file; this is faster and alsothe users .cshrc file; this is faster and alsomakes the script more portable.makes the script more portable. All other lines starting withAll other lines starting with ## are commentsare comments Commands may be continued onto subsequentCommands may be continued onto subsequent

lines withlines with \\ Multiple commands can be placed on a singleMultiple commands can be placed on a singleline withline with ;;

SpacesSpaces around operators and commands arearound operators and commands are

required but there are no other restrictionsrequired but there are no other restrictions


7/22

The basic form of a typical CThe basic form of a typical C--shell script (2)shell script (2)

For neatness, end the script withFor neatness, end the script with exitexit(not(notessential)essential)

Shell variablesShell variables start with astart with a $$ (this is only(this is onlyomitted with usingomitted with using setsetoror @@ -- see below)see below)

The shell variableThe shell variable $#argv$#argv contains thecontains the

number of argumentsnumber of arguments (items) typed on(items) typed onthe command line. The items are referredthe command line. The items are referredto as shell variables $1, $2, $3, , $#argvto as shell variables $1, $2, $3, , $#argv


8/22


For example:For example:myscript jjamyscript jja --4.56 yes4.56 yes

would give:would give:

$#argv = 3$#argv = 3

$1 = jja$1 = jja$2 =$2 = --4.564.56

$3= yes$3= yes


9/22


ShellShell variablesvariables are eitherare either stringstring (text) or(text) or numericnumeric A string variable is set (defined) by:A string variable is set (defined) by:

set var = valueset var = valuee.g.e.g. set x = Fredset x = FredNote: Variables and values areNote: Variables and values are casecase--sensitivesensitive i.e.i.e.x is not the same as Xx is not the same as X

Variable names can be several characters e.g.Variable names can be several characters e.g.set Case2 = 3set Case2 = 3

To print (display) the value of a variable:To print (display) the value of a variable:echo $varecho $vare.g.e.g. echo $xecho $xFredFred


10/22


A numeric variable is set (defined) by:A numeric variable is set (defined) by:@ var = integer@ var = integere.g.e.g. @ k =@ k = --77

Simple calculations may be performedSimple calculations may be performede.g.e.g. @ j = ($k + 1)@ j = ($k + 1) [j would be[j would be 7 + 1 =7 + 1 = --6]6]

@ k = ($k + 1)@ k = ($k + 1) would change k fromwould change k from --7 to7 to --66Could also use C notation:Could also use C notation: @ k ++@ k ++@ j = (2 * $k@ j = (2 * $k -- 5)5) [k=[k= --7 would give j=7 would give j= --19]19]Note: Put spaces around operators like *, + etc.Note: Put spaces around operators like *, + etc.

Floating point operations are not normally used in scriptsFloating point operations are not normally used in scripts

but can be done with the command bcbut can be done with the command bce.g.e.g. echo "2.1 + 6.3" | bcecho "2.1 + 6.3" | bcwould print 8.4 on the screenwould print 8.4 on the screenset y = `echo "2.1 + 6.3" | bc`set y = `echo "2.1 + 6.3" | bc`would save the answer in variable ywould save the answer in variable y


11/22


A shell variable may be given theA shell variable may be given the outputoutputof a command (or user program) as in theof a command (or user program) as in thebc example. In general:bc example. In general:

set var = `command`set var = `command`e.g.e.g. set ff = `ls *.dat`set ff = `ls *.dat`

Remember: ls is used to list filesRemember: ls is used to list files..

If the files are 5.dat 12.dat 13.dat thenIf the files are 5.dat 12.dat 13.dat thenecho $ffecho $ffwould display:would display:

5.dat 12.dat 13.dat5.dat 12.dat 13.dat


12/22

The basic form of a typical CThe basic form of a typical C--shellshell

script (7)script (7) To extract theTo extract the individualindividual elements ofelements of

variable ff we can usevariable ff we can use $#ff$#ffto find theto find the

number of items (similar to $#argv)number of items (similar to $#argv)e.g.e.g. echo $#ffecho $#ff would display 3would display 3echo $ff[1]echo $ff[1] would display 5.datwould display 5.dat

echo $ff[2]echo $ff[2] would display 12.datwould display 12.datecho $ff[3]echo $ff[3] would display 13.datwould display 13.dat


13/22


AA whilewhile loop is used to set up an iterative orloop is used to set up an iterative orrepetitive procedure:repetitive procedure:while (condition is true)while (condition is true)

commandscommandsendende.g.e.g. @ k = 1@ k = 1 # Initialise k# Initialise k

while ($k


14/22


AA foreachforeach loop is used to set up an iterative orloop is used to set up an iterative orrepetitive procedure involvingrepetitive procedure involving filesfiles::foreach var (files)foreach var (files)

commandscommandsendende.g.e.g. foreach f ([Aforeach f ([A--Z]*)Z]*) # f is a file starting with a capital letter# f is a file starting with a capital letter

echo $fecho $f # Print filename# Print filename

mv $f $f.capmv $f $f.cap # Rename file to have extension .cap# Rename file to have extension .capendend


15/22


AnAn ifif--elseelse--endifendifstructure is used to control scriptstructure is used to control scriptbranching:branching:if (condition 1 is true)if (condition 1 is true)

commandscommandselse if (condition 2 is true)else if (condition 2 is true)commandscommands

elseelse

commandscommandsendifendif

Note: use Note: use else ifelse if notnotelseifelseif


16/22


e.g.e.g.if ($x == 6 || $s == sea) thenif ($x == 6 || $s == sea) then

do somethingdo something

else if ($x > 10) thenelse if ($x > 10) thendo something elsedo something else

elseelse

go here if neither of the above conditions is truego here if neither of the above conditions is trueendifendif

Simpler or more complex forms are possibleSimpler or more complex forms are possible

e.g.e.g. if ($y == 2) lsif ($y == 2) ls l *.datl *.dat


17/22


TheThe whilewhile oror ifif(condition) uses C notation(condition) uses C notationfor logical operations:for logical operations:|||| oror

&&&& andand

==== equalequal>=>= greater than or equalgreater than or equal greater thangreater than


18/22


An exampleAn example#!/bin/csh#!/bin/csh ffif ($#argv != 1) thenif ($#argv != 1) then

echo Usage: myscript nameecho Usage: myscript nameexitexit # If incorrect usage end the script# If incorrect usage end the script

elseelse # If only 1 argument, branch here# If only 1 argument, branch hereset n = ($1)set n = ($1) # Put n equal to $1# Put n equal to $1

endifendifecho Your name is $necho Your name is $n# You could do other things here# You could do other things here

exitexit


19/22


Another exampleAnother example#!/bin/csh#!/bin/csh ffset ff = `ls *.dat`set ff = `ls *.dat` # ff contains filenames *.dat# ff contains filenames *.dat@ nf = $#ff@ nf = $#ff # nf equals the no. of files (items in ff)# nf equals the no. of files (items in ff)

@ j = 1@ j = 1 # Set counter j to 1# Set counter j to 1

while ($j


20/22

Introduction to CIntroduction to C--shell programmingshell programming

Refer to the PDF guide:Refer to the PDF guide:http://www.earthsci.unimelb.edu.au/~kevinhttp://www.earthsci.unimelb.edu.au/~kevin/UNIX_Course/Intro_to_C/UNIX_Course/Intro_to_C--Shell_Programming_2008.pdfShell_Programming_2008.pdf


21/22

Advanced UNIX materialsAdvanced UNIX materials

The following are PDF documents thatThe following are PDF documents thatmay be used for reference but wont bemay be used for reference but wont be

discussed in detail during this shortdiscussed in detail during this shortcourse:course:

Part 1 :Part 1 : ScriptsScriptsPart 2 :Part 2 :A summary of useful softwareA summary of useful software

Part 3 :Part 3 : Reanalysis productsReanalysis products


22/22

Lab sessionLab session

Lab session (informal) 2Lab session (informal) 2--3 PM UNIX3 PM UNIXLabLab

See:See:http://www.earthsci.unimelb.edu.au/~kevin/UNIXhttp://www.earthsci.unimelb.edu.au/~kevin/UNIX

_Course/Advanced_UNIX_Lab_Session_2010.pdf_Course/Advanced_UNIX_Lab_Session_2010.pdf

unix advanced

Documents