1© 2012 John Urrutia. All rights reserved.

Chapter 8

The Bourne Again Shell

Halleluiah!

2© 2012 John Urrutia. All rights reserved.

TopicsBackground

Creating a Simple Shell Script

Command Separation and Grouping

Redirecting Standard Error

Job Control

Directory Stack Manipulation

3© 2012 John Urrutia. All rights reserved.

BackgroundThe original Bourne shell was

developed by Steve Bourne of AT&T Bell Laboratories.Many shell scripts have been written to

help manage a UNIX system.

The bash has been written to mimic the Bourne shell

Bourne and bash use sh for invocation

4© 2012 John Urrutia. All rights reserved.

Backgroundbash is POSIX 1003.2 compliant

Efforts are underway to make it fully POSIX compliant.

bash can more closely comply to POSIX with the –posix option.

5© 2012 John Urrutia. All rights reserved.

TopicsBackground

Creating a Simple Shell Script

Command Separation and Grouping

Redirecting Standard Error

Job Control

Directory Stack Manipulation

6© 2012 John Urrutia. All rights reserved.

Shell script is not monetarySet of command stored in a file.

Used to support operational functions by combining many command into one group

Provides flow control commands which can alter the order of command execution.

7© 2012 John Urrutia. All rights reserved.

Script ExecutionEnter the filename on the command

lineMust have execute permission

Must be in the PATH

8© 2012 John Urrutia. All rights reserved.

TopicsBackground

Creating a Simple Shell Script

Command Separation and Grouping

Redirecting Standard Error

Job Control

Directory Stack Manipulation

9© 2012 John Urrutia. All rights reserved.

Command SeparationNewline (nl)X’0D0A’

ends command and initiates execution

Semicolon (;)just separates commands

Backslash (\) X’5C0D0A’at end of line and before you type return

Allows command to be continued

10© 2012 John Urrutia. All rights reserved.

Command Separation (cont.)Ampersand (&)

execute task in the background

Pipe ( | ) pipe

11© 2012 John Urrutia. All rights reserved.

Command GroupingParenthesis used to group

commandscauses Shell to create a subshell

additional processes are created as required when the subshell runs the commands within the parenthesis(ls ; date; w) ; more(ls ; date; w) | more

12© 2012 John Urrutia. All rights reserved.

TopicsBackground

Creating a Simple Shell Script

Command Separation and Grouping

Redirecting Standard Error

Job Control

Directory Stack Manipulation

13© 2012 John Urrutia. All rights reserved.

Streams RevisitedThree streams

standard in < or 0<

standard out > or 1>

standard error 2>

14© 2012 John Urrutia. All rights reserved.

Streams standard I/Ocat x y

if x exists and y does not, contents of x and error message due to y are sent to terminal

both standard out and standard error default to the terminal

15© 2012 John Urrutia. All rights reserved.

Streams Continuedcat x y 2>error.log

standard error is sent to a file to separate it from the expected results of the command

cat x y 2>>newfile 1>>newfilestandard out is redirected to newfile

16© 2012 John Urrutia. All rights reserved.

Here boy

<<

The Here DocumentAllows in-stream data to feed a

script.

Must start with << and a data delimiter character

Data delimiter character on line by itself - terminates

17© 2012 John Urrutia. All rights reserved.

TopicsBackground

Creating a Simple Shell Script

Command Separation and Grouping

Redirecting Standard Error

Job Control

Directory Stack Manipulation

18© 2012 John Urrutia. All rights reserved.

Job ControlAmpersand &

tells the Operating system to run the job in the background

User will still be able to interact with the shell

Pure Bourne shell has limited ability. Can not deal with a specific job it has put into background after initial creation. C shell much better.

19© 2012 John Urrutia. All rights reserved.

Job Control (continued)First two jobs in background, c in foreground

a & b & c

Entire sequence put into backgrounda | b | c &

All three jobs executed in backgrounda & b & c &

jobs – builtin function displays the jobs running in the background

20© 2012 John Urrutia. All rights reserved.

Job Control (continued)…]$ xman&

[1] 1246…]$ date&

[2] 1247…]$ Tue Sep 11 6:17 PDT 2001

[2]+ Done date…]$ find /usr –name ace –print > out &

[2] 1269…]$ jobs

[1]- Running xman &[2]+ Running find /usr –name ace …

21© 2012 John Urrutia. All rights reserved.

Job Control (continued)…]$ (sleep 5;cat>mytext)&

[1] 1343…]$ date Tue Sep 11 6:30 PDT 2001 [1]+ Stopped (tty input) (sleep 5;cat>mytext)…]$ fg

(sleep 5;cat>mytext) Remember to let the cat out!

22© 2012 John Urrutia. All rights reserved.

TopicsBackground

Creating a Simple Shell Script

Command Separation and Grouping

Redirecting Standard Error

Job Control

Directory Stack Manipulation

23© 2012 John Urrutia. All rights reserved.

Directory Stack ManipulationYou can store a list of frequently used

directories in a stack

Push-down (LIFO)The three stack commands

dirspushdpopd

24© 2012 John Urrutia. All rights reserved.

Directory Stack Manipulationdirs – displays all the directories in the

stackWhen stack is empty displays the

Working Directory (~ is your home directory)

25© 2012 John Urrutia. All rights reserved.

Directory Stack Manipulationpushd someDirectoryName–

Change working directory

“pushes” directory onto the stack

Display the directory stack

26© 2012 John Urrutia. All rights reserved.

Directory Stack Manipulationpushd –

“swaps” top of stack with next element

Change working directory to top of stack

Display the directory stack

27© 2012 John Urrutia. All rights reserved.

Directory Stack Manipulationpushd +2 –

“swaps” top of stack with +2 element

Change working directory to top of stack

Display the directory stack

28© 2012 John Urrutia. All rights reserved.

Directory Stack Manipulationpopd –

“pops” removes top entry from stack

Change working directory to top of stack

29© 2012 John Urrutia. All rights reserved.

Directory Stack Manipulationpopd +2 –

Removes the 3rd entry from stack

DOES NOT CHANGE Working Directory

30© 2012 John Urrutia. All rights reserved.

TopicsProcesses

Parameters and Variables

History

Alias

Command-line Expansion

31© 2012 John Urrutia. All rights reserved.

Processes and SubshellsA process is the execution of a command

login to LINUX

execution of a LINUX utility

execution of a shell script creates a new process

script commands each start a new process

Process structure is hierarchicalParent processes spawn or fork children

32© 2012 John Urrutia. All rights reserved.

PID’s … Process ID’s

Sequentially Assigned by the system when a process is started

ps Displays all processes for your userid

33© 2012 John Urrutia. All rights reserved.

ps –Al All PleaseDisplays a long list of all processes

including those not attached to a terminal.

Command preceded by – was initiated by the init process

34© 2012 John Urrutia. All rights reserved.

Process statusAll processes have a status that can

change to:D – Sleeping Do not interrupt (Can’t)

N – Reduced priority

R – Available for execution (running)

S – Sleeping (Waiting)

T – Stopped or being traced

Z – Zombie waiting for child to terminate

35© 2012 John Urrutia. All rights reserved.

Process FlowUser logs in: shell process is created

User issues command, enters returnShell creates a subshell

child process is forked or spawnedunless the command is built into the bourne

shell process

36© 2012 John Urrutia. All rights reserved.

Process flow (cont.)Subshell is a clone of the parent shell

Subshell tries to exec the commandIf it’s a program, the program runsIf it’s a shell script, exec fails and subshell

interprets commands.If it’s neither command fails

37© 2012 John Urrutia. All rights reserved.

Process FlowParent Shell sleeps until child shell

finishes(unless job was executed in background)

Variables that are used in a parent can be sent to a child, but the reverse is not true.

38© 2012 John Urrutia. All rights reserved.

Process FlowShell Scripts need to have execute

permission. You just type the file name as you would a command.

Alternative (new subshell): sh file

Alternative (current shell): • file

39© 2012 John Urrutia. All rights reserved.

Starting bashWhen bash is called, various startup

files are run to issue commands and define environmental variables

Which startup file(s) begin depends upon how bash is called

Use these startup files to make the shell work for you

40© 2012 John Urrutia. All rights reserved.

Login shellsLogin shells are called with the --login

option

We don’t usually do this – it’s done for us

Will first run /etc/profile, which contains global default settings

41© 2012 John Urrutia. All rights reserved.

Login shellsNext, it will attempt to run ~/.bash_profile

~/.bash_login

~./profile

42© 2012 John Urrutia. All rights reserved.

Login shells, con’tCommands in those three files can

override the defaults in /etc/profile

Once one of those files are executed, control is passed to the user

When the user logs out, bash runs ~/.bash_logoutUsually clears temporary information

43© 2012 John Urrutia. All rights reserved.

Interactive nonlogin shellsShells that you spawn yourself by

typing bash

Runs ~/.bashrcThis file is usually called by ~/.bash_profile for login shells

Often this file will also run /etc/bashrc, which again contains system defaults

44© 2012 John Urrutia. All rights reserved.

Noninteractive shellsThese are the shells used to run

scripts

These shells do not run any of the aforementioned startup files

They do however inherit the calling shell’s environmental variables marked for export

45© 2012 John Urrutia. All rights reserved.

Noninteractive shellsSo basically anything you set for the

login shell is set for the noninteractive shell

46© 2012 John Urrutia. All rights reserved.

Working with Startup FilesIn the end, these startup files are just

shell scripts

Obey the same rules and conventions that scripts must use for the particular shell you’re using

Most important files are probably .bashrc and .bash_profile

47© 2012 John Urrutia. All rights reserved.

Startup Files, con’tSimplify – have .bash_profile

call .bashrc

Just edit the startup files in your favorite editor

When done, you can apply changes to your current shell using either . or source

Otherwise, logout and login again

48© 2012 John Urrutia. All rights reserved.

Creating a Shell ScriptUse a text editor like vi

First line should start with #! Followed by the absolute pathname of the shell that is to interpret the script. (default is C shell)#!/bin/sh

Lines which start with a # are comments(except the special line mentioned above)

49© 2012 John Urrutia. All rights reserved.

Dot s de way to Execute itThe exec command

Executes scripts or programs

Runs under the same PID

Provides access to the original environment variables

Terminates current process.

50© 2012 John Urrutia. All rights reserved.

Dot s de way to Exec itThe dot command

Executes only scripts

Runs under the same PID

Provides access to the current environment variables

Returns to next command in script

51© 2012 John Urrutia. All rights reserved.

TopicsProcesses

Parameters and Variables

History

Alias

Command-line Expansion

52© 2012 John Urrutia. All rights reserved.

Parameters & VariablesParameters

Any user accessible variable

Positional on the command line

Two types of VariablesKeyword Shell variables

User Shell variables

53© 2012 John Urrutia. All rights reserved.

Keyword Shell VariablesHOME (contains login directory)

PATH (Used by shell to locate commands you type in)/usr/bin:/usr/sbin:/class/n01/bin:

MAIL (contains name of central post office file for your mail)

PS1, PS2 (primary and secondary prompts)

54© 2012 John Urrutia. All rights reserved.

Keyword Variables (continued)CDPATH

like PATH, except used by cd command

TZtimezone

IFSInternal field separator. Blanks, tabs

and newline are defaults

55© 2012 John Urrutia. All rights reserved.

User Created VariablesCreate a variable by giving a name of your

choice and an optional valuename=charlieNO blanks around the equal sign!!

Remove variableunset name

Keep variable but remove valuename=

56© 2012 John Urrutia. All rights reserved.

Readonly Shell VariablesTwo types: user created variable that

has been declared to be readonlyreadonly name

keeps later statements from changing the value

Special Shell variables Positional VariablesMiscellanous variables

57© 2012 John Urrutia. All rights reserved.

Positional Variables$1 through $9

Keep the first nine arguments entered after the name of the shell script

…]$ myscrpt aardvark dog cat$1 will contain the word aardvark

$2 will contain the word dog

$3 will contain the word cat

58© 2012 John Urrutia. All rights reserved.

Miscellaneous Variables$* contains all arguments (not just

the first one)

$@ similar to $*, except that it internally quotes each argument.

$# total number of arguments

$$ process id number (pid) of current process

59© 2012 John Urrutia. All rights reserved.

Shift commandPromotes values of each positional

variable to the left.

Contents of $1 go to ‘bit bucket’

Contents of $2 go to $1

Contents of $3 go to $2

etc (etcetera, not etci)

60© 2012 John Urrutia. All rights reserved.

Set `em up - Shift `em out

set Populates the $[1-9] variables

shiftMoves each $ variable 1 position

to the left.

61© 2012 John Urrutia. All rights reserved.

[d1@linux2 d1]$ date

Thu Apr 20 11:28:38 PDT 2000

[d1@linux2 d1]$ set `date`

[d1@linux2 d1]$ echo $1

Thu[d1@linux2 d1]$ shift

[d1@linux2 d1]$ echo $1

Apr

The Shift ing sands of time

62© 2012 John Urrutia. All rights reserved.

= or Unset – That’s the ?

=Creates and/or populates any user

variable

unsetRemoves a user variable

63© 2012 John Urrutia. All rights reserved.

export - it to the WorldUser variables are local to the current

process

exportGives child processes copies of user

variables

64© 2012 John Urrutia. All rights reserved.

I do declaredeclare (typeset) – sets

attributes for user variablesf – identify as a function name

i – integer uses binary storage

x – marks for export

r – Read Only

…]$ declare –ix export_var=6

65© 2012 John Urrutia. All rights reserved.

What variables?declare and set

Display all variables and current values

declare [-f -i -x -r]Display all variables and current values

containing one or more of the specified attributes

66© 2012 John Urrutia. All rights reserved.

Read ‘em and weepread var1 var2 var3 …

Takes standard input and populates one or more user variables

IFS (internal field separator) delimitedDefault is space tab newline

67© 2012 John Urrutia. All rights reserved.

Dots impossible!How do I change shell variables

permanently?Create or modify the .profile file the

next time you login the changes will be there.

To do it now execute the . Command

…]$ .profile

68© 2012 John Urrutia. All rights reserved.

Command substitutionReplaces a command with the output

of a commandSimilar to pipe but does not create a file

Two syntaxes`command ` – Old Syntax

$(command ) – New Syntax

69© 2012 John Urrutia. All rights reserved.

Where’s the exit ?exit number

Allows you to set a condition or return code from the process

This value is referenced by $?

70© 2012 John Urrutia. All rights reserved.

TopicsProcesses

Parameters and Variables

History

Alias

Command-line Expansion

71© 2012 John Urrutia. All rights reserved.

HistoryThe history mechanism was adopted

from the C shellIt maintains a list of line commands

Each entry is called an eventEach event can be re-called and re-

executed via a shorthand command by using the event number.

72© 2012 John Urrutia. All rights reserved.

A Historical Re-ExecutionThe built-in command fc (fix command)

Allows viewing of previous commandsAllows correction of a previous command

And re-execution of the culprit

fc –l [ first last ]Lists all commands that meet the criteriafirst and last can be either event # or a

string

73© 2012 John Urrutia. All rights reserved.

A Historical Re-Executionfc –e editor [ first last ]

Edits all commands that meet the criteria with the editor specified.

FCEDIT varaible will set the default editor if one is not specified

As soon as you exit the editor everything in the buffer gets executed!

74© 2012 John Urrutia. All rights reserved.

A Historical Re-Executionfc –s event#

[ oldstring=newstring ]Re-Executes the specified event#

without entering editor mode

If present a string substitution occurs

… ]$ fc -s old.data.file=new.data.file

75© 2012 John Urrutia. All rights reserved.

A Historical Re-ExecutionEvent Number Execution

!! – Re-Executes the previous event

!44 – Re-Executes event 44

!-4 – Re-Executes 4th previous event

!$ Identifies the last token of the previous command line

76© 2012 John Urrutia. All rights reserved.

A Historical Re-ExecutionEvent Text Execution

!cat – Re-Executes the previous event beginning with “cat”

!?cat? – Re-Executes the previous event containing the string “cat”

!$ Identifies the last token of the previous command line

77© 2012 John Urrutia. All rights reserved.

A Historical Re-ExecutionEvent Text substitution with event

modifiers!!:s/car/cat – Re-Executes the

previous event after substituting “cat” for “car”

^car^cat – Shorthand for the above

78© 2012 John Urrutia. All rights reserved.

A Historical Re-ExecutionOther Event Modifiers

P – No execution just print

h – head removes last pathname element

e – removes all but the filename extension

r – remove the filename extension

t – tail removes all but last pathname element

[g]s/old/new/ – Substitute old with the new

79© 2012 John Urrutia. All rights reserved.

TopicsProcesses

Parameters and Variables

History

Alias

Command-line Expansion

80© 2012 John Urrutia. All rights reserved.

AliasThis built-in was borrowed from the C

shell.

Allows substitution of a string for a command.alias [name [=command value] ]

Will not work inside a script.

or any other name would work

81© 2012 John Urrutia. All rights reserved.

Alias or any other name would work

To ‘ (quote) or “ (double quote) ‘– expands shell variables at execution“ – expands shell variables at definition

Let’s analyze the following:…]$ alias p1=“echo my prompt is $PS1”

…]$ alias p2=‘echo my prompt is $PS1’

…]$ PS1=“Hello?”

Hello?

82© 2012 John Urrutia. All rights reserved.

Alias or any other name would work

Hello? p1

My prompt is [\u@\h \W]\$

Hello? P2

My prompt is Hello?

83© 2012 John Urrutia. All rights reserved.

TopicsProcesses

Parameters and Variables

History

Alias

Command-line Expansion

84© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic ShellCommand-Line expansion

Before executionShell parses the command line into tokensTokens are processed to expand the

command line

The entire command line is then executed

85© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic ShellBrace expansion { }

Used to specify filenames when pathname expansion is not required or string substitution is used

Consists of: PreamblePostamble

…]$ echo b{a,e,i,o,u}dbad bed bid bod bud

86© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic ShellTilde expansion ~

Parses the token for a / or a SpaceIf parsed token is not null

Test the value for login name and use if valid If not valid no substitution

If token is nullSubstitute value of HOME for the ~

~- previous working directory

~+ current working directory

87© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic ShellParameter expansion

When a dollar sign ($) is followed by a number The positional parameter from the command

line is substituted

88© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic ShellVariable expansion

Special case – when a dollar sign ($) is followed by a variable name. The shell substitutes the variables value.

General case – ${variable}The braces insulate the variable from what is

around it.Without insulation substitution may not occur

89© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic Shell

…]$ BORK=borken…]$ FORKOLA=$BORKforkola…]$ echo $FORKOLA

…]$ FORKOLA=${BORK}forkola…]$ echo $FORKOLA

borkenforkola

90© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic ShellCommand Substitution

$( command ) or $’command ’Replaces token with the standard output of

the command

…]$ echo $(cat animals)dog cat aardvark Dog mouse cat elephant zebra

91© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic ShellArithmetic Expansion

$[ expression ] Evaluates expression and replaces

token with the valueTreats all variables as integers Converts strings to integersUses the same syntax as the C languageOperators + - * / % =

92© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic ShellArithmetic Expansion

let – Built-in Allows arithmetic evaluation without expansionSets the exit code based on the last expression

evaluated 1 – if the value is zero 0 – for all other values

let a=5+3 b=7-6echo $a $b8 1

93© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic ShellWord Splitting

IFS – Infernal Field SeparatorDefault is one of space tab or newline

Adds additional characters as field separators.

Only works on fields that have some form of expansion

Caution – Changing this will affect the way the shell operates

94© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic Shell

…]$ a=w:x:y:z…]$ cat $acat: w:x:y:z: No such file or directory…]$ IFS=“:”…]$ cat $acat: w: No such file or directorycat: x: No such file or directorycat: y: No such file or directorycat: z: No such file or directory

95© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic ShellPathname Expansion

Uses the wildcard tokens

* - ? - [ - ]Globbing

Ambiguous File reference

96© 2012 John Urrutia. All rights reserved.

The Expand-O-Matic Shell The Order of Expansion

1. Brace – { }

2. Tilde – ~

3. Parameter

4. Variable

5. Command substitution

6. Arithmetic

7. Word splitting

8. Pathname