1
Course StaffB.S.Mahanand .B.E.,(M.Tech).,Dept of Computer Science and EnggS.J.C.E.Mysore.
B.S.Mahanand .B.E.,(M.Tech).,Dept of Computer Science and EnggS.J.C.E.Mysore.
Introduction to Unix 2
Course Texts
Reference: Learning Unix - for those who have not used Unix before.
Introduction to Unix 3
Text Book: Unix Concepts &Applications
By Sumitabha Das
Operating SystemsAn Operating System controls (manages)
hardware and software.provides support for peripherals such as
keyboard, mouse, screen, disk drives, …software applications use the OS to
communicate with peripherals.The OS typically manages (starts, stops,
pauses, etc) applications.
Introduction to Unix 4
Single vs. MultitaskingSome old operating systems could only do
one thing at a time (DOS).Most modern systems can support multiple
applications (tasks) and some can support multiple users (at the same time).
Supporting multiple tasks/users means the OS must manage memory, CPU time, network interfaces, ...
Introduction to Unix 5
User InterfacesThe User Interface is the software that
supports interactions with a human.Some operating systems directly provide a
user interface and some don't.Windows is an example of an Operating
System that includes a user interface.Unix (the OS) does not directly provide a user
interface.
Introduction to Unix 6
What is UNIX?UNIX is a popular operating system (OS) in
both engineering and business worldOS is a “super program” that controls sharing
of resources and communication between machines
E.g., Windows, OS/2, VMS, MacOS, UNIXUNIX is available for all platforms
Introduction to Unix 7
Unix and UsersMost flavors of Unix (there are many) provide
the same set of applications to support humans (commands and shells).
Although these user interface programs are not part of the OS directly, they are standardized enough that learning your way around one flavor of Unix is enough.
Introduction to Unix 8
Flavors of UnixThere are many versions of Unix that are
used by lots of people:SysV (from AT&T)BSD (from Berkeley)Solaris (Sun)IRIX (SGI)AIX (IBM)LINUX (free software)
Introduction to Unix 9
Unix History and MotivationThe first version of Unix came from AT&T in
the early 1970s (Unix is old!).Unix was developed by programmers and for
programmers.Unix is designed so that users can extend the
functionality - to build new tools easily and efficiently (this is important for programmers).
Introduction to Unix 10
Unix History and MotivationK Thomson and D Ritchie at Bell Labs
Wrote first version in assembly then the next version in C; the first “understandable” OS!
UC Berkeley graduate students Enhanced with memory management and
networking capabilities (BDS UNIX)UNIX Today: contain both features
Commercial UNIXLinux
Introduction to Unix 11
Some Basic ConceptsUnix provides a simple interface to
peripherals (it's pretty easy to add support for a new peripheral).
Unix includes a basic set of commands that allow the user to view/change the system resources (filesystem, processes, peripherals, etc.).
Introduction to Unix 12
What we will look atIn this course we will learn about:
Unix user accountsthe core set of Unix commandsthe Unix filesystemA couple of special programs called "shells".A number of commonly used applications:
Window system, text editors, programming tools.
Introduction to Unix 13
The power of Unix is that you can extend the basic commandsWe will also look at how to extend the basic
functionality of Unix:customize the shell and user interface. string together a series of Unix commands to
create new functionality.create custom commands that do exactly what
we want.
Introduction to Unix 14
UNIX ServicesUNIX facts about programs and files
A file is a collection of data stored on diskA program is a collection of bytes representing code
and data that is stored in a fileWhen a program is started, it is loaded from disk
into RAM. A “running” program is called a processProcesses and files have an owner and a group, and
they protected from unauthorized accessUNIX supports a hierarchical directory structureFiles have a location within the directory hierarchy
Introduction to Unix 15
UNIX ServicesSharing resources
UNIX shares CPUs among processes by dividing CPU time into slices (typically 0.1 second) and allocating time slices based on their priorities
UNIX shares memory among processes by dividing RAM into equal-sized pages (e.g., 4K bytes) and allocating them; only those pages of a process that are needed in RAM are loaded from disk while pages of RAM that are not accessed are saved back to disk
UNIX shares disk space among users by dividing disks into equal-sized blocks and allocating them
Introduction to Unix 16
UNIX ServicesUNIX provides communication mechanism
between processes and peripherals Pipe is one-way medium-speed data channel that
allows two processes on the same machine to talk Socket is two-way high-speed data chanel that
allows two processes on different machines to talk
UNIX allows server-client model of communication e.g., X Window systems
Introduction to Unix 17
UNIX for Users
Basic commandsman, ls, cat, more, page, head, tail, lessmv,cp,rm,lnpwd,cd,mkdir,rmdir,filechmod, groups, chgrp
Editing utilities and terminal typevi, emacs
Introduction to Unix 18
Getting Started in UNIXObtain an account and logging-in
passwd: utility for changing passwordShells
A middleman program between you and UNIX OS that executes your commands Shell commands vs. utilities vs. system calls
Three popular shells: Bourne, Korn, CShell has a programming language that are
tailored to manipulate processes and files; such program is called shell script
Introduction to Unix 19
Utilities for Manipulating File mv, cp, rm, ln
move,copy,remove,make links to filesmv [-fi] soutce targetFile .. rename source filenamemv [-fi] soutceList targetDirectory .. move source files into target directorycp [-fip] soutceFile targetFile .. make copy of sourceFile named targetFilecp [-fip] soutceFileList targetDirectory .. copy source files into target directorycp -rR [-fip] soutceDirList targetDirectory .. copy source files into target directory, and copy the directory structure.rm [-fi] fileListrm -rR [-fi] directoryListln [-fns] source [target]ln [-fns] sourceList targetDirectory .. make links to sour files into target directory
file - determine file type $ file hello.c hello.c: c program text
Introduction to Unix 20
Utilities for Hanling Directorypwd, cd, mkdir, rmdir
print working directorychange directory
cd, pushd, popd
make directoryremove directory
Introduction to Unix 21
Vi - Editorvi is a visual text editor. Most of you are too young to understand
what a non-visual editor might be!check out the ed editor!
vi shows you part of your file and allows you to enter commands that change something (add new stuff, delete a char or line, etc).
Introduction to Unix 22
vi modesvi has a couple of modes:
command mode: move the cursor around, move to a different part of the file, issue editing commands, switch to insert mode.
insert mode: whatever you type is put in the file (not interpreted as commands).
when you first start vi you will be in command mode.
Introduction to Unix 23
Cursor Movement Commands(only in command mode!)
h move left one position
l move right one position
j move up one line
k move down one line
Your arrow keys might work (depends on the version of vi and your terminal)
Introduction to Unix 24
spacebar also does this
More Cursor Movementw move forward one word
b move backward one word
e move to the end of the word
) move to beginning of next sentence
( move to beginning of current sentence
Introduction to Unix 25
Scrolling CommandsCTRL-F scroll forward one screen
CTRL-B scroll backward one screen
CTRL-D scroll forward 1/2 screen
CTRL-U scroll backward 1/2 screen
Introduction to Unix 26
Command that delete stuffx delete character (the one at the cursor)
dw delete word
dd delete line
X delete back one character (backspace)
3x delete 3 characters (any number works)
5dd delete 5 lines (any number works)
Introduction to Unix 27
Changing Textcw change word (end with Esc)
cc change line (end with Esc)
C change rest of the line
rx replace character with 'x' (could be anything, not just 'x')
Introduction to Unix 28
Insert ModeIn insert mode whatever you type goes in to
the file. There are many ways to get in to insert mode:
i insert before current position
a append (insert starting after cursor)
A append at end of line
R begin overwriting text
Introduction to Unix 29
Ending Insert ModeTo get out of insert mode (back to command
mode) you press "Esc" (the escape key).
There is a status line (bottom of screen) that tells you what mode/command you are in.
Introduction to Unix 30
Saving and ExitingZZ save if changes were made, and quit.
:wq Write file and quit
:w Write file
:w file Write to file named file
:q Quit
:q! Really quit (discard edits)
Introduction to Unix 31
Searching Commands/text search forward for text
?text search backward for text
n repeat previous search
N repeat search in opposite direction
Introduction to Unix 32
Other StuffCopying and Yanking (Paste)Remembering positionsSwitching filesRepeating commandsDisplay line numbersRun Unix commands (for example: emacs)
Introduction to Unix 33
Emacs -Editor
emacs is every bit as cryptic as vi!emacs allows you to customize it with new
commands and keyboard shortcuts.The emacs commands are written in elisp
(a dialect of Lisp), so you need to understand elisp to do serious customization.
Introduction to Unix 34
Emacs meta keyMany emacs commands are invoked with
sequence of keystrokes.Emacs doesn't have modes like vi – you
can always enter text (at the current cursor position) or commands.
Many commands start with a special keystroke called the metakey. (others use the control key).
The ESC key is (usually) the meta key.
Introduction to Unix 35
Command List SyntaxThe book shows a list of tons of emacs
commands. The syntax used to show this list looks like this:
C-a C-b (means Ctrl-a , Ctrl-b)M-a M-b (means Esc, a, Esc, b)
Introduction to Unix 36
Important CommandsExit: C-x C-cSave file : C-x C-sUndo: C-x uGet out of a command: C-g
Introduction to Unix 37
Cursor movementCursor keys usually work (it depends on how
your terminal is set up and how emacs is configured).
C-f: forward (right arrow)
C-b: backward (left arrow)
C-p: previous line (up arrow)
C-n: next line (down arrow)
Introduction to Unix 38
Other stuff in emacsMove by words, sentences, paragraphsFile handling – save/load file, etc.Delete char, word, sentence, regionBuffer manipulation (multiple buffers)Searching, replacingAutomatic indentation (major mode)Lots more (try the tutorial, read the book!)
Introduction to Unix 39
The Environment
The Unix system environment can be set by user
System variables:Unix system is controlled by number of shell variables set by system called system variables which can be seen by executing $set command
Ex- HOME,PATH,IFS,MAIL,PS1,PS2,SHELL,TERM,LOGNAME
Introduction to Unix 40
.profile : It is the shell script executed during login time.
stty : To set the terminal characteristics
history :It displays previously executed commands.
Introduction to Unix 41
histsize : Used to store more commands
$_ : Stores the last arguments of the last command
Changing Directory : ~(tilde)~ acts as short hand representation of home
Directory.
Introduction to Unix 42
UNIX Utilities: man & ls man
on-line help for UNIX commandsman [-s section] wordman -k keyword
Man pages are divided into 8 sections 1. Commands & application programs 2. System calls, 3. Library functions ...
lslist files
ls [options] {fileName}*
Introduction to Unix 43
UNIX Utilitiesls example
$ ls -alFstotal 151 drwxr-xr-x 2 jaemok mass 512 Mar 2 01:04 ./3 drwxr-xr-x 46 jaemok mass 3072 Mar 2 01:04 ../5 -rwxr-xr-x 1 jaemok mass 5091 Mar 2 01:04 a.out*5 -rwxr-xr-x 1 jaemok mass 5091 Mar 2 01:04 hello*1 -rw-r--r-- 1 jaemok mass 36 Mar 2 01:04 hello.c
5 size in blockdrwxr-xr-x type and file permission1 hard-link countjaemok ownermass group5091 size in bytesMar 2 01:04 modified timea.out filename
Miscellaneous: date, clearIntroduction to Unix 44
Utilities to List File Contentscat,more,page,head,tail,less
cat displays contents of files given on command line (or from standard input) to standard output$ cat > heart blah blah blah ^D$ _
more and page displays contents of files(or input) fitted to rows of terminal, and allows you to scroll.
head and tail only displays given number of lines in the first part or the last part of files(or input).
less is similar to more, but has more functions
Introduction to Unix 45
File Owner and GroupEvery UNIX process has a owner which is
typically the user name who started it My login shell’s owner is my user name
When a process creates a file, its owner is set to the process’ owner
UNIX represents the user name as user IDEvery UNIX user is a member of a group and
an UNIX process belongs to a group My login shell’s group is my group name (ID)
Introduction to Unix 46
File PermissionsFile Permissions (ex: rw-r--r--)
owner: rw-, group: r--, others: r--r: read, w: write, x: execute
When a process executes, it has four values related to file permissiona real user ID, an effective user IDa real group ID, an effective group IDWhen you login, your login shell process’
values are your user ID and group ID
Introduction to Unix 47
File Permission ApplicationWhen a process runs, file permission applies
If process’ effective user ID = file owner Owner permission applies
If process’ effective group ID=file group ID Group permission applies
Otherwise, others permission applies
Super user’s process automatically has all access rights
When creating a file, umask affect permission
Introduction to Unix 48
Effective User and Group IDA process’ effective user ID
depends on who executes the process, not who owns the executable
E.g., if you run passwd (owned by root), the effective user ID is your ID, not root; then how can it update /etc/passwd file owned by root ?
Two special file permissions set user ID and set group ID When an executable with set user ID permission is
executed, the process’ effective user ID becomes that of executable; the real user ID is unaffected
File permission of /bin/passwd is r-sr-sr-x
Introduction to Unix 49
Changing File Permissionschmod
change the permission mode of fileschmod [-fR] <absolute-mode> fileListchmod [-fR] <symbolic-mode-list> fileList
Introduction to Unix 50
chmodabsolute-mode is octal number
if you want a file to have permission setting: rwxr- x--- then, it is an octal number o750 (111 101 000) $ chmod 750 myfile
symbolic-mode-list is comma-separated list of symbolic expressions of form
[who] operator [permissions]who : characters u,g,o,aoperator : -,=.+permissions : r,w,x,s
$ chmod g-rx myfile
Introduction to Unix 51
Text Handling CommandsThere are many Unix commands that handle
textual data:operate on text filesoperate on an input stream
Functions:SearchingProcessing (manipulations)
Introduction to Unix 52
Searching Commandsgrep, egrep, fgrep : search files for text
patterns
strings: search binary files for text strings
find: search for files whose name matches a pattern
Introduction to Unix 53
grep - Get Regular Expressiongrep [options] regexp [files]
regexp is a "regular expression" that describes some pattern.
files can be one or more files (if none, grep reads from standard input).
Introduction to Unix 54
grep ExamplesThe following command will search the
files a,b and c for the string "foo". grep will print out any lines of text it finds (that contain "foo")
grep foo a b c
Without any files specified, grep will read from standard input:
grep I
Introduction to Unix 55
Regular ExpressionsThe string "foo" is a simple pattern.grep actually understands more complex
patterns that are described using regular expressions.
We will look at regular expressions used by grep and other programs later.
In case you can't wait - here is a sample:
grep "[A-Z]0{2,3}" somefile
Introduction to Unix 56
grep options-c print only a count of matched lines.-h don't print filenames-l print filename but not matching line-n print line numbers-v print all lines that don't match!
Introduction to Unix 57
grep, egrep and fgrepAll three search files (or stdin) for a text
pattern.grep supports regular expressionsegrep supports extended regular expressionsfgrep supports only fixed strings (nothing
fancy)All have similar forms and options.
Introduction to Unix 58
stringsThe strings command searches any kind of
file (including binary data files and executable programs) for text strings, and prints out each string found.
strings is typically used to search for some text in a binary file.
strings [options] files
Introduction to Unix 59
The find commandFind searches the filesystem for files whose
name matches a pattern*.Here is a simple example:
find . -name unixtest -print
*Actually find can do lots more!
Introduction to Unix 60
Text ManipulationThere are lots of commands that can read in
text (from files or standard input) and print out a modified version of the input.
Some possible examples: force all characters to lower caseshow only the first word on each lineshow only the first 10 lines
Introduction to Unix 61
Common ConceptsThese commands are often used as filters,
they read from standard input and send output to standard output.
Different commands for different specific functionsanother way is to build one huge complex
command that can do anything. This is not the Unix way!
Introduction to Unix 62
Simple Filtershead tail - show just part of a filecut paste join - deal with columns in a text
file.sort - reorders the lines in a filetr - translate charactersuniq - find repeated or unique lines in a file.
Introduction to Unix 63
head or tails?head shows just the "head" (beginning) of a
file. tail shows just the "tail" (end) of a file.Both commands assume the file is a text file.
Introduction to Unix 64
The head commandhead [options] [files]
By default head shows the first 10 lines.Options: -n print the first n lines. Example:
head -20 /etc/passwd
Introduction to Unix 65
The tail commandtail [options] [files]
By default tail shows the last 10 lines.Options:
-n print the last n lines. -nc print the last n characters+n print starting at line number n+nc print starting at character number n
Introduction to Unix 66
The tail command (cont.)More Options:
-r show lines in reverse order-f don't quit at end of file.
Examples:tail -100 somefile
tail +100 somefile
tail -r -c somefile
Introduction to Unix 67
Not all versions
support this option!
The cut commandcut selects (and prints) columns or fields
from lines of text.
cut options [files]
You must specify an option!
Introduction to Unix 68
cut options-clist cut character positions defined in
list.
list can be:number (specifies a single character position)range (specifies a sequence of positions)comma separated list (specifies multiple positions or ranges)
Introduction to Unix 69
cut -c examplescut -c1 prints first char. (on each line).cut -c1-10 prints first 10 charcut -c1,10 prints first and 10th char.
cut -c5-10,15,20-
prints 5,6,7,8,9,10,15,20,21,… char on each line.
Introduction to Unix 70
more cut options-flist cut fields identified in list.
a field is a sequence of text that ends at some separator character (delimiter).
You can specify the separator with the -d option. -dc where c is the delimiter.
The default delimiter is a tab.
Introduction to Unix 71
Specifying a delimitercut -d: -f1 prints everything before
the first ":" (on each line).
What if we want to use space as the delimiter?
cut -d" " -f1
Introduction to Unix 72
cut -f examplescut -f1 prints everything before the
first tab.cut -d: -f2,3 prints 2nd and 3rd : delimited
columns.cut -d" " -f2 prints 2nd column using space
as the delimiter.
Introduction to Unix 73
The paste commandpaste puts lines from one or more files
together in columns and prints the result.
paste [options] files
The combined output has columns separated by tabs.
Introduction to Unix 74
paste cands votes
Introduction to Unix 75
GoreBradleyBushMcCainTrumpLetterman
GoreBradleyBushMcCainTrumpLetterman
1010101010100
1010101010100
Gore 10Bradley 10Bush 10McCain 10Trump 10Letterman 100
Gore 10Bradley 10Bush 10McCain 10Trump 10Letterman 100
cands votes
paste options-dc separate columns of output with
character c.you can use different c between each column.
-s merge subsequent lines from a single file.
Introduction to Unix 76
Introduction to Unix 77
Gore10Bradley10Bush10Letterman100
Gore10Bradley10Bush10Letterman100
Gore 10Bradley 10Bush 10Letterman 100
Gore 10Bradley 10Bush 10Letterman 100
records
paste -s -c"\t\n" records
Gore 10 Bradley10 Bush 10McCain 10 Letterman100
Gore 10 Bradley10 Bush 10McCain 10 Letterman100
paste -s -c"\t\t\n" records
The join commandjoin combines the common lines of 2 sorted
files. Useful for some text database applications,
but not a very general command.Look at examples in the book if you are
interested.
Introduction to Unix 78
The sort commandsort reorders the lines in a file (or files) and
prints out the result.
sort [options] [files]
Introduction to Unix 79
sort options-b ignore leading spaces and tabs-d sort in dictionary order (ignore
punctuation)-n sort in numerical order-r reverse the order of the sort
tons more options!
Introduction to Unix 80
Numeric vs. AlphabeticBy default, sort uses an alphabetical ordering.
Introduction to Unix 81
381827125687566875
381827125687566875
182738668751256875
182738668751256875
125687518273866875
125687518273866875
sort sort -n
Alphabetic Ordering (uses ASCII)
Introduction to Unix 82
bbbb BBBBaaaaAAAA0000####$$$$
bbbb BBBBaaaaAAAA0000####$$$$
####$$$$0000AAAABBBBaaaabbbb
####$$$$0000AAAABBBBaaaabbbb
sort
'0' < '9' <'A' < 'Z'< 'a' < 'z' <
ASCII codes
32: 33:! 34:" 35:# 36:$ 37:% 38:& 39:'
40:( 41:) 42:* 43:+ 44:, 45:- 46:. 47:/
48:0 49:1 50:2 51:3 52:4 53:5 54:6 55:7
56:8 57:9 58:: 59:; 60:< 61:= 62:> 63:?
64:@ 65:A 66:B 67:C 68:D 69:E 70:F 71:G
72:H 73:I 74:J 75:K 76:L 77:M 78:N 79:O
80:P 81:Q 82:R 83:S 84:T 85:U 86:V 87:W
88:X 89:Y 90:Z 91:[ 92:\ 93:] 94:^ 95:_
96:` 97:a 98:b 99:c 100:d 101:e 102:f 103:g
104:h 105:i 106:j 107:k 108:l 109:m 110:n 111:o
112:p 113:q 114:r 115:s 116:t 117:u 118:v 119:w
120:x 121:y 122:z 123:{ 124:| 125:} 126:~
32: 33:! 34:" 35:# 36:$ 37:% 38:& 39:'
40:( 41:) 42:* 43:+ 44:, 45:- 46:. 47:/
48:0 49:1 50:2 51:3 52:4 53:5 54:6 55:7
56:8 57:9 58:: 59:; 60:< 61:= 62:> 63:?
64:@ 65:A 66:B 67:C 68:D 69:E 70:F 71:G
72:H 73:I 74:J 75:K 76:L 77:M 78:N 79:O
80:P 81:Q 82:R 83:S 84:T 85:U 86:V 87:W
88:X 89:Y 90:Z 91:[ 92:\ 93:] 94:^ 95:_
96:` 97:a 98:b 99:c 100:d 101:e 102:f 103:g
104:h 105:i 106:j 107:k 108:l 109:m 110:n 111:o
112:p 113:q 114:r 115:s 116:t 117:u 118:v 119:w
120:x 121:y 122:z 123:{ 124:| 125:} 126:~
Introduction to Unix 83
The tr commandtr is short for translate.tr translates between two sets of
characters.replace all occurrences of the first character
in set 1 with the first character in set 2, the second char in set 1 with the second char in set 2, …
tr [options] [string1 [string2]]
Introduction to Unix 84
No files! Always standard input!
tr Example
Introduction to Unix 85
GoreBradleyBushMcCainTrumpLetterman
GoreBradleyBushMcCainTrumpLetterman
tr A-Z a-z gorebradleybushmccaintrumpletterman
gorebradleybushmccaintrumpletterman
Replace 'A' with 'a', 'B' with 'b', … 'Z' with 'z'
tr can delete-d option means "delete characters that are
found in string1".
Introduction to Unix 86
GoreBradleyBushMcCainTrumpLetterman
GoreBradleyBushMcCainTrumpLetterman
tr -d aeiou GrBrdlyBshMcCnLttrmn
GrBrdlyBshMcCnLttrmn
Another tr example - remove newlines
Introduction to Unix 87
GoreBradleyBushMcCainTrumpLetterman
GoreBradleyBushMcCainTrumpLetterman tr -d '\n'
GoreBradleyBushMcCainTrumpLettermanGoreBradleyBushMcCainTrumpLetterman
The uniq Commanduniq removes duplicate adjacent lines from a
file.uniq is typically used on a sorted file (which
forces duplicate lines to be adjacent).uniq can also reduce multiple blank lines to a
single blank line.
Introduction to Unix 88
uniq examples
Introduction to Unix 89
GoreBradleyBushMcCainTrumpLetterman
GoreBradleyBushMcCainTrumpLetterman
uniq GoreBradleyBushMcCainTrumpLetterman
GoreBradleyBushMcCainTrumpLetterman
1010101010100
1010101010100
10100
10100
uniq
Shell - a user interfaceA shell is a command interpreter turns text
that you type (at the command line) in to actions:runs a program, perhaps the ls program.allows you to edit a command line.can establish alternative sources of input and
destinations for output for programs.
Introduction to Unix 90
Running a ProgramYou type in the name of a program and
some command line options:The shell reads this line, finds the program
and runs it, feeding it the options you specified.
The shell establishes 3 I/O channels:Standard InputStandard OutputStandard Error
Introduction to Unix 91
Programs and Standard I/O
Introduction to Unix 92
ProgramProgramStandard Input
(STDIN)Standard Output
(STDOUT)
Standard Error(STDERR)
Unix CommandsMost Unix commands (programs):
read something from standard input.
send something to standard output (typically depends on what the input is!).
send error messages to standard error.
Introduction to Unix 93
Defaults for I/OWhen a shell runs a program for you:
standard input is your keyboard.standard output is your screen/window.standard error is your screen/window.
Introduction to Unix 94
Terminating Standard InputIf standard input is your keyboard, you can
type stuff in that goes to a program.To end the input you press Ctrl-D (^D) on a
line by itself, this ends the input stream.The shell is a program that reads from
standard input.What happens when you give the shell ^D?
Introduction to Unix 95
Popular Shellssh Bourne Shell ksh Korn Shell csh C Shellbash Bourne-Again Shell
Introduction to Unix 96
CustomizationEach shell supports some customization.
User promptWhere to find mailShortcuts
The customization takes place in startup files – files that are read by the shell when it starts up
Introduction to Unix 97
Startup filessh,ksh:
/etc/profile (system defaults) ~/.profile
bash:
~/.bash_profile
~/.bashrc
~/.bash_logout
csh:
~/.cshrc
~/.login
~/.logout
Introduction to Unix 98
Wildcards (metacharacters) for filename abbreviationWhen you type in a command line the shell
treats some characters as special.These special characters make it easy to
specify filenames.The shell processes what you give it, using
the special characters to replace your command line with one that includes a bunch of file names.
Introduction to Unix 99
The special character ** matches anything.If you give the shell * by itself (as a command
line argument) the shell will remove the * and replace it with all the filenames in the current directory.
“a*b” matches all files in the current directory that start with a and end with b.
Introduction to Unix 100
Understanding *The echo command prints out whatever you
give it:> echo hi
hiTry this:
> echo *
Introduction to Unix 101
* and lsThings to try:
ls *
ls –al *
ls a*
ls *b
Introduction to Unix 102
Other metacharacters?Matches any single character
ls Test?.doc
[abc…] matches any of the enclosed characters
ls T[eE][sS][tT].doc[a-z] matches any character in a range
ls [a-zA-Z]*
[!abc…] matches any character except those listed.
ls [!0-9]*
Introduction to Unix 103
Input RedirectionThe shell can attach things other than your
keyboard to standard input.A file (the contents of the file are fed to a
program as if you typed it).A pipe (the output of another program is fed as
input as if you typed it).
Introduction to Unix 104
Output RedirectionThe shell can attach things other than your
screen to standard output (or stderr).A file (the output of a program is stored in
file).A pipe (the output of a program is fed as input
to another program).
Introduction to Unix 105
How to tell the shell to redirect thingsTo tell the shell to store the output of your
program in a file, follow the command line for the program with the “>” character followed by the filename:
ls > lsoutthe command above will create a file named lsout and put the output of the ls command in the file.
Introduction to Unix 106
Input redirection To tell the shell to get standard input from
a file, use the “<“ character:sort < nums
The command above would sort the lines in the file nums and send the result to stdout.
Introduction to Unix 107
You can do both!sort < nums > sortednums
tr a-z A-Z < letter > rudeletter
Introduction to Unix 108
Output and Output AppendThe command ls > foo will create a new
file named foo (deleting any existing file named foo).
If you use >> the output will be appended to foo:
ls /etc >> foo
ls /usr >> foo
Introduction to Unix 109
PipesA pipe is a holder for a stream of data.A pipe can be used to hold the output of
one program and feed it to the input of another.
Introduction to Unix 110
prog1prog1 prog2prog2STDOUT STDIN
Asking for a pipeSeparate 2 commands with the “|” character.The shell does all the work!
ls | sort
ls | sort > sortedls
Introduction to Unix 111
Building commandsYou can string together a series of unix
commands to do something new!Exercises:
List all files in the current directory but only use upper case letters.
List only those files that have permissions set so that anyone can write to the file.
Introduction to Unix 112
StderrMany commands send error messages to
standard error.This is a different stream than stdout.
The “>” output redirection only applies to Stdout (not to Stderr).
Try this:ls foo blah gork > savedls
(I’m assuming there are no files named foo, blah or gork!).
Introduction to Unix 113
Capturing stderrTo redirect stderr to a file you need to
know what shell you are using.
When using sh, ksh or bash it’s easy:
ls foo blah gork 2> erroroutput
It’s not so easy with csh...
Introduction to Unix 114
File DescriptorsUnix progams write to file descriptors, small
integers that are somehow attached to a stream.
STDIN is 0
STDOUT is 1
STDERR is 2
“2>” means redirect stream #2 (sh, ksh and bash)
Introduction to Unix 115
Csh and Stderr>& merges STDOUT and STDERR and sends to
a file:ls foo blah >& saveboth
>>& merges STDOUT and STDERR and appends to a file:
ls foo blah >>& saveboth
Introduction to Unix 116
More Csh|& merges STDOUT and STDERR and sends to
a pipe: ls foo blah |& sort
To send STDERR to file “err” and STDOUT to file “out” you can do this:
(ls foo blah > out) >& err
Introduction to Unix 117
Shell Variables*The shell keeps track of a set of parameter
names and values.Some of these parameters determine the
behavior of the shell.We can access these variables:
set new values for some to customize the shell.
find out the value of some to help accomplish a task.
* From now on I'll focus on sh and ignore csh
Introduction to Unix 118
Example Shell Variablessh / ksh / bashPWD current working directoryPATH list of places to look for commandsHOME home directory of userMAIL where your email is storedTERM what kind of terminal you haveHISTFILE where your command history
is saved
Introduction to Unix 119
Displaying Shell VariablesPrefix the name of a shell variable with "$".The echo command will do:
echo $HOME
echo $PATHYou can use these variables on any command
line:
ls -al $HOME
Introduction to Unix 120
Setting Shell VariablesYou can change the value of a shell variable
with an assignment command (this is a shell builtin command):
HOME=/etc
PATH=/usr/bin:/usr/etc:/sbin
NEWVAR="blah blah blah"
Introduction to Unix 121
set command (shell builtin)The set command with no parameters will
print out a list of all the shell varibles.
You'll probably get a pretty long list…
Depending on your shell, you might get other stuff as well...
Introduction to Unix 122
$PS1 and $PS2The PS1 shell variable is your command line
prompt.The PS2 shell variable is used as a prompt
when the shell needs more input (in the middle of processing a command).
By changing PS1 and/or PS2 you can change the prompt.
Introduction to Unix 123
Fancy bash promptsBash supports some fancy stuff in the prompt
string:
\t is replace by the current time
\w is replaced by the current directory
\h is replaced by the hostname
\u is replaced by the username
\n is replaced by a newline
Introduction to Unix 124
Example bash prompt======= [foo.cs.rpi.edu] - 22:43:17 =======
/cs/hollingd/introunix echo $PS1
======= [\h] - \t =======\n\w
You can change your prompt by changing PS1:
PS1="Yes Master? "
Introduction to Unix 125
Making Changes Stick If you want to tell the shell (bash) to always
use the prompt "Yes Master ?", you need to store the change in a shell startup file.
For bash - change the file ~/.bashrc.
Wait a few minutes and we will talk about text editors - you need to use one to do this!
Introduction to Unix 126
The PATHEach time you give the shell a command line
it does the following:Checks to see if the command is a shell built-
in.If not - tries to find a program whose name (the
filename) is the same as the command.The PATH variable tells the shell where to
look for programs (non built-in commands).
Introduction to Unix 127
echo $PATH======= [foo.cs.sjce.edu] - 22:43:17 =======/cs/bsm/introunix echo $PATH/home/hollingd/bin:/usr/bin:/bin:/usr/local/bin:/usr/sbin:/usr/bin/X11:/usr/games:/usr/local/packages/netscape
The PATH is a list of ":" delimited directories.
The PATH is a list and a search order.
You can add stuff to your PATH by changing the shell startup file.
Introduction to Unix 128
Job Control The shell allows you to manage jobs
place jobs in the backgroundmove a job to the foregroundsuspend a jobkill a job
Introduction to Unix 129
Background jobsIf you follow a command line with "&", the
shell will run the job in the background.you don't need to wait for the job to complete,
you can type in a new command right away.you can have a bunch of jobs running at once.you can do all this with a single terminal
(window).ls -lR > saved_ls &
Introduction to Unix 130
Listing jobsThe command jobs will list all background
jobs:
> jobs
[1] Running ls -lR > saved_ls &
>The shell assigns a number to each job (this
one is job number 1).
Introduction to Unix 131
Suspending and Killing the Foreground JobYou can suspend the foreground job by
pressing ^Z (Ctrl-Z). Suspend means the job is stopped, but not
dead.The job will show up in the jobs output.
You can kill the forground job by pressing ^C (Ctrl-C). It's gone...
Introduction to Unix 132
Moving a job back to the foregroundThe fg command will move a job to the
foreground.You give fg a job number (as reported by
the jobs command) preceeded by a %.
> jobs
[1] Stopped ls -lR > saved_ls &
> fg %1
ls -lR > saved_ls
Introduction to Unix 133
Quoting - the problemWe've already seen that some characters
mean something special when typed on the command line: * ? []
What if we don't want the shell to treat these as special - we really mean *, not all the files in the current directory:
echo here is a star *
Introduction to Unix 134
Quoting - the solutionTo turn off special meaning - surround a
string with double quotes:
echo here is a star "*"
echo "here is a star"
Introduction to Unix 135
Careful!You have to be a little careful. Double
quotes around a string turn the string in to a single command line parameter.
> ls
fee file? foo
> ls "foo fee file?"
ls: foo fee file?: No such file or directory
Introduction to Unix 136
Quoting ExceptionsSome special characters are not ignored
even if inside double quotes:$ (prefix for variable names)" the quote character itself\ slash is always something special (\n)
you can use \$ to mean $ or \" to mean "
echo "This is a quote \" "
Introduction to Unix 137
Single quotesYou can use single quotes just like double
quotes.Nothing (except ') is treated special.
> echo 'This is a quote \" 'This is a quote \"
>
Introduction to Unix 138
Backquotes are different!If you surround a string with backquotes the
string is replaced with the result of running the command in backquotes:
> echo `ls`
foo fee file?
> PS1=`date`
Tue Jan 25 00:32:04 EST 2000
Introduction to Unix 139
new prompt!
Communication & E-mailNews : It is invoked by user to read any message
that is sent by system adminstator.
Write : Used for two way communications.
Mesg :Makes the user to block the message.
Talk :It is popular communication program which is superior to write.
Introduction to Unix 140
Communication & E-mailMail : Used when user is not logged in and
message can be viewed,replied,deleted.
Elm and pine : These are mail handlers which are menu driven.
Finger :Gives the details of user with communication features.
Introduction to Unix 141
Shell ProgrammingShell script – Group of commands has to be
executed & stored in file with extension .sh
Exit – Shell Script termination.
Expr – Used for computation of basic four operation
Set – Assigning values to positional parameters.
Introduction to Unix 142
Shell ProgrammingConditional Statements- If- If- elif- Case- While- Until- For
Introduction to Unix 143
Introduction to Unix 144
introduction and historyPractical Extraction and Report Language
Pathologically Eclectic Rubbish Lister?the Swiss Army chainsaw of scripting
languagescombines the best of C, sh, awk, and sedreleased in 1987 by Larry Wallinitially ported to MPE by Mark Kleinre-ported by Mark Bixby in 1997 with
periodic updates since then
Introduction to Unix 145
current statuslatest & greatest Perl release v5.6.0 available
for MPE from bixby.orgPerl is not supported by HP, but if your use of
Perl uncovers any underlying MPE or POSIX bugs, then we certainly want to hear from you!
the best way to get assistance with Perl on MPE is to post your questions to HP3000-L
Introduction to Unix 146
variable namesscalar values
$days # the simple scalar value "days" $days[28] # the 29th element of array
@days$days{'Feb'} # the 'Feb' value from hash
%days$#days # the last index of array @days
entire arrays or array slices (aka lists)@days # ($days[0], $days[1],... $days[n]) @days[3,4,5] # same as @days[3..5]
Introduction to Unix 147
value constructorsscalar values
$abc = 12345;$abc = 12345.67;$abc = 0xffff; # hex$abc = "a string with a newline\n";
list values@abc = ("cat", "dog", $def);($dev, $ino, undef, undef, $uid, $gid) = stat($file);
hash values$abc{'December'} = 12;
$month = $abc{'December'};
Introduction to Unix 148
scalar vs. list contextthe context of some operations will
determine the type of the data returnedscalarlist
assignment to a scalar variable will evaluate the righthand side in a scalar context$onerecord = <STDIN>
assignment to a list variable will evaluate the righthand side in a list context@entirefile = <STDIN>
Introduction to Unix 149
simple statements
terminated with a semicolonmay be followed by one optional modifier
if EXPR unless EXPR while EXPR until EXPR foreach EXPR
$os = 'mpe';$os = 'mpe' if $model == 3000;
Introduction to Unix 150
compound statementsa block is a sequence of statements
delimited by curly brackets (braces) that defines a scope
compound statements that control flow:if (EXPR) BLOCK if (EXPR) BLOCK else BLOCK if (EXPR) BLOCK elsif (EXPR) BLOCK ... else BLOCK
LABEL while (EXPR) BLOCK LABEL while (EXPR) BLOCK continue BLOCK LABEL for (EXPR; EXPR; EXPR) BLOCK
Introduction to Unix 151
subroutinessub max { my $max = shift(@_); foreach $foo (@_) { $max = $foo if $max < $foo; } return $max; }
$bestday = max($mon,$tue,$wed,$thu,$fri);
parameters passed via @_ array@_[0] = parm1, @_[1] = parm2, etc@_ is an alias (i.e. call by reference)private variables declared with my
return or the value of the last expression is the functional return value
Introduction to Unix 152
arithmetic operatorsaddition: +subtraction: -multiplication: *division: /modulus: %exponentiation: **auto-increment and -decrement: ++ --
++$a - increments $a, returns new value$a++ - returns current value, then increments
$a
Introduction to Unix 153
assignment operatorsworks like C
$a += 2; is equivalent to $a = $a + 2;**= += *= &= <<= &&= -= /=
|= >>= ||= .= %= ^= x=
Introduction to Unix 154
relational operatorsnumeric comparisons:
< > <= >= == != <=><=> returns -1, 0, or 1 depending on whether
the left argument is numerically less than, equal to, or greater than the right argument
string comparsions:lt gt le ge eq ne cmpcmp returns -1, 0, or 1 depending on whether
the left argument is stringwise less than, equal to, or greater than the right argument
Introduction to Unix 155
bitwise operatorsshift left: <<shift right: >>AND: &OR: |XOR: ^negation: ~
Introduction to Unix 156
i/o and file handlesopen files are identified via file handlesuppercase handle names by conventionpredefined file handles: STDIN, STDOUT, STDERR
<FILEHANDLE> in a scalar context reads the next record from the file
<FILEHANDLE> in a list context reads ALL of the remaining records from the file
Introduction to Unix 157
opening files with open()open(HANDLE, "/file/path") - open for
readingopen(HANDLE, "< /file/path") - open for
readingopen(HANDLE, "> /file/path") - open for
writingopen(HANDLE, ">> /file/path") - open
for appendingopen(HANDLE, "| shell command") - open
pipe for writing
Introduction to Unix 158
regular expressions
a vast superset beyond standard Unix regexps
a ? modifier to make patterns non-greedy
zero-width lookahead and lookbehind assertions
conditional expressions
Introduction to Unix 159
using regular expressions$showme=`callci showme`;
if ($showme =~ /RELEASE: ([A-Z]\.(\d)(\d)\.\d\d)/) {
$release = $1; # the matching V.UU.FF $mpe = "$2.$3"; # the matching U and U (i.e. 7.0)
}
$showme =~ s/LDev/Logical Device/gi; # global substitution
Introduction to Unix 160
predefined variables$| or $OUTPUT_AUTOFLUSH
By default, all Perl output is buffered (0). To enable automatic flushing, set this variable to 1. Needed when doing MPE I/O which is usually unbuffered.
$$ or $PIDPOSIX PID of the current process
$^O or $OSNAMEoperating system name (mpeix)
@ARGV
Introduction to Unix 161
debugging
invoke the debugger by starting Perl with the -d parameter#!/PERL/PUB/perl -d
examine or modify variablessingle-step executionset breakpointslist source codeset actions to be done before a line is
executed
Introduction to Unix 162
perl extensionsbinary code residing in an external
NMXL loaded at run timea thin layer of C that allows the Perl
interpreter to call compiled code written in other languages
several extension libraries come bundled with Perl (sockets, POSIX, etc)
Introduction to Unix 163
System Administrationfdisk – Dividing a disk into partitions
Useradd – Adding users
Usermod and userdel – Modifying and removing users
Umask – To make default file permissions
Fsck – File system checking
Mkfs – Creating file systems
Introduction to Unix 164
System AdministrationMount - To mount (i.e, attach) a file
system to the root file system.
Init – This daemon maintains the system at one run level.
Lpadmin – Configuring a printer
Lpsat – Obtaining printer and job status
Introduction to Unix 165
TCP\IP Network AdministrationNetconfig – Confiuring the network
interface.
Ifconfig – Interface Configuration
Ping – Checking the network.
Route – Used to build the kernel routing table to indicate the gateways that have to be used for routing packets destined.
Introduction to Unix 166