unix training for beginners
TRANSCRIPT
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 1/71
UNIX
andSHELL Scripting
By
Sanju Kumar1141983000
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 2/71
Contents
What Is UNIX? History
Basic Terminology
Files and Directories File Permissions
User Commands
Vi Editor Shells Scripting
Linux Development
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 3/71
What Is UNIX?
A computer operating system. It is designedto be used by many people at the same time(multi-user).
Runs on a variety of processors
Lends itself well to use in scientific research
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 4/71
History
UNIX was invented in 1969 by AT&T BellLaboratories
Ken Thompson and Dennis Ritchie are
credited as the original architects Written in the C language in 1973
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 5/71
Different Flavors, All UNIX
Solaris (Sun) HP-UX (HP)
Linux (open source).. RH, Mandrake etc
Irix (SGI)
Berkeley (Mac OS X based on this)
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 6/71
The Shell
When you log in to interact with UNIX, yousee a “shell prompt” ($,# or %)
The shell is a program that runs constantly
and executes the commands you give it
You can choose which shell you prefer
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 7/71
Files and Directories
UNIX uses a hierarchy to store files Files are simply a named collection of bytes
Directories contain other files
Main.c
USSD/
TTB/
Config.txt Func.c
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 8/71
System Directories
/root- The source of everything
/bin - binaries (executables) for all the commands
/sbin- generally of interest only to super user
/dev - devices, such as printers. Not strictly part of
UNIX. Memory is also a device.
/etc - utilities for managing user’s environment
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 9/71
System Directories cont….
/tmp - temporary storage space; same as C:\TEMP /home – Home Directory for users
/usr/local - user utilities and local libraries
/var – system logs and other stuff
/proc – has runtime system info
(try: cat /proc/cpuinfo)
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 10/71
Some Basic Terms
• Home directory - top of your file tree
• Root directory – root of file system
• Current directory - directory you’re in now
• Dot (.) - slang for the current directory
• Dot-dot (..) - parent of the current directory
• Path - Where to find file or directory– Absolute: /home/sanju/training.txt
– Relative: ~/training.txt
• Wildcard - * (asterisk) can be used to stand forany character
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 11/71
venus:/home/sanju$ ls -la
commandflags
• Commands are the way to “do things” in unix
(no point and click)
• A command consists of a program name and optionscalled “flags”
prompt
What is command?
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 12/71
Handling Files and Directories
mkdir - creates a directory rmdir - removes a directory (must be empty first)
rm – removes file
rm -fr - removes directory and everything below
touch <filename> - creates an empty file
cat <filename> - displays a file on screen
more <filename> - to see a screenful at a time
mv – Move/Renames a file
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 13/71
User Administration
Group(QAT)
Chandni
Sanju
Mukesh
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 14/71
Create Groups and Users
JURXSDGG ದ $GGDJURXS JURXSGHO 'HOHWHDJURXS
XVHUDGG &UHDWHDQHZXVHUDFFRXQW
XVHUGHO 'HOHWHDXVHUDFFRXQW
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 15/71
• GroupsIn Unix, all users belong to at least one group. The idea is that
each person in one group has a similar amount of access to the
system.
• PermissionsPermissions tell the OS who can do what with your files and
directories. Permissions can be set for three categories, theuser, the group and the rest of the world. The permissions are:
read, write, execute
Groups and Permissions
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 16/71
Groups and Permissions cont…
You “own” your files and directories user-group-other read/write permissions -rw-r--r-- 1 sanju qat 164870 27 Feb 17:58 6.bmp
-rw-r--r-- 1 sanju qat 164870 27 Feb 17:57 5.bmp
-rw-r--r-- 1 sanju qat 164870 27 Feb 17:57 4.5.bmp
-rw-r--r-- 1 sanju qat 164870 27 Feb 17:57 4.bmp
chmod command changes permissions
chmod g-rw 6.bmp-rw----r-- 1 sanju qat 164870 27 Feb 17:58 6.bmp
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 17/71
Groups and Permissions cont…
What they do:
files:read – Allows you to read the file
write – Alows you to modify the fileexecute –Allows you run the file as a script or binary program
directories:
read – lets you get a directory listing of fileswrite – lets you add or remove files from directory
execute – lets you access files in the directory
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 18/71
Groups and Permissions cont…Special Commands
Wants to change the user owner?
$ chown <NewOwner> <FileName>
Wants to change the group owner?
$ Chgrp <Newgroup> <FileName>
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 19/71
User Commands
What’s in my directory?• lsconfig.txt ERICSSONHLRSTUB hlrstub.h Makefile newtelserver.c
• ls -lt-rw-rw-rw- 1 sanju sanju 560 May 10 14:35 config.txt
-rwxrwxr-x 1 sanju sanju 21530 May 16 17:26 ERICSSONHLRSTUB
-rw-rw-rw- 1 sanju sanju 1388 May 10 14:34 hlrstub.h-rw-rw-rw- 1 sanju sanju 177 May 16 17:26 Makefile
-rw-rw-rw- 1 sanju sanju 18614 May 16 17:27 newtelserver.c
Man[ual] page: man ls
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 20/71
User Commands, cont.
Where am I?pwd - print working directory
Who am I?
whoami
Who is on the system? What’s running?
who
w
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 21/71
User Commands, cont.
What time is it?date
Fri Sep 29 09:55:11 IST 2006
• Where is a command?which ls
whereis ls
• How do I use a command?man ls
• The “why” is left to you.
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 22/71
More Useful Commands
cd - change directories (built into shell) wc - word, line, character, and byte count
echo - echo characters back (print)
passwd - change password
sort - sort lines of a text file
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 23/71
More Useful Commands, cont.
cp - copy a file(s) ps - status of processes (what’s running)
history - list of previous commands
alias - create a shorthand for a command
kill – kill a process
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 24/71
More Useful Commands, cont.
• ln – creating links (kind of like shortcuts, butthey work right), HARD and SOFT
• finger – get some information on other users
• diff – find the difference between two (text)
files
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 25/71
Exercise
Let’s copy a file to our directory.
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 26/71
Tar and Compress
Tar stands for “tape archive” but is more often used to gathera files into one bundle
Get in directory above the one you want to bundle
tar -cvf dir_name > diry.tar
To “untar”, usetar -xvf diry.tar
To compress a tar file, use
compress diry.tar or gzip diry.tarTo uncompress, use
uncompress diry.tar.Z or gunzip diry.tar.gz
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 27/71
– used for shell-attempted filename matching
– special characters for wildcards:
*, ?, []
* : matches any string of zero or more characters(special case: a lone * will not match files starting with '.')
? : matches exactly one character
[] : matches any single character within the square
brackets
Wildcards
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 28/71
Finding and Searching
find - a tricky but useful commandfind . -name myfile -print
find . -name ‘*name*’ -print
• grep - stands for “general regular expression print”Search all files for a pattern:
grep -i FUNCTION_FIND *.c
sanju.c: FUNCTION_FIND(*abc, *tg)
Count the number of occurrences of gee whiz in all thefiles:
grep -c FUNCTION_FIND *.[ch]
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 29/71
Input/Output (I/O) andRedirection
Input can be given from the command line, orfrom a file
a.out < inputfile
Output goes to your screen, or you can “redirect”it to a file
echo “hi there” > out.file
• To append, use >>
echo “hi back at you” >> out.file
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 30/71
Pipes
You can string commands together into asingle command using pipes
who | wc – count : how many users on the
systemps | grep sanju - what am I running now
• Save the results in a file:
who | wc > usercount
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 31/71
Pipes and Redirection>& : redirect all output (both stdout and stderr) to a
file
e.g. ls -l sanju>& sanju.alle.g. my.script >& script.output
|& : include stderr in the pipe
e.g. my. script |& error
Advance commands(pipe and redirection)
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 32/71
File Transfer Protocol (FTP)
FTP is can be used via a command lineprogram (“ftp”) or by any number of graphicalFTP tools.
Interaction is similar to command shells (“cd”, “ls”, etc.) with extra commands to transfer files (“get”, “mget”).
Make sure that you are using binary modewhen transferring non-text files.
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 33/71
Editors
vi & Emacs
Advantage of vi is that it is found on every
UNIX system, while other editors may or
may not be available.
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 34/71
vi – How to use it
vi is not user-friendly!
You have to remember all of the
commands, in addition to which modethe editor is in.
However, vi is very powerful and fastonce you have mastered it.
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 35/71
vi – How to use it
Start vi with a command such as:
$ vi filename
There are three modes:command, insert , and colon mode.
The editor starts in command mode.
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 36/71
vi – What it looks like
Hello, this is the file “sanju" and we are
editing it with vi.~
~
~~
~
~
~
~
“sanju" 1L, 61C 1,1
All
File contents
Line and character count
Current editing
position
Empty line
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 37/71
vi – Modes
Command mode – initial state, <ESC>
cancels partial command
Insert mode – entered with ‘a’, ‘i’, etc., exit with <ESC>
Colon mode – entered with ‘:’,
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 38/71
vi – Command mode
dw delete word
dd delete line4dd delete 4 lines
x cut character
^F move forward one page
^B move back one page$ go to end of line
% find matching bracket
i insert
a appendr replace
u undo
. repeat command
: enter last line file mode
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 39/71
vi – Last line mode
: – enter last line file mode
w – save
q – quit
wq – save and quit
q! – force quit, discard changes
e <filename> – open new file
e! – re-edit current file
! <cmd> – execute shell command
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 40/71
vi Demo
Let’s find out what we can do…
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 41/71
Shell Scripting
To automate frequently used commands
As interfaces between off-the-shelf and companydeveloped software
As applications performing tasks all their own, orautomating basic system administration tasks
Scripts are useful for smaller jobs. They areinterpreted by the shell rather than compiled, sothey run fairly slowly
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 42/71
The Shell Script File
The script file will and should contain more information than just the commands needed to produce the desired output.
Comments--non-executablestatements included in a source codefile to explain the source code
A comment is signified by the # (hash/pound) sign.
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 43/71
SCRIPT HEADER
#!/bin/bash#Filename: FindReplace.sh
#Author: Sanju Kumar
#Group: QA & Testiing
#Created: 28-Sep-2006
##Description: This script takes an input as a string and replace its withanother string provided from command prompt.
Note: You will not receive full credit on lab assignments going forward unless your shell scripts contain a header like the example above.
The .sh file extension is not necessary, but is a convention that is widely used. We will use it in our class.
The Shell Script File
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 44/71
Using the Shell as a Programming Language
Shell Variables--symbolic names thatcan access values stored in memory foruse by the script program
Operators--symbols that representmathmatical or logical operations
Logical Structures--structures that
define program logic (loops, if statements, etc.)
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 45/71
C Shell versus Bourne ShellSyntax
C Shell is an older syntax, still commonlyused
More modern C Shell-like shells include
tcsh Bourne Shell added some desirable features
but has different syntax for some things
Newest Bourne Shell variants are KornShell (ksh) and Born Again Shell (bash)
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 46/71
Command arguments
$./MyScript.sh <FirstArgu> <SecondArug>
A shell script to swap files:
#! /bin/bash
cp $1 tmp
cp $2 $1
cp tmp $1
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 47/71
Variables
There are a number of predefined shell variables, suchas $HOME, $PATH
User can also set “environment variables”:
C shell: setenv SANJU TESTING
Bourne: SANJU= TESTINGexport SANJU or export SANJU = TESTING
Korn: export SANJU = TESTING
• To set a variable in C shell, use set x=1• To set a variable in Bourne, just use x=1
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 48/71
Operators
-eq | Equal to -ne | Not equal to
-lt | Less than -le |Less than or equal to
-gt |Greater than -ge |Greater than or equal to
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 49/71
if-then-else
if ( expr ) simple-command
if ( expr ) then
commandlist-1
[else
commandlist-2]
fi
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 50/71
if-then-else cont’d
An example:
if [ $? –eq 2 ] then
echo “we need two parameters!“
elseecho “Two arguments provided!“
endif
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 51/71
Loops
while ( expr )
commandlist
end
For var <worddlist>
commandlist
end
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 52/71
More Shell Commands: basicflow control
– commands:if, else, endif, while, end, for
if ( $user == "sanju" ) then
echo 'the user is SANJU!‘
endif
value = 5
fact = 1
while ( $value > 0 ) #infinite loopEcho “ I am printing this again and again”
End
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 53/71
More Shell Commands: basicflow control
for i in 1 2 3 4 5
doecho $i
done
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 54/71
More Shell Loops
LOGS=mylogfile
TODAY=`date +%m-%d-%y`
for i in `cat $LOGS`
do
mv $i $i.$TODAY
chmod 664 $i.$TODAY
done
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 55/71
Hmmmm.
Okay, shell scripts are useful, but a
little boring.
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 56/71
Awk and Sed
translate input file: “stream editor” (sed)
sed 1,10d filename.txt> New.txt→ delete 1 to 10th linefrom file.
more complex processing of “records”→ awk
ps -ef | awk '{ print $2}‘
Show only second column of “ps –ef”
awk assumes input separated by spaces (can
change)
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 57/71
Homework
Write a shell script that reads a file and replaces a
particular text string with another, all input giveninteractively.
Hint: Use sed for text replacement.
PROCESS
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 58/71
• When you run a command, it starts a new “process”
• Each process has a unique number called the PID(Proccess ID)
• Unix is a multitasking operating system, so you canrun multiple processes simultaneously
• After you start a process, it is usually in what is
called the “foreground.” That is, it takes over yourshell.
PROCESS
PROCESS
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 59/71
• You can suspend processes in the foreground withCtrl-Z
• The process is now frozen. You can pull it to theforeground again by typing “fg”.
• Alternately, you can make it keep running, but put itin the background so can still use the same shell bytyping “bg”.
• You can also start a task in the background by puttinga & at the end of the command.
PROCESS cont…
PROCESS
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 60/71
• You can list all the processes that have been run fromthe current shell with “ps”
• If you want to end a process, you can do “kill <pid>”eg. kill 3724
• If that doesn’t kill it, you can do the “super kill,”
“kill -9 <pid>” eg. kill –9 3724
PROCESS cont…
PROCESS
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 61/71
• You can list all the processes that have been run from the currentshell with “ps”
• To list all the processes on the system, do “ps -ef”
1175 8152 8151 0 17:48 pts/4 00:00:00 -bash
sanju 8204 4145 0 17:50 pts/1 00:00:00 ksh
root 8316 1 0 18:01 ? 00:00:00 httpd
PROCESS cont…
• If you want to end a process, you can do “kill <pid>”
eg. kill 8152
• If that doesn’t kill it, you can do the “super kill,” “kill -9 <pid>”eg. kill –9 8152
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 62/71
Linux Development
What the tools are
What the tools do
Makefile concepts and how-to
Debugging concepts and guidelines
Examples: g++ and gdb
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 63/71
Available Tools
Compilers– C/C++ : cc , gcc , g++
– Java : javac , jikes
– Fortran: f77 , g77
Debuggers
– Text : gdb
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 64/71
Available Tools
Project/Build Management– Versioning Systems : cvs
– Conditional Rebuild: make
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 65/71
Example: A Basic Compile
g++ syntax:
g++ [<options>] <input files>
Some useful options:
-g produce debugging info (for gdb)
-Wall “W arnings all”
-ansi force ANSI compliant compilation
-o <name> output filename
Example:
g++ -g -o hello hello.cpp
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 66/71
The Make Command
make syntax:
make [-f <file>] [<target>]
argument descriptions:
-f <file> use <file> as the makefile
default: Makefile or makefile
<target> compile configuration “target”
default: first target in the makefile
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 67/71
Makefile Syntax
Rule Syntax:
<target>: <requirements>
<command>
The <requirements> may be files and/or other targets
It must be a tab before <command>, or it won’t work
The first rule is the default <target> for make
Variable Syntax:
<variable>=<string value>
All variable values default to the shell variable values
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 68/71
Example Makefile# Example Makefile
CC=gcc
CCOPTS=-g
Application: foo.o bar.o
$(CC) $(CCOPTS) –o Application foo.o
bar.o
foo.o: foo.cpp foo.h
$(CC) $(CCOPTS) –c foo.cpp
bar.o: bar.cpp bar.h
$(CC) $(CCOPTS) –c bar.cpp
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 69/71
Make Power Features
“Fake” targets
– targets that are not actually files
– can do just about anything, not just compile
– like the “clean” target
clean:
rm foo.o bar.o foobar
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 70/71
Anything Else?
7/30/2019 UNIX Training for Beginners
http://slidepdf.com/reader/full/unix-training-for-beginners 71/71
Thanx