notes on unix and shell programming - akcc.kleit.ac.in 3 module.docx · web viewmodule -03. the ....

MODULE -03

The vi editorBasics of vi editor

It is a screen editor which is available on almost all unix systems.

It is a full screen editor.

To start using vi at the UNIX prompt the vi followed by a file name.

The vi uses a curser to indicate where your next command will take.

The vi uses (~) tilde character to indicate lines on screen beyond the end of the line.

While using vi at ant point you are in one of 3 modes of operation. These 3 modes rae

command mode, insert mode, ex or last line mode.

The .exrc File

The vi also supports a configuration file called as .exrc that is read when vi is executed.

When vi is started it looks for the .exrc file in the current directory from which vi was created.

If there is no .exrc file in this directory it checks for the existence of the .exrc file in the home directory

When we open a file using vi, we do some settings depending on our needs say, to set line number, to set indentation, to set the at space etc.. in the file, However these settings will remain till the file is open. Some of these settings any user would always like to retain on opening any file. These settings can be grouped in a file called as .exrc file, and hence the settings become permanent.

Different ways of invoking and quitting vi

1. The editor works with a copy of the file which is placed in a buffer that is simply a temporary storage area which is associated with the file on disk.

2. It is necessary to know how to leave the editor.3. Saving and quitting are handled by the last line mode.4. Every command in this mode is preceded by : (colon) and followed by the enter key.5. We can leave the input mode by pressing Esc key6. The last line mode or ex mode offers 2 ways of saving and quitting are :x and :wq7. The command returns you to shell command prompt after saving your work.8. To abort editing you can use :q(quit) command ,The q command takes you out of the

editor.9. To quit without making changes to the file,use the command :q!

Prepared by,Prof.Shanta K. ,KLEIT,Hubballi

Modes of vi

There are 3 modes as follows

1. Command mode 2. Input mode3. Ex or last line mode

1. Command mode When you start up vi ,you are in command mode. This mode allows you to

use certain commands to edit files or to change other modes. It is default mode. This mode is meant for navigation and cut paste operations. When you press a key in this mode it simply performs its functions The command mode allows you to entry the commands to manipulate text. When you are in command mode letters of the keyboard will be interpreted

as commands.Ex: vi f1.txt $ date

$cal

2. Input mode Before you are able to enter text,you have to change from default

command mode to input mode. When vi is invoked(called) the cursor is always positioned at the first

character of the first line. To insert text at this position press i. The i character doesn’t show up on the screen, but pressing this key

change the mode from command mode to input mode. You can use (backspace or ctrl+w) to correct mistakes in text or erase the

previous character or word. After you have entered a few lines you should press ESC to take you back

to the command mode to save your work. To append text to the right of the cursor position use a character.to append

text at the end of line use A. To open a line below the current line use o.To open a line above the

current line use O. Text can replaced O.with r,s,S,R commands. To replace one single character with another you should use r followed by

the character that replaces the one under the cursor. To replace more than a single character use R followed by text and then

press ESC. S will replace an entire line . s can delete single character n replace one

character.


3. EX mode When in vi command mode typing the “:” allows you to access the ex editor

command set. There are a number of ex commands that you will want to use within the vi

editor. From the vi editor you use the : command to enter in to an Ex mode.

Commands of ex modes

o Saving our work --:wq,:xo Aborting-- :qo Writing selected lines --:10w f1.txt3 ex mode ,it writes to 10th lineo The vi editor is built upon another editor called ax EX.o When in vi command mode typing “:” allows you to access the ex editor

command set.

Navigation commands

1. Cursor movementYou can move the cursor in command mode.The vi has many different cursor movement commands.Key movement directionH move up one line UPJ line move one character to the left DOWNK line move one character to the right LEFTL move down one line RIGHT

MOVING CURSOR

h : one space to the left

j: one line down

k : one line up

l: one space to the right


2. Word navigation commands

b : moves back to beginning of word

e: moves forward to end of the word

w: moves forward to beginning of word.

3. Searching text

A common occurrence intext editing is to replace one word or phase by another. To locate of particular sets of characters use the following commands:

/string = search down for string.

?string = search up for the string

n =repeat last search from present position

4. Inserting or adding text The vi editor supports 2 types of searches: string and character.

For a string search, the / and ? commands are used. The / command searches forwards

or downwards in the file,while ? command searches backwards or upwards in the file.

The f or F commands search for a character on the current line only. f searches

forwards and F searches backwards and the cursor moves to postion of the found

character.

The following commands allow you to insert and add text.

Esc must be typed to move to command mode followed by following commands

a = append string right of the cursor

A=append at theend of the line.

i=insert starting left of the cursor.


I=insert at beginning of current line

5. Deleting textx =delete single character

dd=delete entire line

D=delete from cursor to end of file.

u = undo last change

U=undo all changes to current line.

The set,map and abbr commands

1. The set command

The set command substitutes the string for the abbreviation only when the abbreviation is a separate word.

Set command can be used for

List existing abbreviations Remove an abbreviation Set an abbreviation

For example the command to set the shift width to 4 character is :set sw =4

To display the line numbers,enter the following command

:set number RETURN This command will immediately display the line numbers in the left margin of your vi window.

To turn of thr display of line numbers,simply type :set nonumber press return

2. Mapping keys map command

You can map a simgle keystroke intoseveral characters with the :map command.

The command takes form :map keystroke result

User can use :map,:map!,:ab subcommands to map a keystroke to command or sequence of commands.

:map is used in command mode

:map! and :ab commands can be used in text input mode.

To display a list of current mappings for the command mode enter :map

3. abbr command


The vi can be taught abbreviations so theta each time you type the abbreviations then the vi expands it.The format of defining abbreviations is

:abbreviate abbreviations expand value

To remove a previously defined abbreviation the command unabbreviated is usedto list your currently defined abbreviations type :ab

The Shell Interpretive Cycle

The shell is a type of program called an interpreter.

The following activities are typically performed by the shell in its interpretive

cycle;

i) The shell issues the prompt and waits for you to enter a command.

ii) After a command is entered, the shell scans the command line for meta

characters and expands abbreviations to recreate a simplified command line.

iii) It then passes on the command line to the kernel for execution.

iv) The shell waits for the command to complete and normally cant do any

work while the command is running.

v) After the command execution is complete, the prompt reappears and the

shell returns to its waiting role to start the cycle.

Pattern matching-Wild cards


The special set of characters that the shell uses to match file names are called as

wildcards.

Consider an example ls chap1 chap2 chap3 chap4 chap5 chap7 ,here the file names

are started with the letters chap ,so we can substitute the * as a wildcard symbol for

any number of characters in any filename.

If you type just * after a command, it stands for all files in the current directory.then

the command is executed on the files with names that match the pattern.

Wildcard characters allow you to match multiple filenames.

Character Description

* Matches one or more characters

? Matches exactly one character

[ ] Matches exactly one of a specified set of characters

The pattern is framed using ordinary characters anda meta characters using well

defined rules.

Then the pattern can be used as an argument to the command and shell will expand it

suitably before the command is executed.

Ex1: ls f* here * can be replaced by any characters like f1 ,f2,file1,file2…

f * is an argument to the command ls.

Ex2: ls f? can replace only one character like f1,f2,fk,fl,ff…

File1 file2 will not matched by ?

Ex3: ls f[1-5] can match f1 f2 f3 f4 f5 it cant match f6 f7 file 1 file2.. because [ ]

defines the range .so in the range only the f1 f2 f3 f4 f5 will come .

Where as ls f[!1-5] will display all the files with starting letter as f6 f7 f8 f9.. and

not f1 f2 f3 f4 f5.

Meta characters: Meta characters are characters that specially processed by the shell.when you enter

the command with metacharacters then the shell do the following:I. Scans the metacharacters

II. Process them


III. Executes the command Meta characters are having special meaning.

Characters Description$ To end the line& To provide background execution of the

command* To match one or more characters[ ] To insert wildcards^ To begin a line? To replace a single character/ Used for root directory~ To name home directory Removing special meaning of wildcardsThere are 2 methods to remove the special meaning of wildcards are as follows.

1. Escaping

2. Quoting

Escaping

Provides a \ backslash before a wildcard to remove its special meaning

Ex1: rm chap\*

Here in the above example \ is used it turns off its special meaning.it will not

remove the filenames start with string chap

Quoting

o This is another way of turn off the meaning of a metacharacter.

o When a command argument is enclosed in quotes the meaning of all enclosed

special characters are turned off.

o rm ‘chap*’ it will not remove the file1 chap1 chap2 chap3..

o ls ‘f?’ it will not display the files with letter f ex f1 f2 f3 f4 …are not

displayed.

Three standard files and redirection


The shell associates three files with the terminal areo 2 for displayo 1 for the keyboard

These special files are actually streams of characters which many command see as input output and error.

The stream is sequence of bytes.1. Standard input: it is the file representing input, which is connected to

keyboard.2. Standard output: It is a file representing output which is connected to display.3. Standard error: The file representing error messages that emanate from the

command or shell.this is also connected to the display.

1. Standard input We have used the cat and wc commands to read disk fiels.tehse

commands have an additional method of taking input. When they are used without arguments they read the file representing

the standard input.Syntax: command < source file name

This file is indeed special,it can represent three input resources.I. The keyboard the default resource.

II. The file using redirection with the < symbol (a metacharacter).III. Another program using pipeline.

The shell can redirect the standard input to orginate from file.this requires < symbol Ex1: file f1.txt has “kleit hubballi” 2 words 1 line 14 characters in it. When we give command wc < f1.txtIt gives output as 1 2 14 as answer

Here < symbol will will act as input operator.Wc opens a file f1.txt for reading and performs its action.


2. Standard Output All commands displaying output on the terminal actually write to the

standard output file as a stream of characters. There are 3 possible destinations of this stream are as follows:

i) The terminal, the default destinationii) A file using e\redirection symbol > and >>iii) As input to another program using the pipeline.

Syntax: command name source file > destination file

The shell can redirect the input of command to the destination file when it sees the > symbol.

We can replace any file name as destination file. If it is created already it will be overwritten, if it is not created before it will be automatically created by the shell n output will be redirected.

Ex1: Ex1: file f1.txt has “kleit hubballi” 2 words 1 line 14 characters in it. When we give command wc f1.txt > f2.txt

Wc opens file f1.txt for reading Wc performs its operation It gives output as 1 2 14 as answer in the f2.txt file.

The output will be directed to destination file.nothing will be appeared on the terminal screen.

3. Standard Error Each of these standard fiel represented by the numbers called as file

descriptors. A file is opened by referring by its file name, but subsequent read write

operations can identify the file by this file descriptor. The kernel maintains the table for each process or file in the system. The first 3 slots are allocated to 3 standard streams as

0- standard input 1-standard output 2-standard error.

We need the descriptors when handling the standard error stream.

When we enter in correct command or non-existing file error message will be shown on the screen.

The standard error stream whose default destination is terminal.

Ex1: cat f10.txt


In the above example when we try to open file f10 if it is not existed.then we will get “cant access file” as error message on the terminal.

The error message can be redirected to destination file using descriptor.

Syntax: Command name incorrect command/file 2 > destination file

Ex1: Cat f10.txt 2> f11.txt

Here command is cat try to display the content of f10.txt but it is not existing. So the error message will be produced.

Here we use the descriptor 2 as standard error file with > symbol will redirect the output of source file error message to f11.txt.

We can replace any file name as destination file. If it is created already it will be overwritten, if it is not created before it will be automatically created by the shell n output will be redirected.

Connecting commands: pipes

Pipes connect process together. The input and output of unix programs can be redirected. Pipe is UNIX.LINUX way to connect the output from one program to the

input of another without need of any temporary files. A pipe is a open file connecting two programs. Pipeline is a useful way to use output from one command as input to

another without creating an intermediate file.

Fig 3.1:Structure of Pipe Pipes are used to run more than two commands from same command line. Referring to the above figure the first command will send output to

STDOUT file.it is acting as input to second command so on. Pipe commad uses a buffer to send the piped data to the next command.


When pipe symbol | is used there must be a command on both sides of actual symbol.Syntax : command1 | commad2 | commad3 ……

Ex1: cat f1.txt | wc Here in the above example the cat will display the output of f1.txt and it is

acting as input to wc command. Where wc will read the input from STDIN an perform its action.

The output of an entire command with pipe is the word count character count and line count of f1.txt file.

Then the final output will be displayed default on the terminal screen.

Splitting the output : tee command The tee command handles a character stream by duplicating its input.

It saves copies of data from standard input to each file, and also to standard

output.

Tee can be placed anywhere in a pipeline .It doesn’t perform any filtering

action instead it gives out what it takes.

Syntax: tee options file

Options: a- append to given files and i –ignores interrupt signals

Ex1: ls –l f1.txt | wc | tee f2.txt

In this example the ls command lists the attributes of f1.txt file. This

output is piped to wc command where it counts the words,lines,and

characters. This output is piped too tee command which writes the output to

terminal and as well as to the f2.txt destination file.

Command substitution


The shell enables the connecting of two commands in yet another way using

substitution method.

While a pipe enables a command to obtain its standard input from the standard

output of another command, the shell enables one or more command arguments

to be obtained from the standard output of another command. This feature is

called as command substitution.

Example: suppose you need today’s date with statement like

“The date today is Sat Sep 7 19:09:27 IST 2016”

Here the last part of the statement is output of date command.so we can

incorporate the date command with echo command.as

$ Echo the date today is `date`

Same output as “The date today is Sat Sep 7 19:09:27 IST 2016” will be produced.

When scanning the command line the ` called as back quote is another

metacharacter that shell looks for.

If the ` is found then the shell will execute the command within ` ` .gives the

output of that respective command to the source command.

Basic and Extended regular expressions


A regular expression is a string that describes or matches a set of strings

according to certain syntax rules.

Regular expressions are powerful but they cannot recognize all possible

grammars.

Use of regular expressions provides the ability to perform multiple tasks at one

time, they save time and resources on a server.

Following are some regular expressions:

1. ^ matches the start of the line

2. $ matches the end of the line

3. . matches any single character/

4. [ ] matches a single character that is contained within the brackets.

5. [^] matches the character which is not contained in the brackets.

6. ( ) defines the subexpression.

7. “ .at” matches any three character string like cat mat hat because . can be

replace by single character.

8. “[hc]at” matches hat cat

9. “[^b]at” matches all strings with last letters at except bat.

10. “[hc]at$” matches hat and cat but only at the end of a line.ex:

Example: this is cat.

This is hat.

hat is good.only cat and hat at last lines are matched.

11. “^[hc]at” matches hat cat only at the beginning of a line.

Example: This is cat.

This is hat.

hat is good.

only hat is matched in the 3rd line.

The grep and egrep command


The grep family is a collection of three related programs for finding patterns in files.

Their names are grep,fgrep,egrep .

The name grep has its origin in the phase “Get Regular Expression and Print”

The grep command allows you to search one file or multiple files for linesthat conatin

a pattern.

Grep command display the name of the file that conatins the matched line.

Grep is the original command and it uses basic regular expression to select the lines to

processs.

Egrep is extended grep and it uses extended regular expresssions to select the lines to

process.

Fgrep is fast grep and does not use regular expression but uses literals.

Though grep expects to do the matching on text,it has no limits on input line length .

The syntax of grep command is as follows:

grep [options] pattern files

options

-b display the block or byte number at the beginning of each line

-c display the number of lines matched

-i ignores case sensitivity

-I display the filenames

-v display all lines that do not match

Example1: suppose the file f1.txt has the content

Kleit

Hubballi

India

If we use grep –b Hubblli f1.txt

It dipalys 7 Hubballi as the answer. Here 7 is the byte or block number of pattern.

Example2: Suppose the file f1.txt has the content


Kleit

Hubballi

India

If we use grep –i KLEIT f1.txt

Here –I option ignores the case sensitivity and display the line where kleit is present.

Example3: Suppose the file f1.txt has the content

Kleit

Hubballi

India

grep –c Hubblli f1.txt

Here –c option will display the matched line as 2


notes on unix and shell programming - akcc.kleit.ac.in 3 module.docx · web viewmodule -03. the ....

Documents