The UNIX File SystemThe UNIX File System

IntroductionIntroduction What is File?What is File?

Container for storing information in the form of Container for storing information in the form of sequence of characters.sequence of characters.

A file contains exactly those bytes that you put A file contains exactly those bytes that you put into it, whether it represents a source program, into it, whether it represents a source program, other text or executable code.other text or executable code.

A file neither contains its size nor its attributes.A file neither contains its size nor its attributes. It doesn’t even contain the end-of-file mark.It doesn’t even contain the end-of-file mark. All file attributes are kept in a separate location in All file attributes are kept in a separate location in

the disk specially earmarked for this purpose. the disk specially earmarked for this purpose. When a file is called up by a command or When a file is called up by a command or program, it is this area that is looked up first program, it is this area that is looked up first before the contents are accessed.before the contents are accessed.

UNIX File System in generalUNIX File System in general File system is organized in tree structure.File system is organized in tree structure. File tree can be arbitrarily deep.File tree can be arbitrarily deep. All resources are treated as files. (Devices, All resources are treated as files. (Devices,

shell, Kernel and even RAM also)shell, Kernel and even RAM also) File contains only information, no EOF File contains only information, no EOF

indicator.indicator. Files have access permissions.Files have access permissions.

Types of FilesTypes of Files

Ordinary File / Regular FileOrdinary File / Regular File Directory FileDirectory File Device FileDevice File

Types of FilesTypes of Files1.1. Regular Files:Regular Files:

This is the traditional definition of a file.This is the traditional definition of a file. It consists of a stream of data resident on some It consists of a stream of data resident on some

permanent magnetic media.permanent magnetic media. You can put anything you want into this type of file. This You can put anything you want into this type of file. This

includes all source programs, object and executable includes all source programs, object and executable code, all UNIX programs, as well as any files created by code, all UNIX programs, as well as any files created by the user.the user.

Commands like pwd, ls, etc. Are treated as ordinary files.Commands like pwd, ls, etc. Are treated as ordinary files. All text files also belong to this type. Any text file in UNIX All text files also belong to this type. Any text file in UNIX

contains a sequence of lines, with each line terminated contains a sequence of lines, with each line terminated by the LF(linefeed) character. This character is also by the LF(linefeed) character. This character is also known as the known as the newline newline character, the same character character, the same character used to terminate a command.used to terminate a command.

Types of Files (contd..)Types of Files (contd..)

2. Directory File:2. Directory File: A directory can be defined as a storehouse A directory can be defined as a storehouse

where other programs and sub-directories where other programs and sub-directories reside.reside.

A directory contains no data, but keeps an A directory contains no data, but keeps an account of all the files and sub-directories that account of all the files and sub-directories that it contains.it contains.

The UNIX file system is organised with a The UNIX file system is organised with a number of such directories and sub-directories.number of such directories and sub-directories.

A directory file contains two fields--- the name A directory file contains two fields--- the name of a file and a pointer to a separate disk area of a file and a pointer to a separate disk area which contains the file’s attributes.which contains the file’s attributes.

Types of Files (contd..)Types of Files (contd..)

3. Device File:3. Device File: Allows programs to communicate with Allows programs to communicate with

hardware. hardware. Kernel modules handles device management.Kernel modules handles device management. The device file is special in the sense that any The device file is special in the sense that any

output directed to it will be reflected onto the output directed to it will be reflected onto the respective I/O (input/output) device associated respective I/O (input/output) device associated with the filename.with the filename.

Structure of The File System

/ rootbin Binary executable fileslib Library functionsdev Device related filesetc Binary executable files usually required for system administrationtmp Temporary files created by Unix or usersusr Home directories of all users/usr/bin Additional binary executable files

What is in a File Name?What is in a File Name? File name can be up to 255 characters File name can be up to 255 characters

long.long. Files may or may not have extensions.Files may or may not have extensions. Can consist practically any character Can consist practically any character

(except ‘/’ and NULL).(except ‘/’ and NULL).

Recommended characters in file names:Recommended characters in file names: Alphabetic and numeric characters.Alphabetic and numeric characters. Period (.), Hyphen (-), underscore (_).Period (.), Hyphen (-), underscore (_).

File Handling UtilitiesFile Handling Utilities1. cat:1. cat:

Displays the contents of a file on the terminal.Displays the contents of a file on the terminal.

Ex: $cat abc.txtEx: $cat abc.txt

Options:Options:

-v-v To display non printable characters.To display non printable characters.

-n-n To display line numbers.To display line numbers.

EX: $cat –n abc.txtEX: $cat –n abc.txt

Using cat to create a file:Using cat to create a file:$cat>file1$cat>file1Welcome to Unix ProgrammingWelcome to Unix ProgrammingThis is your first fileThis is your first fileTo save the file Press [CTRL+d]To save the file Press [CTRL+d]$_$_

Using cat to concatenate files:Using cat to concatenate files:

$ cat file1 file2 > newfile

This would create newfile which contains contents of file1 followed by that of file2. If newfile already contains something it would be overwritten.

If you want that it should remain intact and contents of file1 and file2 should get appended to it then you should use the ‘append output redirection operator’, >>, as

$ cat file1 file2 >> newfile

File Handling UtilitiesFile Handling Utilities2. cp:2. cp:

Duplicates a file or group of files.Duplicates a file or group of files.

Ex:Ex: $cp abc.txt b.txt $cp abc.txt b.txt

$cp chap1 chap2 chap3 $cp chap1 chap2 chap3 prgsprgs

$cp chap* $cp chap* prgsprgs

Options:Options:

-i-i To prompt user before overwriting the destination file. To prompt user before overwriting the destination file.

-R-R To copy complete directory structure.To copy complete directory structure.

EX: $cp -i chap1 chap2EX: $cp -i chap1 chap2

Eg:

[itlakshmi@itlab2 ~]$ cp -R d1 d2

$ cd d1

$ ls

file1 file2 [itlakshmi@itlab2 d1]$ cd\ > [itlakshmi@itlab2 ~]$ cd d2 [itlakshmi@itlab2 d2]$ ls file1 file2

File Handling UtilitiesFile Handling Utilities3. rm:3. rm:

Deletes one or more files.Deletes one or more files.

Ex:Ex: $rm abc.txt $rm abc.txt $rm chap1 chap2 chap3$rm chap1 chap2 chap3

Options:Options:-i-i To prompt user before deleting To prompt user before deleting

the file.the file.-r or -R-r or -R Recursive deletion.Recursive deletion.-f -f Forcing for deletion.Forcing for deletion.

EX:EX: $rm –r dir1 $rm –r dir1

Eg: $ ls

a abc.txt b c d1 d2

$ rm -i a

rm: remove regular file `a'? y

$ ls

abc.txt b c d1 d2 In DOS, to remove a directory, you are first required to

empty the directory, and then delete it. Unix offers a single command for the same.

Eg: $ rm -r d1 rm used with the –f option removes files forcibly,

irrespective of whether you have write permission to them or not.

File Handling UtilitiesFile Handling Utilities4. mv:4. mv:

Renames a file or Directory.Renames a file or Directory. Moves a group of files to different Directory.Moves a group of files to different Directory.

Ex:Ex: $mv abc.txt b.txt $mv abc.txt b.txt

$mv chap1 chap2 chap3 $mv chap1 chap2 chap3 prgs prgs ( chap1 chap2 chap3 ( chap1 chap2 chap3 no longer present in their original location but are moved to no longer present in their original location but are moved to the directory prgs).the directory prgs).

Options:Options:-i-i To prompt user before moving/renaming the To prompt user before moving/renaming the

File/Directory.File/Directory.

Ex: $mv -i abc.txt b.txtEx: $mv -i abc.txt b.txt

Moving a file implies removing it from its current location and copying it to a new location.

File Handling UtilitiesFile Handling Utilities

5. more:5. more: While viewing large files, you often find the

output scrolling offf your screen. This sometimes happens so fast that, before you hit <Control-s> to stop it, quite a bit of the output would have scrolled off. Using <Control-s> and <Control-q> alternatively is certainly not a comfortable method of halting the output. It also doesn’t let you see what has already been seen before. UNIX offers the more command as a paging tool, so that you can view one page at a time.

Unix offers this command for pager which has Unix offers this command for pager which has today replaced with ‘pg’ command.today replaced with ‘pg’ command.

Ex:Ex: $more chap1 $more chap1

you will the see the contents of chap1 on the you will the see the contents of chap1 on the screen, one page at a time. Normally, the message “—Morescreen, one page at a time. Normally, the message “—More—” , along with the % of the file that has been viewed, is —” , along with the % of the file that has been viewed, is displayed at the bottom of the screen.displayed at the bottom of the screen.

To quit the To quit the moremore program , simply press q (quit). program , simply press q (quit).

More also works with multiple filenames.More also works with multiple filenames.

Ex: $more chap1 chap2 chap3Ex: $more chap1 chap2 chap3

$ more +10 -15 –s -d myfile yourfile$ more +10 -15 –s -d myfile yourfile

File Handling UtilitiesFile Handling Utilities6. ls:6. ls:

Lists all the names of all files and sub directories in the Lists all the names of all files and sub directories in the current current

directory.directory.

Ex:Ex: $ls $ls

Options:Options:-x -x Multicolumnar output. Multicolumnar output. -F -F Marks executables with *, directories with / .Marks executables with *, directories with / .-a -a Shows all file names beginning with a dot including . and ..Shows all file names beginning with a dot including . and ..-R -R Recursive list.Recursive list.-r -r Sorts file names in reverse order. Sorts file names in reverse order.-l -l Long listing showing 7 attributes of a file.Long listing showing 7 attributes of a file.-d -d Lists only directory names.Lists only directory names.-t -t Sorts file names by last modification time.Sorts file names by last modification time.-1 -1 One file per line.One file per line.-u -u Sorts file names by last access time.Sorts file names by last access time.-i -i Displays inode number.Displays inode number.

1. Ex: $ ls

d1 d2 df1 example f1

2. Ex: $ ls -a

. .bash_history .bash_profile d1 df1 f1 .kde

.. .bash_logout .bashrc d2 example .gnome2 .zshrc

3. Ex: ls -F

d1/ d2/ df1 example/ f1

4. Ex: $ ls -r

f1 example df1 d2 d1

5. Ex: ]$ ls -l

total 20

drwxrwxr-x 2 itlakshmi itlakshmi 4096 2009-07-11 13:16 d1

drwxrwxr-x 2 itlakshmi itlakshmi 4096 2009-07-11 13:20 d2

-rw-rw-r-- 1 itlakshmi itlakshmi 14 2009-07-11 15:23 df1

drwxrwxr-x 2 itlakshmi itlakshmi 4096 2009-07-11 14:20 example

-rw-rw-r-- 1 itlakshmi itlakshmi 31 2009-07-11 15:22 f1

In above eg., ‘total 20’ indicates that the total number of disk blocks that the files in the current directory have occupied is 20.

Unix treats all entities – files, directories, devices – as files. In all the lines listed in the output of the command ‘ls –l ‘ the leftmost character indicates this type.

A ‘-’ indicates that the file is an ordinary file. Other possible filetypes are given below.

File Type Meaning

- Ordinary file

d Directory file

c Character special file

b Block special file

l symbolic link

s Semaphore

p Named pipe

m Shared memory file

The character special files and the block special files are normally present in the /dev directory. These files are used to handle character oriented devices like terminals or block oriented devices like disks. When these files are listed the file size entry for these files is replaced by major and minor device numbers. Usually these files are never found in user’s directory.

The next nine characters following the file type character are the file permissions.

Each column in succession gives the number of links, owner name, group name, size of file in bytes, date and time when the file was last modified, and finally the filename.

The person who creates the file is the owner of that file. A ‘group’ may be formed of a given number of users, who may be working on similar data.

How many links the file has essentially determines by how many different names the file is accessible. When a file has two links, it is not physically present at two places, but can be referred to by either of the names.

6. Eg: $ ls -R.:d1 d2 df1 example f1

./d1:a2 b2

./d2:file1 file2 samp

./example:l_file

File Handling UtilitiesFile Handling Utilities

7.lp :

UNIX provides a spooling facility by which you can queue up jobs for printing. This is done with the lp (line printing) command.

Ex:Ex: $lp a.lst $lp a.lst request id is lp-320 (1 file)request id is lp-320 (1 file)

$_$_ Options:Options:

-d<printer name> <file name> -d<printer name> <file name> To specify printer name. To specify printer name. -t ”title” <file name> -t ”title” <file name> To specify page title. To specify page title. -m -m To notify user after the file has been To notify user after the file has been

printed. printed. -n<number> -n<number> To specify number of copies. To specify number of copies.

Ex:Ex: $lp –d laser a.list $lp –d laser a.listEx:Ex: $lp –t”first chapter” a.txt $lp –t”first chapter” a.txtEx:Ex: $lp –n3 –m a.lst $lp –n3 –m a.lst

The file is not actually printed at the time the The file is not actually printed at the time the command is invoked, but later depending on the command is invoked, but later depending on the number of jobs already lined up in the queue. number of jobs already lined up in the queue. Several users can print their files in this way Several users can print their files in this way without conflict. There is a without conflict. There is a demondemon (a process (a process which runs periodically ) which monitors this which runs periodically ) which monitors this queue and prints each job in turn. queue and prints each job in turn.

The output from the printer will be a hard copy of The output from the printer will be a hard copy of the actual file, preceded by a title page the actual file, preceded by a title page mentioning the user name, request-id and date.mentioning the user name, request-id and date.

File Handling UtilitiesFile Handling Utilities8. lpstat:8. lpstat:

To view print queue.To view print queue.

Ex:Ex: $lpstat $lpstat 9. cancel:9. cancel:

To remove a print job from print queue.To remove a print job from print queue.

Ex:Ex: $cancel laser $cancel laser $cancel lp-320$cancel lp-320

10. file:10. file:To know file type.To know file type.

Ex: Ex: $ file f1$ file f1f1: ASCII textf1: ASCII text

File Handling UtilitiesFile Handling Utilities11. wc:11. wc:

To count lines, words and characters in a file.To count lines, words and characters in a file.

Ex:Ex: $wc file1 $wc file1 33 2020 103103 file1 file1 $_$_

Options:Options:-l -l To print number of lines. To print number of lines.-w -w To print number of words. To print number of words.-c -c To print number of characters. To print number of characters.

Ex:Ex: ]$ wc f1 df1]$ wc f1 df1 3 4 31 f13 4 31 f1

3 3 14 df13 3 14 df1

6 7 45 total6 7 45 total [itlakshmi@itlab2 ~]$ wc -wl f1[itlakshmi@itlab2 ~]$ wc -wl f1

3 4 f13 4 f1

File Handling UtilitiesFile Handling Utilities

12. cmp:12. cmp:

Comparing two files byte by byte.Comparing two files byte by byte.Ex:Ex: $cmp file1 file2$cmp file1 file2 File1 file2 differ: char 9, line 1File1 file2 differ: char 9, line 1 $_$_The two files are compared byte by byte The two files are compared byte by byte

and the location of the first mismatch is and the location of the first mismatch is echoed to the screen.echoed to the screen.

When two files are identical, then cmp When two files are identical, then cmp displays no message, but simply returns displays no message, but simply returns the $ prompt.the $ prompt.

File Handling UtilitiesFile Handling Utilities13. comm:13. comm:

Checks for commonality between two sorted files by Checks for commonality between two sorted files by comparing each line of the first file with its corresponding comparing each line of the first file with its corresponding line in the second.line in the second.

Displays three columnar output:Displays three columnar output: lines unique to first file.lines unique to first file. lines unique to second file.lines unique to second file. Lines common to both the files.Lines common to both the files.

Ex:Ex: $comm file1 file2 $comm file1 file2

Options:Options:

-1 -1 To drop first column. To drop first column.

-2 -2 To drop second column. To drop second column.

-3 -3 To drop third column. To drop third column.

Ex:Ex: $comm -12 file1 file2 $comm -12 file1 file2

Ex: $ cat fname1abcghijkl

$ cat fname2defghijklmno

$ comm fname1 fname2 abc def ghi jkl mno

File Handling UtilitiesFile Handling Utilities14. diff:14. diff:

To display file differences.To display file differences. Unlike cmp, comm it also displays lines to be changed in Unlike cmp, comm it also displays lines to be changed in

file1 to make two files identical.file1 to make two files identical. diff uses special symbols and instructions to indicate the diff uses special symbols and instructions to indicate the

changes that are to required to make two files identical.changes that are to required to make two files identical.

Ex: Ex: $diff file1 file2$diff file1 file20a1,20a1,2 append after line 0 of first fileappend after line 0 of first file> anil aggarwal> anil aggarwal this linethis line> raj rajiv> raj rajiv and this lineand this line2c42c4 change line 2 of first filechange line 2 of first file< chanchal singhal< chanchal singhal replacing this linereplacing this line---- withwith> lalit rajiv> lalit rajiv this linethis line4d54d5 delete line 4 of first filedelete line 4 of first file< sumit chakrobarty< sumit chakrobarty containing this linecontaining this line

File Handling UtilitiesFile Handling Utilities

15. dos2unix and unix2dos:15. dos2unix and unix2dos:To convert files between DOS and UNIX.To convert files between DOS and UNIX.

EX:EX: $dos2unix file1 file1.dos $dos2unix file1 file1.dos

16. dos:16. dos:To convert files between DOS and UNIX.To convert files between DOS and UNIX.

EX:EX: $dos2unix file1 file1.dos $dos2unix file1 file1.dos

File Handling UtilitiesFile Handling Utilities

COMPRESSING AND ARCHIVING FILESCOMPRESSING AND ARCHIVING FILES

Unix system comes with some or all of the following Unix system comes with some or all of the following compression and compression and decompression utilities.decompression utilities.

Apart form compressing, you’ll also need to group a set of Apart form compressing, you’ll also need to group a set of files into a files into a single file, called an “archive”.single file, called an “archive”.

gzip and gunzip (.gz).gzip and gunzip (.gz). bzip2 and bunzip2 (.bz2)bzip2 and bunzip2 (.bz2) zip and unzip (.zip)zip and unzip (.zip) tartar

File Handling UtilitiesFile Handling Utilities17. gzip:17. gzip:

To compress one or more files.To compress one or more files. It provides the extension .gz to the compressed file and It provides the extension .gz to the compressed file and

removes the original file.removes the original file.Ex: $gzip libc.html Ex: $gzip libc.html

Options:Options:-d -d Uncompressing a “gzipped” fileUncompressing a “gzipped” file-r -r Recursive compression compressing files in sub Recursive compression compressing files in sub

directories also.directories also.

Ex:Ex: $gzip –d libc.html $gzip –d libc.html $gzip –r $gzip –r prgsprgs

gunzip:gunzip:To decompress one or more files.To decompress one or more files.Ex:Ex: $gunzip –r $gunzip –r prgs prgs or gzip –dr or gzip –dr prgsprgs

File Handling UtilitiesFile Handling Utilities18. tar:18. tar:

For creating disk archive that contains a group of files or an For creating disk archive that contains a group of files or an entire entire

directory. directory.

Options:Options:-c -c Create an archive/write operation.Create an archive/write operation.-x -x Extract files from archive.Extract files from archive.-v -v Display the progress while tar works (Verbose).Display the progress while tar works (Verbose).-t -t Display files in archive.Display files in archive.-f arch -f arch Specify the archive name.Specify the archive name.

Creating an Archive:Creating an Archive: (-c) (-c)$tar -cvf archive.tar libc.html user_guide.ps$tar -cvf archive.tar libc.html user_guide.ps$gzip archive.tar$gzip archive.tar

Extracting files from Archive:Extracting files from Archive: (-x) (-x)$tar -xvf archive.tar$tar -xvf archive.tar

Viewing the Archive:Viewing the Archive: (-t) (-t)$tar -tvf archive.tar$tar -tvf archive.tar

File Handling UtilitiesFile Handling Utilities19. zip:19. zip:

It combines the compressing (gzip) and archival (tar). It combines the compressing (gzip) and archival (tar).

Options:Options:-r -r Recursive compression.Recursive compression.-v -v Viewing the archive.Viewing the archive.

Ex:Ex: $zip archive.zip libc.html userguide.ps $zip archive.zip libc.html userguide.ps

unzip:unzip:It combines the decompressing (gzip -d) and extrcting (tar -It combines the decompressing (gzip -d) and extrcting (tar -

x). x).

Ex:Ex: $unzip -v archive.zip $unzip -v archive.zip