linux system programmingcfs3.tistory.com/upload_control/download.blog?fhandle... · unix system...
TRANSCRIPT
2Unix System Programming
References
Advanced Programming in the UNIX Environment Linux manual pages Understanding the Linux Kernel
3Unix System Programming
Contents 1. Overview 2. UNIX Standards and Implementations 3. File I/O 4. Files and Directories 5. Standard I/O Library 6. System Data Files and Information 7. Environment of a Process 8. Process Control 9. Process Relationships 10. Signal
5Unix System Programming
Role of the Linux Operating System
virtual machine operates on the physical machine which contains the low
level programming interface provides the high-level abstractions as well as a more
advanced programming and user interface interfaces between applications and the machine
applications
kernel
hardware
6Unix System Programming
Role of the Linux Operating System
sharing the processor Linux system operates in multi-tasking mode
run several programs(processes) at the same time system implements a scheduling system
scheduling system connects each of the processes in turn to the processor.
by cycling the processes rapidly, machine users have the impression that the processes are being carried out in parallel.
7Unix System Programming
Role of the Linux Operating System
memory management in a multi-user and multi-tasking computer, very rigorous
control of memory is necessary. each process has its own address space. the kernel must be capable of efficiently controlling memory in
order to meet the demands of the different processes as quickly as possible.
zones of memory allocated to different processes are protected so as to prevent unauthorized modifications
8Unix System Programming
Role of the Linux Operating System
memory management (cont’d) physical memory is typically insufficient
the system uses part of the disk as auxiliary memory(swap area)
9Unix System Programming
Role of the Linux Operating System
resource manager the processor and the memory are particular instances of
resource provides processes with an interface allowing resources
(such as disks, printers, etc.) to be shared prevent conflicts of access
implements a protection system which allow users and system administrators to protect access to their data
10Unix System Programming
Role of the Linux Operating System
communication hub of the machine the OS should manage different events
events arising from either the physical layer (interrupts), or from applications(system calls)
the system must deal with the events if necessary send them to the processes concerned.
the system must be able to put several processes in communication with each other
signal, message queue, semaphore, shared memory, pipe, socket
11Unix System Programming
System Structureprocesses
System calls interface
File systems Central kernelext2fs, proc,
minix, ...task manage, schedulerIPC, memory manage
machine
machine interface
Network manageripv4, ethernet, ...
peripheral managerblock character
buffer cache
12Unix System Programming
Kernel mode & User mode
kernel mode privileged mode no restriction is imposed on the kernel of the system may use all the instructions of the processor manipulate the whole of the memory talk directly to the peripheral controllers
13Unix System Programming
Kernel mode & User mode
user mode normal execution mode for a process has no privileges
certain instructions are forbidden only allowed to zones allocated to it cannot interact with the physical machine
process carries out operations in its own environment, without interfering with other processes
process may be interrupted at any moment
14Unix System Programming
System Calls
system call system call is a request transmitted from the process to the
kernel process in user mode cannot directly access the machine
resources the kernel deals with the request in kernel mode, without
any restrictions, and sends the result to the process trap instruction
causes the CPU to switch into kernel mode Intel CPU: int 0x80
15Unix System Programming
System Calls
wrapper routine
#define _syscall0(type,name) \type name(void) \{ \long __res; \__asm__ volatile ("int $0x80" \ : "=a" (__res) \ : "0" (__NR_##name)); \__syscall_return(type,__res); \}
16Unix System Programming
System Calls
syscall( ) int syscall(int syscall_number, arg0, arg1, ...);
#include <asm/unistd.h>main(){ int p1, p2;
p1 = syscall(__NR_getpid); p2 = getpid();
printf("syscall: %d\n", p1); printf("getpid: %d\n", p2);}
17Unix System Programming
Structure of the Ext2 file system
Block group 0 Block group 1 ... Block group nBootblock
Superblock
Groupdescriptors
Blockbitmap
Inodebitmap
Inodetable Data blocks
partition 2 partition 3
hard disk
Ext2 file system
block group
partition 1
18Unix System Programming
Structure of the Ext2 file system
boot block contains bootstrap code
the machine code needed for loading the kernel when the system is started up
block group a file system is divided into a number of block groups reduce the disk seek time (exploit locality)
keep data blocks close to their inodes keep file inodes close to their directory inode
19Unix System Programming
Block Group
superblock contains the control information of the file system
number of inodes and blocks number of free inodes and blocks block size blocks per group inodes per group
duplicated in each group of blocks enable the corruption of the file system to be easily overcome
20Unix System Programming
Block Group
group descriptor contains control information of a block group
block number of the inode bitmap block number of the block bitmap block number of the first block of inode table the number of free inodes and free blocks the number of directories in this block group
used by the inode allocation algorithm for the directories attempts to spread directories as evenly as possible over
the block groups
21Unix System Programming
Block Group
block bitmap block allocation status a bit indicates whether the block is allocated or available
inode bitmap inode allocation status a bit indicates whether the inode is allocated or available
22Unix System Programming
Block Group
inode table contains a part of a table of inodes of the file system
data blocks used to store data contained in files and directories also used to store indirect blocks
23Unix System Programming
Inode
inode internal representation of a file every file has one inode inode contains
type/permission user(UID), group(GID) file size, number of blocks, link counter access time, modification time, change time list of addresses of data blocks
24Unix System Programming
Inode and Data Blocks
direct 0
direct 1
direct 2
direct 9
direct 10
direct 11
double indirect
triple indirect
single indirect
Address List in Inode Data Blocks
25Unix System Programming
File and Directory Blocks
inodetable
datablock
datablock
directoryblock
datablock
inodenumber filename
inode
26Unix System Programming
Ext2 Directory Entry
2 12 1 . 2017 20 10 lost+found
0 4 6 8 12 16 18 20 32
struct ext2_dir_entry {unsigned long inode; /* inode number */unsigned short rec_len; /* length of directory entry */unsigned short name_len; /* length of filename */char name[EXT2_NAME_LEN]; /* filename */
};
27Unix System Programming
Kernel Data Structure for Open Files
fd flag ptrfd 0fd 1fd 2
user file descriptor table file status flagcurrent file offset
inode ptr
file status flagcurrent file offset
inode ptr
file tableinode table
28Unix System Programming
Kernel Data Structure for Open Files
user file descriptor table allocated per process identifies all open files for a process when a process “open” or “creat” a file, the kernel allocates
an entry return value of “open” and “creat” is the index into the user
file descriptor table contains pointer to file table entry
29Unix System Programming
Kernel Data Structure for Open Files
file table global kernel structure contains the description of all open files in the system
file status flag (open mode) current file offset
contains pointer to in-core inode table entry
30Unix System Programming
Kernel Data Structure for Open Files
in-core inode table global kernel structure when a process opens a file, the kernel converts the
filename into an identity pair(device number, inode number) the kernel then loads the corresponding inode into in-core
inode table
31Unix System Programming
Idea of Process
process an instance of a program in execution
at any given time a single instruction is carried out within the process
processes are often called “tasks” in Linux source code. process descriptor
task_struct contains all the information related to a single process
32Unix System Programming
Process Descriptor
state flagsprioritynext_taskprev_tasknext_runprev_runp_pptrp_optr...ttyfsfilesmmsig...
tty_struct
fs_struct
files_struct
mm_struct
signal_struct
tty associated with the process
current directory
pointers to file descriptors
pointers to memory area descriptors
signals received
task_struct
33Unix System Programming
Process States
process states state of a process is defined by its current activity executing
the process is being executed by the processor ready
the process could be executed, but another process is currently being executed
34Unix System Programming
Process States
process states (cont’d) suspended
the process is suspended (sleeping) until some condition becomes true.
hardware interrupt releasing a system resource the process is waiting for delivering a signals, etc.
35Unix System Programming
Process States
process states (cont’d) stopped
process execution has been stopped caused by signals
SIGSTOP, SIGTSTP, SIGTTIN, SIGTTOU zombie
the process has finished execution, but it is still referenced in the system
36Unix System Programming
Process States
stopped
suspended
ready executing zombie
creation
scheduling
signal signal
termination
input/outputend ofinput/output
37Unix System Programming
Attributes of a Process
attributes state identification (unique number) values of the registers, including the program counter user identity under whose name the process is executing information used by the kernel to establish the schedule of
the processes priority, execution time
38Unix System Programming
Attributes of a Process
attributes (cont’d) information concerning the address space of the process
segments for the code, data, stack information concerning the inputs/outputs carried out by the
process descriptions of open files, current directory, ...
information summarizing resources used by the process
39Unix System Programming
Login
/etc/passwd login name:password:UID:GID:comment:home directory:shell
Shell : command interpreter Bourne shell: /bin/sh C shell: /bin/csh Korn shell: /bin/ksh Tenex shell: /bin/tcsh
File and directory file system directory file name
40Unix System Programming
User Identity
user identifiers real user id
identifier of the user who started up the process effective user id
identifier which is used by the system for access control can be different from the real user, especially in the case of
programs with the setuid bit set
41Unix System Programming
User Identity
user identifiers (cont’d) saved user id
effective user id is saved when a process executes a new program
saved user id is used to authorize the modification of the effective user id
setuid( ), seteuid( ), setreuid( )
42Unix System Programming
User Identity
user identifiers (cont’d) file system user id
used to check for all accesses to the file system normally equal to the effective user ID whenever the effective user id is changed, file system user id
will also be changed to new value of effective user id effective user id is not changed when file system user id is
changed
43Unix System Programming
User Identity
group identifiers real group id
identifier of the primary group of the user who started the process
effective group id identifier which is used by the system for access control can be different from the real group, especially in the case of
programs with the setgid bit set
44Unix System Programming
User Identity
group identifiers (cont’d) a list of group identifiers
any user can belong to several groups simultaneously the kernel keeps a list of groups associated with each process
in order to carry out access control process can possess up to 32 groups
saved group id effective group id is saved when a process executes a new
program
45Unix System Programming
Shell
Shell Command line interpreter Reads and executes user command, and then shows the
result Bourne shell: /bin/sh C shell: /bin/csh Korn shell: /bin/ksh Tenex shell: /bin/tcsh
46Unix System Programming
File and Directory File System, File name, and Pathname Working Directory, Home Directory Ex : myls.c
#include <sys/types.h>#include <dirent.h>#include “ourhdr.h”
int main(int argc, char *argv[]){ DIR *dp; struct dirent *dirp;
if (argc != 2) err_quit(“\”the directory name\” is required”);
if ((dp == opendir(argv[1])) == NULL) err_sys(“ca’t open %d”, argv[1]); while ((dirp = readdir(dp)) != NULL) printf(“%s\n”, dirp->d_name); closedir(dp); exit(0);}
47Unix System Programming
How to build and run myls.c
# cc myls.c - or - #gcc myls.c# a.out /dev
# cc –o myls myls.c - or - # gcc –o myls myls.c# myls /dev
48Unix System Programming
Input and Output stdin, stdout, stderr Unbuffered I/O and
standard I/O(buffered I/O)
# ls -vs-# ls > file.list
#include “ourhdr.h”
#define BUFFSIZE 8192
int main(void){ int n; char buf[BUFFSIZE];
while ((n = read(STDIN_FILENO, buf, BUFFSIZE)) > 0) if (write(STDOUT_FILENO, buf, n) != n) err_sys(“write error”);
if (n < 0) err_sys(“read error”); exit(0);}
49Unix System Programming
Input and Output
Another example
#include “ourhdr.h”
int main(void){ int c; while ((c = getc(stdin)) != EOF) if (putc(c, stdout) == EOF) err_sys(“output error”); if (ferror(stdin)) err_sys(“input error”); exit(0);}
50Unix System Programming
Program and Process
Process ID
#include “ourhdr.h”
int main(void){ printf(“hello world from process ID %d\n”, getpid()); exit(0);}
# a.outhello world from process ID 851# a.outhello world from process ID 854
51Unix System Programming
Process control fork(), exec(), waitpid()
#include <sys/types.h>#include <sys/wait.h>#include “ourhdr.h”
int main(void){ char buf[MAXLINE]; pid_t pid; int status;
printf(“%% “);
while (fgets(buf, MAXLINE, stdin) != NULL) { buf[strlen(buf) -1] = 0; if ((pid = fork()) < 0) err_sys(“form error”); else if (pid == 0) { execlp(buf, buf, (char *) 0); err_ret(“couldn’t execute: %s”, buf); exit(127); } if ((pid = waitpid(pid, &status, 0)) < 0) err_sys(“waitpid error”); printf(“%% “); }}
52Unix System Programming
ANSI C
Functionsssize_t read(int, void *, size_t);ssize_t write(int, const void *, size_t);pid_t getpid(void);
Generic pointer void * : char * is used in early UNIX system.
Primitive System Data Types
53Unix System Programming
Error handling• Related Variable and Functions
extern int errno;
#include <string.h> char *strerror(int errnum);
#include <stdio.h> void perror(const char *msg);
#include <error.h>#include “ourhdr.h”
Int main(int argc, char *argv[]){ fprintf(stderr, “EACCES: %s\n”, strerror(EACCES)); errno = ENOENT; perror(argv[0]); exit(0);}
54Unix System Programming
User Identification
UserID, GroupID
#include “ourhdr.h”
Int main(void){ printf(“uid = %d, gid = %d\n”, getuid(), getgid()); exit(0);}
55Unix System Programming
Signal Signal Actions
Ignore Default User specified function
#include <sys/types.h>#include <sys/wait.h>#include “ourhdr.h”
int main(void){ char buf[MAXLINE]; pid_t pid; int status;
if ((signal(SIGINT, sig_int) == SIG_ERR) err_sys(“signal error”); printf(“%% “);
while (fgets(buf, MAXLINE, stdin) != NULL) { buf[strlen(buf) -1] = 0; if ((pid = fork()) < 0) err_sys(“form error”); else if (pid == 0) { execlp(buf, buf, (char *) 0); err_ret(“couldn’t execute: %s”, buf); exit(127); } if ((pid = waitpid(pid, &status, 0)) < 0) err_sys(“waitpid error”); printf(“%% “); }}void sig_int(int signo){ printf(“interrupt\n%% ”); }
56Unix System Programming
UNIX time
Calendar time Time elapsed since 1970/1/1 00h00m System data type : time_t Ex) file modification time, …
Process time (clock time, user CPU time, system CPU time) System data type : clock_t Typically 50Hz, 60Hz, 100Hz
$ cd /usr/include$ time grep _POSIX_SOURCE */*.h > /dev/null
real 0m19.81suser 0m0.43ssys 0m4.53s
57Unix System Programming
System call
Application
code
Memory allocation function malloc
sbrk
system call
KERNEL
USER PROCESS
58Unix System Programming
System call
Application
code
C library
functions
System calls
USER PROCESS
KERNEL
60Unix System Programming
ANSI C
ANSI Standard X3. 159-1989 ISO/IEC 9899:1990 Goal : Portable C program for various OS
61Unix System Programming
OS standards
IEEE POSIX POSIX :Portable Operating System Interface for Computer
Environments IEEE Standard 1003.1-1988 : OS Interface 1003.2 : Shell and utilities 1003.7 : System management
X/Open XPG3 X/Open : International Consortium of Computer Manufacturers X/Open Portability Guide, Issue 3 (XPG3)
FIPS (Federal Information Processing Standards)
62Unix System Programming
UNIX Implementations Developed for PDP-7 by Ben Tompson in Bell Lab AT&T UNIX System V (SVR 3, SVR 4)
Complies POSIX 1003.1 and XPG3 Provides ANSI C Compiler
Berkeley Software Distributions (BSD) 4.3 BSD Tahoe – BSD Networking Software, Release 1.0
Microsoft Xenix, System V SunOS
4.2 BSD + SVR 4 CMU Mach
Support multi-processor, multi-thread
68Unix System Programming
sysconf, pathconf, fpathconf
Synopsis #include <unistd.h> long sysconf(int name); long pathconf(const char *pathname, int name); long fpathconf(int filedes, int name);
Ex)#include <errno.h>#include “ourhdr.h”static void pr_sysconf(char *, int);static void pr_pathconf(char *, char *, int);
int main(int argc, char * argv[]){ if (argc != 2) err_quit(usage:a.out <dirname>”);
pr_sysconf(“CHILD_MAX = “, _SC_CHILD_MAX); pr_pathconf(“PATH_MAX = “, argv[1], _PC_PATH_MAX);}
static void pr_sysconf(char *mesg, int name){ fputs(msg, stdout); printf(“%ld\n”, sysconf(name));}
static void pr_pathconf(char *mesg, char *path, int name){ fputs(mesg, stdout); printf(“%ld\n”, pathconf(path, name));}
70Unix System Programming
Indeterminate runtime limits#include <errno.h>#include <limits.h>#include “ourhdr.h”
#ifdef PATH_MAXstatic int pathmax = PATH_MAX;#elsestatic int pathmax = 0;#endif
#define PATH_MAX_GUESS 1024
char *path_alloc(int *size){ char *ptr;
if (pathmax == 0) { if ((pathmax = pathconf(“/”, _PC_PATH_MAX)) < 0) { pathmax = PATH_MAX_GUESS; else pathmax++; } ptr = malloc(pathmax+1); *size = pathmax+1; return ptr; }
maxpath
71Unix System Programming
Indeterminate runtime limits(con’t)
max open fileEx1)
#include <sys/param.h>
for (i = 0; i < NOFILE, i++) close(i);
Ex2)#include <unistd.h>
#define OPEN_MAX_GUESS 256
openmax = sysconf(_SC_OPEN_MAX);
if (openmax < 0) openmax = OPEN_MAX_GUESS;
for (I = 0; I < openmax; i++) close(i);