tdc368 unix and network programming
DESCRIPTION
TDC368 UNIX and Network Programming. Week 1: Course Introduction Overview of UNIX Operating Systems Architecture UNIX process concepts. Camelia Zlatea, PhD Email: [email protected]. Overview of UNIX operating system architecture. - PowerPoint PPT PresentationTRANSCRIPT
TDC368UNIX and Network Programming
Camelia Zlatea, PhD
Email: [email protected]
Week 1:
Course Introduction
Overview of UNIX Operating Systems Architecture
UNIX process concepts
UNIX Network Programming – TDC368-901 Page 2Spring 2003
Course Outline
Overview of UNIX operating system architecture. UNIX process concepts. Process control primitives
(fork, exec, exit, wait, waitpid). Filters and I/O Redirection. Interprocess
Comunications (IPC): pipes, FIFO's. Asynchronous Events. Signals. Signal Functions
(kill, system, sleep, raise, alarm, pause, abort). UNIX system calls: C library functions, C++ classes,
POSIX API's. UNIX File System: files, catalogs (inodes; types,size,
permissions). UNIX File I/O: unbuffered I/O functions
(open ,close,seek,umask,read,write). Standard I/O buffering (fopen, fclose, fflush). File
Sharing (dup,dup2,fcntl).
UNIX Network Programming – TDC368-901 Page 3Spring 2003
Course Outline (cont.)
Interprocess Communication: Shared Memories, Semaphores, POSIX Semaphores, lock files.
Socket API (TCP, UDP). Design Methods for Client/Server Applications UNIX shells. C-shell commands, shell variables. C-Shell
Programming. Managing UNIX projects (make). Compilation and
Debugging. UNIX shells: I/O redirection, filters(grep, sed), awk.
UNIX Network Programming – TDC368-901 Page 4Spring 2003
Objectives
Upon completion of this course, students– will understand the fundamental concepts of UNIX operating
system and use them to create various multitasking applications.
– Will be familiar with UNIX programming environment, system call interface, ANSI C and ANSI/ISO C++ functions and classes
– Will be familiar with Client/Server methods and Socket APIs
– Will be proficient in UNIX shell programming
UNIX Network Programming – TDC368-901 Page 5Spring 2003
Prerequisites
CSC 309 Basic understanding of operating systems and
programming techniques C/C++ will be used to illustrate the concepts presented in
the lecture material and for the implementation of the programming assignments and the project.
UNIX Network Programming – TDC368-901 Page 6Spring 2003
Texts
Dave Curry, UNIX Systems Programming for SVR4, O'Reilly & Assoc., Sebastopol, CA, 1996.
W. Richard Stevens, UNIX Network Programming - Interprocess Communication, Volume 2, Second Edition, Prentice Hall PTR, Upper Saddle River, NJ, 1999.
W. Richard Stevens, Advanced Programming in the UNIX Environment, Addison-Wesley, Reading, MA, 1992.
John Shapley Gray, Interprocess Communications in Unix, The Nooks & Crannies, 2nd ed. Prentice Hall PTR, NJ, 1998.
HP-UX man pageshttp://docs.hp.com/hpux/onlinedocs/B2355-90682/B2355-90682.html (system calls)
UNIX Network Programming – TDC368-901 Page 7Spring 2003
Other References
Daniel Gilly, Unix in a Nutshell, O'Reilly and Associates, Inc., Sebastopol, CA, 1992.
K.A.Robbins and S. Robbins, Practical UNIX Programming - A Guide to Concurrency, Communication, and Multithreading , Prentice Hall PTR, NJ, 1996
Ellie Quingley, UNIX Shells by Example, Prentice Hall PTR, NJ, 1997
Prabhat K. Andleigh, UNIX System Architecture, Prentice-Hall, Englewood Cliffs, NJ, 1990.
Andrew Oran and Steve Talbot, Managing Projects with make, O'Reilly and Associates, Inc., Sebastopol, CA, 1991.
Brian W. Kernighan and Dennis M. Ritchie, The C Programming Language, Second Edition, Prentice-Hall, Englewood Cliffs, NJ, 1989.
UNIX Network Programming – TDC368-901 Page 8Spring 2003
Class Work and Grading
Programming Assignment 1 20%
due by April 22th
Programming Assignment 2 20%
due by May 6th
Midterm 20% May 13th
Programming Assignment 3 20%
due by May 27th
Final Project 20% due by June 10th
UNIX Network Programming – TDC368-901 Page 9Spring 2003
Programming Environment
All students will get an account on hawk.depaul.edu server (running HP-UX).
Optional, you can use Linux system
Project submissions via COL (dlweb.cti.depaul.edu )– use Submit Assignments link
Grading will be done on HP-UX or Linux– An experimental report is required for each programming project
» The the report should include the program design description and the test cases and test scenarios for the application.
UNIX Network Programming – TDC368-901 Page 10Spring 2003
TDC368 – Network Programming
Instructor: Camelia Zlatea, PhDOffice Hours: T 4:45-5:45PM; 9:00-9:30PM
Where: Rm. 429, ext. x26149 (during office hours)
or leave a message to setup a conference: +630-979-3621
Email: [email protected]
Web: http://condor.depaul.edu/~czlatea/TDC368/ (*)
Note:
* In addition, course materials will be available from dlweb.cti.depaul.edu
UNIX Network Programming – TDC368-901 Page 11Spring 2003
Class Communication
COL (dlweb.cti.depaul.edu) will be used primarily for:– Links to course materials/assignments
– Grades
– Announcements
– Email/Communication
– Newsgroup
Asking questions – Send me an email at [email protected] OR
– Use dlweb.cti.depaul.edu and post questions to class forum
• Anybody may reply (including your classmates)
• Everyone benefits from common issues
UNIX Network Programming – TDC368-901 Page 12Spring 2003
UNIX - Brief History
1969, Ken Thompson, Bell-Labs.
– UNIX OS written in Assembly on PDP-7 (DEC) 1973, K.Thompson&D.Richie
– UNIX re-written in C 1975, Bell-Labs. offered UNIX to educational
institutions, at minimal cost
UNIX Network Programming – TDC368-901 Page 13Spring 2003
UNIX Features
Portability Multi-process architecture (multitasking) Multi-user capability Ability to initiate asynchronous processes A hierarchical file system Device independent I/O operations User interface: Shell; selectable per user basis
UNIX Network Programming – TDC368-901 Page 14Spring 2003
UNIX Major Versions
AT&T UNIX version V
– 1983 /SV, 1986 /SVR3.0 AT&T and Sun Microsystems
– 1988 /SVR4.0 - tries to unify Berkeley UNIX & other Berkeley UNIX, CSRG at U. of California
– BSD (Berkeley Software Distribution)
UNIX Network Programming – TDC368-901 Page 15Spring 2003
UNIX Standards
System V Interface Definition (SVID), AT&T Portable Operating System Interface for Computer
Environments (POSIX), based on SVID, IEEE ANSI C, American National Standard Institute ANSI/ISO C++ Standard (draft)
UNIX Network Programming – TDC368-901 Page 16Spring 2003
UNIX Implementations
Solaris - Sun Microsystems
– SunOS (later called Solaris), Solaris 2.x based on SVR4
HP-UX, Hewlett-Packard, SVR2 Linux, Linus Torvalds, free distribution, PC-based AIX, IBM, similar to SVR4 IRIX, Silicon Graphics, SVR4
UNIX Network Programming – TDC368-901 Page 17Spring 2003
UNIX Implementations
UNIX-like– same facilities as SV, does not use entirely the same
code– Mach (CMU), kernel is based on OO model server– Next (implementation inspired by Mach with MacOS
UI) UNIX-alike
– portability is no longer supported – attributes not compliant with SV or BSD– OS/9(MicroWare, Motorola 680xx, RT kernel)– QNX(Quantum Software, Intel, RT processing)
UNIX Network Programming – TDC368-901 Page 18Spring 2003
UNIX Architectural Overview
Hardware Architecture
Kernel UNIX
System Calls Interface
Shell’s Executable Programs
pipe, filters Commands
Applications Network ApplicationsDBMS Utilities
UNIX Network Programming – TDC368-901 Page 19Spring 2003
UNIX Features
Portability Multi-process architecture (multitasking) Multi-user capability Ability to initiate asynchronous processes A hierarchical file system Device independent I/O operations User interface: Shell; selectable per user basis
UNIX Network Programming – TDC368-901 Page 20Spring 2003
Multi-process/Multi-user architecture
Virtual Machine– timesharing OS
– process, process quantum, process states
Kernel, base OS– manages all HW dependent functions
– users have no direct access to it
System Calls Interface– service routine performing user requests
UNIX Network Programming – TDC368-901 Page 21Spring 2003
UNIX Processes
A process may create sub-processes– fork();
A process may terminate– exit();
A process may put itself to sleep temporarily– sleep(20);
– pause();
– wait();
Processes– synchronization mechanisms
– communication mechanisms
UNIX Network Programming – TDC368-901 Page 22Spring 2003
UNIX Threads
Multiple Processes - concurrency at OS level Multiple Threads - concurrency at process level
– thread = flow of control in a process
– multiple threads (stream of instructions) are executed within the same process
– threads share code & data (address space)
– threads have their own execution stack, PC, register set and states
– context switches are avoided
– efficient mapping on multi-processor machines
UNIX Network Programming – TDC368-901 Page 23Spring 2003
Some Unix Terms
TRAP INSTRUCTION– switches a system call from user to kernel mode
– handles error conditions
– synchronous execution
– Ex: division by zero/overflow
INTERRUPT– used by peripherals to request services from the devices handles
– asynchronous execution
SIGNALS software notification of an event example:
– when typing “ctrl-c” an interrupt is generated for the keyboard driver, which notifies the process by sending a signal
UNIX Network Programming – TDC368-901 Page 24Spring 2003
UNIX Kernel - model
Hardware
System Call Interface (Library Routines)
Process Memory File System I/O Services Mgmt. Mgmt.
Scheduler Device Drivers I/O Buffers
User Processes
Kernel
Space
User
Space
UNIX Network Programming – TDC368-901 Page 25Spring 2003
Apps./Utilities
System Calls
Hardware InterruptsUNIX KERNEL
Hardware
UNIX Network Programming – TDC368-901 Page 26Spring 2003
UNIX Kernel - low level
Process representation, scheduling, dispatching Memory allocation and de-allocation Interrupt handling Low level device control Disk Mgmt., data buffering Process synchronization and IPC
UNIX Network Programming – TDC368-901 Page 27Spring 2003
Kernel - services level
Maps user-level requests with device driver actions A user system call is translated to a call of the kernel routine,
providing that requested service Type of Services:
– process creation and termination
– I/O services
– UNIX file system services
– terminal handling services
UNIX Network Programming – TDC368-901 Page 28Spring 2003
Kernel - services level
User program:
system(“ps -e”);
System calls
actions:
fork();
//process creation service
execl(“/bin/ps”, “ps -e”);
//code execution
Executable Programs
Library SystemFunctions Calls
Kernel
Hardware
UNIX Network Programming – TDC368-901 Page 29Spring 2003
System Call Interface level
A user mode process is translated into a protected kernel mode process
Now, program can call kernel routines
UNIX Network Programming – TDC368-901 Page 30Spring 2003
User Processes level
User processes running:– shells
– Unix commands
– utilities
– application programs
UNIX Network Programming – TDC368-901 Page 31Spring 2003
UNIX and POSIX API Library Functions ANSI C Standard ANSI/ISO C++ Standard
UNIX Standards
UNIX Network Programming – TDC368-901 Page 32Spring 2003
UNIX and POSIX API
UNIX API - system calls UNIX API are called by
– C library functions and
– C++ standard classes
Example: iostream class
Program confirms POSIX.1?#define _POSIX_SOURCE
or
% CC -D_POSIX_SOURCE *.C
UNIX Network Programming – TDC368-901 Page 33Spring 2003
UNIX and POSIX API
API set to perform:– determine system configuration and user information
– file management
– process creation and management
– inter-process communication
– network communication
UNIX Network Programming – TDC368-901 Page 34Spring 2003
UNIX and POSIX API
User Process (User Mode of Execution)
Kernel mode of execution
UNIX API’s level
an API is invoked
API executed
in protected mode
API execution
completed
UNIX Network Programming – TDC368-901 Page 35Spring 2003
UNIX and POSIX API
Context Switch from user to kernel mode – more overhead than library functions, for the same task
– I/O lib.functions are buffered
UNIX Network Programming – TDC368-901 Page 36Spring 2003
UNIX and POSIX API
UNIX API - system calls UNIX API are called by
– C library functions and
– C++ standard classes
Example: iostream class
Program confirms POSIX.1?#define _POSIX_SOURCE
or
% CC -D_POSIX_SOURCE *.C
UNIX Network Programming – TDC368-901 Page 37Spring 2003
UNIX and POSIX API
Development elements: Headers
– <unistd.h>
– <sys> (/usr/include/sys directory)
– <stdio.h>
function perror - system diagnostic message for an API execution failure
Object Code – libc.a and libc.so (on most systems)
– other special libraries, libsocket.a and libsocket.so
UNIX Network Programming – TDC368-901 Page 38Spring 2003
UNIX and POSIX API
#include <stdio.h>
void perror(const char *s);
#include <errno.h>
int errno;
#include <string.h>
char *strerror(int errnum);
UNIX Network Programming – TDC368-901 Page 39Spring 2003
Library Functions
Library of functions
– object code of the functions stored in archives (library files)
– run-time library routines UNIX archive ar utility manage library files.
Displaying the table of contents of the standard library:
ar -t /usr/lib/libc.a
UNIX Network Programming – TDC368-901 Page 40Spring 2003
ANSI C Standard
Standard C library functions:
– memory management
– get system information
– get/set time information
– data conversion, processing Advantages:
– portability
– maintainability
– cost effective for application design Disadvantages:
– duration of execution
– weak support for time-critical apps.
UNIX Network Programming – TDC368-901 Page 41Spring 2003
Standard C library functions
Header files directory: /usr/include Header files:
- <stdio.h> - <stdlib.h>
- <string.h>
- <memory.h> - <malloc.h>
- <time.h>
- <assert.h> - <stdarg.h>
- <getopt.h> - <setjmp.h>
Object code directory: /usr/lib– libc.a , libc.so
UNIX Network Programming – TDC368-901 Page 42Spring 2003
Standard C library functions
<stdio.h> - declares FILE data type functions/macros -manage stream files
open/close stream files: fopen/fclose
read/write block data: fread/fwrite
read/write lines: fgets/fputs
write formatted data: fprintf
positions/returns stream file location:fseek/ftell
macros: feof, ferror
unnamed pipes: popen/pclose
UNIX Network Programming – TDC368-901 Page 43Spring 2003
Example: ls -l|grep tdc368
#include <stdio.h>int main(){ /* execute the command */
FILE *cmd = popen(“ls -l|grep tdc368”,”r”); if (!cmd) {
perror(“popen”); return 1; }
char answer[256];/* read grep process execution output */while(fgets(answer, sizeof(answer),cmd))
fputs(answer, stdout); /* echo */pclose(cmd);return 0;
}
UNIX Network Programming – TDC368-901 Page 44Spring 2003
Standard C library functions
<stdlib.h> - declares set of functions for data conversions, get/set env. variables, shell command execution, process termination– atoi, atof, atol, strtod, strtol
– rand, srand
– getenv, putenv
– system
– exit
UNIX Network Programming – TDC368-901 Page 45Spring 2003
Example: ls -l|grep tdc368
#include <stdlib.h>
int main()
{ /* execute the command */
if (system(“ls -l|grep tdc368”)== -1) {
perror(“system”); return 1;
}
return 0;
}
UNIX Network Programming – TDC368-901 Page 46Spring 2003
Example: getenv/putenv
#include <iostream.h>
#include <stdlib.h>
int main()
{
char *env = getenv(“REMOTEHOST”);
cout << “\REMOTEHOST\” is:”<<env<<‘\n’;
if (putenv(“CC=c++”))
cer<<“putenv of CC failed\n”);
return 0;
}
UNIX Network Programming – TDC368-901 Page 47Spring 2003
Example: <malloc.h>
//C style
char* adr1 = (char *)malloc(4096);
free(adr1);
//C++ style , no use of malloc !
char* adr2 = new char[4096];
delete adr2;
UNIX Network Programming – TDC368-901 Page 48Spring 2003
Standard C library functions
<time.h> - declares set of functions for system clock query
time - #sec since 01/01/70 (UNIX birth)
ctime - current local time
localtime
gmtime
asctime
mktime
clock - (ANSI C) , #microsec since the process first called clock
UNIX Network Programming – TDC368-901 Page 49Spring 2003
Example: <time.h>
// evaluate process execution time
#include <iostream.h>
#include <time.h>
main() {
time_t tick = CLOCKS_PER_SECOND;
clock_t start_time = clock(); // start timer
/* process code here */
clock_t duration = clock() - start_time;
cout <<“Duration:”<<(duration/tick)<<endl;
}
UNIX Network Programming – TDC368-901 Page 50Spring 2003
Standard C library functions
<assert.h> - declares a macro to assert conditions or invariants of the program– easier program debugging
– compiler option -DNDEBUG take them out
or
#ifndef NDEBUG
#define assert(p) {if (!(p)) {\
fprintf(stderr,”Assertion failed: \
file:\”%s\”,line %d\n”,\
_FILE_, _LINE_); exit(1); }
#endif
UNIX Network Programming – TDC368-901 Page 51Spring 2003
Example: <assert.h>
#include <fstream.h>
#include <string.h>
#include <assert.h>
int main(int argc, char * argv[])
{
assert(argc >1); // should have 1 arg
ifstream ifs(argv[1])
assert(ifs.good()); // should be opened OK
char *adr = new char[strlen(argv[1])+1];
assert(adr); // should not be NULL
}
UNIX Network Programming – TDC368-901 Page 52Spring 2003
ANSI/ISO C++ Standard
1980, Bjarne Stroustrup, C++ 1994, ANSI/ISO C++ standard (draft) C++ compiler (AT&T C++ V3.0) support:
– C++ classes, derived classes
– virtual functions, operator overloading
– templates classes, functions
– exception handling
– iostream classes
UNIX Network Programming – TDC368-901 Page 53Spring 2003
Compilation
cc -o file file.c
file
Man Pages
man cc
man sys_call
man shell_cmd
UNIX Network Programming – TDC368-901 Page 54Spring 2003
Processes
Process - a program in execution– process - active entity
– program - passive entity (binary file)
Address Space - list of memory locations from where a process reads/writes (code/data/stack)
Set of registers (PC, SP, ...) Process Table - linked list of structures associates w/ processes System Calls - interface between OS and User process
UNIX Network Programming – TDC368-901 Page 55Spring 2003
Process Control Block (process attributes)
Process State – new, ready, running, blocked, terminated
Process Image Map Pointer Process ID
– assigned at process creation
Program Counter (PC) – address of next instruction to be executed
CPU Registers (saved process context) List of Open File Descriptors I/O Devices Attached CPU Scheduling Info (priority)
UNIX Network Programming – TDC368-901 Page 56Spring 2003
Process Image Map
Proc. n
Proc. 1Process Image
Process Table
Text/Code Segment
Data Segment
Stack Segment
Process Control Block
UNIX Network Programming – TDC368-901 Page 57Spring 2003
Process Memory Map
High Address
System Memory Map
process 0process 1.....
process i....
program text segment
initialized static data
uninitialized static data
heap
stack
argc argv; environment
System Process Info.System Stack
User addressablearea
Systemaddressablearea
Low Address
UNIX Data Structures
u area
Buffers
Kernel code
Low level device drivers
UNIX Network Programming – TDC368-901 Page 58Spring 2003
Example:
/* Display Segment Address Information
*/
#include <stdio.h>
extern int etext,edata,end;
void main(void) {
printf(“etext: %6X\t edata: %6X \t end: %6X \n”, &etext, &edata, &end);
}
UNIX Network Programming – TDC368-901 Page 59Spring 2003
Process States
New - process created ( Ex: fork(); ) Ready - process is waiting to be
assigned to processor (inserted in ready queue) Running - instructions are being executed Blocked - wait for events to occur (inserted
in queue) Ex: wait(); pause(); Terminated - normal/abnormal termination
(exit();)
UNIX Network Programming – TDC368-901 Page 60Spring 2003
Process Model
New
Ready
Blocked/Suspended
RunningUser Mode
Terminated
exitsleep
wakeup
dispatch
created
QuantumExpired
RunningKernel Mode
System Call
InterruptReturn
InterruptInterrupt return
UNIX Network Programming – TDC368-901 Page 61Spring 2003
Context of a Process– process state (defined by it’s code)
– value of u-area
– values of registers the process uses
– contents of user and kernel stacks
– is associated with process image map
Context Switching– system executes a process in the context of the process
– when the kernel decides to execute another process, it does context switching
– kernel saves enough information such that it can later switch back to the first process and resumes its execution
Mode Switching– moving from user to kernel mode
– kernel save information to return to user mode
UNIX Network Programming – TDC368-901 Page 62Spring 2003
User mode– processes in use mode can access their own instructions and data;
NOT kernel or other process’s code or data
Kernel mode– process can access system(kernel) code and data and user addresses
– Kernel is part of each process
– Kernel executes on behalf of the process
P1 P2 P3 P4
Kernel Mode
User Mode
K K
U U
OSHW
UNIX Network Programming – TDC368-901 Page 63Spring 2003
Context Switching
P1 P2
Save state in PCB1
Reload state from PCB2
Save state in PCB2
Reload state from PCB1
OS
UNIX Network Programming – TDC368-901 Page 64Spring 2003
Switching the CPU to another process by saving the state of the old process (PCB) and load the state of the new process (PCB)
Pure Overhead Performance Bottleneck Avoid Overhead of Context Switching by introducing new
structures:
THREADS
Context Switching
UNIX Network Programming – TDC368-901 Page 65Spring 2003
Multitasking
Sequential Execution
Context Switching