COMP 2007: UNIX Operating System
Dr Eleni Mangina
http://www.cs.ucd.ie/staff/emangina
OPERATING SYSTEM
An operating system (OS) is a program that is loaded into the computer’s memory when the system is booted, and it always remains there. It interacts with two agencies:
• Applications (like word-processing software) that run under its auspices
• Users who pass commands to it using a command language interpreter. This too is a program; it translates commands to instructions that can be understood by the operating system.
Why Study Operating System?
• Argument against studying OS
Very few designers/implementers needed
All I need to know is in the manual pages
We are all just going to use Windows’?? anyway
Why Study Operating System?• Arguments in favour
OS is the most used program Need to know about system design in general OS includes all design/implementation problems
• design of efficient interfaces• flexibility, robustness and performance• trade-off between performance and simplicity
Mini-OS in many applications We are not all using Windows-??...
What is an OS?
• Is a program that
manages physical resources acts as an interface between the user and the hardware controls the execution of application programs provides an environment for the execution of programs provides mechanisms and policies to manage the whole
resources of the system controls programs and users interactions
Major Issues in OS• Structure -- how is an operating system organised?
• Sharing -- how are resources shared between users and programs?
• Naming -- how are resources named (by users or programs)?
• Protection -- how is a user/program protected from another?
• Security -- how to restrict the flow information?
• Performance -- why it is slow?
• Reliability & fault tolerance -- when something goes wrong
• Extensibility -- how to add new features?
Major Issues in OS (ctd)• Communication -- how and with whom can we communicate
(exchange information)?
• Concurrency -- how are parallel activities created & controlled?
• Scalability & Growth -- what will happened when demands and/or resources increase?
• Persistence -- how to make data last longer than programs?
• Compatibility -- can we ever do anything new?
• Distribution -- accessing the world of information?• Accounting -- who pays the bills, and how to control resource
usage?
OS Views• An operating system can be seen as:
– a virtual machine (extending the hardware functionality)
– resource manager (managing the hardware resources)
• Virtual Machine– Hide all the low-level details of programming the hardware
– Construct layers of software that provide more and more functionality
– Present a nice and simple view the computer use
OS as Resource Manager• A computer can be seen as a set of resources for
– movement, storage, and processing of data and– control of these functions
• The hardware resources can be– CPU (computation)– Main memory– Secondary memory (disks, tapes, etc)– Network– I/O devices (keyboard, mouse, printers, display, etc)– ...
OS Principles• OS as illusionist
– make hardware resources unlimited– provide illusion of dedicated machine with infinite
memory and infinite processors
• OS as government– protect users from each other– allocate resources fairly and efficiently
• OS as history teacher– learn from the past to predict the future
A brief history of OS• With the earliest computers, the OS was just “code” to which
you linked your program, loaded as a whole into the main memory and ran it– It is just a run-time library
• Simple batch systems were the first real OSs – the OS is stored in the main memory– a single job is loaded (from card reader) into the memory– the control cards of the job tell the OS what to do– run the job (compute, print the outputs, etc.)
• Spooling & buffering allow more than one job to be loaded into tape/disk or main memory
UNIX HISTORY
• 1965: Bell Telephone Laboratories & General Electric Company & Project MAC of MIT new OS Multics
• 1969: primitive version of Multics on a GE 645 & Bell end the participation
• Ken Thopson & Dennis Ritchie design of a file system early version of UNIX
• UNIX Brian Kernighan (Computing Science Research centre)• 1971: Real project: PDP-11, 16K bytes for the system, 8K for
user program, a disk of 512K bytes, limit 64K bytes per file• 1973: Operating system re-written in C 25 installations at Bell
Labs
UNIX HISTORY (cont.)
• AT & T could not market due to a 1956 Consent Decree BUT Universities could use it for educational purposes POPULARITY INCREASED
• 1974: Thompson and Ritchie publication of UNIX description
• 1977: UNIX 500 (125 were universities)• 1977: “ported” to a non-PDP machine• 1984: Developers enhanced UNIX their own way!
variants, 100.000 UNIX system installations!!
WHY UNIX?• The system is written in a high level language• Simpler user interface• Permit complex programs to be built from simpler
programs• Hierarchical file system• Consistent format of files• Simple, consistent interface to peripheral devices• It is a multi-user, multi-process system• It hides the machine architecture from the user
Versions of UNIX
• System V UNIX: AIX, Irix, Solaris, Tru64 Unix, Unicos, UnixWare
• BSD UNIX: BSD/OS, Dynix/ptx, FreeBSD, Mac OS X, NetBSD, OpenBSD, ULTRIX
• UNIX- like systems: Hurd, Linux, Minix, XINU
Multiprogramming• Idea: Expand memory to hold 2 or more programs and switch
among all of them. This process is known as Multitasking or Multiprogramming
• Multiprogramming Systems– keep multiple runnable jobs loaded in memory– overlap I/O operations of a job with computations of another– benefit from I/O devices that can operate asynchronously– require the use of interrupts and DMA– increase the processor utilisation and attempt to optimise
throughput• Central theme of modern operating systems
Timesharing• Idea: is to have multiple users simultaneously using terminals,
with the OS interleaving the execution of each user program in a short quantum of computation
• Timesharing Systems– based on time-slicing– each user feels like using the computer on his/her own– try to optimise response time– allow the users to view, edit, debug, and run their programs
interactively • MIT Multics system (mid-late 60s) was the first large
timesharing system
Real-Time OS
• Idea: OS must guarantee response to physical events in a fixed interval of time
• Real-Time OSs– are used for specialised applications: subway systems, flight
control, factories, nuclear power, etc.– their problem is to schedule all activities in order to meet all
their critical requirements– perform operations at fixed, predetermined times (time
intervals)– allow more user control
• With the use of video applications on modern PCs, all OSs will have real-time requirements
Distributed OS
• Idea: a common operating system shared by a network of independent computers
• Distributed OS– facilitates the use of geographically distributed resources– looks to its users like an ordinary centralised OS– supports communication between parts of a job or
different jobs – supports sharing of distributed resources (hardware and
software)– allows some kind of parallelism, but the speedup is not
an issue
Parallel OS• Idea: a system that can run and manage parallel applications
efficiently on parallel computers
• Parallel OS– support parallel applications composed of several time
consuming subtasks– needs basic primitives for dividing a task into multiple
parallel activities– needs scheduling primitives to map parallel activities into
processors– supports efficient communication between activities– supports synchronisation of activities to co-ordinate
sharing of data
OS Structure
• To understand any OS structure, let’s have a look first at its components and, then, how they are composed and organised
• Note that– it is never simple as it looks like
– the basic concepts exist in some forms in all OSs
– each OS implements them in a slightly different way
– the division between components is not clearly defined
UNIX System Structure
Center: operating system interacts directly with the hardware, providing common services to programs. System kernel: operating system; isolated from user programs. Programs such as shell and editors (ed, vi) interact with kernel with a set of system calls commands!
SCCS: Source Code Control System
Major Components of UNIX:
• Kernel: The master control program of the computer. It resides in the computer’s main memory and it manages the computer’s resources. It is the kernel that handles the switching necessary to provide multitasking
• File System: UNIX organises data into collections calles files. Files may be grouped together into collection called directory files or directories
• Shell: The part of UNIX that interprets user commands and passes them on to the kernel is called the shell. A typical shell provides a command-line interface, where the user can type commands.
• Utilities: A utility is a useful software tool that is included as a standard part of the UNIX operating system. Utilities are often called commands
User Perspective
File system:• Hierarchical structure• Consistent treatment of file data• Ability to create & delete files• Dynamic growth of files• Protection of the file data• Treatment of peripheral devices The file system is organised as a tree with a single root node called
root (written “/”) every non-leaf node of the file system structure is a directory of files and files are either directories, regular files or special device files.
Pathname: how to locate the file in the file system hierarchy Permission: Controlled by access permissions associated with the
file: read, write, execute
File Management• Secondary storage devices are too crude to be used directly
for long-term storage
• A file is – the basic long-term storage entity
– a collection of permanent information that can be read or written
• File system – provides logical objects
– provides logical operations on these objects
– supports directories -- special files that contain other files
File Management• Secondary storage devices are too crude to be used directly
for long-term storage
• A file is – the basic long-term storage entity
– a collection of permanent information that can be read or written
• File system – provides logical objects
– provides logical operations on these objects
– supports directories -- special files that contain other files
File Hierarchy
• File system provides standard file operations– create/delete files– create/delete directories– copy/move files/directories– other basic operations like: read, write, protect, extend,
…
• File system provide also general services– backup– maintaining mapping information– accounting and quotas
Process Management
• An OS executes many kinds of activities– user programs– batch jobs and command scripts– system programs; print spooler, name servers,
file servers, etc
• Each of these “execution entities” is encapsulated in a process
• The OS schedules processes to run
Processes
• A program is just a file on a disk with code that is potentially executable
• A process is one instance of a program in execution. At any time, there may be many processes running copies of a single program (e.g. an editor): each one is a separate and independent process
• A process includes the execution context (program counter, registers, virtual memory, resources, etc.) and all information the activity (program) needs to run
Process Operations• Processes are fundamental OS-provided objects
• The OS operations on processes include:– create a process
– delete a process
– suspend a process
– inter-process communication
– inter-process synchronisation
– create/delete a sub-process
Processing Environment
Program: executable fileProcess: instance of a program in executionexample:
main(argc,argv)int argc;char *argv[];
{/*assume 2 args: source file and target file */if (fork() == 0)
execl(“copy”,”copy”, argv[1],argv[2],0);wait((int *) 0);printf(“copy done\n”);
}
Processing Environment (cont.)
Shell: the command interpreter program that users typicall execute after logging into the system The shell interpretes the first word of a command line as a command name, treating the remaing words as parameters to the commands. It allows three trtype of commands:
• Executable file that contains object code produced by compilation of source code
• Command can be any executable file that contains a sequence of shell command lines
• Internal shell command
example:Who: causes the system to execute the programs stored in the file /bin/who, which prints a list of people who are currently logged into the systemWho &: in the background
Memory Management• The main memory is a direct-access storage device• Programs must be stored in memory to be executed• OS must
– allocate memory to processes (implicitly and explicitly)
– de-allocate it when they are done
– maintain (update) the mapping between virtual and physical memory (page tables)
– Keep track of which parts of memory are in use and which parts are not
– decide how much memory to allocate to each process and when a process should be removed from memory
I/O Management• An important part of the OS kernel is dedicated to I/O
• The OS provides a standard interface between processes (user’s or system’s) and devices
• Device driver – is a process responsible for a given device type
– encapsulate device-specific knowledge, e.g. for device initiation and control, interrupt handling and errors
– there may be a process for each device, or for each I/O request, depending on the OS
Secondary Storage Management• Secondary Storage
– permanent memory– endures system failures (we hope)– block access (Disk), sequential access (tape)
• OS must – provide low-level routines responsible for low-level disk
functions such as disk scheduling, head movement, error handling, etc
– manage the disk space
• The space management can be implemented as low-level routines or handled by the file system
Building Block Primitives
Redirect I/O: processes conventionally have access to three files:• Read from their standard input file (<)• Write to their standard output file (>)• Write error messages to their standard error file (2>)$ ls$ ls > output$ mail mjb < letter$ nroff –mm <doc1> doc1.out 2> errorsPipe: a mechanism that allows a stream of data to be passed
between reader and writer processes$ grep main a.c b.c c.c$ grep main a.c b.c c.c | wc -l
Operating System Services
• Controlling the execution of processes by allowing their creation, termination or suspension and communication
• Scheduling processes fairly for execution on the CPU• Allocating main memory for an executing process• Allocating secondary memory for efficient storage
and retrieval of user data• Allowing processes controlled access to peripheral
devices
System Protection• Protection is a mechanism for controlling the access of
processes or users to the resources of the system
• All resource objects need to be protected– CPU, memory– processes– files– devices, networking
• Protection mechanisms– help to detect errors and
– prevent malicious destruction
Assumptions about Hardware
User mode – Kernel mode
The differences between the two modes are:• Processes in user mode can access their own
instructions and data but not kernel instructions and data processes in kernel mode, however, can access kernel and user addresses
• Some machine instructions are priviliged and result in an error when executed in user mode
Lecture 2:Introduction to the KERNEL
UNIX system supports the illusion that:• File system has “places” &• Processes have “life”
Two central concepts in the UNIX system model!!
KERNEL• The kernel of UNIX is the hub of the operating system: It
allocates time and memory to programs and handles the filestore and communications in response to system calls.
• As an illustration of the way the shell and the kernel work together, suppose a user types rm myfile (which has the effect of removing the file myfile). The shell searches the filestore for the file containing the program rm, and then requests the kernel, through system calls, to execute the program rm on myfile. When the process rm myfile has finished running, the shell then returns the UNIX prompt to the user, indicating that it is waiting for further commands.
Block Diagram of Kernel
Hardware
Hardware controlKernel Level Hardware Level
libraries
System call interface
File subsystem
Buffer cache
Character | block
Device drivers
process
control
subsystem
Inter-process communication
scheduler
Memory management
User programs
User Level
Kernel Level
File subsystem
• The internal representation of a file is given by an “inode”• File table is a global kernel structure but the user file descriptor
table is allocated per process (when a process opens or creates a file, the kernel allocates an entry from each table, corresponding to the file’s inode. These three entries maintain the state of the file and the user’s access to it
• A file system consists of a sequence of logical blocks each containing 512, 1024, 2048 or any convenient multiple of 512 bytes, depending on the system implementation
“block”: logical stock contains 1K bytes of data
Overview of the File Subsystem
User File Descriptor
Table
File Table Inode Table
File System Layout
• Boot block: occupies the beginning of a file system, typically the first sector and may contain the bootstrap code that is read into the machine to boot, or initialise the operating system
• Super block: describes the state of a file system. How large it is, how many files it can store, where to find free space on the file system and other information
• The inode list is a list of inodes. One inode is the root inode of the file system: it is the inode by which the directory structure of the file system is accessible after execution of the system call
• The data blocks start at the end of the inode list and contain file data and administrative data. All allocated data blocks can belong to one & only one file in the file system.
Boot block Super block Inode list Data blocks
Process Process is the execution of a program and consists of
a pattern of bytes that the CPU interprets as machine instructions, data and stacks.
• A process executes by following a strict sequence of instructions that is self-contained and does not jump to that of another process;
• It reads and writes its data and stack sections, but it cannot read or write the dats and stack of other processes;
• Processes communicate with other processes and with the rest of the world via system calls.
Process (cont.)
• In practical terms, a process on a UNIX system is the entity that is created by the fork system call. Every process except process 0 is created when another process executes the fork system call. The process that invoked the fork system call is the parent process and th enewly created is the child process.
• Every process can be identified by the kernel by its process number, called the process ID (PID).
• Process 0 is a special process that is created “by hand” when the system boots.
Process (cont.)
A user compiles the source code of a program to create an executable file, which consists of several parts:
• A set of “headers” that describe the attributes of a file• The program text• A machine language representation of data that has
initial values when the program starts execution, and an indication of how much space the kernel should allocate for uninitialized data
• Other sections, such as symbol table information
Process (cont.)
The kernel loads and executable file into memory during an exec system call and the loaded process consists of at least three partscalled regions
• text, data correspond to sections of executable file and • the stack automatically created and its size is dynamically
adjusted by the kernel at run time
Stack:
a. Logical stack of frames that are pushed when calling a function and popped when returning
b. Stack pointer: indicates the current stack depth
c. Stack frame: parameters to a function, local variables, data necessary to recover the previous stack frame
User and Kernel Stack for copy program
Local not
Vars shown
Direction of stack growth
Addr of Frame 2
Ret addr after write call
Parmss to write
(new buffer count)
Frame 3
Call write() Frame 3
Local count
Vars
Local
Vars
Addr of Frame 1 Addr of Frame 1
Ret addr after copy call Ret addr after func2 call
Parms to copy (old new) Frame 2 Frame 2 Parms to kernel func2
Local fdold
Vars fdnew
call copy() call func2() Local
Vars
Addr of Frame 0 Addr of Frame 0
Ret addr after main call Ret addr after func1 call
Parms to main (argc argv) Frame 1 Frame 1 Parms to kernel func1
User Stack Kernel Stack
call main()
Frame 0
Start
call func1()
Frame 0
System call interface
Process (cont.)
• Every process has an entry in the kernel process table, and each process is allocated a u area that contains private data manipulated only by the kernel
• The process table contains a per process region table, whose entries point to entries in a region table
• A region is a contiguous area of a process’s address space, such as text, data and stack
• Region table entries describe the attributes of the region, such as whether it contains text or data, whether it is shared or private and where the data of the region is located in the memory
• The process table points to a per process region table with pointers to the region table entries for the text, data and stack regions of the process
Data structures for processes
The u-area contains information describing the process that needs to be accessible only when the process is executing:
• A pointer to the process table slot of the currently executing process
• Parameters of the current system call, return values or error codes
• File descriptors for all open files• Internal I/O parameters• Current directory and current root• Process and file size limits
u area
Main memoryProcess table
Per process region table
region table
Process states (cont.)
The lifetime of a process can be divided into a set of states, each with certain characteristics that describe the process:
• The process is currently executing in user mode• The process is currently executing in kernel mode• The process is not executing but it is ready to run as
soon as the schedule chooses it. Many processes may be in this state and the scheduling algorithm determines which one will execute next
• The process is sleeping. A process puts itself to sleep when it can no longer continue executing, such as when it is waiting for I/O to complete
State transitions
• Process move continuously between states according to well-defined rules.
• A state transition diagram is a directed graph whose nodes represent the states a process can enter and whose edges represent the events that cause a process to move from one state to another.
• State transitions are legal between two states if there exists an edge from the first state to the second.
• Several process can execute simultaneuously in a time-shared manner and they may all run simultaneuously in kernel mode.
Process states & Transitions
1
2
34
User running
Sys call or interrupt
return Interrupt, interrupt return
Schedule process
Ready to runWake up
sleep
Asleep
Context switch permissible