ceng334 introduction to operating systems 1 erol sahin dept of computer eng. middle east technical...

CENG334Introduction to Operating Systems

1

Erol Sahin

Dept of Computer Eng.Middle East Technical University

Ankara, TURKEY

URL: http://kovan.ceng.metu.edu.tr/~erol/Courses/CENG334

Operating System OverviewTopics•Brief History•OS Services•System calls•Basic Operation•OS structures

Some of the following slides are adapted from Matt Welsh, Harvard Univ.

2

In the Beginning... There was no OS – just libraries

Computer only ran one program at a time, so no need for an OS

Programming through wiring..

Harvard Mark I, 1944IBM 360, 1960's

ENIAC, 1945

3

In the Beginning... There was no OS – just libraries

Computer only ran one program at a time, so no need for an OS

And then there were batch systems Programs printed on stacks of punchhole cards OS was resident in a portion of machine memory When previous program was finished, OS loaded next program to run

4

Punch Card

5

In the Beginning... There was no OS – just libraries

Computer only ran one program at a time, so no need for an OS

And then there were batch systems Programs printed on stacks of punchhole cards OS was resident in a portion of machine memory When previous program was finished, OS loaded next program to run

6

In the Beginning... There was no OS – just libraries

Computer only ran one program at a time, so no need for an OS

And then there were batch systems Programs printed on stacks of punchhole cards OS was resident in a portion of machine memory When previous program was finished, OS loaded next program to run

Disk spooling Disks were much read stack onto disk while previous program is running With multiple programs on disk, need to decide which to run next! But, CPU still idle while program accesses a peripheral (e.g., tape or disk!)

7

MultiprogrammingTo increase system utilization, multiprogramming OS’s were

invented keeps multiple runnable jobs loaded in memory at once

Overlaps I/O of a job with computing of another While one job waits for I/O to compile, CPU runs instructions from another job

To benefit, need asynchronous I/O devices need some way to know when devices are done performing I/O

Goal: optimize system throughput perhaps at the cost of response time…

Dennis Ritchie and Ken Thompson at a PDP11, 1971

8

TimesharingTo support interactive use, timesharing OS's were created

multiple terminals connected to one machine

each user has illusion of entire machine to him/herself

optimize response time, perhaps at the cost of throughput

Timeslicing divide CPU fairly among the users

if job is truly interactive (e.g. editor), then can switch between programs and users faster than users can generate load

MIT Multics (mid-1960’s) was the first large timeshared system nearly all modern OS concepts can be traced back to Multics

9

Personal ComputingTotally changed the computing industry.

CP/M: First personal computer OS IBM needed OS for their PCs, CP/M behind schedule Bill Gates to the rescue: Bought 86-DOS and made MS-DOS

DOS is basically a subroutine library!

Many popular personal computers follow Apple, Commodore, TRS-80, TI 99/4, Atari, etc...

Bill Gates and Paul Allen, c.1975

Commodore VIC-20

IBM PC, 1981

Apple LISA, 1983

Apple I, 1976

10

11

Parallel Computing and ClustersHigh-end scientific apps want to use many CPUs at once

Parallel processing to crunch on enormous data sets

Need OS and language primitives for dividing program into parallel activities

Need OS primitives for fast communication between processors

degree of speedup dictated by communication/computation ratio

Many kinds of parallel machines: SMPs: symmetric multiprocessors – several CPUs accessing the same memory MPPs: massively parallel processors – each CPU may have its own memory Clusters: connect a lot of commodity machines with a fast network

12

Distributed OSGoal – Make use of geographically distributed resources

workstations on a LAN

servers across the Internet

Supports communication between applications interprocess communication (on a single machine):

message passing and shared memory

networking procotols (across multiple machines): TCP/IP, Java RMI, .NET SOAP

“The Grid”, .NET, and OGSA Idea: Seamlessly connect vast computational resources across the Internet

13

Embedded OSThe rise of tiny computers everywhere – ubiquitous computing

Processor cost low enough to embed in many devices PDAs, cell phones, pagers, ...

How many CPUs are in your car? On your body right now?

Gets more interesting with ubiquitous networking! Wireless networks becoming pervasive Sensor networks are an exciting new direction here

Little “motes” with less 4KB of RAM, some sensors, and a radio

Typically very constrained hardware resources slow processors

very small amount of memory (e.g. 8 MB)

no disk – but maybe quasi-permanent storage such as EEPROM

14

Operating System Overview

User application

User application

User application

KernelMemory management

Disk I/O

Process management

Device drivers

Filesystem TCP/IP stackAccounting

CPU support

Protection boundary

Hardware/software interface

15

Operating System Services(What things does the OS do?)

Services that (more-or-less) map onto components Program execution

How do you execute concurrent sequences of instructions? I/O operations

Standardized interfaces to extremely diverse devices File system manipulation

How do you read/write/preserve files? Looming concern: How do you even find files???

Communications Networking protocols/Interface with CyberSpace?

User interface- Almost all operating systems have a user interface (UI) Varies between Command-Line (CLI), Graphics User Interface (GUI), Batch

Cross-cutting capabilities Error detection & recovery Resource allocation Accounting Protection

16

User Operating System Interface - CLI

CLI allows direct command entry Sometimes implemented in kernel, sometimes by

systems programs Sometimes multiple flavors implemented – shells Primarily fetches a command from user and

executes it Sometimes commands built-in, sometimes just

names of programs If the latter, adding new features

doesn’t require shell modification

17

User Operating System Interface - GUI

User-friendly desktop metaphor interface• Usually mouse, keyboard, and monitor• Icons represent files, programs, actions, etc• Various mouse buttons over objects in the interface

cause various actions • provide information, options, • execute function, open directory (known as a folder)

• Invented at Xerox PARC

• Many systems now include both CLI and GUI interfaces

• Microsoft Windows is GUI with CLI “command” shell• Apple Mac OS X as “Aqua” GUI interface with UNIX

kernel underneath and shells available• Solaris is CLI with optional GUI interfaces (Java Desktop,

KDE)

18

Xerox PARC Alto

19

System Calls

Programming interface to the services provided by the OS

Typically written in a high-level language (C or C++)

Mostly accessed by programs via a high-level Application Program Interface (API) rather than direct system call use

Three most common APIs are • Win32 API for Windows, • POSIX API for POSIX-based systems (including

virtually all versions of UNIX, Linux, and Mac OS X), and

• Java API for the Java virtual machine (JVM)

• Why use APIs rather than system calls?

20

Example of Standard API

Consider the ReadFile() function in the

Win32 API—a function for reading from a file

A description of the parameters passed to ReadFile() HANDLE file—the file to be read LPVOID buffer—a buffer where the data will be read into and written from DWORD bytesToRead—the number of bytes to be read into the buffer LPDWORD bytesRead—the number of bytes read during the last read LPOVERLAPPED ovl—indicates if overlapped I/O is being used

21

System Call Implementation

Typically, a number associated with each system call

• System-call interface maintains a table indexed according to these numbers

• The system call interface invokes intended system call in OS kernel and returns status of the system call and any return values

• The caller need know nothing about how the system call is implemented

• Just needs to obey API and understand what OS will do as a result call

Most details of OS interface hidden from programmer by API Managed by run-time support library (set of functions built into libraries included with compiler)

22

API – System Call – OS Relationship

23

System Programs

System programs provide a convenient environment for program development and execution. They can be divided into:

File manipulation Status information File modification Programming language support Program loading and execution Communications Application programs

Most users’ view of the operation system is defined by system programs, not the actual system calls

24

System Programs

Provide a convenient environment for program development and execution

Some of them are simply user interfaces to system calls; others are considerably more complex

File management - Create, delete, copy, rename, print, dump, list, and generally manipulate files and directories

Status information Some ask the system for info - date, time, amount of available

memory, disk space, number of users Others provide detailed performance, logging, and debugging

information Typically, these programs format and print the output to the

terminal or other output devices Some systems implement a registry - used to store and

retrieve configuration information

25

System Programs (cont’d)

File modification Text editors to create and modify files Special commands to search contents of files or

perform transformations of the text

Programming-language support - Compilers, assemblers, debuggers and interpreters sometimes provided

Program loading and execution- Absolute loaders, relocatable loaders, linkage editors, and overlay-loaders, debugging systems for higher-level and machine language

Communications - Provide the mechanism for creating virtual connections among processes, users, and computer systems

Allow users to send messages to one another’s screens, browse web pages, send electronic-mail messages, log in remotely, transfer files from one machine to another

26

Memory

Operating System operationThe OS kernel is just a bunch of code that sits around in memory,

waiting to be executed

OS Kernel(device drivers,file systems,

virtual memory, etc.)

EmacsFirefox

xmms

sshd

27

Operating System operationThe OS kernel is just a bunch of code that sits around in memory,

waiting to be executed

OS is triggered in two ways: system calls and hardware interrupts

System call: Direct “call” from a user program For example, open() to open a file, or exec() to run a new program

Hardware interrupt: Trigger from some hardware device For example, when a disk block has been read or written

OS Kernel(device drivers,file systems,

virtual memory, etc.)

Memory

EmacsFirefox

xmms

sshd

System call(open network socket)

Interrupt (disk block read)

28

Interrupts – a primer

An interrupt is a signal that causes the CPU to jump to a pre-defined instruction – called the interrupt handler

• Interrupt can be caused by hardware or software

• Hardware interrupt examples• Timer interrupt (periodic “tick” from a programmable timer)• Device interrupts

• e.g., Disk will interrupt the CPU when an I/O operation has completed

• Software interrupt examples (also called exceptions)• Division by zero error• Access to a bad memory address• Intentional software interrupt – e.g., x86 “INT” instruction

• Can be used to trap from user program into the OS kernel!• Why might this be useful?

29

Interrupt handler example

Interrupt handler tableInterrupt handlerfor interrupt 4

Interrupt handlerfor interrupt 5

1) OS fills in interrupt handlertable (usually at boot time)

2) Interrupt occurs – e.g., hardwaresignal

!!!

3) CPU state saved to stack

30

Interrupt handlerfor interrupt 4Interrupt handlerfor interrupt 4

Interrupt handler example

Interrupt handler table

Interrupt handlerfor interrupt 5

1) OS fills in interrupt handlertable (usually at boot time)

2) Interrupt occurs – e.g., hardwaresignal

!!!

3) CPU state saved to stack

4) CPU consults interrupt tableand invokes appropriate handler

31

ProtectionA major job of the OS is to enforce protection

Prevent malicious or buggy programs from: Allocating too many resources (denial of service) Corrupting or overwriting shared resources (files, shared memory, etc.)

Prevent different users, groups, etc. from: Accessing or modifying private state (files, shared memory, etc.) Killing each other's processes

How does the OS enforce protection boundaries?

32

Enforcing Resource Limits

The OS limits what resources user programs can access For example, Emacs can't modify memory in use by Mozilla. xmms can't hog the CPU and prevent other programs from running. One user cannot read/write another user's files

(Unless permissions are set appropriately)

How does the OS enforce these limits? This implies that regular user programs cannot “break out” of these limits! We'll see how on the next slide.

A lot of viruses, worms, etc. exploit security holes in the OS Overrunning a memory buffer in the kernel can give a non-root process root privileges

Kernel code needs to be rock solid in order to be secure!!!

33

User mode vs. kernel modeWhat makes the kernel different from user programs?

Kernel can execute special privileged instructions

Examples of privileged instructions: Access I/O devices

Poll for IO, perform DMA, catch hardware interrupt Manipulate memory management

Set up page tables, load/flush the TLB and CPU caches, etc. Configure various “mode bits”

Interrupt priority level, software trap vectors, etc. Call halt instruction

Put CPU into low-power or idle state until next interrupt

These are enforced by the CPU hardware itself. CPU has at least two protection levels: Kernel mode and user mode CPU checks current protection level on each instruction What happens if user program tries to execute a privileged instruction?

34

Boundary Crossing

Kernel mode

User mode

Mozilla calls read() system call

Kernel trap handler

read() system call

Perform internal read()

Trap to kernel modeSave application registers and state

Lookup read() in system call tableInvoke internal read() function

Return to trap handler

Restore app registersReturn CPU to user mode

35

Web surfing homework for Wednesday!

Learn

• More about XEROX PARC • What else had they invented

• More about Ken Thomson and Dennis Ritchie

• What are they known for

• More about Microsoft• How did MS-DOS become so successful?

• More about Apple• What’s the relation between XEROX PARC GUI and

Apple GUI?

• Use Wikipedia, and google the web..

36

OS design and implementation

There is no ultimate OS that would satisfy all requirements: • Trade-offs have to made at each level and for all aspects.

Important principle to separate • Policy: What will be done? • Mechanism: How to do it?

• Mechanisms determine how to do something, policies decide what will be done

• The separation of policy from mechanism is a very important principle, it allows maximum flexibility if policy decisions are to be changed later

37

Operating Systems Structure(What is the organizational

Principle?)Simple

Only one or two levels of code

Layered Lower levels independent of upper levels

Microkernel OS built from many user-level processes

Modular Core kernel with Dynamically loadable modules

38

Simple Structure

MS-DOS – written to provide the most functionality in the least space

Not divided into modules Interfaces and levels of

functionality not well separated

39

Monolithic KernelsMost common OS kernel design (used in UNIX and Linux)

Kernel code is privileged and lives in its own address space User applications are unprivileged and live in their own separate address spaces All kernel functions loaded into memory as one large, messy program

Pros and cons???

User application

System call

User application

User application

KernelMemory management

Disk I/O

Process management

Device drivers

Filesystem TCP/IP stackAccounting

CPU support

Protection boundary

40

Monolithic KernelsMost common OS kernel design

Kernel code is privileged and lives in its own address space User applications are unprivileged and live in their own separate address spaces All kernel functions loaded into memory as one large, messy program

Pros and cons Pro: Overhead of module interactions within the kernel is low (function call) Pro: Kernel modules can directly share memory Con: Very complicated and difficult to organize Con: A bug in any part of the kernel can crash the whole system!

User application

System call

User application

User application

KernelMemory management

Disk I/O

Process management

Device drivers

Filesystem TCP/IP stackAccounting

CPU support

Protection boundary

41

Layered kernels

Operating system is divided many layers (levels)• Each built on top of lower layers• Bottom layer (layer 0) is hardware• Highest layer (layer N) is the user interface

• Each layer uses functions (operations) and services of only lower-level layers

• Advantage: modularity Easier debugging/Maintenance • Not always possible: Does process scheduler lie above or

below virtual memory layer?• Need to reschedule processor while waiting for paging• May need to page in information about tasks

• Important: Machine-dependent vs independent layers

• Easier migration between platforms• Easier evolution of hardware platform

42

MicrokernelsUse a very small, minimal kernel, and implement all other functionality

as user level “servers” Kernel only knows how to perform lowest-level hardware operations Device drivers, filesystems, virtual memory, etc. all implemented on top Use inter-process procedure call (IPC) to communicate between

applications and servers

Pros and Cons???

User application

User application

User application

Protection boundary

Microkernel

Memory management

Disk I/O

Process management

Filesystem TCP/IP stackAccounting

CPU supportDevice drivers

Inter-process procedure call

43

Microkernels - 2

Pros and cons• Pro: Kernel is small and simple, servers are protected from each other• Con: Overhead of invoking servers may be very high

• e.g., A user process accessing a file may require inter-process communication through 2 or 3 servers!

• Microkernels today• Lots of research in late 80's and early 90's• Windows NT uses “modified microkernel”:

• Monolithic kernel for most things, OS APIs (DOS, Win3.1, Win32, POSIX) implemented as user-level services

• Mac OS X has reincarnated the microkernel architecture as well:• Gnarly hybrid of Mach (microkernel) and FreeBSD (monolithic)

44

Modules-based Structure

Most modern operating systems implement modules Uses object-oriented approach Each core component is separate Each talks to the others over known interfaces Each is loadable as needed within the kernel

Overall, similar to layers but more flexible

45

Virtual Machines

A virtual machine takes the layered approach to its logical conclusion. It treats hardware and the operating system kernel as though they were all hardware

A virtual machine provides an interface identical to the underlying bare hardware

The operating system creates the illusion of multiple processes, each executing on its own processor with its own (virtual) memory

Virtualbox.org Vmware.comparallels.com

46

VMware Architecture

47

Virtualization

48

The Java Virtual Machine