advanced operating systems university of tehran dept. of ee and computer engineering by: dr. nasser...

Advanced Operating Systems

Advanced Advanced

Operating SystemsOperating Systems

University of TehranDept. of EE and Computer Engineering

By:Dr. Nasser Yazdani

Lecture 3: OS designOS design

Univ. of Tehran 1


How to design an OSHow to design an OS Some general guides and experiences. References

“Exokernel: An Operating System Architecture for Application Level Resource Management”, Dawson R., Engler M, Frans Kaashoek, et al.

“On Micro-Kernel Constructions“,

Univ. of Tehran 2


OutlineOutline New applications/requirements Organizing operating systems Some microkernel examples Object-oriented organizations

Spring Organization for multiprocessors

Univ. of Tehran 3


New visionNew vision Two important problems: location and

scale. Ubiquitous computing: tiny kernels of

functionality Virtual Reality Mobility Intelligent devices distributed computing" make networks

appear like disks, memory, or other nonnetworked devices.

Univ. of Tehran 4


What is the big deal?What is the big deal? Performance Border crossings are expensive

Change in locality Copying between user and kernel

buffers Application requirements differ in terms

of resource management

Univ. of Tehran 5


Operating System Operating System OrganizationOrganization

What is the best way to design an operating system?

Put another way, what are the important software characteristics of an OS?

What should be in OS kernel or application or partitioning. Is there a minimal set for kernel?

Univ. of Tehran 6


Important OS Software Important OS Software CharacteristicsCharacteristics

Correctness and simplicity Power and completeness Performance Extensibility and portability

Flexibility Scalability

Suitability for distributed and parallel systems

Compatibility with existing systems Security and fault tolerance

Univ. of Tehran 7

Common OS Common OS OrganizationsOrganizations

Monolithic Virtual machine Structured design

Layered designs Object-Oriented

Microkernels Trade off between generality and

specialization

Advanced Operating SystemsUniv. of Tehran 8

Monolithic OS DesignMonolithic OS Design Build OS as single combined module

Hopefully using data abstraction. OS lives in its own, single address

space Examples

DOS early Unix systems most VFS file systems


Pros/Cons of Monolithic Pros/Cons of Monolithic OS OrganizationOS Organization

+ Highly adaptable (at first . . .)+ Little planning required+ Potentially good performance– Hard to extend and change– Eventually becomes extremely

complex– Eventually performance becomes

poor– Highly prone to bugs


Virtual Machine Virtual Machine OrganizationsOrganizations

A base operating system provides services in a very generic way

One or more other operating systems live on top of the base system Using the services it provides To offer different views of system to users

Examples - IBM’s VM/370, the Java interpreter


Pros/Cons of Virtual Pros/Cons of Virtual Machine OrganizationsMachine Organizations

+ Allows multiple OS personalities on a single machine

+ Good OS development environment+ Can provide good portability of

applications– Significant performance problems– Especially if more than 2 layers– Lacking in flexibility



Old ideaOld idea VM 370

Virtualization for binary support for legacy apps

Why resurgence today? Companies want a share of everybody’s pie

IBM zSeries “mainframes” support virtualization for server consolidation

Enables billing and performance isolation while hosting several customers

Microsoft has announced virtualization plans to allow easy upgrades and hosting Linux!

Univ. of Tehran 13

Layered OS DesignLayered OS Design

Design tiny innermost layer of software Next layer out provides more functionality

Using services provided by inner layer Continue adding layers until all

functionality required has been provided Examples

Multics Fluke layered file systems and comm. protocols


Pros/Cons of Layered Pros/Cons of Layered OrganizationOrganization

+ More structured and extensible+ Easy model and development– Performance: Layer crossing can be

expensive– In some cases, unnecessary layers,

duplicated functionality.


Two layer OS DesignsTwo layer OS Designs Only two OS layers

Kernel OS services Non-kernel OS services

Move certain functionality outside kernel file systems, libraries

Unlike virtual machines, kernel doesn’t stand alone

Examples - Most modern Unix systems


Pros/Cons of two layer Pros/Cons of two layer OSOS

+ Many advantages of layering, without disadvantage of too many layers

+ Easier to demonstrate correctness– Not as general as layering– Offers no organizing principle for

other parts of OS, user services– Kernels tend to grow to monoliths


Object-Oriented OS Object-Oriented OS DesignDesign

Design internals of OS as set of privileged objects, using OO methods

Sometimes extended into application space

Tends to lead to client/server style of computing

Examples Mach (internally) Spring (totally)



Object-Oriented Object-Oriented OrganizationsOrganizations

Object-oriented organization is increasingly popular

Well suited to OS development, in some ways OSes manage important data

structures OSes are modularizable Strong interfaces are good in OSes

Univ. of Tehran 19


Object-Orientation and Object-Orientation and ExtensibilityExtensibility

One of the main advantages of object-oriented programming is extensibility

Operating systems increasingly need extensibility

So, again, object-oriented techniques are a good match for operating system design

Univ. of Tehran 20


How object-oriented How object-oriented should an OS be?should an OS be?

Many OSes have been built with object-oriented techniques E.g., Mach and Windows NT

But most of them leave object orientation at the microkernel boundary No attempt to force object orientation

on out-of-kernel modules

Univ. of Tehran 21

Pros/Cons of Object Pros/Cons of Object Oriented OS Oriented OS OrganizationOrganization+ Offers organizational model for entire system

+ Easily divides system into pieces+ Good hooks for security– Can be a limiting model– Must watch for performance

problemsNot widely used yet


Microkernel OS DesignMicrokernel OS Design Like kernels, only less number of abstractions

exported (threads, address space, communication channel)

Try to include only small set of required services in the microkernel

Moves even more out of innermost OS part Like parts of VM, IPC, paging, etc.

System services (e.g. VM manager) implemented as servers on top

High comm overhead between services implemented at user level and microkernel limits extensibility in practice

Examples - Mach, Amoeba, Plan 9, Windows NT, Chorus, Spring, etc.


Pros/Cons of Pros/Cons of Microkernel Microkernel OrganizationOrganization+ Those of kernels, plus:

+ Minimizes code for most important OS services

+ Offers model for entire system– Microkernels tend to grow into

kernels– Requires very careful initial design

choices– Serious danger of bad performance



Organizing the Total Organizing the Total SystemSystem

In microkernel organizations, much of the OS is outside the microkernel

But that doesn’t answer the question of how the system as a whole gets organized

How do you fit together the components to build an integrated system? While maintaining all the advantages of the microkernel

Univ. of Tehran 25

Micro-ness is in the eye of the beholder Spin X-kernel Exokernel Mach Spring Amoeba Plan 9 Windows NT

Some Important Some Important Microkernel DesignsMicrokernel Designs


MachMach Mach didn’t start life as a

microkernel Became one in Mach 3.0

Object-oriented internally Doesn’t force OO at higher levels

Microkernel focus is on communications facilities

Much concern with parallel/distributed systems


Mach ModelMach Model

Kernelspace

UserspaceSoftware

emulationlayer

4.3BSDemul.

SysVemul.

HP/UXemul.

otheremul.

Userprocesses

Microkernel


What’s In the Mach What’s In the Mach Microkernel?Microkernel?

Tasks & Threads Ports and Port Sets Messages Memory Objects Device Support Multiprocessor/Distributed Support


Mach Task ModelMach Task Model

Processport

Bootstrapport

Exceptionport

Registeredports

Addressspace

Thread

Process

Use

r sp

ace

Ker

nel


Mach PortsMach Ports Basic Mach object reference mechanism

Kernel-protected communication channel Tasks communicate by sending

messages to ports Threads in receiving tasks pull messages

off a queue Ports are location independent Port queues protected by kernel;

bounded



Port RightsPort Rights mechanism by which tasks control

who may talk to their ports Kernel prevents messages being set

to a port unless the sender has its port rights

Port rights also control which single task receives on a port

Univ. of Tehran 32


Port SetsPort Sets A group of ports sharing a common

message queue A thread can receive messages from

a port set Thus servicing multiple ports

Messages are tagged with the actual port

A port can be a member of at most one port set

Univ. of Tehran 33

Mach MessagesMach Messages Typed collection of data objects

Unlimited size Sent to particular port May contain actual data or pointer to

data Port rights may be passed in a

message Kernel inspects messages for

particular data types (like port rights)


Mach Memory ObjectsMach Memory Objects A source of memory accessible by

tasks May be managed by user-mode

external memory manager a file managed by a file server

Accessed by messages through a port Kernel manages physical memory as

cache of contents of memory objects


Mach Device SupportMach Device Support Devices represented by ports Messages control the device and its

data transfer Actual device driver outside the

kernel in an external object


Mach Multiprocessor Mach Multiprocessor and DS Supportand DS Support

Messages and ports can extend across processor/machine boundaries Location transparent entities

Kernel manages distributed hardware Per-processor data structures, but also

structures shared across the processors Intermachine messages handled by a

server that knows about network details



Mach’s NetMsgServerMach’s NetMsgServer User-level capability-based

networking daemon Handles naming and transport for

messages Provides world-wide name service

for ports Messages sent to off-node ports go

through this server

Univ. of Tehran 38


NetMsgServer in ActionNetMsgServer in Action

User space

Kernel space

Sender

User process

NetMsgServer

User space

Kernel space

Receiver

User process

NetMsgServer

Univ. of Tehran 39

Mach and User Mach and User InterfacesInterfaces

Mach was built for the UNIX community UNIX programs don’t know about ports,

messages, threads, and tasks How do UNIX programs run under Mach? Mach typically runs a user-level server

that offers UNIX emulation Either provides UNIX system call

semantics internally or translates it to Mach primitives


Windows NTWindows NT More layered than some microkernel

designs NT Microkernel provides base services Executive builds on base services via

modules to provide user-level services User-level services used by

privileged subsystems (parts of OS) true user programs


Windows NT DiagramWindows NT Diagram

Hardware

MicrokernelExecutive

UserProcesses

ProtectedSubsystems

User Mode

Kernel Mode

Win32 POSIX


NT MicrokernelNT Microkernel Thread scheduling Process switching Exception and interrupt handling Multiprocessor synchronization Only NT part not preemptible or

pageable All other NT components runs in

threads


NT ExecutiveNT Executive Higher level services than

microkernel Runs in kernel mode

but separate from the microkernel itself ease of change and expansion

Built of independent modules all preemptible and pageable


NT Executive ModulesNT Executive Modules Object manager Security reference monitor Process manager Local procedure call facility (a la

RPC) Virtual memory manager I/O manager


Typical Activity in NTTypical Activity in NT

Hardware

KernelExecutive

Client Process

Win32ProtectedSubsystem


More On MicrokernelsMore On Microkernels Microkernels were the research

architecture of the 80s But few commercial systems of the

90s really use microkernels To some extent, “microkernel” is

now a dirty word in OS design Why?


Main IssueMain Issue What should be in the Kernel?

Different designs give different answers.

How to implement the system efficiently? Some people think Micro kernel is slow Micro kernel construction paper argue

other way.



ExokernelExokernel

FIXED

Hardware

Apache

InterfaceAbstractions

SQL Server

Traditional OS

Traditional operating systems fix the interface and implementation of OS abstractions.

Abstractions must be overly general to work with diverse application needs.

Univ. of Tehran 49


The Issues The Issues Performance

Denies applications the advantages of domain-specific optimizations

Flexibility Restricts the flexibility of application builders Difficult or impossible to implement own

resource management abstractions. Functionality

Discourages changes to the implementations of existing abstractions since it is used by different applications

Univ. of Tehran 50


Performance Performance Example: A DB can have predictable data

access patterns, that doesn't fit with OS LRU page replacement, causing bad performance.

Cao et al. Found that application-controlled file caching can reduce running time by as much as 45%.

There is no single way to abstract physical resources or to implement an abstraction that is best for all applications.

OS is forced to make trade-offs Performance improvements of application-

specific policies could be substantial

Univ. of Tehran 51


The SolutionThe Solution Separate protection from management

Allow user level to manage resources Application libraries implement OS abstractions

Exokernel exports resources Low level interface Protects, does not manage Expose hardware

Univ. of Tehran 52


Applications know better than Operating Systems what the goal of their resource management decisions should beApplications should be given as much control as possible over those decisionsImplementation view

ExokernelExokernel PhilosophyPhilosophy

Frame Buffer | TLB | Network | Memory | DiskExokernel

HW

Univ. of Tehran 53


ExampleExample

Hardware

Exokernel – Application level resource management

SQL Server

Library OS Customized for SQLServer


Library OSChosen from available

Apache


Exokernel

Univ. of Tehran 54


Library O.S., which uses the low-level exokernel interface to implement higher-level abstractions.

Implementation Implementation OverviewOverview


HW

Library O.S.

Univ. of Tehran 55


Applications link to library kernel, leveraging their higher-level abstractions.

Implementation Implementation OverviewOverview


HW

Library O.S.

Application

Library O.S.

Application

Univ. of Tehran 56


End-to-End ArgumentEnd-to-End Argument “if something has to be done by the

user program itself, it is wasteful to do it in a lower level as well.”

Why should the OS do anything that the user program can do itself?

In other words - all an OS should do is securely allocate resources.

Univ. of Tehran 57


Exokernel designExokernel design

Univ. of Tehran 58


Exokernel tasksExokernel tasks Track ownership Guard all resources through bind

points Revoke access to resources Abort

Univ. of Tehran 59


Design principleDesign principle Expose hardware (securely) Expose allocation Expose names Expose revocation

Univ. of Tehran 60


Secure bindingSecure binding Decouples authorization from use Allows kernel to protect resource without

understanding their semantics Example: TLB entry

Virtual to physical mapping performed in the library (above exokernel)

Binding loaded into the kernel; used multiple times

Example: packet filter Predicates loaded into the kernel Checked on each packet arrival

Univ. of Tehran 61


Implementing secure Implementing secure bindingsbindings

Hardware mechanisms Capability for physical pages of a file Frame buffer regions (SGI)

Software caching Exokernel large software TLB overlaying

the hardware TLB Downloading code into kernel

Avoid expensive boundary crossings

Univ. of Tehran 62


Examples of secure Examples of secure bindingbinding

Physical memory allocation (hardware supported binding) Library allocates physical page Exokernel records the allocator and the permissions

and returns a “capability” – an encrypted cypher Every access to this page by the library requires this

capability

Page fault:•Kernel fields it•Kicks it up to the library•Library allocated a page – gets an encrypted capability•Library calls the kernel to enter a particular translation into the TLB by presenting the capability

Univ. of Tehran 63


Download code into kernel to establish secure binding Packet filter for demultiplexing network

packets How to ensure authenticity? Only trusted servers (library OS) can

download code into the kernel Other use of downloaded code

Execute code on behalf of an app that is not currently scheduled

E.g. application handler for garbage collection could be installed in the kernel

Univ. of Tehran 64


Visible resource Visible resource revocationrevocation

Most resources are visibly revoked E.g. processor; physical page Library can then perform necessary

action before relinquishing the resource E.g. needed state saving for a processor E.g. update of page table

Univ. of Tehran 65


Abort protocolAbort protocol Repossession exception passed to

the library OS Repossession vector

Gives info to the library OS as to what was repossessed so that corrective action can be taken

Library OS can seed the vector to enable exokernel to autosave (e.g. disk blocks to which a physical page being repossessed should be written to)

Univ. of Tehran 66


Aegis – an exokernelAegis – an exokernel

Univ. of Tehran 67


Secure BindingsSecure Bindings Secure Binding – a protection

mechanism that decouples authorization from actual use of a resource Allows the kernel to protect resources

without having to understand them

Univ. of Tehran 68


Aegis – processor time Aegis – processor time sliceslice

Linear vector of time slots Round robin An application can mark its “position” in

the vector for scheduling Timer interrupt

Beginning and end of time slices Control transferred to library specified handler

for actual saving/restoring Time to save/restore is bounded

Penalty? loss of a time slice next time!

Univ. of Tehran 69


Aegis – processor Aegis – processor environmentsenvironments

Exception context Program generated

Interrupt context External: e,g. timer

Protected entry context Cross domain calls

Addressing context Guaranteed mappings implemented by

software TLB mimicking the library OS page table

Univ. of Tehran 70


Aegis performanceAegis performance

Univ. of Tehran 71

pointerArithmetic Protected page


Aegis - Address Aegis - Address translation translation

On TLB miss Kernel installs hardware from software

TLB for guaranteed mappings Otherwise application handler called Application establishes mapping TLB entry with associated capability

presented to the kernel Kernel installs and resumes execution

of the application

Univ. of Tehran 72


ExOS – library OSExOS – library OS IPC abstraction VM Remote communication using ASH

(application specific safe handlers) using message.

Takeaway:significant performance improvement possible compared to a monolithic implementation

Univ. of Tehran 73


Library operating Library operating systemssystems

Use the low level exokernel interface Higher level abstractions Special purpose implementations

An application can choose the library which best suits its needs, or even build its own.

Univ. of Tehran 74


Exokernel vs. Exokernel vs. MicrokernelMicrokernel

A micro-kernel provides abstractions to the hardware such as files, sockets, graphics etc.

An exokernel provides almost raw access to the hardware.

Univ. of Tehran 75


Design ChallengeDesign ChallengeHow can an Exokernel allow libOSes to

freely manage physical resources while protecting them from each other? Track ownership of resources

Secure bindings – libOS can securely bind to machine resources

Guard all resource usage Revoke access to resources

Univ. of Tehran 76


Secure BindingsSecure Bindings Exokernel allows libOSes to bind

resources using secure bindings Multiplex resources securely Protection for mutually distrusted apps Efficient

Univ. of Tehran 77


ConclusionConclusion An Exokernel securely multiplexes

available hardware raw hardware among applications

Application level library operating systems implement higher-level traditional OS abstractions

LibOSes can specialize an implementation to suit a particular application

Univ. of Tehran 78


ConclusionConclusion The lower the level of a primitive…

…the more efficiently it can be implemented… the more latitude it gives to higher level abstractions

So, separate management from protection and……implement protection at a low level

(exokernel)… implement management at a higher level

(libOS)

Univ. of Tehran 79

Microkernel Microkernel ConstructionConstruction

Most Microkernels do not perform well Is it inherent in the approach or Implementation?

IPC, microkernel bottleneck, can implemented an order of magnitude faster. Not supervise memory Minimal address space management, grant,

map, flush. Fast kernel-User Switch, usually 20-30 us but

3 in L3 implementationAdvanced Operating SystemsUniv. of Tehran 80


Micro Kernel Micro Kernel constructionconstruction

Microkernel should provide minimal abstractions Address space, threads, IPC

Abstractions machine independent but implementation hardware dependent for performance

Myths about inefficiency of micro-kernel stem from inefficient implementation and NOT from microkernel approach

Univ. of Tehran 81


What abstractions?What abstractions? Determining criterion:

Functionality not performance Hardware and microkernel should be

trusted but applications are not Hardware provides page-based virtual

memory Kernel builds on this to provide protection for

services above and outside the microkernel Principles of independence and integrity

Subsystems independent of one another Integrity of channels between subsystems

protected from other subsystems

Univ. of Tehran 82


Microkernel ConceptsMicrokernel Concepts

Hardware provides address space mapping from virtual page to a physical page implemented by page tables and TLB

Microkernel concept of address spaces Hides the hardware address spaces and

provides an abstraction that supports Grant? Map? Flush?

These primitives allows building a hierarchy of protected address spaces

Univ. of Tehran 83


Address spacesAddress spaces

A1, P1 V1, R

map

A2, P2 V2, R

R

(P1, v1)

R

(P1, v1)

(P2, v2)

grant

A2, P2 V2, NILR

(P1, v1)

(P2, v2)

A3, P3 V3, R

(P3, v3)

flush

A3, P3 V3, NILR

(P1, v1)

Univ. of Tehran 84

Mem. Manger, process

Page tablePhy. Mem


Power and flexibility of address spaces Initial memory manager for address space

A0 appears by magic (outside the kernel) and encompasses the physical memory

Allow creation of stackable memory managers (all outside the kernel)

Pagers can be part of a memory manager or outside the memory manager

All address space changes (map, grant, flush) orchestrated via kernel for protection

Device driver can be implemented as a special memory manager outside the kernel as wellUniv. of Tehran 85


Microkernelprocessor

M0, A0, P0

PT

M1, A1, P1

PT

M2, A2, P2

PT

Map/grant

Univ. of Tehran 86


Threads and IPCThreads and IPC Executes in an address space

PC, SP, processor registers, and state info (such as address space)

IPC is cross address space communication Supported by the microkernel

Classic method is message passing between threads via the kernel

Sender sends info; receiver decides if it wants to receive it, and if so where

Address space operations such as map, grant, flush need IPC

Higher level communication (e.g. RPC) built on top of basic IPC

Univ. of Tehran 87


Interrupts? Each hardware device is a thread from kernel’s

perspective Interrupt is a null message from a hardware thread

to the software thread Kernel transforms hardware interrupt into a

message Does not know or care about the semantics of the

interrupt Device specific interrupt handling outside the kernel Clearing hardware state (if privileged) then carried out by

the kernel upon driver thread’s next IPC TLB handler?

In theory software TLB handler can be outside the microkernel

In practice first level TLB handler inside the microkernel or in hardware

Univ. of Tehran 88


Unique IDsUnique IDs Kernel provides uid over space and

time for Threads IPC channels

Univ. of Tehran 89


Breaking some Breaking some performance mythsperformance myths

Kernel user switches Address space switches Thread switches and IPC Memory effects

Base system: 486 (50 MHz) – 20 ns cycle time

Univ. of Tehran 90


Kernel-user switchesKernel-user switches Machine instruction for entering and exiting

107 cycles Mach measures 900 cycles for kernel-user switch

Why? Empirical proof

L3 kernel ~ 123 cycles (accounting for some TLB, cache misses)

Where did the remaining 800 cycles go in MACH? Kernel overhead (construction of the kernel, and inherent

in the approach)

Univ. of Tehran 91


Address space switchesAddress space switches Primer on TLBs

AS tagged TLB (MIPS R4000) vs untagged TLB (486)

Untagged TLB requires flush on AS switch Instruction and data caches

Usually physically tagged in most modern processors so TLB flush has no effect

Address space switch Complete reload of Pentium TLB ~ 864

cycles

Univ. of Tehran 92


Do we need a TLB flush always? Implementation issue of “protection domains” SPIN implements protection domains as Modula

names within a single hardware address space Liedtke suggests similar approach in the

microkernel in an architecture-specific manner PowerPC: use segment registers => no flush Pentium or 486: share the linear hardware address

space among several user address spaces => no flush

There are some caveats in terms of size of user space and how many can be “packed” in a 2**32 global space

Univ. of Tehran 93


Upshot? Address space switching among medium or

small protection domains can ALWAYS be made efficient by careful construction of the microkernel

Large address spaces switches are going to be expensive ALWAYS due to cache effects and TLB effects, so switching cost is not the most critical issue

Univ. of Tehran 94


Thread switches and Thread switches and IPCIPC

Univ. of Tehran 95


Segment switch (instead of AS switch) makes cross domain calls cheap

Univ. of Tehran 96


Memory Effects – Memory Effects – SystemSystem

Univ. of Tehran 97


Capacity induced MCPICapacity induced MCPI

Univ. of Tehran 98


Portability Vs. Portability Vs. PerformancePerformance

Microkernel on top of abstract hardware while portable Cannot exploit hardware features Cannot take precautions to avoid

performance problems specific to an arch

Incurs performance penalty due to abstract layer

Univ. of Tehran 99


Examples of non-Examples of non-portabilityportability

Same processor family Use address space switch implementation

TLB flush method preferable for 486 Segment register switch preferable for Pentium

=> 50% change of microkernel! IPC implementation

Details of the cache layout (associativity) requires different handling of IPC buffers in 486 and Pentium

Incompatible processors Exokernel on R4000 (tagged TLB) Vs. 486

(untagged TLB)

=> Microkernels are inherently non-portable

Univ. of Tehran 100


SummarySummary Minimal set of abstractions in

microkernel Microkernels are processor specific

(at least in implementation) and non-portable

Right abstractions and processor-specific implementation leads to efficient processor-independent abstractions at higher layers

Univ. of Tehran 101


Next LectureNext Lecture Process and Thread

“Cooperative Task Management Without Manual Stack Management”, by Atul Adya, et.al.

“Capriccio: Scalable Threads for Internet Services”, by Ron Von Behrn, et. al.

“The Performance Implication of Thread Management Alternative for Shared-Memory Multiprocessors”, Thomas E. Anderson, et.al.

Univ. of Tehran 102

advanced operating systems university of tehran dept. of ee and computer engineering by: dr. nasser...

Documents

tehran2 slide

tehran1 slide

tehran3 slide

new applicationsrequirements

computer engineering

microkernel constructions

frans kaashoek

general guides