operating system concepts - uni-hamburg.de · 28/10/15 os concepts 1 . what is an os • hard to...

Introduc)ontoOpera)ngSystemConcepts

Prak)kumKernelProgrammingUniversityofHamburgScien)ficCompu)ng

Wintersemester2015/2016

Konstan)nosChasapisKonstan)nos.chasapis@informa)k.uni-hamburg.de

Outline

•  WhatisanOpera)ngSystem•  HistoryofOpera)ngSystems•  ComputerHardware•  TypesofOpera)ngSystems•  ProcessManagement•  MemoryManagement•  StorageManagement*disclaimer:someofthetopicspresentedhereareincomplete.28/10/15 OS Concepts 1

WhatisanOS•  Hardtodefine•  Abstractsasetofhardwareresources– Highlevelinterfaceinsteadofmachinecode

•  e.g.filestorageontopofblockdevices•  Resourcemanagement

– Mul)plexing(sharing)resources•  e.g.assignCPU)metoapplica)ons

Hardware

Operating System

Libraries

Applications

28/10/15 OS Concepts 2

1stGenera)on

•  VacuumTubes(1945-55)– ~20,000vacuumtubeswereused– Programmingwasdoneinabsolutemachinecode

– Assemblylanguagewasunknown– Eachprogramusedthemachineexclusively

– MostfamousENIAC•  Announcedin1946•  Solvelargeclassnumericalproblems


2ndGenera)on•  Transistorsandbatchsystems(1955-65)–  Designers/Builders/Operators/Programmers/Maintainers

–  Programmersfirstwrotetheprogramonpaper,thenpuncheditoncards

–  Cardreaderstoreadtheprogramsource

–  Outputstoredontapesandalsoprinted–  1stuseofcompilers(FORTRAN)


3rdGenera)on•  ICsandMul)programming(1965-1980)

–  IBM360Mainframe•  Mul)programming•  Severalprogramsinmemoryatoncewithseparatememory,overlapI/Owithcomputa)on

–  Timesharing•  Eachuserhasanonlineterminal•  CTSS(Compa)bleTimeSharingSystem)•  MULTICS(MULTiplexInforma)onandCompu)ngSystem)•  UNIX,astripped-downversionofMULTICS•  BSC(BerkeleySodwareDistribu)on)


4thGenera)on

•  PersonalComputers(1980-today)– SYSTEMV,1stcommercialUNIXopera)ngSystem(1983)

– LSI(LargeScaleIntegra)on)–  IBMPC(early1980s)

•  Intel80286CPU•  DOS(DiskOpera)ngSystem)•  MS-DOS(MicrosodDOS)

– LISA•  FirstcomputerwithGUI•  Protectedmemory,preemp)vemul)tasking


5thGenera)on

•  Smartphones(1990-today)•  SymbianOS•  RIM’sBlackberryOS•  WindowsMobile•  Android•  iOS


ModernOpera)ngSystems


Outline

•  WhatisanOpera)ngSystem•  HistoryofOpera)ngSystems•  ComputerHardware•  TypesofOpera)ngSystems•  ProcessManagement•  MemoryManagement•  StorageManagement


ComputerHardware

•  Processor•  Mainmemory•  I/Odevices•  Disk•  Busses


Processor(CPU)

•  Specificinstruc)onset

•  Basiccycle•  fetch•  decode•  execute•  Mul)plecores•  Mul)plelevelsofcachememory

https://commons.wikimedia.org/wiki/File:AMD_Bulldozer_block_diagram_%288_core_CPU%29.PNG 28/10/15 OS Concepts 11

MainMemory

•  Randomaccessmemory(RAM)– Largebutslow– Vola)le,losesdatainpowerloss

•  Sta)c(SRAM)•  Fastbutexpensive•  UsedasCPUcache

– Dynamic(DRAM)•  Storebitsincapacitors•  Requirerefreshtoretainstate•  LargerthanSRAMbutslower

– Non-Vola)le,keepstatewithoutpower•  Notyetmaturetechnology

RAM Disk

CPU Cache Registers


Disks

•  Mechanicaldevice•  Highcapacity

– 200xsizeofRAM•  Slow

– 1000xslowerthanRAM•  Magne)crecording•  Movingparts

– Rota)ngplaiers– Head

•  Seek)me28/10/15 OS Concepts 13

I/ODevices

•  Usuallytwoparts– Theactualdevice– Controller

•  Chip(s)thatphysicallycontrolsthedevice•  “Talks”totheopera)ngsystem(OS)•  Devicedriver

–  SodwarethatconnectsOSwiththecontroller


I/ODevices(cont.)

•  Formsofcommunica)on– Busywai)ng

•  OSwaitsun)ldeviceresponse–  Interrupts

•  DeviceinformtheOSthat“something”happened– Polling

•  TheOSregularlychecksthedevice– DMA(DirectMemoryAccess)

•  Specialhardware•  DatamovementfrommemorytocontrollerwithoutgoingthroughtheCPU


Busses•  Connectcomputercomponents

–  Parallel(mul)plewires)•  Carrydatawordsinparallel

–  Serial(lanes)•  Carrydatainserialformineachlane

•  Differentspeeds•  Differentfunc)onality•  Examples

–  PCIe(PeripheralComponentInterconnectexpress)–  SATA(SerialATA)


Outline



TypesofOSs•  Mul)-user

–  Mul)pleusersaccessthecomputersimultaneously•  Single-tasking

–  Onlyonerunningprogram•  Mul)-tasking

–  Allowsmorethanoneprogramtoruninparallel•  Twotypes:

–  Pre-emp)ve,theOSinterruptstherunningprogramandassignstheCPUtothenext

–  Co-opera)ve,eachprocessgive)metotheothers•  Real-)me

–  Aimsatexecu)ngreal-)meapplica)ons


TypesofOSs(cont.)

•  Distributed– Managesagroupofindependentcomputersandmakesthemappeartobeasinglecomputer

•  Templated– Asinglevirtualmachineimageasaguestopera)ngsystem,thensavingitasatoolformul)plerunningvirtualmachines

•  Embedded– Designedtobeusedinembeddedcomputersystems


Monolithickernel•  Singleimagethatrunsinasingleaddressspace

– Asetofprimi)veopera)onsareimplementedintheopera)ngsystemlevel

•  Processmanagement•  Memorymanagement•  DeviceDrivers

– Trivial(IPC)InterProcessCommunica)on– Easytodesign– Difficulttomaintainandextend– Examples:

•  MULTICS,SunOS,Linux,BSD


Micro-kernel•  TheminimumamountofsodwarethatprovidesthemechanismsneededtoimplementanOS– Alsoknownasμ-kernel– Provides

•  Built-inIPC•  Lowleveladdressspacemanagement•  Threadmanagement

– Easytoextend– Performancepenal)es(requiresIPCcalls)– Examples

•  Symbian,MacOS,WinNT28/10/15 OS Concepts 21

Monolithicvs.μ-kernelEverythingthatrunsinkernelmodedefinestheOS

Source: http://en.wikipedia.org/wiki/Microkernel#mediaviewer/File:OS-structure.svg


Outline



Processes•  Anabstrac)onofarunningprogram•  Everyprocess“thinks”thatitrunsalone

– Assumededicatedresources(addressspace,CPU)•  Hasasinglecontrolflow(programcounter)

– Whichopera)oniscurrentlyexecuted•  Programmer’srole

– Defineswhattheprocesswilldo•  Opera)ngsystemrole

–  Processmanagement28/10/15 OS Concepts 24

ProcessManagement•  Processcrea)on

–  System(ini)aliza)on,systemcall)– User(executesanewprogram)

•  Processtermina)on– Normalexit–  Error(voluntary),Fatalerror(involuntary)–  Killedbyanotherprocess

•  ProcessScheduling– AssignCPUtotheprocesses– Determinewhichprocesswillexecutenext–  Switchbetweendifferentprocesses(contextswitch)


ProcessHierarchies

•  Oneprocessinvokesthecrea)onofanother– Parentprocess,ini)atedthecrea)on– Childprocess,thecreatedprocess

•  Canalsocreateprocesses•  Daemonprocess

– Runsinthebackground– Notcontrolledbytheuser– Crea)on

•  Createanewprocess(child)•  Killyourparent•  Con)nuewiththeexecu)on


ProcessStates

•  Running– CurrentlyusingtheCPU

•  Ready– Readytoexecute

•  Blocked– Unabletorun– Waitsun)l“something”externaltohappen

Running

Blocked Ready


ProcessScheduling

•  Holdprocessesinqueues–  jobqueue,allprocesses–  runningprocesses,abletorun– devicequeue,wai)ngforI/Ocomple)on


ProcessScheduling

•  Scheduler– Processthatmigratesprocessesbetweenqueues

•  PolicyConsidera)ons– Priori)za)on,whichwillexecutefirst– Fairness,everyprocessgetsafairamountofCPU– Starva)on,doesnotgetCPU)me– MaximizeCPUu)liza)on


ContextSwitch

•  SwitchtheCPUtoanotherprocess/thread– Saveprogram“context”

•  Registers•  Stack•  Memory

–  Impliesoverhead


Threads

•  Whynothavingmorethanonecontrolflowinsideaprocess?– Sameproblemsolvedbymorethanone

•  Threadsare“mini”orlightweightprocesses


Threadvs.Process•  Perthread

–  Stack–  Registers–  Privatecontrolflow(Programcounter)

•  Perprocess– Addressspace– Globalvariables–  Filedescriptors–  Childprocesses–  Signals,signalhandling– Accoun)nginforma)on


Threadsconcurrency

•  Manythreadsrunninginparallel•  Cooperatetosolveasingleproblem

– Splittheprobleminsub-problems•  Requirecoordina)on/synchroniza)on

– ControlCPUusage– Controlaccesstosharedresources


ArrayMul)plica)on•  Assumethatwehavetomul)plytwoarrays•  Singlethreadedsolu)on

–  Startfromthebeginningofthematrixandcalculateeverycelloneata)me

•  Mul)threadedsolu)on–  Splitthearrayinsub-arrays–  Assigneachsub-arraytoadifferentthread–  Runthreadsinparallel– Waitun)lallthreadsfinishthecalcula)ons–  Outputmatrixisready


Synchronous/Asynchronousexecu)on

•  Synchronousexecu)on–  Opera)onreturnswhentheexecu)onfinishes–  Example

•  Iwillcon)nuecookingwhenthewaterhasboiled•  Asynchronousexecu)on

–  Opera)onreturnsimmediatelyaderexecu)onini)aliza)on

–  Checklaterorno)fywhentheexecu)onfinishes–  Example

•  Iputthewatertoboilandcon)nuewiththerestoftherecipe

•  Ader5minutesIcheckifthewaterhasboiled


Webserverexample•  Webservertasks

– Acceptsrequestsforawebpage–  Fetchesthedatafromthestoragesystem–  Repliesbysendingthedata

•  Singlethreadsolu)on–  Eachrequestisexecutedserially– Onlyasinglerequestisbeingprocessedata)me–  Everyrequesthastowaitfortheprevioustocompletealltasks

–  I/Oisslow,lotsof)mewai)ng–  Cannotu)lizemul)plecores


Webserverexample•  Mul)-threaded

–  Createathreadforeachrequest– Mul)plerequestscanbeservedinparallel–  Threadexecu)on

•  Acceptrequest•  I/Orequest,fetchdatafromdisk•  Thisrequestcallcanbeasynchronous•  Replyrequest,senddata

– Mul)-coreu)liza)on– OverlapI/Owithcomputa)on


Synchroniza)onissues•  Cri)calregion

–  Partoftheprogramthataccessessharedresources•  Globalvariables•  Sharedmemory•  Filedescriptors

–  Itissafetobeexecutedbyonlyoneprocess/threadata)me

•  Racecondi)on–  Thesuccessfulexecu)ondependsonthesequenceor)mingoftheotherthreads/processes


Synchroniza)onissues•  Assumetwotasksruninparallel

•  Task1readsaglobalvariableandupdatesthevalue

•  Task2usesthesamevariableasTask1


MutualExclusion•  Preventparallelexecu)onsofthe“cri)calregion”

•  Basicmethodsformutualexclusion–  Sleepandwakeup

•  Onetaskexplicitlyputstheotheronetosleepwhenitentersthecri)calregion

•  Wakesitupwhenitexitsthecri)calregion–  Semaphores

•  Specialintegervariablestocountsleepsandwakeups– Mutex

•  Simplifiedsemaphoreonlytwostateslock,unlock


Producer/Consumer

procedure producer() { while (true) { item = produceItem(); if (count == N) sleep(); insert(item); count = count + 1; if (count == 1) wakeup(consumer); } }

procedure consumer() { while (true) { if (count == 0) sleep(); item = remove(); itemCount = Count - 1; if (count == N - 1) wakeup(producer); consumeItem(item); } }

#define N 100 int count=0;


Deadlock

•  Oneprocesswaitsfortheother•  Competeforresources

– P1needsresourcesfromP2– P2needsresourcesfromP1

•  Alwaysallocatewiththesameorder•  Hardtodebug


Outline



AccessPhysicalMemory

•  Useabsolutephysicaladdresses•  Nomemoryabstrac)on•  Runmul)pleprocesses

– OSsavestheen)rememory– OSloadsthenextprocess’smemory– Runsthenextprocess


MemoryAbstrac)on•  Allowmul)pleprocessestoexistinmemory•  Protec)on

–  Askforpermissiontoaccessanaddressinmemory•  Reloca)on

–  Allocatephysicalmemorydynamically–  Relocatetheprocesstoadifferentregion

•  Sharing–  Controlwhichdatawillbesharedbetweenprocesses

•  Dis)nguishphysicaltologicalmemory–  Managethememoryhierarchy


AddressSpace

•  Setofaddressthataprocesscanuse– Memoryabstrac)on– Mapeachprocess’saddressspaceintodifferentpartsofphysicalmemory

•  Useoftworegisters– Base,startaddressoftheprogram– Limit,lengthoftheprogram– OnlyOSmodifiestheseregisters

•  Processmemoryaccess– Logicaladdress+Baseregister


Swapping

•  Ifmainmemoryisnotenough– Usedisktotemporarilystoreprocessdata

•  Movingfrommemorytodiskandviceversaiscalled“swapping”

•  Slowprocess–  InvolvesdiskI/O


VirtualMemory

•  Maplogicaladdressestophysicaladdresses•  Eachprocesshasaprivateaddressspace•  Addressspacedividedtopages•  Pagesmappedtophysicalmemory•  OSkeepsapagetable

– Transla)onfromvirtualtophysicalpages


Outline



StorageManagement

•  Applica)onsneedpermanentdatastorage– Persistency

•  Hardwareprovidesblocklevelaccesses– Usefixed-sizeblocks– Everyblockhasitsownaddress– Transfersinmul)plesofblocks– Seagateprovidesobjectbasedaccess(key/value)

•  OSprovidesmethodstostoreandretrievedatafromdisks


FileSystem•  Movedatafrommemorytodiskandviceversa•  Exportfilesabstrac)on

– Namedatacollec)onswithnames->filename– Maplogicalcon)nuousunitstoarbitrarydiskblocks– Accesstheminbytelevel

•  Exportfilesystemtree(hierarchy)– Groupfilescollec)onindirectories


FileSystem

•  Diskspacemanagement– Keepstrackofusedspace– Freespacethatisnotusedbytheusers

•  Holdsystemstate– Systemmightbeininconsistentstateduringanunexpectedfailure

•  Updatesystemvariables–  Journalusedtokeeppreviouslyconsistentstate– Recoverfromsystemcrash


FileSystem-Dataprotec)on

•  Eachusercontrolsfiles/directoriespermission•  Linuxpermissionthreegroups

– owner,thefile/directoryowner– group,thegroupthattheuserbelongs– allusers,therestoftheusers

•  Threebasicpermissiontypes–  read,canreadthefile– write,canmodifythefile– execute,canexecuteafile(denotesexecutables)


Ques)ons?

Konstan)nosChasapis

Konstan)nos.chasapis@informa)k.uni-hamburg.de

OS Concepts 28/10/15 54

operating system concepts - uni-hamburg.de · 28/10/15 os concepts 1 . what is an os • hard to...

Documents