kt6213 lecture 14: operating systems – file management & internal os computer organization and...
TRANSCRIPT
KT6213
Lecture 14: Operating Systems – File Management & Internal OS
Computer Organization Computer Organization and Architectureand Architecture
KT6213
Introduction to FilesIntroduction to Files Collection of (usually related) data Block
◦ Typically 256, 512, or 1024 bytes◦ Files usually require a 1-block minimum
Cluster◦ Groups of one or more blocks
Blocks or clusters correspond to one or more sectors on a disk’s single track or cylinder
File extension◦ Name of file includes identification of what type of file it is
File Association◦ The file specifies which program with which it is to be
used
KT6213
TerminologyTerminology Logical view
◦ Contents and attributes of files viewed by the user Physical view
◦ The actual way a file is stored within the computer system Sequential access
◦ Data files whose records always have to be retrieved from the beginning
Random or relative access◦ Data files whose records can be retrieved from anywhere in the
file in random sequence Contiguous
◦ The blocks that hold a particular file are stored together Noncontiguous
◦ The blocks that hold a particular file are scattered all over the device
KT6213
Database File – Table ImageDatabase File – Table Image
KT6213
Database File – Form ImageDatabase File – Form Image
KT6213
Database File – Stream ImageDatabase File – Stream Image
Closer to physical representation of file Example: YouTube video
KT6213
Logical View vs. Physical ViewLogical View vs. Physical View
KT6213
File Management SystemFile Management System Provides a logical view for the user and hides the
physical implementation Consistent set of commands that are translated to a
form appropriate for the device Consistent view of files regardless of file type, file
characteristics, or device Supports manipulation of data within the file Manages directory structures which are presented in a
logical view Command shell takes user file commands and program
file requests and translates them for the file manager Requests data transfers from I/O device drivers File security and protection of file integrity
KT6213
File Manager Request HandlingFile Manager Request Handling
KT6213
File Operations [1/3]File Operations [1/3]
File as a whole◦ Copy, Move◦ List, Print◦ Load and execute a program◦ Load file into memory◦ Store file from memory◦ Append data from memory to file◦ Compile, assemble a file
KT6213
File Operations [2/3]File Operations [2/3]
Within a file◦ Open a file◦ Read a number of bytes from file◦ Write a number of bytes to a file◦ Move the file pointer forward or backward◦ Move file pointer to beginning of a file◦ Close a file
KT6213
File Operations [3/3]File Operations [3/3]
Record Storage◦ Retrieve a record (read)◦ Store a record (write)◦ Add a record to a file◦ Delete a record◦ Modify contents of a record
KT6213
File Directory OperationsFile Directory Operations
File Directory◦ Create a new (empty) file◦ Move a file from one directory to another◦ Rename a file◦ Append one file to another◦ Delete a file
KT6213
File Management and I/O FunctionsFile Management and I/O Functions
Separation between the two allows1. I/O devices can change while keeping the
file system the same2. Redirecting of data is simple
KT6213
File Access MethodsFile Access Methods
Sequential Access◦ File is read in sequence from beginning to end◦ Majority of all files◦ Program source and binary files
Random Access◦ Assumes file is made up of fixed length logical records◦ Hashing is a common method used to calculate the
location of an internal logical record Indexed Access
◦ Additional means for accessing and viewing records in a file
◦ Key indexes◦ ISAM – indexed sequential access method
KT6213
Physical File StoragePhysical File Storage
ContiguousNon-contiguous
◦ Linked◦ Indexed
Examples◦ DOS/Windows FAT◦ UNIX i-nodes◦ Windows NTFS
Free space management
KT6213
Contiguous Storage AllocationContiguous Storage Allocation
Assign blocks (all in a row) to hold the fileAccess is simple for both sequential and
random methodsDisadvantages
◦ Space must be large enough◦ Have to take into account file growth◦ May need to be moved if it outgrows its
space◦ Fragmentation of disk
Allocation strategies to minimize fragmentation◦ First-fit, best-fit
Eventually disk becomes fragmented
KT6213
Linked AllocationLinked Allocation
Non-contiguousEach block contains a link to the next physical
blockVariant – links in both directionsAdvantages
◦ no fragmentation◦ Adding to a file is easy
Disadvantages◦ Not usable for random access◦ Additional disk head searching◦ Overhead in storing the pointers◦ Recovery of a defective block is difficult
KT6213
File Allocation
Contiguous Storage Allocation
Linked Allocation
KT6213
Windows FATWindows FAT
File Allocation Table (FAT)◦ Linked allocation with links stored in a
table◦ Table contains the first block of each file on
the disk or disk partition◦ Successive blocks contain a link to the next
blockDisadvantages
◦ Requires a tremendous amount of space◦ File integrity can be easily compromised
KT6213
File Allocation TableFile Allocation Table
Linked Allocation and File Allocation Table
KT6213
Indexed AllocationIndexed AllocationNon-contiguousAll link pointers for a file are stored together in
a single block called the index blockOne index block per fileAdvantages
◦ No fragmentation◦ Can be used for random access
Disadvantage◦ Slower due to additional access of the index
block◦ Additional disk head searching◦ Recovery of a defective block is difficult
KT6213
Indexed AllocationIndexed Allocation
Index blocks for indexed allocation of linked files
KT6213
Unix i-nodesUnix i-nodes
Indexed file allocation Index block contains
◦ File attributes◦ 10 direct blocks◦ 1 single indirect◦ 1 double indirect◦ 1 triple indirect
Advantages◦ Fast for small blocks◦ Can accommodate very large files – 100’s
of gigabytes
KT6213
Unix i-nodesUnix i-nodes
KT6213
Windows NTFSWindows NTFS
Dynamically sized volumesVolumes may be a fraction of a disk or span
many disksMaster File Table (MFT) of 1kb records
◦ First 16 records are metadata files which describe the volume
◦ First record stores the MFT attributes◦ Each file has an MFT entry
File records made up of attributes, including file information and data
KT6213
NTFS Volume LayoutNTFS Volume Layout
KT6213
Free Space ManagementFree Space Management
Bit map method◦ one bit for each block
to indicate if it is usedor free
Linked list method◦ Pointer to first free block◦ Each free block has a pointer to the next◦ Blocks are allocated from the beginning◦ Deleted files are placed at the end
KT6213
Other Secondary Storage AllocationOther Secondary Storage Allocation
Tape Allocation◦ Not practical to reallocate space in the middle
of the tape◦ Files that grow must be re-written◦ Files are stored contiguously whenever
possibleOptical drives and flash drive file allocation
◦ Similar to that of hard disks◦ UDF (Universal Data Format) supports up to 2
terabytes of data◦ Hierarchical directory format◦ Support for both HD-DVD and Blu-Ray DVD
formats
KT6213
Partitions, File Systems, Volumes, and Partitions, File Systems, Volumes, and PoolsPools
KT6213
Directory StructureDirectory Structure
Provides a means of organization so that files can be located easily and efficiently
Hide the physical devices from the logical view of the files
Partitions◦ Independent subsections of a device
Volume◦ Directory structure for a particular partition◦ Needs to be mounted to be incorporated
into the overall file system structureContain file attributes
KT6213
Tree-Structure DirectoryTree-Structure Directory Hierarchical with a top-level root directory from which all
other directories stem All directories and files have names Separator
◦ Used to indicate subdirectories and files located in a directory
◦ / UNIX◦ \ DOS, Windows
Pathname◦ Absolute – full pathname starting from the root directory◦ Relative – pathname is created starting from the current
directory Search Paths
◦ Directory locations that the operating system uses to locate files
KT6213
Tree-Structure DirectoryTree-Structure Directory
Pathname Examples:
Absolute:C:\FINANCE\QUICKEN\Q.EXE
Relative from the FINANCE directory: QUICKEN\Q.EXE
Search Path: PATH=C:\DOS;C:\FINANCE\QUICKEN◦ Now the programs in the two directories can be run by specifying the
name of the program without the absolute or relative pathnames
For our examples, root is on disk device C:
KT6213
Acyclic Directory StructuresAcyclic Directory Structures
Tree-structure that permits links between separate branches of the tree
Advantage◦ Easy user access
Disadvantages◦ Cycles and dangling links
Examples◦ Windows shortcuts◦ Unix hard and symbolic links◦ MacIntosh aliases
KT6213
An Acyclic-Graph DirectoryAn Acyclic-Graph Directory
KT6213
Graph with a CycleGraph with a Cycle
KT6213
Hard Links vs. Symbolic LinksHard Links vs. Symbolic Links
KT6213
Network File AccessNetwork File Access
FTP◦ File Transfer Protocol◦ Part of the TCP/IP protocol family
Network file systems◦ Windows
Drive letters aliased to remote file systems
◦ UNIX Network File System (NFS) Remote Procedure Call (RPC)
KT6213
Typical NFS ConfigurationTypical NFS Configuration
KT6213
File ProtectionFile Protection
Logins and passwordsRead, write, and execute protectionsACL – access control list, permissionsUNIX/Linux – owner, group, everyone
KT6213
Data Storage ApproachesData Storage Approaches
Standard client server configuration
Storage area network configuration
KT6213
File ProtectionFile Protection
Most systems provide three forms of protection on files◦ Read protection◦ Write protection◦ Execution protection
Access control list (ACL)◦ List of users who may access the file for
each of the forms of protection◦ Tremendous overhead if there are a lot of
usersOwner/Group/Everyone protection method
◦ UNIX, Linux
KT6213
File Directory Showing ProtectionFile Directory Showing Protection
ls –lF list files in directory using a long format and indicate file type
10-char code for file protection
1st char d for directory, - for file, s for symbolic link
2nd to 4th char permissions for the owner
5th to 7th char permissions for the group
8th to 10th char permissions for everyone
r - read permission, w - write permission, x - execute permission
KT6213
Journaling File SystemsJournaling File Systems
Log file records every system transaction that requires a write access to the file system
Two levels of capability1. Protect the integrity of the file system
structure only Example: Windows NTFS file system
2. Also guarantees the integrity of data that has not yet been written to the disk
Examples: Linux ext3 and ext4, IBM JFS
KT6213
Primary Kernel FunctionsPrimary Kernel Functions
File manager translates logical file requests into specific physical I/O requests
I/O Control System (IOCS) performs resource allocation and device management
Memory management determines if it is possible to load programs and data into memory and if so where in memory
Scheduler allocates time for the program to execute
KT6213
Miniature Operating SystemMiniature Operating System
Block Diagram
MemoryMap
Process Dispatch
KT6213
BootstrappingBootstrapping
Execution begins with bootstrap loader (mini-loader, IPL) stored in ROM
Looks for OS program in a fixed location (possibly on the network)
Loads OS into RAMTransfers control to starting location of OS Loader program in OS used to load and
execute user programs
KT6213
BootstrappingBootstrapping
Cold vs. warm boot (does not retest the system)
KT6213
Process [1/2]Process [1/2]
Process: basic unit of work in the OS◦ A program together with all the resources
that are associated with it as it is executed◦ Program: a file or listing◦ Process: a program being executed
Independent vs. cooperating processesPID (process ID): a unique identifier for each
process
KT6213
Process [2/2]Process [2/2]
Process creation◦ Forking, spawning, cloning a new process◦ Parent and child processes
Process context◦ All relevant register data including the
program counter◦ Allows interruption and restart invisibly
KT6213
Two Processes Sharing a Single Two Processes Sharing a Single ProgramProgram
KT6213
Process Control BlockProcess Control Block
A block of data for each process in the system
Contains all relevant information about the process
Typical process control block on the right
KT6213
Process StatesProcess States
Three primary process operating states◦ Ready state◦ Running state◦ Blocked state
Dispatching - Move from ready state to running state
Wake-up - Move from blocked state to ready stateTime-out - Move from running state to ready stateProcess completion
◦ killed, terminated, destroyedAdditional states – suspend, swapResumption – Move from suspended state to
ready state
KT6213
Process State DiagramProcess State Diagram
KT6213
ThreadsThreads
‘Miniprocess’ that can be run independent of other parts of the process◦ Event-driven programs◦ Shares resources allocated to its parent process
including primary storage, files and I/O devices◦ Each thread has its own context
Advantage of process/thread families over multiple independent processes:◦ Reduced OS overhead for resource allocation
and process management◦ Substantially less information than a normal
PCB
KT6213
Loading and Executing a ProcessLoading and Executing a Process
KT6213
CPU SchedulingCPU Scheduling
High-level scheduling Adding a program to the pool of programs to be executed
Short-term scheduling(dispatcher)
Deciding which process shall be executed next by the processor
Mid-level scheduling Swapping processes
I/O scheduling Deciding which process’s pending I/O request shall be handled by an available I/O device
KT6213
Dispatching ObjectivesDispatching Objectives
Ensure FairnessMaximize throughputMinimize turnaround timeMaximize CPU utilizationMaximize resource allocationPromote graceful degradationProvide minimal and consistent response timePrevent starvation
KT6213
Nonpreemptive DispatchingNonpreemptive Dispatching
First in, first out (FIFO)◦ Unfair to short processes and I/O based
processesShortest Job First (SJF)
◦ Longer jobs can be starvedPriority Scheduling
◦ Job with the highest priority is selected◦ If multiple jobs have the highest priority then
dispatcher selects among them using FIFO
KT6213
Preemptive DispatchingPreemptive DispatchingRound robin
◦ Inherently fair and maximizes throughput
Dynamic Priority◦ Based on ratio of CPU time to total time process has been in the system◦ Smallest ratio has highest priority◦ Linux; Windows 2000, XP, Vista, 7
KT6213
Preemptive DispatchingPreemptive Dispatching
Multilevel feedback queues◦ Favors short jobs, I/O bound jobs◦ Each level assigns more CPU time
KT6213
Virtual Memory – Basic IdeasVirtual Memory – Basic IdeasVirtual memory increases the apparent amount
of memory by using far less expensive hard disk space
Provides for process separationDemand paging
◦ Pages reside on hard disk and are brought into memory as needed
Page table◦ Keeps track of what is in memory and what is
still out on hard disk Inverted page table
◦ Representation of physical memory with the page that resides in each frame
KT6213
Pages and Frames [1/2]Pages and Frames [1/2]
Program Memory
Unit Page Frame
Address Logical Physical
Size 2 to 4KB 2 to 4KB
Amount# of bits in instruction word
Installed memory
KT6213
Pages and Frames [2/2]Pages and Frames [2/2]
Each program has its collection of pages.
The total number of pages can exceed the number of frames (physical memory).
A Simple Page Table TranslationPages and Frames
KT6213
Page Translation ProcessPage Translation Process
KT6213
Disk
Virtual Memory Pages
Page FramePages not in main memory:page fault when accessed
1 2 3 4
5 6 7 8
9 10 11
1234567891011
64
8
1012
7
Page TablePage Table
Swap space
KT6213
Mapping for Three ProcessesMapping for Three Processes
KT6213
Inverted Page TableInverted Page Table
Inverted Page Table for the previous slideThe table represents what is in physical memory
KT6213
Steps in Handling a Page FaultSteps in Handling a Page Fault
KT6213
Concept of LocalityConcept of Locality
Most memory references confined to small region
Well-written program in small loop, procedure or function
Data likely in array and variables stored togetherWorking set
◦Number of pages sufficient to run program normally, i.e., satisfy locality of a particular program
KT6213
Page Replacement AlgorithmsPage Replacement Algorithms
Page fault◦ Page is not in memory and must be loaded from disk
Algorithms to manage swapping◦ FIFO – First-In, First-Out
Belady’s Anomaly – when increasing number of pages results in more page faults
◦ LRU – Least Recently Used◦ LFU – Least Frequently Used◦ NUR – Not Used Recently
Referenced bit Modified (dirty) bit
◦ Second Chance Replacement algorithms Thrashing
◦ too many page faults affect system performance
KT6213
Frame Lookup ProceduresFrame Lookup Procedures
KT6213
SegmentationSegmentation
KT6213
Virtual Memory TradeoffsVirtual Memory Tradeoffs
DisadvantagesSWAP file takes up space on diskPaging takes up resources of the CPU
AdvantagesPrograms share memory spaceMore programs run at the same timePrograms run even if they cannot fit into
memory all at onceProcess separation
KT6213
Virtual Memory vs. CachingVirtual Memory vs. Caching
Cache speeds up memory accessVirtual memory increases amount of
perceived storage ◦ Independence from the configuration and
capacity of the memory system◦ Low cost per bit compared to main
memory
KT6213
Secondary Storage SchedulingSecondary Storage Scheduling
FCFS - First-Come, First-ServedShortest Distance First
◦ Indefinite postponement problemScan
◦ Middle of disk gets serviced twiceN-Step C-Scan
◦ Disk seek in only one direction◦ Return after last request in queue served◦ Two queues
Queue of requests being processed Queue of new requests
KT6213
Scan Scheduling AlgorithmScan Scheduling Algorithm
KT6213
Comparison of Different Disk Comparison of Different Disk AlgorithmsAlgorithms
KT6213
Network OS ServicesNetwork OS Services
File transfer programs Access to data files on other computers on the network
◦ Computer naming scheme is required for some network systems
Print services◦ Print requests are redirected by the OS to the
network station that manages the requested printer Other peripheral sharing services Web services Messaging services API network services Security and network management services Remote processing services
KT6213
Access for a Networked OSAccess for a Networked OS
KT6213
Other OS IssuesOther OS Issues
Deadlock◦ Two or more processes simultaneously
have resources that are required by one another in order to proceed
◦ Prevention◦ Avoidance◦ Detection and recovery
Process Synchronization◦ Required by cooperating processes when
one process is dependent on the other
KT6213
Virtual MachinesVirtual MachinesVirtualization
◦ Using a powerful computer to simulate a number of computers
Virtual machines◦ A simulated computer
Hypervisor◦ Layer of software and/or hardware that separates
one or more operating systems from the hardware
◦ Type 1 (native) – hypervisor software interfaces directly with the computer hardware
◦ Type 2 (hosted) – hypervisor software runs as a program on the operating system
KT6213
Virtual Machine ConfigurationVirtual Machine Configuration