file system basics adopted from fred b. schneider department of computer science cornell university...
Post on 21-Dec-2015
214 views
TRANSCRIPT
![Page 1: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/1.jpg)
File System Basics
Adopted fromFred B. Schneider
Department of Computer ScienceCornell University
1
![Page 2: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/2.jpg)
Agenda
• Definition• Position• Responsibilities• Design Principles• Examples
2
![Page 3: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/3.jpg)
Storing Information
• Applications can store it in the process address space
• Why is it a bad idea?– Size is limited to size of virtual address space
• May not be sufficient for airline reservations, banking, etc.
– The data is lost when the application terminates• Even when computer doesn’t crash!
– Multiple process might want to access the same data• Imagine a telephone directory part of one process
3
![Page 4: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/4.jpg)
File Systems
• 3 criteria for long-term information storage:– Should be able to store very large amount of information– Information must survive the processes using it– Should provide concurrent access to multiple processes
• Solution:– Store information on disks in units called files– Files are persistent, and only owner can explicitly delete it
• Files are managed by the OS
• File Systems: How the OS manages files!
4
![Page 5: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/5.jpg)
File System: Definition
• A software component that imposes structure on the address space of one or more physical or virtual disks so that applications may deal more conveniently with abstract named data objects of variable size (files). [SNIA]
• A method of storing and organizing computer files and their data. Essentially, it organizes these files into a database for the storage, organization, manipulation, and retrieval by the computer's operating system. [WIKI]
5SNIA: http://www.snia.org/education/dictionary/f/ Wiki: http://en.wikipedia.org/wiki/File_system
![Page 6: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/6.jpg)
Storage Hierarchy
6
![Page 7: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/7.jpg)
Abstraction Layer
• Typically, file systems are used on storage devices (e.g. hard drives) to maintain the physical locations of files.
• As an important software component of OS, a file system provides the OS abstraction that average user is most familiar with
• Files, Directories, Access controls, …
7
![Page 8: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/8.jpg)
Files
• A file is a collection of data with system maintained properties like– Owner, size, name, last read/write time, etc.
• Files often have “types” which allow users and applications to recognize their intended use
• Some file types are understood by the file system (mount point, symbolic link, directory)
• Some file types are understood by applications and users (.txt, .jpg, .html, .doc, ...)
8
![Page 9: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/9.jpg)
File Naming
• Motivation: Files abstract information stored on disk– You do not need to remember block, sector, …– We have human readable names
• How does it work?– Process creates a file, and gives it a name
• Other processes can access the file by that name
• Naming conventions are OS dependent– Usually names as long as 255 characters is allowed– Digits and special characters are sometimes allowed– MS-DOS and Windows are not case sensitive, UNIX family
is
9
![Page 10: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/10.jpg)
File Extensions
• Name divided into 2 parts, second part is the extension
• On UNIX, extensions are not enforced by OS– However C compiler might insist on its extensions• These extensions are very useful for C
• Windows attaches meaning to extensions– Tries to associate applications to file extensions
10
![Page 11: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/11.jpg)
File Attributes• File-specific info maintained by the OS
– File size, modification date, creation time, etc.– Varies a lot across different OSes
• Some examples:– Name – only information kept in human-readable form– Identifier – unique tag (number) identifies file within file system– Type – needed for systems that support different types– Location – pointer to file location on device– Size – current file size– Protection – controls who can do reading, writing, executing– Time, date, and user identification – data for protection,
security, and usage monitoring
11
![Page 12: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/12.jpg)
Directories
• Directories provide a way for users to organize their files *and* a convenient way for users to identify and share data
• Logically directories store information like file name, size, modification time etc (Not always kept in the directory though..)
• Most file systems support hierarchical directories (/usr/local/bin or C:\WINNT)– People like to organize information hierarchically
12
![Page 13: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/13.jpg)
Directories are special files
• Directories are files containing information to be interpreted by the file system itself
• List of files and other directories contained in this directory
• Some attributes of each child including where to find it
13
![Page 14: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/14.jpg)
Path Name Translation
• To find file “/foo/bar/baz” – Find the special root directory – In special root directory file, look for entry foo and
that entry will tell you where foo is – Read special directory file foo and look for entry bar
to tell you where bar is– Find special directory file bar and look for entry baz to
tell you where baz is– Finally find baz
• FS can cache common prefixes for efficiency
14
![Page 15: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/15.jpg)
Basic File Operations
• UNIX – create (name)– open (name, mode) – read (fd) – write(fd) – sync(fd)– seek(fd, pos) – close(fd) – unlink (name) – rename(old,new)
15
![Page 16: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/16.jpg)
Basic File Operations• Windows– CreateFile (name, CREATE) – CreateFile (name, OPEN) – ReadFile (handle) – WriteFile (handle) – FlushFileBuffers(handle) – SetFilePointer(handle) – CloseHandle(handl) – DeleteFile (name) – CopyFile (name)– MoveFile(name)
16
![Page 17: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/17.jpg)
Core Modules
• Data organization• Freespace management• Metadata management
17
![Page 18: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/18.jpg)
Design Principles
• Data organization– Sequentiality– Spatial Locality
18
![Page 19: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/19.jpg)
“Unwritten Contract”
• Sequential accesses are best, much better than non-sequential.
• An access to a block near the previous access in LBN space is usually considerably more efficient than an access to a block farther away.
• Ranges of the LBN space are interchangeable, such that bandwidth and positioning delays are affected by relative LBN addresses but not absolute LBN addresses.
19Source: MEMS-based storage devices and standard disk interfaces: A square peg in a round hole? FAST’04
![Page 20: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/20.jpg)
Ext2 FS Data Layout
20Source: http://www.linux-tutorial.info/modules.php?name=MContent&pageid=272
![Page 21: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/21.jpg)
Freespace Management• Freespace Allocation and Reclamation• Linked list of free space – Just put freed blocks on the end and pull blocks from front
to allocate– Hard to manage spatial locality
• Bitmap – Divide all space into blocks – Bit per block (0 = free; 1 = allocated) – Easy to find groups of nearby blocks – Useful for disk recovery – How big? If had 40 GB disk, then have 10M of 4K blocks is
each needs 1 bit then 10M/8 = 1.2 MB for the bitmap
21
![Page 22: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/22.jpg)
Metadata Management
• Metadata: the data used to describe the data• Superblocks, Block bitmaps, Inode bitmaps,
Directories, … • Occupies a small amount of storage capacity• Absorbs a large numbers of IO requests
22
![Page 23: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/23.jpg)
File Buffer Cache• Cache Data Read
– Exploit temporal locality of access by caching pathname translation information
– Exploit temporal locality of access by leaving recently accesses chunks of a file in memory in hopes that they will be accessed again
– Exploit spatial locality of access by bringing in large chunks of a file at once
• Data written is also cached– For correctness should be write-through to disk– Normally is write-behind
• FS periodically walks the buffer cache and “flushes” things older than 30 seconds to disk
• Unreliable! • Usually LRU replacement
23
![Page 24: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/24.jpg)
File Buffer Cache
• Typically cache is system wide (shared by all processes)– Shared libraries and executables and other
commonly accessed files likely to be in memory already
• Competes with virtual memory system for physical memory– Processes have less memory available to them to
store code and data (address space)– Some systems have integrated VM/FS caches
24
![Page 25: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/25.jpg)
Examples
• Ext2/Ext3/Ext4 (Linux)• FAT/NTFS (Windows)• HFS/HFS+ (Mac OS)• UFS (FreeBSD)
• In development:• Btrfs: snapshots, checksums, multi-disk
spanning, optimized for SSD
25
![Page 26: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/26.jpg)
• Disk FS:• Network FS: NFS client-server• Distributed FS: Google FS• Special-Purpose: procfs /proc• ……
26
![Page 27: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/27.jpg)
Research Links
• File Systems and Storage Lab (FSL)– http://www.fsl.cs.sunysb.edu/– FiST, Tracefs, Avfs, VersionFS, I3FS, RAIF, UnionFS,
NcryptFS,…
• The ADvanced Systems Laboratory (ADSL) – http://www.cs.wisc.edu/adsl/Publications/– SDS, D-GRAID, IRON FS, EIO, SQCK, EnvyFS, …
27
![Page 28: File System Basics Adopted from Fred B. Schneider Department of Computer Science Cornell University 1](https://reader030.vdocuments.us/reader030/viewer/2022032704/56649d545503460f94a306c7/html5/thumbnails/28.jpg)
Acknowledgement
• http://www.cs.cornell.edu/Courses/cs414/2004su/slides/11_fsbasics.pdf
28