![Page 1: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/1.jpg)
Chapter 20
Distributed File SystemsCopyright © 2008
![Page 2: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/2.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.2Operating Systems, by Dhananjay Dhamdhere 2
Introduction
• Design Issues in Distributed File Systems• Transparency• Semantics of File Sharing• Fault Tolerance• DFS Performance• Case Studies
![Page 3: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/3.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.3Operating Systems, by Dhananjay Dhamdhere 3
Design Issues in Distributed File Systems
![Page 4: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/4.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.4Operating Systems, by Dhananjay Dhamdhere 4
Overview of DFS Operation
• Remote file processing model
• File server agent and client agent are analogous to RPC’s stub processes
• For efficiency, the client agent and the cache manager are typically rolled into a single unit
![Page 5: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/5.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.5Operating Systems, by Dhananjay Dhamdhere 5
Transparency
• In a conventional file system, a user identifies a file through a path name– User is aware that file belongs in a specific directory, but
is not aware of its location in the system
• Location info field of the file’s directory entry indicates the file’s location on disk
• Location transparency can be provided in a DFS through a similar mechanism– Location info: (node id, location)
• Location independence requires information in location info field to vary dynamically
![Page 6: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/6.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.6Operating Systems, by Dhananjay Dhamdhere 6
Semantics of File Sharing
• Semantics determine manner in which effect of file manipulations performed by concurrent users of a file are visible to one another
![Page 7: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/7.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.7Operating Systems, by Dhananjay Dhamdhere 7
Semantics of File Sharing (continued)
• A session consists of some clients of a file that are located in the same node of a system
• Problem with session semantics: poor portability• Session semantics are easy to implement in a DFS
employing file caching– File changes are not visible to clients in other nodes
![Page 8: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/8.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.8Operating Systems, by Dhananjay Dhamdhere 8
Fault Tolerance
• File system reliability has several facets:– A file must be robust, recoverable, available
• Robustness is achieved using techniques for reliable storage of data
• Robustness and recoverability depend on how files are stored and backed up, respectively
• Availability depends on how files are opened and accessed
• Only defense against client node crashes is use of transaction semantics in file server
![Page 9: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/9.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.9Operating Systems, by Dhananjay Dhamdhere 9
Fault Tolerance (continued)
![Page 10: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/10.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.10Operating Systems, by Dhananjay Dhamdhere 10
Availability
• File is available if a copy can be opened and accessed by client– Ability to open file depends on path name resolution
– Access requires functional client and server nodes
• An anomalous situation may arise when path names span many nodes– If a node in path crashes, file operation will fail even if the
node that contains the file has not crashed
• Solution: cached directories
• File replication is transparent to clients– Updating techniques: 2PC, use of primary copies
![Page 11: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/11.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.11Operating Systems, by Dhananjay Dhamdhere 11
Client and Server Node Failures
• File server can maintain FCBs and OFT in memory– Stateful design
– Good performance
– Problems in event of client and server crashes
• Solution: client and file server share a virtual circuit– Virtual circuit “owns” the file processing actions and
resources like file server metadata
– Actions and resources become orphans after crash• Actions are rolled back and metadata destroyed
– Client–server protocol implementing transaction semantics may be used to ensure this
![Page 12: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/12.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.12Operating Systems, by Dhananjay Dhamdhere 12
Stateless File Servers
• File server does not maintain state information about file processing activity
• Client must:– Keep state information about file processing activity
– Provide all relevant information in a file system call• read (“alpha”, <record/byte id>, <io_area address>);
• Many actions traditionally performed only at file open time are repeated at every file operation
• If file server crashes, time-outs and retransmissions occur in client
• Cannot employ file caching
![Page 13: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/13.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.13Operating Systems, by Dhananjay Dhamdhere 13
DFS Performance
• DFS design is scalable if DFS performance doesn’t degrade with increase in size of distributed system
![Page 14: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/14.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.14Operating Systems, by Dhananjay Dhamdhere 14
Efficient File Access
• Inherent efficiency of file access depends on how the operation of a file server is structured
• Two server structures that provide efficient file access:– Multithreaded file server
– Hint-based file server• State information is used as a hint• Server operation is stateless if hint is not available
![Page 15: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/15.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.15Operating Systems, by Dhananjay Dhamdhere 15
File Caching
• File cache and copy of file on disk in server node form a memory hierarchy– Operation of the file cache and its benefits are analogous
to those of a CPU cache
• Chunks of file data are loaded from the file server into the file cache
• Studies of file size distributions indicate small average file size– Whole-file caching is feasible
• File server may use a separate attributes cache
![Page 16: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/16.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.16Operating Systems, by Dhananjay Dhamdhere 16
File Caching (continued)
• Key issues:– Location of the file cache: memory or disk
– File updating policy: write-through or delayed write
– Cache validation policy: client- or server- initiated
– Chunk size: large or small? Fixed or variable?
![Page 17: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/17.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.17Operating Systems, by Dhananjay Dhamdhere 17
Scalability
• DFS scalability achieved through techniques that localize most data traffic generated by file processing activities within clusters– Clusters typically represent subnets like high-speed LANs
– An increase in the number of clusters does not lead to degradation of performance
• It does not add much network traffic
![Page 18: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/18.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.18Operating Systems, by Dhananjay Dhamdhere 18
Case Studies
• Sun Network File System• Andrew and Coda File Systems• GPFS• Windows
![Page 19: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/19.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.19Operating Systems, by Dhananjay Dhamdhere 19
Sun Network File System
• VFS implements mount protocol and creates a system-wide unique vnode for each file
• NFS layer interacts with remote node containing file through NFS protocol
![Page 20: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/20.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.20Operating Systems, by Dhananjay Dhamdhere 20
Sun Network File System (continued)
• Several techniques to improve performance– A directory names cache is used in each client node
– A file attributes cache caches inode information• Cached attributes are discarded after 3 seconds for files
and after 30 seconds for directories
– File blocks cache is the conventional file cache• Server uses large (8 Kbytes) data blocks• Cache validation performed through timestamps associated
with each file, and cache block
• File server is stateless• Neither Unix semantics nor session semantics
![Page 21: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/21.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.21Operating Systems, by Dhananjay Dhamdhere 21
Andrew and Coda File Systems
• Targeted at gigantic distributed systems• All clients have an identical shared name space
– Is location transparent in nature
– Implemented by dedicated servers (Vice)
• Clusters localize file processing activities– Traffic within cluster reduced by caching entire file on
local disk
• A volume typically contains files of a single user• 64 KB chunks (size adapted on a per-client basis)• User process called Venus performs open/close
![Page 22: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/22.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.22Operating Systems, by Dhananjay Dhamdhere 22
Andrew and Coda File Systems (continued)
• Server-initiated cache validation using callbacks• Path name resolution performed on a component-by-
component basis– Venus maintains a mapping cache
• File servers are multithreaded• Client–server communication uses RPCs• Two features to achieve high availability:
– Replication and disconnected operation• Read one, write all policy• Supports hoarding of files
![Page 23: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/23.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.23Operating Systems, by Dhananjay Dhamdhere 23
GPFS
• General parallel file system: high-performance shared-disk file system– For large computing clusters operating under Linux
• Uses data striping across all disks in cluster– A large-size block (strip) used to minimize seek overhead
during a file read/write• A smaller subblock is used for small files
– Locking used to maintain consistency of file data• Lock granularity is as coarse as possible, but as fine as
necessary• Centralized lock manager and few distributed lock
managers
![Page 24: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/24.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.24Operating Systems, by Dhananjay Dhamdhere 24
GPFS (continued)
• Notion of lock tokens to reduce latency and overhead of locking
• Race conditions may arise over metadata of a file– Solution: one of the nodes is designated as the metanode
for the file; it performs file updates
• Central allocation manager partitions free space map and gives one partition to each node
• Each node writes a separate journal for recovery• If network is partitioned, only nodes in the majority
partition can perform file processing at any time
![Page 25: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/25.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.25Operating Systems, by Dhananjay Dhamdhere 25
Windows
• Windows Server 2003 provides two features for data replication and data distribution:– Remote differential compression (RDC)
– DFS namespaces
• Replication organized using notion of a replication group
• DFS namespace is created by a system administrator• Other key concepts: referrals and hot standbys
![Page 26: Chapter 20 Distributed File Systems Copyright © 2008](https://reader030.vdocuments.us/reader030/viewer/2022032709/56649ecb5503460f94bd95b7/html5/thumbnails/26.jpg)
Operating Systems, by Dhananjay Dhamdhere Copyright © 2008 20.26Operating Systems, by Dhananjay Dhamdhere 26
Summary
• Transparency concerns association between path name of a file and location of the file
• File sharing semantics may differ between DFSs:– Unix semantics
– Session semantics
– Transaction semantics (atomic transactions)
• Stateless server design provides high availability– Notion of a hint used to improve performance
• DFS uses file caching to improve performance– Cache coherence techniques are needed