energy-efficiency and storage flexibility in the blue file system e. b. nightingale and j. flinn...
TRANSCRIPT
![Page 1: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/1.jpg)
ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY
IN THE BLUE FILE SYSTEM
E. B. Nightingale and J. FlinnUniversity of Michigan
![Page 2: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/2.jpg)
Key ideas
• Reduce power consumption of portable devices– Two big culprits
• Disk drive• Wireless
• Integrate flash drives
![Page 3: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/3.jpg)
Dynamic storage hierarchy
• We should power down drives when they are idle– But this causes powering up delays
• To mask these delays introduce a dynamic storage hierarchy – Includes flash drives and remote server– Takes into consideration state of local disk drive
![Page 4: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/4.jpg)
Examples
• Disk is powered up • Disk is powered down
Flash
Disk
Server
Flash
Server
Disk
The storage hierarchy depends on the status of the hard disk.
![Page 5: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/5.jpg)
Incorporating flash drives
• To satisfy read requests, Blue FS looks first in the flash drive
• Implies– A write all/read any policy– Treating flash drives as caches
• Need a quick way to query flash drive contents
![Page 6: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/6.jpg)
More on caches
• Unit of caching is block
• Can cache some blocks of a file but not all– Must maintain information on validity and
currency of each cached block• See details in paper
![Page 7: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/7.jpg)
Write to all/read any
• All writes are propagated to all devices
Disk
Server
![Page 8: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/8.jpg)
Advantages
• Write all allows read any– Future power savings
• Requires efficient writes– Propagates all updates to the server
• Unless in disconnected mode
![Page 9: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/9.jpg)
Aggregating writes (I)
• Blue FS maintains one write queue per device
Disk
Server
![Page 10: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/10.jpg)
Aggregating writes (II)
• When it writes to a device, it flushes the whole queue
Disk
Server
![Page 11: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/11.jpg)
Aggregating writes (III)
• After the writes, each queue is empty
Disk
Server
![Page 12: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/12.jpg)
More on device queues
• Device queues share among themselves all common blocks– Save space
• Blocks in device queues can be accessed through a hashing scheme– Always access most recent version of a block
even when it is not yet saved on any device
![Page 13: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/13.jpg)
Why?
• Makes a lot of sense for disk drive and remote server– Power up disk, do a few updates, then power
down– Power up wireless connection, send a few
updates then power down• Delaying updates allows BlueFS to coalesce
multiple updates to the same block
![Page 14: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/14.jpg)
Using flash drives as caches
• Small flash drives cannot contain whole contents of file system– Especially true in 2004!
• Must have a fast way to find whether a file is cached or not– Enode
![Page 15: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/15.jpg)
E-node
• Captures all the information Blue FS has about the validity of an object
• E-nodes are– Hashed by file id– Stored in an e-node cache managed by LRU
replacement policy• Default size is one MB
![Page 16: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/16.jpg)
Cache consistency
• Uses– Optimistic replica control (Coda)– Callbacks (AFS)
• Changes– Blue FS maintains callbacks on a per-device
basis, instead of on a per-client basis– Server queues invalidation messages when
a device is disconnected.
![Page 17: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/17.jpg)
Optimistic replication
• Optimistic replication control protocols allow access in disconnected mode– Tolerate temporary inconsistencies– Promise to detect them later– Provide much higher data availability
Updated forFall 2012
Optimistic replication control putsdata availability above data consistency
![Page 18: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/18.jpg)
Callbacks (I)
• When a client opens a file for the first time, server promises to notify it whenever it receives a new version of the file from any other client– Promise is called a callback
• Relieves the server from having to answer a call from the client every time the file is opened– Significant reduction of server workload
Updated forFall 2012
![Page 19: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/19.jpg)
If machines could talk (I)
Client
I should contact the server each time I open
any file
Server
99% of these requests are useless!
![Page 20: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/20.jpg)
If machines could talk (II)
Client
??
Server
You do not need to call me:I promise to call you back if anyone has modified the file
![Page 21: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/21.jpg)
If machines could talk (III)
Client
What if I do not receive your
callback?
Server
Though luck!
![Page 22: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/22.jpg)
Callbacks (II)
• Callbacks can be lost!
– Client will call the server every tauminutes to check whether it received all callbacks it should have received
– Cached copy is only guaranteed to reflect the state of the server copy up to tau minutes before the time the client opened the file for the last time
![Page 23: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/23.jpg)
If machines could talk (IV)
Client
I will contact you from time to time to check
for lost callbacks
Server
Sure as long as you do not do it too frequently
![Page 24: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/24.jpg)
Disconnected mode
• Like Coda, Blue FS works in disconnected mode
• User has access to all files cached– On local disk drive– On flash drive if any
• Can even specify that some files must always be cached– Affinity
![Page 25: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/25.jpg)
Disconnected mode (II)
• When a device is disconnected, server queues all callbacks that could not be delivered to the device– Speeds up reintegration
• If inconsistencies are discovered at reconnection time, will run a resynchronization process
![Page 26: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/26.jpg)
Accessing large files
• When disk is powered down, Blue FS get first file blocks from remote server
![Page 27: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/27.jpg)
Deciding which device to use
• If decisions are made for each individual access– Will never activate a disk that’s powered down
• Blue FS uses ghost hints– Measure penalty for not using a disk that is
powered down for a given access– When sum of ghosts hints exceeds the cost of
powering up the disk, disk is powered up
![Page 28: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/28.jpg)
Blue FS architecture (II)
Applications
BlueFS KernelModule
Wolverine
Linux Kernel
Linux File Cache
USBStick
CardFlash
Micro-drive
Local Disk
To BlueFS Server
VFS Operatio
ns
Up-call
Updated forFall 2012
![Page 29: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/29.jpg)
Blue FS architecture (I)
• Most of Blue FS functionality is implemented in a user-level process – Wolverine
• Kernel-resident part of Blue FS intercepts I/O calls that have to be forwarded to Wolverine
![Page 30: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/30.jpg)
Blue FS architecture (II)
Applications
BlueFS KernelModule
Wolverine
Linux Kernel
Linux File Cache
USBStick
CardFlash
Micro-drive
Local Disk
To BlueFS Server
VFS Operations
Up-call
![Page 31: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/31.jpg)
Performance
• Will use a standard benchmark– Andrews/AFS benchmark
• Will compare interactive delays andpower consumption against– NFS – Coda with synchronous server updates (Coda)– Coda with asynchronous server updates
(Coda WD)
![Page 32: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/32.jpg)
Interactive delay
Ten times faster than NFS, 16% faster than Coda WD
6000.0
![Page 33: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/33.jpg)
Adding a 16MB flash drive
• Assumes that cache is fully loaded• 48% shorter interactive delay 48%, 25% less energy
0
20
40
60
80
100
120
140
160
180
200
0 ms delay 30 ms delay
Inte
ract
ive
Del
ay (
seco
nd
s)
Coda BlueFS BlueFS w/ Flash
0
100
200
300
400
500
600
700
800
0 ms delay 30 ms delayD
FS
En
erg
y (J
ou
les)
![Page 34: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/34.jpg)
Energy efficiency
• Different benchmark run with half-full cache• 76% shorter interactive delay, 55% less energy than Coda
0
200
400
600
800
1000
1200
1400
1600
0 ms delay 30 ms delayD
FS
En
erg
y (J
ou
les)
Coda
BlueFS
0
100
200
300
400
500
600
700
800
0 ms delay 30 ms delay
Inte
rac
tiv
e D
ela
y (
se
co
nd
s)
![Page 35: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/35.jpg)
Adding a 16MB flash drive
• Assumes that cache is fully loaded• 48% shorter interactive delay, 25% less energy
0
20
40
60
80
100
120
140
160
180
200
0 ms delay 30 ms delay
Inte
ract
ive
Del
ay (
seco
nd
s)
Coda BlueFS BlueFS w/ Flash
0
100
200
300
400
500
600
700
800
0 ms delay 30 ms delayD
FS
En
erg
y (J
ou
les)
![Page 36: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/36.jpg)
Discussion
• Comparison with NFS is unfair– NFS was designed for LANs, not WANs!
• Comparison with Coda is fair but– Coda did not incorporate flash drives
• Did not exist then!– Coda did not try to minimize power
consumption• Portable devices did not exist then!
![Page 37: ENERGY-EFFICIENCY AND STORAGE FLEXIBILITY IN THE BLUE FILE SYSTEM E. B. Nightingale and J. Flinn University of Michigan](https://reader035.vdocuments.us/reader035/viewer/2022070412/5697bf8b1a28abf838c8b47b/html5/thumbnails/37.jpg)
Conclusion
• Blue FS updates Coda by taking into consideration the arrival of– Wireless networks (near ubiquitous access)– Portable devices (limited autonomy)– Flash drives
• Retains Coda features such as optimistic replication, callbacks and disconnected mode operation