x-ray: a non-invasive exclusive caching mechanism for raids lakshmi n. bairavasundaram muthian...
TRANSCRIPT
![Page 1: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/1.jpg)
X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs
Lakshmi N. Bairavasundaram
Muthian Sivathanu
Andrea C. Arpaci-Dusseau
Remzi H. Arpaci-Dusseau
ADvanced Systems Laboratory
Computer Sciences Department
University of Wisconsin – Madison
![Page 2: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/2.jpg)
Introduction Caching in modern systems
Multiple levels Storage: 2-level hierarchy
Level 1: File system (FS) cache Software-managed Main memory of host/client LRU-like cache replacement
Level 2: RAID cache Firmware-managed Memory inside RAID system Usually LRU replacement .......
File system cache
RAID cache
RAID
Application
Host
![Page 3: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/3.jpg)
Introduction – contd. LRU
Replace LRU block Cache placement on read
Read Block no. 10
LRU MRU
Read Block no. 10
39 …….. 4523 10…….. 4523
![Page 4: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/4.jpg)
Introduction – contd. LRU
Replace LRU block Cache placement on read
2 levels of LRU Redundant contents
……..
……..
Read Block no. 10
Read Block no. 10
Read Block no. 10
10
10
MRU
MRU
LRU
LRU10
LRU 10 MRU
MRULRU
11
11
12
12
….
….
FS Cache
RAID Cache
![Page 5: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/5.jpg)
Introduction – contd. LRU
Cache placement on read Replace LRU block
2 levels of LRU Redundant contents
Goal: Exclusive caching 10
LRU 10 MRU
MRULRU
11
11
12
12
….
….
FS Cache
RAID Cache
![Page 6: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/6.jpg)
Improved RAID Caching Multi-Queue (Zhou et al. 2001)
Add frequency component to cache policy Not strictly exclusive!
DEMOTE (Wong and Wilkes 2002) Change interface to disk File system issues “cache place” command Has perfect information and hence perfectly exclusive caches Interface changes – difficult to deploy
![Page 7: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/7.jpg)
Ideal RAID Cache Exclusive caching
File system and RAID caches should have different contents
Global LRU Known to work well RAID cache should be a victim cache
No interface changes….
……
FS Cache
RAID Cache Block ReadVictim Block
LRU
MRU
![Page 8: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/8.jpg)
X-RAY Observes disk traffic
Reads and writes to data and metadata
Builds a model of the FS cache Uses semantic knowledge Predicts size and contents of FS cache
Identifies set of exclusive blocks Recent victims of the FS cache
Reads blocks from disk into cache Result
A nearly exclusive cache without interface changes
File system cache
RAID cache
RAID
Host
Model of FS cacheX-RAY
![Page 9: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/9.jpg)
Talk Outline Introduction File Systems Information and Inferences X-RAY Cache Design Results Conclusion
![Page 10: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/10.jpg)
File System Operation Applications perform file reads and writes File system (Unix)
Translates file accesses to disk block requests Metadata
To maintain application data on disk and manage disk blocks Periodically written to disk Examples: inodes, bitmap blocks
![Page 11: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/11.jpg)
File System Operation Inode
Pointers to data blocks File access information
Inode
Data Blocks
Latest access time
Pointers to data blocks
File
![Page 12: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/12.jpg)
File System Operation File access
Use inode to obtain pointers to disk data blocks Read corresponding blocks from disk if they are not in FS cache Update the access time information in inode
Metadata updates Periodically check for “dirty” inodes and write to disk
![Page 13: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/13.jpg)
The Problem To observe disk traffic and infer
the contents of FS cache Why difficult?
FS cache size changes over time Shares main memory with virtual
memory system
![Page 14: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/14.jpg)
The Problem To observe disk traffic and infer
the contents of FS cache Why difficult?
FS cache size changes over time Disk cannot observe all FS-level
accesses
Read block:
10
10
Disk Read
11
11
101112
12
12
LRU
LRU
MRU
MRU
FS Cache
FS Cache Model
RAID
![Page 15: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/15.jpg)
The Problem To observe disk traffic and infer
the contents of FS cache Why difficult?
FS cache size changes over time Disk cannot observe all FS-level
accesses
Read block:
10
10
Disk Read
11
11
10
12
12
LRU
LRU
MRU
MRU
13
FS Cache
FS Cache Model
RAID
![Page 16: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/16.jpg)
The ProblemRead block:
10
11 12
12 13
13
LRU
LRU
MRU
MRU
FS Cache
FS Cache Model
RAID
To observe disk traffic and infer the contents of FS cache
Why difficult? FS cache size changes over time Disk cannot observe all FS-level
accesses
![Page 17: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/17.jpg)
The ProblemRead block:
10
11 12
12 13
13
LRU
LRU
MRU
MRU
FS Cache
FS Cache Model
RAID
To observe disk traffic and infer the contents of FS cache
Why difficult? FS cache size changes over time Disk cannot observe all FS-level
accesses
Key observation We need information about
accesses that hit in FS cache File system maintains access
information in inodes
![Page 18: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/18.jpg)
Talk Outline Introduction File Systems Information and Inferences X-RAY Cache Design Results Conclusion
![Page 19: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/19.jpg)
Information Obtain information from observing disk traffic Knowledge of file system structures and operations
File system maintains time of last access in inodes Periodic inode writes Assuming whole file access, all blocks are in FS cache
Assume file system cache policy is LRU
![Page 20: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/20.jpg)
Inferences Read for data block
Block will be placed in file system cache (MRU block)
Read for previously read data block Block became victim in file system cache Blocks with an earlier access time should also be victims
Inode write: new access time , no disk read observed All blocks belonging to file are in FS cache Other blocks with later access time should also be present
![Page 21: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/21.jpg)
Talk Outline Introduction File Systems Information and Inferences X-RAY Cache Design Results Conclusion
![Page 22: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/22.jpg)
Design
Recency list (R-list) List of data blocks ordered
by access time Cache Begin (CB) pointer
Divides R-list into inclusive and exclusive regions
RAID Cache contents Subset of blocks in exclusive
region
LRU MRUA, 1 B, 1 D, 3C, 2 F, 5E, 3
CBInclusive regionExclusive region
Block number Access time
Blocks the RAID should cache
Blocks expected to be in FS cache
![Page 23: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/23.jpg)
Disk Read
LRU MRUA , 1 B , 1 C , 2 D , 3 E , 3 F , 4
CBInclusive regionExclusive region
Read Block ‘D’ ; time = 6
![Page 24: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/24.jpg)
Disk Read
LRU MRUA , 1 B , 1 C , 2 D , 3 E , 3 F , 4
CBInclusive regionExclusive region
Read Block ‘D’ ; time = 6
![Page 25: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/25.jpg)
Disk Read
LRU MRUA , 1 B , 1 C , 2 D , 6E , 3 F , 4
CBInclusive regionExclusive region
Read Block ‘D’ ; time = 6
![Page 26: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/26.jpg)
Inode Write – Access time change
LRU MRUA , 1 B , 1 C , 2 D , 3 E , 4 F , 5
CBInclusive regionExclusive region
G , 7
Inode “23” : access time = 6
Semantic knowledge Inode “23” == blocks D & E
Blocks D, E : access time = 6
![Page 27: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/27.jpg)
Inode Write – Access time change
LRU MRUA , 1 B , 1 C , 2 D , 3 E , 4 F , 5
CBInclusive regionExclusive region
G , 7
Blocks D, E : access time = 6Inode “23” : access time = 6
![Page 28: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/28.jpg)
Inode Write – Access time change
LRU MRUA , 1 B , 1 C , 2 F , 5
D , 6 E , 6
CBInclusive regionExclusive region
G , 7
Blocks D, E : access time = 6Inode “23” : access time = 6
![Page 29: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/29.jpg)
X-RAY Cache
LRU MRUA , 1 B , 1 C , 2 F , 5 D , 6 E , 6
CBInclusive regionExclusive region
G , 7
RAID Cache (size = 2 blocks)
Keep track of additions to window in exclusive region
![Page 30: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/30.jpg)
X-RAY Cache
Read newly-added blocks from disk Replace blocks no longer in the window Additional disk bandwidth
Idle time, extra internal bandwidth, freeblock scheduling
LRU MRUA , 1 B , 1 C , 2 F , 5 D , 6 E , 6
CBInclusive regionExclusive region
G , 7
RAID Cache (size = 2 blocks)
![Page 31: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/31.jpg)
Talk Outline Introduction File Systems Information and Inferences X-RAY Cache Design Results
Tracking FS Cache Contents RAID Cache Performance
Conclusion
![Page 32: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/32.jpg)
Results – Tracking Accurate size and content prediction Highly responsive to FS cache size changes Tolerates changes in inode write interval Partial file reads
X-RAY performs well if percentage of partially accessed files is < 40% (typical traces have less than 30%)
![Page 33: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/33.jpg)
Results – Cache Performance
Performs better than LRU and Multi-Queue
Close to DEMOTE, in spite of imperfect information
Hit rate advantage translates to lower read latency
![Page 34: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/34.jpg)
Additional Results File system cache policy is not LRU
Clock, 2Q X-RAY performs nearly as well as before It performs better than both LRU and Multi-Queue
Idle time requirements X-RAY reads blocks into cache only during idle time It performs well if idle time is greater than one-third of actual idle time
observed in the trace
More in the paper …
![Page 35: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/35.jpg)
Conclusion Easy deployment is an important goal in developing technology
Avoid interface changes – use non-invasive mechanisms
Higher-level systems maintain various pieces of information about data they manage Provide low-level systems with basic semantic knowledge
Semantic intelligence for managing RAID caches Use access information in metadata to track file system cache contents
and cache exclusive blocks In spite of imperfect information, X-RAY performs nearly as well as
changing the interface
Semantically-smart Disk Systems Availability, security and performance improvements
![Page 36: X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs Lakshmi N. Bairavasundaram Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau](https://reader035.vdocuments.us/reader035/viewer/2022081501/56649ef05503460f94bfff16/html5/thumbnails/36.jpg)
Questions ?
ADvanced Systems Laboratory (ADSL)
Computer Sciences, University of Wisconsin-Madison
http://www.cs.wisc.edu/adsl