IO-Lite:IO-Lite:A Unified Buffering and Caching SystemA Unified Buffering and Caching System
By Pai, Druschel, and Zwaenepoel (1999)By Pai, Druschel, and Zwaenepoel (1999)
Presented by Justin KligerPresented by Justin Kligerfor CS780: Advanced Techniques in Cachingfor CS780: Advanced Techniques in Caching
Professor Zhang (Summer 2005)Professor Zhang (Summer 2005)
OutlineOutline
Problem & SignificanceProblem & Significance Literature ReviewLiterature Review Proposed Solution Proposed Solution Design, Implementation, & OperationDesign, Implementation, & Operation Experimental Design Experimental Design ResultsResults Conclusion Conclusion Further ResearchFurther Research
The ProblemThe Problem
The I/O subsystem and various The I/O subsystem and various applications all tend to use their own applications all tend to use their own private I/O buffersprivate I/O buffers
Redundant data copyingRedundant data copying
Multiple bufferingMultiple buffering
Lack of cross-subsystem optimizationLack of cross-subsystem optimization
Problem’s SignificanceProblem’s Significance
Wastes memoryWastes memory
Reduces space Reduces space available for cachingavailable for caching
Causes higher cache Causes higher cache miss ratesmiss rates
•High CPU overhead
Limits server throughput
Literature ReviewLiterature Review
1.1. POSIX I/OPOSIX I/O-Problem: -Problem:
double-bufferingdouble-buffering
2.2. Memory-mapped files (Memory-mapped files (mmapmmap))
--Problem:Problem:Not generalized to network I/ONot generalized to network I/O
Literature ReviewLiterature Review
3.3. Transparent Copy AvoidanceTransparent Copy Avoidance-Problem:-Problem:
VM page alignment problemsVM page alignment problems
Copy-on-write faultsCopy-on-write faults Genie (emulated copy)Genie (emulated copy)
Lack of full transparency leads to same problemsLack of full transparency leads to same problems
4.4. Copy Avoidance with Handoff SemanticsCopy Avoidance with Handoff Semantics-Problem:-Problem:
Lack of concurrent sharing reduces effectivenessLack of concurrent sharing reduces effectiveness
Literature ReviewLiterature Review
5.5. Fast buffers (Fast buffers (fbufsfbufs))Designed by DruschelDesigned by Druschel
-Problem:-Problem:Does not support filesystem access,Does not support filesystem access,
or a file cacheor a file cache
6.6. Extensible kernelsExtensible kernels-Problem:-Problem:
More overhead, not OS-portableMore overhead, not OS-portable
IO-Lite SolutionIO-Lite Solution
UnifiedUnified buffering and caching buffering and caching
Allow all applications and subsystems share Allow all applications and subsystems share the same buffered I/O datathe same buffered I/O data
Very simple at face value, Very simple at face value,
very complex to implementvery complex to implement
Basic DesignBasic Design
Immutable buffersImmutable buffers Initial allocated data cannot be modifiedInitial allocated data cannot be modified Effectively read-only sharingEffectively read-only sharing
Advantages?Advantages? Eliminates synchronization and protection Eliminates synchronization and protection
problemsproblems
Disadvantages?Disadvantages? I/O data cannot be modified in placeI/O data cannot be modified in place
Further Design ConsiderationsFurther Design Considerations
To make up for immutable buffers:To make up for immutable buffers: Create buffer aggregate abstraction (an ADT)Create buffer aggregate abstraction (an ADT)
mutablemutable
Reference to Reference to IO-Lite WindowIO-Lite Window in VM in VM
Aggregates contain ordered list of form <address, length>Aggregates contain ordered list of form <address, length> Aggregates passed by valueAggregates passed by value Buffers passed by referenceBuffers passed by reference
Further Design ConsiderationsFurther Design Considerations
Buffer sharing must be Buffer sharing must be concurrentconcurrent
To achieve this, use similar method to To achieve this, use similar method to fbufsfbufs Expand to include the filesystemExpand to include the filesystem
Adapts for general purpose OSAdapts for general purpose OS
Worst case scenario (in terms of overhead):Worst case scenario (in terms of overhead): Page remapping Page remapping
(when last buffer is allocated before first is deallocated)(when last buffer is allocated before first is deallocated)
IO-Lite ImplementationIO-Lite Implementation
New read & write API which supersedes New read & write API which supersedes the regular read & write the regular read & write
size_t IOL_read(int fd, IOL_Agg **aggr, size_t size);size_t IOL_read(int fd, IOL_Agg **aggr, size_t size); size_t IOL_write(int fd, IOL_Agg *aggr);size_t IOL_write(int fd, IOL_Agg *aggr);
IOL_Agg is buffer aggregate data typeIOL_Agg is buffer aggregate data type
Both operations are Both operations are atomicatomic
IO-Lite ImplementationIO-Lite Implementation
Applications:Applications: Recommends implementation in runtime I/O Libraries Recommends implementation in runtime I/O Libraries
to avoid modifying all programsto avoid modifying all programs
Filesystem:Filesystem: File cache data structure: <file-id, offset, length>File cache data structure: <file-id, offset, length>
Network:Network: Need to modify network device drivers to allow Need to modify network device drivers to allow early early
demultiplexingdemultiplexing (using a packet filter) (using a packet filter)
IO-Lite OperationIO-Lite Operation
With regards to the cache:With regards to the cache:
Cache replacement basically LRUCache replacement basically LRU Allows for application customizationAllows for application customization
Cache eviction controlled by VM daemonCache eviction controlled by VM daemon Do >½ replaced pages contain I/O data?Do >½ replaced pages contain I/O data?
IO-Lite OperationIO-Lite Operation
Impact of immutable buffers:Impact of immutable buffers:
Case 1: Entire object is modifiedCase 1: Entire object is modified Lack of in-place modification has no ill effectLack of in-place modification has no ill effect
Case 2: Subset of object needs to be modifiedCase 2: Subset of object needs to be modified Rather than recopy entire object, use Rather than recopy entire object, use chainingchaining
Performance loss is small if blocks are localizedPerformance loss is small if blocks are localized
Case 3: Scattered subset needs modificationCase 3: Scattered subset needs modification IO-Lite incorporates IO-Lite incorporates mmapmmap interface for this interface for this
Experimental DesignExperimental Design
Compared:Compared:
Apache 1.3.1Apache 1.3.1Widely used web serverWidely used web server
Flash (event-driven HTTP server)Flash (event-driven HTTP server)Designed by authors in previous yearDesigned by authors in previous year
Flash-Lite (Flash modified to use IO-Lite API)Flash-Lite (Flash modified to use IO-Lite API)New design by authorsNew design by authors
Experimental DesignExperimental Design
1.1. General: varied requested file sizeGeneral: varied requested file size 40 requests for same file40 requests for same file File size ranged from 500 bytes – 200 KbytesFile size ranged from 500 bytes – 200 Kbytes
2.2. Persistent connectionsPersistent connections Reduces overheadReduces overhead
3.3. CGICGI Additional I/O traditionally slows serversAdditional I/O traditionally slows servers
Experimental DesignExperimental Design
4.4. Real workloadsReal workloads Shows performance benefits by allowing more Shows performance benefits by allowing more
space for cachingspace for caching Based on Rice’s CSCI department logsBased on Rice’s CSCI department logs
5.5. Wide Area Network (WAN)Wide Area Network (WAN) Test throughput with 0-256 slow clients connectingTest throughput with 0-256 slow clients connecting
6.6. ApplicationsApplications Incorporated API into UNIX programsIncorporated API into UNIX programs
ResultsResults
1.1. General test:General test: Bandwidth increase of 43% over Flash, 137% over ApacheBandwidth increase of 43% over Flash, 137% over Apache No real difference for files less than 5KBytesNo real difference for files less than 5KBytes
ResultsResults
2.2. Persistent ConnectionsPersistent Connections Flash-Lite even more effective at smaller file sizesFlash-Lite even more effective at smaller file sizes
3.3. CGICGI All servers slow, but Flash-Lite still much betterAll servers slow, but Flash-Lite still much better
4.4. Real workloadReal workload Flash-Lite throughput 65% greater than ApacheFlash-Lite throughput 65% greater than Apache
5.5. WANWAN Flash-Lite does not suffer from slow clientsFlash-Lite does not suffer from slow clients
6.6. ApplicationsApplications Varied improvement for all programs testedVaried improvement for all programs tested
ConclusionConclusion
IO-Lite consistently improved IO-Lite consistently improved performance in all contexts testedperformance in all contexts tested
Requires modification to numerous Requires modification to numerous libraries and network device driverslibraries and network device drivers EG: see Peng, Sharma, & Chiueh (2003)EG: see Peng, Sharma, & Chiueh (2003)
Further ResearchFurther Research
There have been 42 citations There have been 42 citations Almost all fell between 2001-2003Almost all fell between 2001-2003
Authors have not written any follow-upsAuthors have not written any follow-ups
Lack of papers that involve implementation Lack of papers that involve implementation of IO-Lite or a variation of itof IO-Lite or a variation of it Probably because of complexity and number Probably because of complexity and number
of modifications that are necessaryof modifications that are necessary