IO-Lite: A Unified Buffering and Caching System By Pai, Druschel, and Zwaenepoel (1999) Presented by Justin Kliger for CS780: Advanced Techniques in Caching Professor Zhang (Summer 2005)

Outline Problem & Significance Problem & Significance Literature Review Literature Review Proposed Solution Proposed Solution Design, Implementation, & Operation Design, Implementation, & Operation Experimental Design Experimental Design Results Results Conclusion Conclusion Further Research Further Research

The Problem The I/O subsystem and various applications all tend to use their own private I/O buffers The I/O subsystem and various applications all tend to use their own private I/O buffers Redundant data copying Redundant data copying Multiple buffering Multiple buffering Lack of cross-subsystem optimization Lack of cross-subsystem optimization

Problem’s Significance Wastes memory Wastes memory Reduces space Reduces space available for caching Causes higher cache Causes higher cache miss rates High CPU overhead Limits server throughput

Literature Review 1.POSIX I/O -Problem:double-buffering 2.Memory-mapped files (mmap) - Problem: - Problem: Not generalized to network I/O

Literature Review 3.Transparent Copy Avoidance -Problem: VM page alignment problems Copy-on-write faults Genie (emulated copy) Genie (emulated copy) Lack of full transparency leads to same problems 4.Copy Avoidance with Handoff Semantics -Problem: Lack of concurrent sharing reduces effectiveness

Literature Review 5.Fast buffers (fbufs) Designed by Druschel -Problem: Does not support filesystem access, or a file cache or a file cache 6.Extensible kernels -Problem: More overhead, not OS-portable

IO-Lite Solution Unified buffering and caching Unified buffering and caching Allow all applications and subsystems share the same buffered I/O data Allow all applications and subsystems share the same buffered I/O data Very simple at face value, Very simple at face value, very complex to implement

Basic Design Immutable buffers Immutable buffers Initial allocated data cannot be modified Initial allocated data cannot be modified Effectively read-only sharing Effectively read-only sharingAdvantages? Eliminates synchronization and protection problems Eliminates synchronization and protection problemsDisadvantages? I/O data cannot be modified in place I/O data cannot be modified in place

Further Design Considerations To make up for immutable buffers: Create buffer aggregate abstraction (an ADT) Create buffer aggregate abstraction (an ADT)mutable Reference to IO-Lite Window in VM Aggregates contain ordered list of form Aggregates contain ordered list of form Aggregates passed by value Aggregates passed by value Buffers passed by reference Buffers passed by reference

Further Design Considerations Buffer sharing must be concurrent Buffer sharing must be concurrent To achieve this, use similar method to fbufs To achieve this, use similar method to fbufs Expand to include the filesystem Expand to include the filesystem Adapts for general purpose OS Adapts for general purpose OS Worst case scenario (in terms of overhead): Worst case scenario (in terms of overhead): Page remapping Page remapping (when last buffer is allocated before first is deallocated) (when last buffer is allocated before first is deallocated)

IO-Lite Implementation New read & write API which supersedes the regular read & write New read & write API which supersedes the regular read & write size_t IOL_read(int fd, IOL_Agg **aggr, size_t size); size_t IOL_read(int fd, IOL_Agg **aggr, size_t size); size_t IOL_write(int fd, IOL_Agg *aggr); size_t IOL_write(int fd, IOL_Agg *aggr); IOL_Agg is buffer aggregate data type IOL_Agg is buffer aggregate data type Both operations are atomic Both operations are atomic

IO-Lite Implementation Applications: Recommends implementation in runtime I/O Libraries to avoid modifying all programs Recommends implementation in runtime I/O Libraries to avoid modifying all programsFilesystem: File cache data structure: File cache data structure: Network: Need to modify network device drivers to allow early demultiplexing (using a packet filter) Need to modify network device drivers to allow early demultiplexing (using a packet filter)

IO-Lite Operation With regards to the cache: With regards to the cache: Cache replacement basically LRU Cache replacement basically LRU Allows for application customization Allows for application customization Cache eviction controlled by VM daemon Cache eviction controlled by VM daemon Do >½ replaced pages contain I/O data? Do >½ replaced pages contain I/O data?

IO-Lite Operation Impact of immutable buffers: Impact of immutable buffers: Case 1: Entire object is modified Case 1: Entire object is modified Lack of in-place modification has no ill effect Lack of in-place modification has no ill effect Case 2: Subset of object needs to be modified Case 2: Subset of object needs to be modified Rather than recopy entire object, use chaining Rather than recopy entire object, use chaining Performance loss is small if blocks are localized Performance loss is small if blocks are localized Case 3: Scattered subset needs modification Case 3: Scattered subset needs modification IO-Lite incorporates mmap interface for this IO-Lite incorporates mmap interface for this

Experimental Design Compared: Apache Apache Widely used web server Flash (event-driven HTTP server) Flash (event-driven HTTP server) Designed by authors in previous year Flash-Lite (Flash modified to use IO-Lite API) Flash-Lite (Flash modified to use IO-Lite API) New design by authors

Experimental Design 1.General: varied requested file size 40 requests for same file 40 requests for same file File size ranged from 500 bytes – 200 Kbytes File size ranged from 500 bytes – 200 Kbytes 2.Persistent connections Reduces overhead Reduces overhead 3.CGI Additional I/O traditionally slows servers Additional I/O traditionally slows servers

Experimental Design 4.Real workloads Shows performance benefits by allowing more space for caching Shows performance benefits by allowing more space for caching Based on Rice’s CSCI department logs Based on Rice’s CSCI department logs 5.Wide Area Network (WAN) Test throughput with slow clients connecting Test throughput with slow clients connecting 6.Applications Incorporated API into UNIX programs Incorporated API into UNIX programs

Results 1.General test: Bandwidth increase of 43% over Flash, 137% over Apache Bandwidth increase of 43% over Flash, 137% over Apache No real difference for files less than 5KBytes No real difference for files less than 5KBytes

Results 2.Persistent Connections Flash-Lite even more effective at smaller file sizes Flash-Lite even more effective at smaller file sizes 3.CGI All servers slow, but Flash-Lite still much better All servers slow, but Flash-Lite still much better 4.Real workload Flash-Lite throughput 65% greater than Apache Flash-Lite throughput 65% greater than Apache 5.WAN Flash-Lite does not suffer from slow clients Flash-Lite does not suffer from slow clients 6.Applications Varied improvement for all programs tested Varied improvement for all programs tested

Conclusion IO-Lite consistently improved performance in all contexts tested IO-Lite consistently improved performance in all contexts tested Requires modification to numerous libraries and network device drivers EG: see Peng, Sharma, & Chiueh (2003) EG: see Peng, Sharma, & Chiueh (2003)

Further Research There have been 42 citations Almost all fell between Almost all fell between Authors have not written any follow-ups Authors have not written any follow-ups Lack of papers that involve implementation of IO-Lite or a variation of it Probably because of complexity and number of modifications that are necessary Probably because of complexity and number of modifications that are necessary

Appendix: Figures 2) 4) 5) 3) 6)

Questions?