Page Cache and Page Writeback Chris Gill, Brian Kocoloski CSE 422S - Operating Systems Organization Washington University in St. Louis St. Louis, MO 63130

System Architecture struct page * pages CPU DRAM System Bus Peripheral Devices (hard drives, keyboards, adapters, etc. Bridge I/O buses CSE 422S – Operating Systems Organization

Main memory (RAM) Characteristics Fast Limited Capacity CPU DRAM System Bus Peripheral Devices (hard drives, keyboards, adapters, etc. Secondary memory (e.g., hard drive) Characteristics Slow Large Capacity Persistent Bridge I/O buses CSE 422S – Operating Systems Organization

Hardware Performance Differences Performance comparison of most block devices with other hardware Reading/writing a CPU register: O(nanoseconds) Access memory in CPU cache: DRAM memory access: O(microseconds) Access to spinning hard drive: O(milliseconds) CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization I/O Without Caching Read/write block to/from I/O device Without any caching, all reads/writes go to a (slow) storage device along a (relatively slow) I/O bus Disk reads on the order of ms in time (~ 2 orders of magnitude slower than main memory) CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization I/O With Caching Page Read/write block to/from I/O device Read block from storage Allocate page of memory Copy block contents into the page Store page in the page cache Memory write Disk read CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization I/O With Caching Page On subsequent read/write of the same block Is the block located somewhere in the page cache? If yes, read/write directly from memory Else, read/write from storage Memory write Disk read CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization Page Cache Simple high level idea: save file contents in memory for faster future access Q: does the overhead of checking the page cache matter? Do we care about the overhead for when the cache misses? What about when the cache hits? Q: what other questions does this create? CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization Page Cache Simple high level idea: save file contents in memory for faster future access Q: does the overhead of checking the page cache matter? Q: what other questions does this create? How large should the cache be? How to maintain consistency between memory and disk? How frequently should cache/disk contents be synchronized? What should happen on writes? CSE 422S – Operating Systems Organization

Cache Size and Consistency Size of the page cache grows/shrinks dynamically based on system workload Only data that have been accessed are mapped Page cache may hold different portions of different files Write caching Writes may bypass the cache entirely (so-called “no-write” approach) Writes may update both page cache and disk (“write-through” approach) Writes may update page cache and mark as “dirty” and then written back to disk later (“write-back” approach) Allows requests to be merged and sorted, per previous lecture CSE 422S – Operating Systems Organization

Write Caching Approaches No-write Only makes sense if you know the data will not be read/written again in the future Write-through Writes to the page cache are immediately written through to storage Benefit: keeps cache coherent with physical storage Downside: all writes to storage issued independently Write-back Writes to the page cache are NOT written to the backing store immediately Downside: inconsistency between memory and storage contents (what happens if the system crashes?) Benefit: multiple writes to backing store can be coalesced and performed in bulk, which the block layer can handle more efficiently (via I/O scheduling approaches) CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization Cache Eviction Clean vs dirty If not enough memory available, Linux replaces clean page(s) first If no clean pages are available, Linux forces a writeback of a dirty page, and then replaces that page Eviction Clean page; no writeback needed Dirty page; ”dirty” contents need to be written back to persistent storage, then the page can be released Eviction Policy Assuming all pages are dirty, what should be evicted? CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization Linux’ Cache Eviction Cache eviction LRU strategy: if we need to evict, find the least recently used clean page, and remove it from the cache; no writeback needed Clean Pages Dirty Pages Sorted by LRU Sorted by LRU CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization Linux’ Cache Eviction Cache eviction LRU strategy: if we need to evict, find the least recently used clean page, and remove it from the cache; no writeback needed Clean Pages Dirty Pages Sorted by LRU Sorted by LRU CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization Linux’ Cache Eviction Cache eviction If no clean pages are available, Linux forces a writeback of a dirty page, and frees the page or marks it clean Clean Pages Dirty Pages CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization Linux’ Cache Eviction Cache eviction If no clean pages are available, Linux forces a writeback of a dirty page, and frees the page or marks it clean Clean Pages Dirty Pages CSE 422S – Operating Systems Organization

CSE 422S – Operating Systems Organization Cache Eviction Least recently used strategy (LRU) Evicts page with the oldest access timestamp Based on idea that “least recently used” is a good approximation for “least likely to be used in the near future” An effective heuristic (Linux isn’t really clairvoyant) Linux current strategy: Two-list strategy (LRU/2) Addresses issue where file contents are only ever read once Linux modifies LRU slightly by keeping two separate lists Pages move from inactive list to active one when they are accessed Within each list, pages are ordered by access timestamp If active list is larger than inactive one, older pages move to inactive CSE 422S – Operating Systems Organization

Linux’ Cache Eviction Two-List LRU Active List (clean, but cannot be evicted) Inactive List (clean, and can be evicted) Considered “hot” Considered “cold” CSE 422S – Operating Systems Organization

Linux’ Cache Eviction Clean Pages Page accessed Active List (clean, but cannot be evicted) Inactive List (clean, and can be evicted) Page accessed CSE 422S – Operating Systems Organization

Linux’ Cache Eviction Clean Pages Page accessed (moves to active list) Active List (clean, but cannot be evicted) Inactive List (clean, and can be evicted) Page accessed (moves to active list) CSE 422S – Operating Systems Organization

Reading file contents into memory Page Memory write Q: What is the Linux system call we’ve used that performs very similar operations? Disk read CSE 422S – Operating Systems Organization