1. Caching for File Systems Andy Wang COP 5611 Advanced Operating Systems

2. Caching for File Systems  Conventional role of caching Performance improvement Assumptions  Locality  Scarcity of RAM  Shifting role of caching Shaping disk access patterns Assumptions  Locality  Abundance of RAM

3. Performance Improvement  Essentially all file systems rely on caching to achieve acceptable performance  Goal is to make FS run at the memory speeds Even though most of the data is on disk

4. Issues in I/O Buffer Caching  Cache size  Cache replacement policy  Cache write handling  Cache-to-process data handling

5. Cache size  The bigger, the fewer the cache misses  More data to keep in sync with disk

6. What if….  RAM size = disk size?  What are some implications in terms of disk layouts? Memory dump? LFS layout?

7. What if….  RAM is big enough to cache all hot files  What are some implications in terms of disk layouts? Optimized for the remaining files

8. Cache Replacement Policy  LRU works fairly well Can use “stack of pointers” to keep track of LRU info cheaply Need to watch out for cache pollutions  LFU doesn’t work well because a block may get lots of hits, then not be used So, it takes a long time to get it out

9. What is the optimal policy?  MIN: Replacing a page that will not be used for the longest time… Hmm…

10. What if your goal is to save power?  Option 1: MIN replacement RAM will cache the hottest data items Disks will achieve maximum idleness… Hmm…

11. What if you have multiple disks?

12. And access patterns are skewed Access patterns

13. Better Off Caching Cold Disks Access patterns Spin down cold disks

14. Handling Writes to Cached Blocks  Write-through cache: update propagate through various levels of caches immediately  Write-back cache: delayed updates to amortize the cost of propagation

15. What if….  Multiple levels of caching with different speeds and sizes?  What are some tricky performance behaviors?

16. istory’s Mystery Puzzling Conquest Microbenchmark Numbers… Geoff Kuenning: “If Conquest is slower than ext2fs, I will toss you off of the balcony…”

17. With me hanging off a balcony…  Original large-file microbenchmark: one 1-MB file (Conquest in-core file)

18. Odd Microbenchmark Numbers  Why are random reads slower than sequential reads?

19. Odd Microbenchmark Numbers  Why are RAM-based FSes slower than disk-based FSes?

20. A Series of Hypotheses  Warm-up effect? Maybe Why do RAM-based systems warm up slower?  Bad initial states? No  Pentium III streaming I/O option? No

21. Effects of L2 Cache Footprints Large L2 cache footprint Small L2 cache footprint write a file sequentially footprintfile end footprint read the same file sequentially footprint flush file end file read write a file sequentially footprintfile end footprint read the same file sequentially footprint flush file end read file

22. LFS Sprite Microbenchmarks  Modified large-file microbenchmark: ten 1-MB files (in-core files)

23. More Lessons Learned  Effects of L2 caching become highly visible in memory workloads (modern workloads)  Cannot blindly apply disk-based microbenchmarks to measure memory performance of file systems  Need to consider states of L2 cache and memory behaviors at each stage of microbenchmarking

24. Additional Lessons Learned  Don’t discuss your performance numbers next to a balcony…unless…

25. What if….  Multiple levels of caching with similar characteristics? (via network)

26. A Cache Miss  Multiple levels of caching with similar characteristics? (via network)

27. A Cache Miss  Multiple levels of caching with similar characteristics? (via network) Why cache the same data twice?

28. What if….  A network of caches?

29. Cache-to-Process Data Handling  Data in buffer is destined for a user process (or came from one, on writes)  But buffers are in system space  How to get the data to the user space? Copy it Virtual memory techniques Use DMA in the first place