U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Matthew Hertz, Yi Feng, & Emery Berger Department of Computer Science University of Massachusetts Hippocratic Garbage Collection

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 2 Dude, where’s my performance? Sun HotSpot JDK 1.4.1, Java Beans benchmark

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 3 Ever-Steeper Memory Hierarchy Higher = smaller, faster, closer to CPU A real desktop machine (mine) registers L1 cache L2 cache RAM Disk 8 integer, 8 floating-point; 1-cycle latency 8K data & instructions; 2-cycle latency 512K; 7-cycle latency 1GB; 100 cycle latency 40 GB; 38,000,000 cycle latency (!)

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 4 Program Throughput: Ideal vs. Real

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 5 GC Pause Time From 55 milliseconds to 30 seconds!..Can we avoid this?

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 6 Memory Prices Over Time “Soon it will be free…”

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 7 Memory Prices: Inflection Point

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 8 Memory Not So Free Servers: buy 4GB, get 1 CPU free! Sun E10000: 4GB extra = $75,000! Fast RAM Cosmic rays… 4GB Sun RAM = ½ Ferrari Modena “Buy more RAM” = “Ferraris for Everyone” Desktops: most 256MB 1GB = 50% more $ Laptops = 70%, if possible

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 9 Fast DDR2 modules today: $500/GB

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 10 Outline Motivation GC Paging Behavior Hippocratic Garbage Collection Cooperation Bookmarking Results Future Work

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 11 What’s really going on here?

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 12 GC Performance While Paging RAM Hard Disk GC: Touch evicted page

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 13 RAM Hard Disk GC Performance While Paging VM: brings page in-core

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 14 RAM Hard Disk GC Performance While Paging VM: evicts different page (LRU)

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 15 RAM Hard Disk GC Performance While Paging GC: Touches page on disk

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 16 RAM Hard Disk GC Performance While Paging VM: Evicts another page…and so on…

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 17 Oblivious Memory Management Garbage collection: VM-oblivious Cannot adjust to changing memory pressure Touches non-resident pages Virtual memory: GC-oblivious Likely to evict pages needed by GC

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 18 Outline Motivation GC Paging Behavior Hippocratic Garbage Collection Cooperation Bookmarking Results Future Work

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 19 Our Inspiration As to diseases, make a habit of two things: to help, or at least, to do no harm. Hippocrates

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 20 Hippocratic Garbage Collection Goal: GC triggers no additional paging Key ideas: Minimize space consumption Exploit page information from VM Residency, eviction notifications Avoid touching evicted pages Garbage collectorVirtual memory manager Evacuates pages Selects victim pages Page replacement page eviction notification victim page(s)

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 21 HC Collector Overview Generational Focused activity in nursery: improves locality Mark-sweep (w/o pressure) Compaction Shrink nursery Segregated size classes Eliminates fragmentation Reduces memory pressure  Essentially page-oriented GenMS + compaction

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 22 Segregated Size Classes Mark objects

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 23 Segregated Size Classes Copy into empty spaces

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 24 Cooperative Collection GC and VM communication Before-paging callback Per-page eviction negotiation Modify collector Manage heap memory at page granularity GC on page eviction notification When possible, return empty pages Otherwise, process pages before eviction: “bookmarking”

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 25 Process pages before they are evicted… RAM Hard Disk Bookmarking

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 26 …by “bookmarking” target objects RAM Hard Disk Bookmarking

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 27 then allow page eviction RAM Hard Disk Bookmarking

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 28 process objects as normal except… RAM Hard Disk Collection with Bookmarking

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 29 …ignore references to evicted pages RAM Hard Disk Collection with Bookmarking

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 30 bookmarks remember references from disk RAM Hard Disk Collection with Bookmarking

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 31 Result: no paging! RAM Hard Disk Collection with Bookmarking

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 32 Bookmarking Details Cheap summary of connectivity One bit per object: stolen from object header One word per page: count of pages that hold bookmarks on this page Clear bookmarks when zero Invoke VM-oblivious collection only when heap is exhausted Common case for all other collectors! In practice: hasn’t happened yet

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 33 Outline Motivation GC Paging Behavior Hippocratic Garbage Collection Cooperation Bookmarking Results Future Work

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 34 Methodology Jikes RVM, MMTk, extended Linux kernel “OPT + reset” Opt-compile first, full collection, second timing run Benchmarks: SPECjvm98, DaCapo suite (ipsixql, jython) SPECjbb variant (pseudojbb) Signalmem for memory pressure Dynamic (30MB, then 60MB) Static (40% of heap available)

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 35 Execution time: No Memory Pressure Generally runs in smaller heaps, fast as GenMS

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 36 Execution Time: No Memory Pressure – pseudoJBB CopyMS, etc. at least 6%-89% slower

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 37 Execution Time: Dynamic Memory Pressure Startup process: 30MB, then 60MB more

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 38 Pause Time: Dynamic Memory Pressure Startup process: 30MB, then 60MB more

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 39 Execution Time: Constant Memory Pressure Memory sufficient to hold 40% of heap

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 40 Pause Times: Constant Memory Pressure Memory sufficient to hold 40% of heap

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 41 Future Work Different eviction strategies Exploit type info, fewer pointers Richer summaries Shrink heap vs. evict pages; grow heap Requires more VM info Static analyses Combine with automatic heap sizing [Yang et al., ISMM 04]

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 42 Conclusion Hippocratic Garbage Collection Cooperates with VM Competitive when not paging Performs up to 13% faster with small heaps As fast as GenMS for larger heaps Greatly improves performance when paging Throughput up to 3x greater Average pause times up to 30x smaller than CopyMS, 70x smaller than GenMS

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 43 Backup Slides

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 44 Some Related Work Compaction: reduce working set Linearize lists [Bobrow & Murphy] Semispace collection [Baker, Fenichel & Yochelson] Reduce frequency of whole-heap collection Ephemeral [Moon] & generational collectors [Lieberman & Hewett, Ungar, Appel] Cooperation Shrink heap [Alonso & Appel] Give up empty pages [Cooper, Nettles et al.]

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 45 Tracking Page Value Maintain histogram per page position Provides value to application of n pages (for any n) unprotected protected