21 September 2005Rotor Capstone Workshop Parallel, Real-Time Garbage Collection Daniel Spoonhower Guy Blelloch, Robert Harper, David Swasey Carnegie Mellon University

Rotor Capstone Workshop2Parallel, Real-Time Garbage Collection21 September 2005 Motivation: Support Real-Time in C#  Real-time requires predictability  Focus on garbage collection in this work  Real-time collection must be:  predictable – allow programmers to reason about code  efficient – RT apps must meet deadlines  scalable – multi-threaded apps  multi-threaded GC

Rotor Capstone Workshop3Parallel, Real-Time Garbage Collection21 September 2005 Our Results Collector implementation available online  incremental GC w/ stacklets real-soon-now  C# benchmarks also available Published work in VEE ’05  using residency to balance GC tradeoffs  dynamically adapt to application behavior

Rotor Capstone Workshop4Parallel, Real-Time Garbage Collection21 September 2005 Outline of Talk  Implementing a real-time collector  incremental collection  challenges in Rotor  Using residency to balance GC tradeoffs  between copying and non-copying GC  semantic constraints (e.g. pinning)  use of space and time (i.e. overhead)

Rotor Capstone Workshop5Parallel, Real-Time Garbage Collection21 September 2005 Incremental Collection Divide collector work into small pieces  interleave collector execution with application  avoid long interruptions of application  divide call stack into fixed size stacklets  typical metric: (minimum) mutator utilization

Rotor Capstone Workshop6Parallel, Real-Time Garbage Collection21 September 2005 Minimum Mutator Utilization (MMU)

Rotor Capstone Workshop7Parallel, Real-Time Garbage Collection21 September 2005 Building on Previous Work We extend replicating copying collection  Nettles & O’Toole; Cheng & Blelloch  based on semi-space copying collection  good time bounds, high space overhead Rotor requires GC to support pinned objects  part of foreign function interface  determined by programmer Incompatible with pure copying GC!

Rotor Capstone Workshop8Parallel, Real-Time Garbage Collection21 September 2005 Outline of Talk 1.Implementing a real-time collector  incremental collection  challenges in Rotor  Using residency to balance tradeoffs  between copying and non-copying GC  semantic constraints (e.g. pinning)  use of space and time (i.e. overhead)

Rotor Capstone Workshop9Parallel, Real-Time Garbage Collection21 September 2005 Adaptive Hybrid Collection Bartlett: mostly copying collection  reduce fragmentation in constrained GC We extend this to uniformly account for:  pinned objects  large objects  densely populated pages

Rotor Capstone Workshop10Parallel, Real-Time Garbage Collection21 September 2005 Mostly Copying Collection “Copy objects...most of the time.” Key ideas:  divide heap into pages  determine strategy for each page dynamically: objects may be evacuated or promoted in place Gain benefits of both copying and non- copying collection

Rotor Capstone Workshop11Parallel, Real-Time Garbage Collection21 September 2005 Mostly Copying Collection divide heap into pages……reserve pages for allocation…...allocate objects……begin collection……flip……some objects are unreachable… …evacuate reachable objects……pinned……in-place promotion!... and reclaim unused space.

Rotor Capstone Workshop12Parallel, Real-Time Garbage Collection21 September 2005 Don’t Relocate Large Objects Large objects often relegated to separate heap ... because they are expensive to copy We use in-place promotion instead.  avoids usual penalties of copying large objects  no additional heap space required  use small objects to fill in gaps

Rotor Capstone Workshop13Parallel, Real-Time Garbage Collection21 September 2005 Don’t Relocate Large Objects They don’t cause (much) fragmentation  little benefit to moving them around Similarly for densely populated pages:  expensive to evacuate  little fragmentation vs.

Rotor Capstone Workshop14Parallel, Real-Time Garbage Collection21 September 2005 Page Residency as Guiding Principle Residency = density of reachable objects  avoid evacuating objects from dense pages  focus on fragmented parts of the heap Large objects occupy pages w/ high residency.  therefore, they are promoted in place

Rotor Capstone Workshop15Parallel, Real-Time Garbage Collection21 September 2005 Revisiting Allocation Copying collectors use simple space check  constant time when there is sufficient space  requires contiguous unused space In-place promotion requires more sophisticated management of free space.  need to find an free gap of appropriate size

Rotor Capstone Workshop16Parallel, Real-Time Garbage Collection21 September 2005 Overview of Our Hybrid Algorithm Collection strategy determined for each page Two data structures for heap traversal:  Cheney queues for evacuated objects  mark stack for objects promoted in place Allocation using a modified first-fit  favor free pages (treated as large gaps)  other free list algorithms also applicable

Rotor Capstone Workshop17Parallel, Real-Time Garbage Collection21 September 2005 Two Mechanisms to Reclaim Space Reclaim space both...  via evacuation use residency to determine which objects should be evacuated  during allocation use residency to determine which pages have enough space to make allocation worthwhile

Rotor Capstone Workshop18Parallel, Real-Time Garbage Collection21 September 2005 Collector Configurations Two residency parameters:  r evac = evacuation threshold, maximum residency at which objects are evacuated  r alloc = allocation threshold, maximum residency at which free space is reused (both measured as % of page) Classic algorithms fall out as special cases  semi-space: r evac = 100%, r alloc = 0%  mark-sweep: r evac = 0%, r alloc = 100%

Rotor Capstone Workshop19Parallel, Real-Time Garbage Collection21 September 2005 A Continuum of Configurations r evac = when objects are evacuated r alloc = when free space is reused

Rotor Capstone Workshop20Parallel, Real-Time Garbage Collection21 September 2005 Adaptive Hybrid Collection So far...  outlined a hybrid algorithm for garbage collection  shown how residency thresholds give rise to a continuum of configurations Next: how to determine residency...

Rotor Capstone Workshop21Parallel, Real-Time Garbage Collection21 September 2005 Measuring Residency Residency can be computed during a heap traversal at little additional cost. However, we need to know residency at beginning of collection...

Rotor Capstone Workshop22Parallel, Real-Time Garbage Collection21 September 2005 Use Past Measurements One possibility: explicitly measure residency during each collection (using an extra traversal). Instead, measure and predict future values  historical measurements avoid a second pass  measured residency still accounts for changes in program behavior over time

Rotor Capstone Workshop23Parallel, Real-Time Garbage Collection21 September 2005 Existing GCs Also Predict Residency Other hybrids fix residency estimates in advance.  syntactic properties to distinguish objects  e.g. size, type, allocation point Age: “Most objects die young.”  “Nursery is sparsely populated.”  i.e. using age to predict residency  however, generational collectors are often inconsistent in using age (e.g. large objects)

Rotor Capstone Workshop24Parallel, Real-Time Garbage Collection21 September 2005 Recovering from Poor Predictions Adaptive GC recovers naturally from poor predictions.  at most two collections to recover space  without changes to algorithm

Rotor Capstone Workshop25Parallel, Real-Time Garbage Collection21 September 2005 Recovering from Poor Predictions Consider the impact of overestimation:  many sparse pages remain – high fragmentation  measurements give true residency for each page  next collection will relocate objects to reclaim space GC with poor predictions behaves like an explicit measurement phase.  in effect, only corrects inaccurate estimates Allocation uses true residency, not predictions.  actual residency is known at the end of collection

Rotor Capstone Workshop26Parallel, Real-Time Garbage Collection21 September 2005 Experimental Results If we know usage patterns of an application in advance, we choose an appropriate algorithm.  adaptive hybrid GC relieves us of this choice Consider the impact of heap size on GC overhead  caveat: other effects of GC not shown here

Rotor Capstone Workshop27Parallel, Real-Time Garbage Collection21 September 2005 Impact of Heap Size Heap size = memory available to collector We expect copying collectors to perform well when there is plenty of space. For small heaps, semi-space collectors may not satisfy all allocation requests. Hybrid yields good performance in all cases.

Rotor Capstone Workshop28Parallel, Real-Time Garbage Collection21 September 2005 Impact of Heap Size

Rotor Capstone Workshop29Parallel, Real-Time Garbage Collection21 September 2005 Impact of Heap Size

Rotor Capstone Workshop30Parallel, Real-Time Garbage Collection21 September 2005 Other Measurements Same configuration performs well in many different environments  different applications  small and large footprints (100 kB to 100 MB)

Rotor Capstone Workshop31Parallel, Real-Time Garbage Collection21 September 2005 Related Work  Replicating Collection – Nettles & O’Toole; Cheng & Blelloch  Mostly Copying – Bartlett; Smith & Morrisett  Garbage First – Detlefs, Flood, Heller, & Printezis (also uses residency as a measure of cost)  Metronome – Bacon, Cheng, & Rajan (different point in the “mostly” design space)

Rotor Capstone Workshop32Parallel, Real-Time Garbage Collection21 September 2005 Summary Incremental collection  divide up collector work over time Adaptive hybrid collection  use page residency to guide runtime decisions  continuous spectrum of tracing collectors Measure and adapt to application behavior  predict residency based on past measurements  naturally recovers from poor estimates

Rotor Capstone Workshop33Parallel, Real-Time Garbage Collection21 September 2005 Useful Links VEE ’05 paper: Collector implementation: C# benchmarks: