Garbage Collection Memory Management Garbage Collection –Language requirement –VM service –Performance issue in time and space

Garbage Collection Outline Motivation for GC Basics of GC –How it works –Key design choices –Basic GCs A taxonomy of sorts GC Performance

Garbage Collection Outline Briefly introduce the challenges and key ideas in Memory Management –Explicit vs Automatic –Memory Organization Contiguous allocation (bump pointer) Free lists (analogous to pages in memory) –Reclamation Tracing Reference counting

Garbage Collection Memory Management Program Objects/Data occupy memory How does the runtime system efficiently create and recycle memory on behalf of the program? –What makes this problem important? –What makes this problem hard? –Why are researchers (e.g. me) still working on it?

Garbage Collection Dynamic memory allocation and reclamation Heap contains dynamically allocated objects Object allocation: malloc, new Deallocation: –Manual/explicit: free, delete –automatic: garbage collection

Garbage Collection Explicit Memory Management Challenges More code to maintain Correctness –Free an object too soon - core dump –Free an object too late - waste space –Never free - at best waste, at worst fail Efficiency can be very high Gives programmers “control”

Garbage Collection Garbage collection: Automatic memory management reduces programmer burden eliminates sources of errors integral to modern object-oriented languages, i.e., Java, C#,.net now part of mainstream computing Challenge: –performance efficiency

Garbage Collection Key Issues For both –Fast allocation –Fast reclamation –Low fragmentation (wasted space) –How to organize the memory space Garbage Collection –Discriminating live objects and garbage

Garbage Collection GC: How? Automatically collect dead objects Liveness  reachability –Root sets Unreachable  non-live  garbage –Compared to dead in compiler ‘Once garbage always garbage’ –(Always safe to collect unreachable objects)

Garbage Collection Liveness: the GC and VM/Compiler Contract GC Maps - identify what is live –Root set Live registers, walk the stack to enumerate stack variables, globals at any potential GC point –JIT/compiler generates a map for every program point where a GC may occur –Can constrain optimizations (derived pointers) –Required for type-accurate GC

Garbage Collection VM/Compiler GC contract Write/Read barriers for generational & incremental collection –JIT must insert barriers in generated code –Usually inlines barriers –Barriers trade-off GC and mutator costs Cooperative scheduling –In many VMs, all mutator threads must be stopped at GC points. One solution requires JITs to inject GC yieldpoints at regular intervals in the generated code

Garbage Collection Basic GC Techniques Reference Counting Tracing –Mark-sweep –Mark-compact –Copying

Garbage Collection Tracing/Reference Counting ‘Tracing’ (reachability) –Trace reachability from program ‘roots’ Registers Stacks Statics –Objects not traced are unreachable A Reference Count for each object −#incoming pointers −incremental −count = 0 (unreachability) –Notice removal of last reference to an object –Write barrier for implementation –Concerns –cycles are an issue –space for reference count –Efficiency (deferred reference counting)

Garbage Collection Mark-Sweep How it works –Tracing to “mark” live objects –Sweep to reclaim dead objects Dead objects linked to free-lists Concerns –Fragmentation –Cost proportional to heap size –Locality

Garbage Collection Mark-Compact How it works –Tracing to “mark” live (reachable) objects –Sliding compacting marked objects Concerns –Fragmentation solved –Cost Two or more scans of live objects –Locality problem ameliorated

Garbage Collection Copying GC Copy reachable objects to a contiguous area –e.g., Semispace with Cheney scan Fromspace Root set Tospace

Garbage Collection Copying GC Copy reachable objects to a contiguous area –e.g., Semispace with Cheney scan From space Root set To space

Garbage Collection Copying Garbage Collection ‘from space’‘to space’‘from space’‘to space’ ‘from space’ ‘to space’ ‘from space’

Garbage Collection Cheney Scan Root set A B C D E F scan free AB scan free AB scan free AB C DC

Garbage Collection Space Management Two broad approaches: –Copying Bump allocation & en masse reclamation –Fast allocation & reclaim –Space overhead, copy cost –Non-copying Free-list allocation & reclamation –Space efficiency –Fragmentation

Garbage Collection Bump-Pointer Fast (increment & bounds check)  Can't incrementally free & reuse: must free en masse  Relatively slow (consult list for fit) Can incrementally free & reuse cells Free-List Allocation Choices

Garbage Collection Allocation Choices Bump pointer – ~70 bytes IA32 instructions, 726MB/s Free list – ~140 bytes IA32 instructions, 654MB/s Bump pointer 11% faster in tight loop – < 1% in practical setting – No significant difference (?) Second order effects? – Locality?? – Collection mechanism??

Garbage Collection Generational GC Observation –Most objects die young –A small percentage long lived objects Avoid copying long-lived objects several times Generational GC segregates objects by age –Older object collects less often

Garbage Collection Generational GC Design Issues –Advancement policies When to advance a live object into next generation –Heap organization –Collection scheduling –Intergenerational references Remembered sets Page marking, word marking, card marking, store list

Garbage Collection Incremental/Concurrent Approaches For real time and interactive application –Guarantee pause time –Can generational collection work? Techniques –Reference counting –Tracing concurrency

Garbage Collection Incremental/Concurrent Approaches Approaches to coordinate the collector and the mutator –Tri-color marking Black, gray, white –Mutator cannot install a pointer from black to white Read barrier –Color an white object gray before the mutator access it Write barrier –Trap an object when a pointer is write into it

Garbage Collection Write Barrier Algorithms Snap-shot-at-beginning –No objects ever become in in accessible to the GC while collection is in progress Yussa’s: An overwriten value is first saved for later examination Objects are allocated black Incremental Update –Objects that die during GC before being reached by GC traversal are not traversed and marked –Objects are allocated white –Iteratively traverse black objects got pointers store into

Garbage Collection Baker’s Read Barrier Algorithms Incremental copying with Cheney scan Any fromspace object accessed by the mutator is copy to tospace –Use read barrier New objects allocated in tospace –black

Garbage Collection Taxonomy of Sorts or: Key Design Dimensions Incrementality Composability Concurrency Parallelism Distribution

Garbage Collection Incrementality ‘Full heap’ tracing: –‘Pause time’ goes up with heap size Incremental tracing: –Bounded tracing time –Conservative assumption: All objects in rest of heap are live –Remember pointers from rest of heap Add ‘remembered set’ to roots for tracing

Garbage Collection Composability Hybrids –Copy younger objects –Non-copying collection of older objects Hierarchies –Copying intra-partition (increment) –Reference counting inter-partition

Garbage Collection Concurrency ‘Mutator’ and GC operate concurrently ?

Garbage Collection Parallelism Concurrency among multiple GCs –Load balancing –Race conditions when tracing –Synchronization

Garbage Collection Distribution Typically implies: –Incrementality –Concurrency –Parallelism –Composability Detecting termination –When has a partition become isolated?

Garbage Collection GC Performance Three key dimensions: –Throughput (bandwidth) –Responsiveness (latency) –Space Measurement issues: –Selecting benchmarks –Understanding space time tradeoff