1. Dynamic Compilation Vijay Janapa Reddi
The University of Texas at Austin
Garbage Collection 1

2. Today Garbage Collection Why use garbage collection? What is garbage?
Reachable vs live, stack maps, etc.
Allocators and their collection mechanisms
Semispace
Marksweep
Performance comparisons
Incremental age based collection
Write barriers: Friend or foe?
Generational
Beltway
More performance

3. Basic VM Structure Program/Bytecode Executing Program
Class Loader Verifier, etc.
Heap
Thread Scheduler
Dynamic Compilation Subsystem
Garbage Collector

4. True or False?
Real programmers use languages with explicit memory management?
I can optimize my memory management much better than any garbage collector

5. True or False?
Real programmers use languages with explicit memory management.
I can optimize my memory management much better than any garbage collector
Scope of effort?

6. Why Use Garbage Collection?
Software engineering benefits
Less user code compared to explict memory management (MM)
Less user code to get correct
Protects against some classes of memory errors
No free(), thus no premature free(), no double free(), or forgetting to free()
Not perfect, memory can still “leak”
Programmers still need to eliminate all pointers to objects the program no longer needs
Performance: space/time tradeoff
Time proportional to dead objects (explicit mm, reference counting) or live objects (semispace, marksweep)
Throughput versus pause time
Less frequent collection, typically reduces total time but can increase space requirements and pause times
Hidden locality benefits?

7. GC, A tool for all occasions?
When might you NOT be willing to use a garbage collector?

8. What is Garbage?
In theory, any object the program will never reference again
But compiler & runtime system cannot figure that out
In practice, any object the program cannot reach is garbage
Approximate liveness with reachability
OK, so how do we what data is reachable?
Keep track of pointers
They tell you how to “reach” some other piece of data
What about programming languages like C?
X = *(<arbitrary address >)
Everything is (potentially) reachable
It’s up to the programmer… malloc() & free()

9. What is Garbage? Managed languages couple GC with “safe” pointers
Programs may not access arbitrary addresses in memory
The compiler can identify and provide to the garbage collector all the pointers, thus
“Once garbage, always garbage”
Runtime system can potentially relocate objects by updating pointers

10. Reference Counting
If we know whenever we assign a pointer, we update a reference count. When it is decremented to 0, it is freed.
Consider: Pop of a stack…
Head 1 1 1

11. Reference Counting
If we know whenever we assign a pointer, we update a reference count. When it is decremented to 0, it is freed.
Consider: Pop of a stack…
Head 2 1

12. Reference Counting
If we know whenever we assign a pointer, we update a reference count. When it is decremented to 0, it is freed.
Consider: Pop of a stack…
Head 1 1

13. Reference Counting What if we want to delete this deque?
Head = Tail = NULL;
Head
Tail 2 2 2

14. Reference Counting What if we want to delete this deque?
Head = Tail = NULL;
Head
NULL
Tail 1 2 1

15. Reference Counting What if we want to delete this deque?
Head = Tail = NULL;
Head
NULL
Tail 1 2 1
Cycles lead to
Orphaned Garbage

16. Reference Counting Reference Counting is used in C++ “smart pointers”
“Shared” pointers that cause reference counting
“Weak” pointers won’t keep an object alive (don’t affect reference count)
Programmers need to pay attention to right kind of pointer
Sort-of a half-way between manual new/delete and “real” garbage collection
Head
Tail 1 1 2

17. Tracing Collectors More robust solution are “tracing” collectors
Start with a “root set” of all references
Objects you “know” without having to have a pointer to them (e.g. globals)
See what the objects in the root set point to, follow those, etc.
Need to know how to find pointers within an object
Bascially, traces through to find all the reachable objects
Everything else must be garbage

18. { Tracing Collectors .... r0 = obj globals stack registers heap A B C
Compiler produces a stack-map at GC safe-points and Type Information Blocks
GC safe points: new(), method entry, method exit, & back-edges (thread switch points)
Stack-map: enumerate global variables, stack variables, live registers -- This code is hard to get right! Why?
Type Information Blocks: identify reference fields in objects A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap

19. { Tracing Collectors .... r0 = obj globals stack registers heap A B C
Compiler produces a stack-map at GC safe-points and Type Information Blocks
Type Information Blocks: identify reference fields in objects
for each type i (class) in the program, a map
TIBi 2 3 A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap

20. { Tracing Collectors Tracing collector (semispace, marksweep) mark
Marks the objects reachable from the roots live, and then performs a transitive closure over them
mark A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap

21. { Tracing Collectors Tracing collector (semispace, marksweep) mark
Marks the objects reachable from the roots live, and then performs a transitive closure over them
mark A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap

22. { Tracing Collectors Tracing collector (semispace, marksweep) mark
Marks the objects reachable from the roots live, and then performs a transitive closure over them
mark A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap

23. { Tracing Collectors Tracing collector (semispace, marksweep)
Marks the objects reachable from the roots live, and then performs a transitive closure over them
All unmarked objects are dead, and can be reclaimed
mark A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap

24. { Tracing Collectors Tracing collector (semispace, marksweep)
Marks the objects reachable from the roots live, and then performs a transitive closure over them
All unmarked objects are dead, and can be reclaimed
sweep A B C {
....
r0 = obj
PC -> p.f = obj
globals
stack
registers
heap

25. Conservative Collectors
What if we didn’t have the type information block?
That is, we can’t identify the pointers within an object

26. Conservative Collectors
What if we didn’t have the type information block?
That is, we can’t identify the pointers within an object
e.g. with C or vanilla C++ pointers
Answer: Do “Conservative Collection”
Treat every value like it might be a pointer
If it looks like it might point to a memory region in the heap, assume it is a pointer
Trace the block of data that was “malloc’d” that contains that address
In some architectures, pointers must be word-aligned (least significant two bits are zero) which helps filter out random integers
But “unfortunate integers” can keep memory alive
Also, can’t move objects since we can’t safely backpatch pointers (since, they might really be integers)

27. Today Garbage Collection Why use garbage collection? What is garbage?
Reachable vs live, stack maps, etc.
Allocators and their collection mechanisms
Semispace
Marksweep
Performance comparisons
Incremental age based collection
Write barriers: Friend or foe?
Generational
Beltway
More performance

28. Semispace Fast bump pointer allocation Requires copying collection
Cannot incrementally reclaim memory, must free en masse
Reserves 1/2 the heap to copy in to, in case all objects are live
to space
from space
heap

29. Semispace Fast bump pointer allocation Requires copying collection
Cannot incrementally reclaim memory, must free en masse
Reserves 1/2 the heap to copy in to, in case all objects are live
to space
from space
heap

30. Semispace Fast bump pointer allocation Requires copying collection
Cannot incrementally reclaim memory, must free en masse
Reserves 1/2 the heap to copy in to, in case all objects are live
to space
from space
heap

31. Semispace Fast bump pointer allocation Requires copying collection
Cannot incrementally reclaim memory, must free en masse
Reserves 1/2 the heap to copy in to, in case all objects are live
to space
from space
heap

32. Semispace Mark phase: copies object when collector first encounters it
installs forwarding pointers
from space
to space
heap

33. Semispace Mark phase: copies object when collector first encounters it
installs forwarding pointers
performs transitive closure, updating pointers as it goes
from space
to space
heap

34. Semispace Mark phase: copies object when collector first encounters it
installs forwarding pointers
performs transitive closure, updating pointers as it goes
from space
to space
heap

35. Semispace Mark phase: copies object when collector first encounters it
installs forwarding pointers
performs transitive closure, updating pointers as it goes
from space
to space
heap

36. Semispace Mark phase: copies object when collector first encounters it
installs forwarding pointers
performs transitive closure, updating pointers as it goes
reclaims “from space” en masse
from space
to space
heap

37. Semispace Mark phase: from space to space heap
copies object when collector first encounters it
installs forwarding pointers
performs transitive closure, updating pointers as it goes
reclaims “from space” en masse
start allocating again into “to space”
from space
to space
heap

38. Semispace Mark phase: from space to space heap
copies object when collector first encounters it
installs forwarding pointers
performs transitive closure, updating pointers as it goes
reclaims “from space” en masse
start allocating again into “to space”
from space
to space
heap

39. Semispace Notice: fast allocation
locality of contemporaneously allocated objects
locality of objects connected by pointers
wasted space
from space
to space
heap

40. Marksweep Free-lists organized by size
blocks of same size, or
individual objects of same size
Most objects are small < 128 bytes 4 8 12 16
...
128
...
heap
...
free lists

41. Marksweep Allocation heap free lists
Grab a free object off the free list 4 8 12 16
...
128
...
heap
...
free lists

42. Marksweep Allocation heap free lists
Grab a free object off the free list 4 8 12 16
...
128
...
heap
...
free lists

43. Marksweep Allocation heap free lists
Grab a free object off the free list 4 8 12 16
...
128
...
heap
...
free lists

44. Marksweep heap free lists Allocation
Grab a free object off the free list
No more memory of the right size triggers a collection
Mark phase - find the live objects
Sweep phase - put free ones on the free list 4 8 12 16
...
128
...
heap
...
free lists

45. Marksweep heap free lists Mark phase Sweep phase
Transitive closure marking all the live objects
Sweep phase
sweep the memory for free objects populating free list 4 8 12 16
...
128
...
heap
...
free lists

46. Marksweep heap free lists Mark phase Sweep phase
Transitive closure marking all the live objects
Sweep phase
sweep the memory for free objects populating free list 4 8 12 16
...
128
...
heap
...
free lists

47. Marksweep heap free lists Mark phase Sweep phase
Transitive closure marking all the live objects
Sweep phase
sweep the memory for free objects populating free list 4 8 12 16
...
128
...
heap
...
free lists

48. Marksweep heap free lists Mark phase Sweep phase
Transitive closure marking all the live objects
Sweep phase
sweep the memory for free objects populating free list
can be made incremental by organizing the heap in blocks and sweeping one block at a time on demand 4 8 12 16
...
128
...
heap
...
free lists

49. Marksweep heap free lists space efficiency
Incremental object reclamation
relatively slower allocation time
poor locality of contemporaneously allocated objects 4 8 12 16
...
128
...
heap
...
free lists

50. How do these differences play out in practice?
Marksweep
space efficiency
Incremental object reclamation
relatively slower allocation time
poor locality of contemporaneously allocated objects
Semispace
fast allocation
locality of contemporaneously allocated objects
locality of objects connected by pointers
wasted space

51. Methodology [SIGMETRICS 2004]
Compare Marksweep (MS) and Semispace (SS)
Mutator time, GC time, total time
Jikes RVM & MMTk
replay compilation
measure second iteration without compilation
Platforms
1.6GHz G5 (PowerPC 970)
1.9GHz AMD Athlon 2600+
2.6GHz Intel P4
Linux with perfctr patch & libraries
Separate accounting of GC & Mutator counts
SPECjvm98 & pseudojbb

52. Allocation Mechanism Bump pointer Free list
~70 bytes IA32 instructions, 726MB/s
Free list
~140 bytes IA32 instructions, 654MB/s
Bump pointer 11% faster in tight loop
< 1% in practical setting
No significant difference (?)

53. Mutator Time

54. jess

55. jess

56. jess

57. jess

58. javac

59. pseudojbb

60. Geometric Mean Mutator Time

61. Garbage Collection Time

62. Garbage Collection Time
javac
pseudojbb
jess
Geometric
mean

63. Total Time

64. Total Time
javac
pseudojbb
jess
Geometric
mean

65. MS/SS Crossover: 1.6GHz PPC

66. MS/SS Crossover: 1.9GHz AMD

67. MS/SS Crossover: 2.6GHz P4

68. MS/SS Crossover: 3.2GHz P4

69. MS/SS Crossover
locality
space
2.6GHz
1.6GHz
3.2GHz
1.9GHz