1 The Garbage Collection Advantage: Improving Program Locality Xianglong Huang (UT), Stephen M Blackburn (ANU), Kathryn S McKinley (UT) J Eliot B Moss (Umass), Zhenlin Wang (MTU), Perry Cheng (IBM) Presented by Na Meng Many thanks to authors and the anonymous speaker on MM course last time

2 Motivation Memory gap problem OO programs exacerbates memory gap problem –Automatic memory management Pointer data structures Goal: improve OO program locality

3 Opportunity Copying garbage collector reorders objects at runtime

Copying of Linked Objects Breadth First

Copying of Linked Objects Breadth First Depth First

Copying of Linked Objects Depth First Online Object Reordering 1 4 Breadth First

7 Outline Motivation Online Object Reordering (OOR) Methodology Experimental Results Conclusion

8 Online Object Reordering Where are the cache misses? How to identify hot field accesses at runtime? How to reorder the objects?

9 Where Are The Cache Misses? VM ObjectsStack Older Generation Heap structure: Nursery Not to scale

10 Where Are The Cache Misses?

11 Where Are The Cache Misses? Two opportunities to reorder objects in the older generation –Promote nursery objects –Full heap collection

12 How to Find Hot Fields? Runtime info (intercept every read)? Compiler analysis? Runtime information + compiler analysis Key: Low overhead estimation

13 Which Classes Need Reordering? Step 1: Compiler analysis –Excludes cold basic blocks –Identifies field accesses Step 2: JIT adaptive sampling identifies hot methods –Mark as hot field accesses in hot methods

14 Example: Compiler Analysis Compiler Hot BB Collect access info Cold BB Ignore Compiler Access List: 1. A.b 2. …. …. Method Foo { Class A a; try { …=a.b; … } catch(Exception e){ …a.c }

15 Example: Adaptive Sampling Method Foo { Class A a; try { …=a.b; … } catch(Exception e){ …a.c } Adaptive Sampling Foo is hot Foo Accesses: 1. A.b 2. …. …. A.b is hot A B b ….. c A’s type information cb

Copying of Linked Objects Online Object Reordering Type Information Hot space Cold space

17 OOR System Overview Baseline Compiler Source Code Executing Code Adaptive Sampling Optimizing Compiler Hot Methods Access Info Database Register Hot Field Accesses Look Up Adds Entries GC: Copies Objects Affects Locality Advice GC: Copies Objects OOR addition JikesRVM componentInput/Output Optimizing Compiler Adaptive Sampling Improves Locality

18 Outline Motivation Online Object Reordering Methodology Experimental Results Conclusion

19 Virtual Machine Jikes RVM –VM written in Java –High performance –Timer based adaptive sampling –Dynamic optimization Experiment setup –Pseudo-adaptive –2 nd iteration [Eeckhout et al.]

20 Memory Management Memory Management Toolkit (MMTk) –Allocators and garbage collectors –Multi-space heap Boot image Large object space (LOS) Immortal space Experiment setup –Generational copying GC with 4M bounded nursery

21 Overhead: OOR Analysis Only BenchmarkBase Execution Time (sec) w/ only OOR Analysis (sec) Overhead jess % jack % raytrace % mtrt % javac % compress % pseudojbb % db % antlr % hsqldb % ipsixql % jython % ps-fun % Mean -0.19%

22 Detailed Experiments Separate application and GC time Vary thresholds for method heat Vary thresholds for cold basic blocks Three architectures –x86, AMD, PowerPC x86 Performance counter: –DL1, trace cache, L2, DTLB, ITLB

23 Performance javac

24 Performance db

25 Performance jython Is the improvement significant?

26 Phase Changes

Algorithm: Decay Field Heat 27 DECAY-HEAT(method) 1 for each fieldAccess in method do 2 if PotentiallyHot(fieldAccess)then 3 hotField  fieldAccess.field 4 class  hotField.instantiatingClass 5 class.hasHotField  true 6 for each field in class do 7 period  Now() – class.lastUpdate 8 decay  HI/(HI + period) 9 field.heat  field.heat * decay 10 if field.heat < LO then 11 field.heat = 0 12 hotField.heat  HI 13 class.lastUpdate  Now() Will the latest access pattern erase the earlier access pattern(s)? m1(){ for(… …){ … … a.b = … } m2(){ for(… …){ … … = a.c; } for(… …){ m1(); //GC works m2(); //GC works }

OOR w/o vs. w phase change 28 Almost all hot fields within an object are visited around the same time The standard benchmarks have few, if any, traversal order phases.

Copying Advantage (javac) 29 GenCopy vs. MS Mutator time? GC time? Total time?

A Possible Comparison 30 GenCopy vs. GenOOR ?

Discussion Any other solution to improve the locality while doing copying collection 31

32 Questions? Thank you!