1. Dynamic Object Sampling for Pretenuring Maria Jump Department of Computer Sciences The University of Texas at Austin mjump@cs.utexas.edu Stephen M. Blackburn Department of Computer Science Australia National University Steve.Blackburn@anu.edu.au Kathryn S. McKinley Department of Computer Sciences The University of Texas at Austin mckinley@cs.utexas.edu

2. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley2 Current Trends  Object-oriented languages Lots of small objects Runtime system treats them uniformly  Objects have distinctive properties Lifetime, access frequency, locality, calling context, thread usage, etc.  PROBLEM: How can we discover and exploit object properties?

3. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley3 Gathering Object Statistics  Offline statistics Inconsistent with Java Obscure phase behavior  Online statistics Profiling all objects Dynamic Object Sampling  Statistics only for selected objects  Use to estimate behavior of all objects

4. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley4 Example: Using Object Lifetimes for Pretenuring PROBLEM: Long-lived objects are allocated into the nursery SOLUTION: Allocate long-lived objects directly into mature space CHALLENGE: Correlating lifetime behavior with allocation site

5. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley5 Outline  Motivation  Dynamic Object Sampling Mechanism  Dynamic Lifetime Prediction Accuracy vs. Cost  Dynamic Pretenuring Allocation Survival Phases Coverage, Accuracy, and Performance  Conclusions

6. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley6 384256128 Dynamic Object Sampling  Modify a bump-pointer allocator  Sample one object every n bytes Memory chunk: Bump Pointer Sample Tag (for n=128)

7. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley7 VM_Address alloc(int bytes) throws VM_PragmaInline { VM_Addresss oldCursor = cursor; VM_Address newCursor = oldCursor.add(bytes); if (newCursor.GT(sampleLimit)) return sample(bytes); cursor = newCursor; return oldCursor; } Sample Path (Intermediate) : 1) adds sample tag2) collects statistics Slow Path Bump-Pointer Allocator VM_Address alloc(int bytes) throws VM_PragmaInline { VM_Addresss oldCursor = cursor; VM_Address newCursor = oldCursor.add(bytes); if (newCursor.GT(limit)) return allocSlow(bytes); cursor = newCursor; return oldCursor; }

8. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley8 During Garbage Collection  Use GC to update statistics Trace live objects Update transient and total statistics SAMPLE TAG FOUND! 1. decode tag word 2. collect statistics survivor What’s in the Sample Tag?

9. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley9 Dynamic Lifetime Prediction  Encode allocation site identifier into the object sampling sample tag Allocation site is a good lifetime predictor [Blackburn et al.] Compiler maps identifiers to sites  Accuracy vs. Cost tradeoff

10. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley10 Implementation and Experimental Methodology  Jikes RVM 2.3.0.1 with MMTk  SPEC JVM benchmarks, SPEC JBB 2000 with constant workload  3.2 GHz Intel Pentium 4 w/ hyperthreading, 1GB main memory, Linux 2.6.0  Generational/Mark-Sweep (GenMS) 4MB bounded nursery  Pseudoadaptive compilation  Report application time only [Eeckhout et al.] 2 nd run methodology

11. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley11 Accuracy of Sampling javac Accuracy 92.5%

12. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley12 Cost of Sampling

13. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley13 Object Sampling for Dynamic Pretenuring  Accurately and efficiently estimate allocation-site lifetimes  At garbage collection time Calculate survival rate per allocation site Compute transient and total statistics if (survival rate >= threshold) then change the allocation target Pretenure:

14. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley14 if (shortLived(site)) region = nursery.alloc(…); else region = matureAlloc(…); break; Dynamic Allocation Target case NURSERY_SPACE: region = nursery.alloc(…); break; table lookup

15. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley15 Dynamic Allocation Target Overhead

16. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley16 Dynamic Pretenuring Overhead

17. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley17 Allocation Phases  Current survival rate is not a guarantee of future survival  Sampling in older space is ineffective Collections are infrequent Small amount of allocation Periodically allocate site in nursery BACKSAMPLING

18. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley18 Backsampling  Protects the system from bad choices  Reduces effectiveness of good choices When to Backsample? Backsample Trigger = n * f Number of allocations for pretenuring decision Growth function: Constant Linear Exponential

19. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley19 Dynamic Pretenuring Policies Parameters Configurations cfg1cfg2cfg3 sampling interval256 minimum samples8410 pretenuring threshold80% 85% backsampling policylinearexponentiallinear backsampling shift141 decay shift001

20. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley20 Coverage: How much? None 100% 95% 90% 85% 80% 75% Pretenuring Threshold Long-Lived Pretenured

21. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley21 Accuracy: Are we right? None 100% 95% 90% 85% 80% 75% Pretenuring Threshold Incorrect Correct

22. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley22 GC Time: javac

23. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley23 Total Time: javac

24. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley24 Conclusions  Lifetime estimation for dynamic pretenuring Accurately and efficiently estimates allocation- site lifetimes Pretenuring long-lived allocation-sites can improve runtime Few opportunities in our programs  Dynamic Object Sampling Low-overhead High accuracy Generic mechanism

25. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley25 Questions? Thank You

26. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley26 GC Time: jess

27. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley27 Total Time: jess

28. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley28 Coverage: How much?

29. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley29 Accuracy: Are we right?

30. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley30 Coverage: How much?

31. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley31 Coverage: How much?

32. ISMM, Oct. 25, 2004Jump, Blackburn, McKinley32 Coverage: How much?