1. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 1 Automatic Heap Sizing: Taking Real Memory into Account Ting Yang, Emery Berger, Matthew Hertz, Scott Kaplan ¶, Eliot Moss Department of Computer Science Department of Computer Science ¶ University of Massachusetts Amherst College Amherst MA 01003 Amherst MA 01002 {tingy,emery,hertz,moss}@cs.umass.edu sfkaplan@cs.amherst.edu

2. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 2 Appel _213_javac 60MB real memory Too small: GC a lot Too large: page a lot Optimal Problem & Motivation Heap size vs Running time

3. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 3 Problem & Motivation Multiprogramming makes it harder: Amount of available real memory changes Impossible to select heap size a priori Strategy: adjust adaptively during execution

4. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 4 Outline Problem & Motivation Problem & Motivation Paging behavior of Garbage Collection Paging behavior of Garbage Collection VMM: collecting the information we need VMM: collecting the information we need Collector: adjusting heap size adaptively Collector: adjusting heap size adaptively Experimental results Experimental results Conclusion & Future work Conclusion & Future work

5. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 5 GC Paging Behavior For strategy to work, need to relate: GC algorithm, heap size, and footprint Analysis methodology: Obtain reference trace: simulate Jikes RVM under DSS Process with LRU stack  # faults at all memory sizes GCs and programs traced: Mark-Sweep (MS), Semi-Space (SS), and Appel GCs SPECjvm98, ipsixql, and pseudojbb benchmarks

6. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 6 Heap Size = 240Mb Memory = 145Mb # of Faults ≈ 1000 50 seconds extreme paging substantial paging: “looping” behavior fits into memory Fault curve: Relationship of heap size, real memory & page faults Heap size= 0.5 second

7. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 7 Relationship between heap size and footprint Page fault threshold = Our definition of footprint: The amount of memory needed so that the time spent on page faults is lower than a certain percentage of total execution time

8. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 8 A Linear Model: Heap size vs. Footprint Heap footprint model : Heaputil : SS 0.5; MS: 1 base : Jikes RVM plus live data size How the GC can use this model:

9. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 9 Strategy Overview VMM ( in OS) Knows memory allocation / available memory Needs to track/calculate application footprint Garbage Collector (in User Space) Has ability to change heap size Needs info: available memory, footprint Request: mem alloc footprint Send: mem alloc footprint

10. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 10 Outline Problem & Motivation Problem & Motivation Paging behavior of Garbage Collection Paging behavior of Garbage Collection VMM: collecting the information we need VMM: collecting the information we need Collector: adjusting heap size adaptively Collector: adjusting heap size adaptively Experimental results Experimental results Conclusion & Future work Conclusion & Future work

11. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 11 defghijklmncklmncbcdefghijklmncklmnabaabcdefghijklmnabcdefghijklmnabdefghijcklnmabcdefghijkmnlabcdefghijlmnkabdefghijklmnc4n3211 Approach to Measuring Footprint Memory reference sequence LRU Queue Pages in Least Recently Used order Hit Histogram Fault Curve 00000000000000mn 114 lmnklmncklmnabcdefghijklmncklmn 5 1 1 114 12 pages5 pages Associated with each LRU position

12. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 12 VMM design: SegQ [SKW’99,KMC’02] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 LRU Queue Hit Histogram Hot setCold set Evicted set 0 0 0 0 0 0 0 0 0 0 0 Hot / Cold Boundary 0 0 0 0 0 0 0 0 0 0 0 CLOCK algorithm Strict LRU Decay histogram periodically Adaptive control of hot set size Footprint Major fault (on disk)Minor fault (in memory)

13. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 13 VMM design: SegQ [SKW’99,KMC’02] Hot setCold set Evicted set Hot / Cold Boundary CLOCK algorithm Strict LRU Footprint What is the footprint w.r.t 5%? 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

14. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 14 Outline Problem & Motivation Problem & Motivation Paging behavior of Garbage Collection Paging behavior of Garbage Collection VMM: collecting the information we need VMM: collecting the information we need Collector: adjusting heap size adaptively Collector: adjusting heap size adaptively Experimental results Experimental results Conclusion & Future work Conclusion & Future work

15. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 15 Collector Design: Communicate with VMM after GC First GC: Appel, SS: HeapUtil = 0.5 Following GCs: Calculate HeapUtil from history

16. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 16 Outline Problem & Motivation Problem & Motivation Paging behavior of Garbage Collection Paging behavior of Garbage Collection VMM: collecting the information we need VMM: collecting the information we need Collector: adjusting heap size adaptively Collector: adjusting heap size adaptively Experimental results Experimental results Conclusion & Future work Conclusion & Future work

17. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 17 Experiments Experimental setting: Jikes RVM 2.0.3 Dynamic SimpleScalar: extended with new VMM model Major fault = 5 million instructions = 5 ms @ 1 Gips Minor fault = 2000 instructions = 2 µs Page fault cost threshold = 5% - 10% Histogram collecting cost threshold = 1% Adapting to fixed memory pressure Adapting to dynamic memory pressure Add/Remove 15MB real memory after 2 billion insts

18. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 18 Memory under-utilized Optimal heap Paging a lot Larger heap Fewer GCs Less Paging 60MB Real Memory Appel _213_javac

19. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 19 60MB Real Memory Appel _213_javac Increase memory: 15MB at 2 billion instructions

20. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 20 60MB Real Memory Appel _213_javac Decrease memory: 15MB at 2 billion instructions

21. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 21 Conclusion: Automatic Heap Sizing New collector usually picks heap size that: Maximizes memory utilization (reducing GCs) While avoiding paging Linear model works well in practice Improves performance by up to 8x under pressure Cost of collecting information is low: around 1% New collector adapts quickly to steady and to changing real memory allocations Within 1 or 2 major GCs

22. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 22 Ongoing Work Implement in real kernel Extend to more collectors Adjust during allocation, not just after GC Detailed graphs & tech report: http://www-ali.cs.umass.edu/~tingy/CRAMM