1. Complutense University, Madrid. Spain. * IMEC, Heverlee. Belgium. SEMINARIO 5-Marzo-2004 Garbage Collectors Refinement for New Dynamic Multimedia Applications on Embedded Systems Garbage Collectors Refinement for New Dynamic Multimedia Applications on Embedded Systems José M. Velasco, David Atienza, Francky Catthoor*, Francisco Tirado, Katzalin Olcoz, José M. Mendias

2. Seminario 5 Marzo 2004 2 Motivation  New application domains of embedded systems: Mobile terminals ( all sort of multimedia applications) Mobile terminals ( all sort of multimedia applications) AutomotiveAutomotive Game processors, Game processors, PDA, MP3 Player, etc.PDA, MP3 Player, etc.  Java is present everywhere. Java is a General Purpose OOP Language !!!!!! Java is a General Purpose OOP Language !!!!!!

3. Seminario 5 Marzo 2004 3 Java Popularity  Portability  JVM  OOP  Reusable Code  Dynamic Class Loading  Security  Multi-Thread  Automatic Memory Management

4. Seminario 5 Marzo 2004 4 Manual MM versus Automatic MM C++ malloc ( ) Free ( )  Destroy object Java Java { Class1 Object = new Class1 Class1 Object = new Class1 }  Take reference out from stack }  Take reference out from stack

5. Seminario 5 Marzo 2004 5 Automatic Memory Management Stack Pointer 2 Pointer 1 Heap Object 2 Object 1

6. Seminario 5 Marzo 2004 6 Automatic Memory Management Stack Pointer 1 Heap Unreacheable Object 1

7. Seminario 5 Marzo 2004 7 Automatic Memory Management Stack Pointer 3 Pointer 1 Heap Object 3 GARBAGE Object 1

8. Seminario 5 Marzo 2004 8 Manual MM versus Automatic MM Alive Internal Frag. External Frag. Alive Dead Internal Frag. C ++ Java

9. Seminario 5 Marzo 2004 9 Java

10. 10 Virtual Machine Organization Compiler Subsystem Bytecodes.class JVM Dynamic class loading:.class Thread and synchronization Subsystem Run-Time Subsystem Memory Managment Subsystem

11. Seminario 5 Marzo 2004 11 Execution Profile

12. Seminario 5 Marzo 2004 12 A Mono-Application Virtual Machine JVM App. 1 App. 2 App. 3 Memory JVM JVM

13. Seminario 5 Marzo 2004 13 Automatic Memory Management  Allocation. It is the process of assigning resources of memory when requested by the program.  Garbage Collection. It is the automatic recycling of this dynamically allocated memory when it is no longer needed.  A lot of GC strategies  Reference Count, Mark&Sweep, M&S compact, Copying, Generational, Older First, Connectivity, Incremental,... All of them focused in one Objective.  Simplicity, performance, real time...All of them focused in one Objective.  Simplicity, performance, real time...  Why GC Refinements are important? Performance. Memory Footprint. Real Time requirements. Energy consumption. Tradeoff

14. Seminario 5 Marzo 2004 14 Presentation Overview  Garbage Collection Distinguishing Garbage.Distinguishing Garbage. Reclaiming Garbage.Reclaiming Garbage. AllocationAllocation Heap Organization.Heap Organization. Write BarriersWrite Barriers  The goal: JVM on Embedded SystemsJVM on Embedded Systems Concurrent Multimedia applicationsConcurrent Multimedia applications  Our proposal Multi-Application JVMMulti-Application JVM Custom GCsCustom GCs Adaptive techniquesAdaptive techniques  Future Work

15. Seminario 5 Marzo 2004 15 Distinguishing Garbage Phase.

16. Seminario 5 Marzo 2004 16 Distinguishing Garbage Phase. Reference Count. 2 StackHeap

17. Seminario 5 Marzo 2004 17 Distinguishing Garbage Phase. Reference Count. 1 StackHeap

18. Seminario 5 Marzo 2004 18 Distinguishing Garbage Phase. Reference Count. 0 StackHeap

19. Seminario 5 Marzo 2004 19 Distinguishing Garbage Phase. Reference Count. 2 1 2 1 1 StackHeap

20. Seminario 5 Marzo 2004 20 Distinguishing Garbage Phase. Reference Count. 1 1 1 1 1 StackHeap

21. Seminario 5 Marzo 2004 21 Distinguishing Garbage Phase. Tracing Collection. StackHeap

22. Seminario 5 Marzo 2004 22 Distinguishing Garbage Phase. Tracing Collection. StackHeap

23. Seminario 5 Marzo 2004 23 Distinguishing Garbage Phase. Tracing Collection. StackHeap

24. Seminario 5 Marzo 2004 24 Distinguishing Garbage Phase. Tracing Collection. StackHeap

25. Seminario 5 Marzo 2004 25 Cache Behavior during Tracing Collection 1 32 546 78 257 678 Cache Misses Misses Misses Misses

26. Seminario 5 Marzo 2004 26 Reclaiming Garbage Phase

27. Seminario 5 Marzo 2004 27 Reclaiming Garbage Phase. Tracing Collection. StackHeap Free lists

28. Seminario 5 Marzo 2004 28 Free Lists Pool

29. Seminario 5 Marzo 2004 29 Free Lists -- Fragmentation Pool

30. Seminario 5 Marzo 2004 30 Semispace Copying Policy Stack Pointer 2 Pointer1Semispace

31. Seminario 5 Marzo 2004 31 Semispace Copying Policy Stack Pointer 2 Pointer1Semispace

32. Seminario 5 Marzo 2004 32 Semispace Copying Policy Stack Pointer 2 Pointer1Allocation Semispace Free Semispace

33. Seminario 5 Marzo 2004 33 BumpPointer

34. Seminario 5 Marzo 2004 34 BumpPointer

35. Seminario 5 Marzo 2004 35 BumpPointer

36. Seminario 5 Marzo 2004 36 BumpPointer

37. Seminario 5 Marzo 2004 37  Mark&Sweep. Free List.Free List. Fragmentation. Fragmentation. Poor Locality. Poor Locality. Small Heap size?. Small Heap size?.  Copying Policy Bump Pointer.Bump Pointer. Available memory / 2. Available memory / 2. High Cost for copying. High Cost for copying. Big Heap Size?. Big Heap Size?.

38. Seminario 5 Marzo 2004 38 Execution Mode  Stop-the-World. Parallel.  Concurrent.  Incremental Program Program ProgramProgram Program ProgramGC GC GC GCGCGC ProgramProgramProgram ProgramProgramProgram

39. Seminario 5 Marzo 2004 39 Barriers Collected SubHeap SubHeap Not Collected SubHeap Remember

40. Seminario 5 Marzo 2004 40 Heap Organization Copying policy Mark&SweepMark&Sweep Large Objects Heap

41. Seminario 5 Marzo 2004 41 Mark&Sweep

42. Seminario 5 Marzo 2004 42 SemiSpace Copying Policy

43. Seminario 5 Marzo 2004 43 Copy + Mark&Sweep

44. Seminario 5 Marzo 2004 44 Generational Garbage Collection Strategy Generational Collection. Weak Generational Hypothesis.Weak Generational Hypothesis. Java Objects Lifetime. Java Objects Lifetime. AdultYoungQuasi-Inmortal 60% 60%10%30% Java Distribution Gauss Distribution

45. Seminario 5 Marzo 2004 45 Generational Collection  The memory is divided into multiple subheaps with varying scavenge frequencies  Objects begin their life in the higher collection frequency area.  The objects that survive a number of collections move to a less frequently collected area.  This collector avoids to copy repeatedly the inmortal or quasi-inmortal objects. Mature Nursery Two generations Heap

46. Seminario 5 Marzo 2004 46 Generational – Mature  Mark&Sweep Policy

47. Seminario 5 Marzo 2004 47 Generational – Mature  Copying Policy

48. Seminario 5 Marzo 2004 48 Pauses Distribution through Execution Time

49. Seminario 5 Marzo 2004 49 Minimum Program Utilization

50. Seminario 5 Marzo 2004 50 Java on Embedded Systems Minimum Heap Size  Max Number of living ObjectsMinimum Heap Size  Max Number of living Objects Copying  x2 Copying  x2 Mark&Sweep  + Frag Mark&Sweep  + Frag Soft Real Time  PerformanceSoft Real Time  Performance Pause Time  MPU Pause Time  MPU Write Barriers Write Barriers  Incremental  Concurrent Tracing  Cache Misses Tracing  Cache Misses Mono-Application Virtual MachineMono-Application Virtual Machine

51. Seminario 5 Marzo 2004 51 Embedded Systems  Multimedia Applications  Internal Dynamism Unpredictable data behavior.Unpredictable data behavior.  External Dynamism Several applications working concurrently.Several applications working concurrently. Different scenarios of available memory.Different scenarios of available memory. How can we handle these dynamic systems?

52. Seminario 5 Marzo 2004 52 Our proposal

53. Seminario 5 Marzo 2004 53 A Multi-Application Virtual Machine JVM App. 1 App. 2 App. 3 Min Min x 3 Min x 1.5 Memory

54. Seminario 5 Marzo 2004 54 A Multi-Application Virtual Machine JVM App. 1 App. 2 App. 3 Min x 2.5 Min Min x 2 Memory

55. Seminario 5 Marzo 2004 55 A Multi-Application Virtual Machine JVM App. 1 App. 2 App. 3 Min Min x 1.5 Min x 4 Memory

56. Seminario 5 Marzo 2004 56 Overview of Our Approach In this paper we propose a new method that allows the design of custom GCs for new dynamic applications (e.g. multimedia), taking into account the application behavior and the constraints of the final embedded device. - Analysis (mainly profiling) of the application - Characterizing DM behaviour pattern

57. Seminario 5 Marzo 2004 57 Overview of Our Approach Application Design Search Space  All possible decisions to select an optimal GC This Search Space is formalized as a set of orthogonal trees with basic interdependencies a set of orthogonal trees with basic interdependencies

58. Seminario 5 Marzo 2004 58 Overview of Our Approach Application We propose a new system-level method to explore the vast design search space of possible garbage collector mechanisms, Our long term Goal is to develop an automatic tool to traverse the trees and produce a tailor made GC for a specific multimedia application with a tradeoff between performance, memory footprint, real time requirements and energy consumption. Our long term Goal is to develop an automatic tool to traverse the trees and produce a tailor made GC for a specific multimedia application with a tradeoff between performance, memory footprint, real time requirements and energy consumption.

59. Seminario 5 Marzo 2004 59 Overview of Our Approach Application GCRefinements As a result of our Refinements As a result of our Refinements a set of modules that form a custom GC a set of modules that form a custom GC These modules are linked to the Java Virtual Machine that will run over the final device

60. Seminario 5 Marzo 2004 60 Overview of Our Approach Application GCRefinements Linked Modules Linked Modules The custom GC can be a mixed hardware/ Software implementation. The custom GC can be a mixed hardware/ Software implementation. - Write Barriers. - Real-Time Restrictions. -Hardware Design Tool.

61. Seminario 5 Marzo 2004 61 Overview of Our Approach Application GCRefinements Linked Modules Linked Modules Dedicated Hardware And finally, a convenient custom GC design for a specific embedded system without a complex engineering process.

62. Seminario 5 Marzo 2004 62 Overview of Our Approach – Matisse Project (IMEC) Application GCRefinements Linked Modules Linked Modules Dedicated Hardware MicroProcessorFPGA

63. Seminario 5 Marzo 2004 63 Summary of our proposed approach Main objective: - New design method to design custom GCs for new dynamic applications (e.g. multimedia), taking into account the application behavior and the constraints of the final embedded device. Phases: - Analysis (mainly profiling) of the application - taking into account the application behavior and the constraints of the final embedded device. - Systematic exploration of possible GC solutions from structured (orthogonalized) design space, taking into account the application behavior and the constraints of the final embedded device. - Implementation of promising GC solutions and empirical evaluation of them.

64. Seminario 5 Marzo 2004 64 Our Design Space  Design Search Space of Orthogonal Decisions for Garbage Collectors AllocationAllocation Distinguishing Garbage.Distinguishing Garbage. Reclaiming Garbage.Reclaiming Garbage. Heap Organization.Heap Organization. Write BarriersWrite Barriers  Any Combination of decisions in Orthogonal trees  Valid Solution  All possible solutions in the search space should be spanned by a combination of leaves in the orthogonal trees.

65. Seminario 5 Marzo 2004 65 The different zones distinguished by our custom GCs A-ZoneB-ZoneC-Zone

66. Seminario 5 Marzo 2004 66 Experimental Results.

67. Seminario 5 Marzo 2004 67 Experimental Results.

68. Seminario 5 Marzo 2004 68 Adaptive Techniques

69. Seminario 5 Marzo 2004 69 Minimum Heap Size  Different Benchmarks Minimum Heap Size  Different Benchmarks

70. Seminario 5 Marzo 2004 70 Appel Collector

71. Seminario 5 Marzo 2004 71 Adaptive Management of Reserved Space

72. Seminario 5 Marzo 2004 72 Dynamic Management of Nursery Organization. Jack.

73. Seminario 5 Marzo 2004 73 Dynamic Strategies  Dynamic “Worst Case” strategy.  Dynamic “Average” strategy.

74. Seminario 5 Marzo 2004 74 Mispredictions

75. Seminario 5 Marzo 2004 75 Nursery Threshold

76. Seminario 5 Marzo 2004 76 Experimental Results  Metrics: Amount of Copied memory.Amount of Copied memory. Number of Collections.Number of Collections. Global Collection Time.Global Collection Time. Execution Time.Execution Time.

77. Seminario 5 Marzo 2004 77 Amount of copied memory Jess Jack

78. Seminario 5 Marzo 2004 78 Summary

79. Seminario 5 Marzo 2004 79 Reduction in the number of collections. Jess

80. Seminario 5 Marzo 2004 80 Summary

81. Seminario 5 Marzo 2004 81 Total Collection Time SpeedUp

82. Seminario 5 Marzo 2004 82 Summary

83. Seminario 5 Marzo 2004 83 Execution Time SpeedUp

84. Seminario 5 Marzo 2004 84 Summary

85. Seminario 5 Marzo 2004 85 Conclusions Generational Collector that Dynamically:Generational Collector that Dynamically: Readjust the reserved space in the nursery. Readjust the reserved space in the nursery. Changes the nursery threshold that triggers a Full Heap Collection. Changes the nursery threshold that triggers a Full Heap Collection. Novel Field of Research.Novel Field of Research. Can be used jointly with others strategies based in Copying Policy.Can be used jointly with others strategies based in Copying Policy.

86. Seminario 5 Marzo 2004 86 Current work  Incremental Copy

87. Seminario 5 Marzo 2004 87 Write Barriers  Remembered Sets LOS Root Set

88. Seminario 5 Marzo 2004 88 Write Barriers  Remembered Sets LOS Root Set

89. Seminario 5 Marzo 2004 89 Is Not Complete !!! Is Not Complete !!!

90. Seminario 5 Marzo 2004 90 Is Not Complete !!!

91. Seminario 5 Marzo 2004 91 Adaptive  Incremental

92. Seminario 5 Marzo 2004 92 Future Work – BrainStorming Heap CompressionHeap Compression Selective Selective  Object fields  References  Generations Compiler AnalysisCompiler Analysis Objects  Used Fields Objects  Used Fields Fixed Regions  Stack Fixed Regions  Stack Data Connectivity Data Connectivity Hardware SupportHardware Support Write Barriers Write Barriers Compression Compression Tracing Tracing On-Line ProfilingOn-Line Profiling Class behavior Class behavior Data Connectivity behavior Data Connectivity behavior Gehringer & Chang Mushroom (Wolczko&Williams) L1 L2Co-processorReferenceCounting L1Nursery TaggedMemoryFixed regions regions