Memory Management for Real-Time Java Wes Beebee and Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Supported by: DARPA Program Composition for Embedded Systems (PCES) Program

Goal: Enable Use of Java for Real- Time and Embedded Systems

Vision Downloaded Java Applets Standard Java Applications Real-Time Computation In Java Unified Language/Environment Facilitates Interaction

Why Java? Type safe language, no memory corruption Reasonably modern approach Object oriented Garbage collected memory management Popular and supported… Programmers available Tools available Libraries available

Implications and Issues Heterogeneous components with different needs and goals Real-time computation User interface Data management Issues Memory management Scheduling Event management and delivery Processor allocation

Why NOT Java Unpredictable memory usage Dynamic memory allocation Allocation hidden in extensive set of libraries and native methods Allocation hidden in exception model Unpredictable execution times Garbage collection No scheduling guarantees Thread scheduling Event delivery Complex libraries and native methods

Why NOT Java Impoverished set of abstractions Threads, mutex locks, signal and wait No good way express relationship between Events in system Corresponding pieces of computation No good way to express timing expectations Real-Time Java Approach Extend library Native methods for new mechanisms

Real-Time Java Standard Goal: Augment Java to better support real-time systems Augment memory model to enable threads to avoid garbage collection pauses Augment thread scheduling model to add more control over task scheduling Augment synchronization model to include lightweight event delivery mechanism

Our View Real-time Java is a work in progress Many of extensions generate More complex programming model More possibilities for errors Our goal Isolate general principles/concepts we think will last Develop new program analyses and implementation mechanisms That help programmers use real-time extensions safely and effectively

Java Memory Models Java: single garbage-collected heap Real-time Java: multiple kinds of memories Garbage-collected heap memory Immortal memory (live for full computation) Scoped memories (live for specific subcomputations) Linear-time allocation ( LTMemory ) Variable-time allocation ( VTMemory )

Problems/Issues with Memory Model Scoped memory issues Scoped memory reference checks Scoped memory sizes Avoiding garbage collection interaction issues No-heap real-time thread access checks Priority inversions caused by indirect interactions with garbage collector

Scoped Memory Overview Standard Java Computation

Scoped Memory Overview Standard Java Computation Objects in GC Heap

Scoped Memory Overview Standard Java Computation Objects in GC Heap

Objects in GC Heap Scoped Memory Overview New Computation Typically new thread Maybe even real-time thread Standard Java Computation

Objects in GC Heap Scoped Memory Overview New Computation Typically new thread Maybe even real-time thread Standard Java Computation Scoped Memory New Thread Runs In Scoped Memory

Objects in GC Heap Scoped Memory Overview New Computation Typically new thread Maybe even real-time thread Standard Java Computation Scoped Memory Thread’s New Objects Allocated in Scoped Memory Objects

Objects in GC Heap Scoped Memory Overview New Computation Typically new thread Maybe even real-time thread Standard Java Computation Scoped Memory Objects Thread’s New Objects Allocated in Scoped Memory

Objects in GC Heap Scoped Memory Overview New Computation Typically new thread Maybe even real-time thread Standard Java Computation Scoped Memory Objects Thread’s New Objects Allocated in Scoped Memory

Objects in GC Heap Scoped Memory Overview New Computation Typically new thread Maybe even real-time thread Standard Java Computation Scoped Memory Objects Computation Terminates

Objects in GC Heap Scoped Memory Overview New Computation Typically new thread Maybe even real-time thread Standard Java Computation Scoped Memory Objects Objects in Scoped Memory Deallocated as a Unit without GC

Scoped Memory Motivation Dynamic memory allocation without GC Tie object lifetimes to computation lifetimes Eliminate need to dynamically trace out reachable objects Warning: Example illustrates primary intended use Specification allows more behaviors Scoped memories shared by multiple threads Nested scoped memories Scoped memories entered multiple times

Safety Issue for Scoped Memories: Dangling References Lifetimes of objects in scoped memory determined by lifetime of computation Must ensure that no reference goes from long-lived object to short-lived object

Nested Scoped Memories Scoped Memory Object

Referencing Constraints Scoped Memory Object Referencing Down Scopes Is NOT OK Referencing Up Scopes Is OK

Preventing Downward References Dynamic Reference Checks At every write of a reference into an object field or array element Check that written object is allocated in a scope with a lifetime at least as long as that of referred object If not, throw an exception Drawbacks Dynamic checking overhead New class of dynamic errors

Static Analysis Goal Eliminate need for dynamic checks by Statically checking that program does not violate referencing constraints Basic approach: escape analysis

What Escape Analysis Provides void compute(d,e) ———— void multiplyAdd(a,b,c) ————————— void multiply(m) ———— void add(u,v) —————— Control Flow Graph Nodes = methods Edges = invocation relationships

What Escape Analysis Provides void compute(d,e) ———— void multiplyAdd(a,b,c) ————————— void multiply(m) ———— void add(u,v) —————— Allocation Site Control Flow Graph Nodes = methods Edges = invocation relationships

What Escape Analysis Provides void compute(d,e) ———— void multiplyAdd(a,b,c) ————————— void multiply(m) ———— void add(u,v) —————— Allocation Site Object Allocated Here Does Not Escape Computation of multiplyAdd method Control Flow Graph Nodes = methods Edges = invocation relationships

Our Escape Analysis Interprocedural Analyzes interactions between methods Recaptures objects in callers of allocating methods Compositional Analyzes each method once Single analysis result that can be specialized for use in different calling contexts Suitable for multithreaded programs Analyzes interactions between threads Recaptures objects that do not escape a given multithreaded computation

Using Escape Analysis to Verify Correct Use of Scoped Memories For each computation that runs in scoped memory Check that allocated objects do not escape

Implementation FLEX compiler infrastructure ( Full Java compiler Lots of utilities and packages Support for deep program analyses and transformations Implemented scoped memories and checks Implemented escape analysis Used results to eliminate checks In applications, eliminated all checks

Experimental Results Array (Heap) Array (Scope) Tree (Heap) Tree (Scope) Water (Heap) Water (Scope) Barnes (Heap) Barnes (Scope) Benchmarks Time (sec) Scope Checks Application

Scoped Memory Sizes Scoped memory creation and size MemoryArea ma = new LTMemory(10000); “create a new scoped memory with 10,000 bytes” If try to allocate more than 10,000 bytes, implementation throws an exception Problems Java does not specify object sizes Size of given object may change during its lifetime in computation So how big to make scoped memory?

Objects in GC Heap Modularity Problems Scoped Memory Size Determined Here Standard Java Computation Scoped Memory Objects Required Size Determined by Behavior of Code in this Computation

Objects in GC Heap Modularity Problems Scoped Memory Size Determined Here Standard Java Computation Scoped Memory Objects If change program, size may need to change! Amount of allocated memory becomes part of interface!

More Issues Different executions may allocate different amounts of data Lots of hidden allocation in libraries Difficult to find out how much memory is really allocated If change implementation, may need to change scoped memory size in clients

Analysis Solution Analyze program to symbolically compute allocated memory sizes Input variables Object sizes Compiler knows object sizes, can conservatively generate scoped memory sizes

Interaction with Garbage Collector Standard Collector Assumptions Can interrupt computation at any point Can suspend for unbounded time Real-Time Java extension No-Heap Real-Time Threads Can Access Immortal memory Scoped memory Do not interact with GC heap AT ALL Can run asynchronously with GC Immortal Scoped GC Heap

No-Heap Real-Time Thread Checks Dynamically check that no-heap real-time threads never access a location containing a reference into garbage-collected heap At every read, check to make sure result does not point into garbage-collected heap At every write, check to make sure not overwriting reference into GC heap If check fails, throw exception Drawbacks Dynamic checking overhead New class of dynamic errors

Implementation FLEX compiler infrastructure ( Implemented no-heap real-time threads Implemented access checks Measured performance with and without checks

Experimental Results ArrayTreeWater Benchmark Time (sec) Scope Checks Heap Checks Application

Program Analysis for Eliminating Checks Control-flow analysis to identify code that may execute in no-heap real-time thread Global value-flow analysis Tags each value that points to GC heap Identifies all locations into which these values may flow Combine results Look at all no-heap real-time thread code Check statically for access violations

Analysis Issues and Solutions Complicated, whole-program analysis Simple annotations can enable local analysis Annotate each reference with source Scoped memory Immortal memory Heap memory Annotation checker validates annotations Result: Scalable analysis Eliminate checks, eliminate programming errors

Indirect Priority Inversions Standard Java Thread No-heap Thread Lock Acquire Garbage Collector Lock Acquire Interaction Between Resource Sharing and Garbage Collection

Indirect Priority Inversions Standard Java Thread No-heap Thread Lock Acquire Garbage Collector Lock Acquire Interaction Between Resource Sharing and Garbage Collection Blocks

Indirect Priority Inversions Standard Java Thread No-heap Thread Lock Acquire Garbage Collector Lock Acquire No-heap thread must wait for standard Java thread to release lock Standard thread must wait for GC to finish (heap inconsistent until it finishes) No-heap thread must wait for GC!

Using Non-Blocking Synchronization to Eliminate Indirect Priority Inversions Standard Java Thread No-heap Thread Start Atomic Region Garbage Collector Start Atomic Region End Atomic Region Does Not Block! Abort, Retry

Implementation Status Non-blocking synchronization implemented for memory management primitives Useful when threads share scoped memory Uses non-blocking synchronization instructions from processor Software implementation underway for general atomic regions

Goal Dangling references for scoped memories Resource needs of computations Isolating computations from garbage collector Ensure threads with real-time constraints don’t access garbage collected data Eliminate indirect interactions Our view Dynamic checks inadequate Statically verify correct use, eliminate checks Enable safe real-time code to interact successfully with code that accesses GC data Issues and Complications

Broader View of Real-Time Java Java is best suited to express “batch” computations on objects Not so good for control in asynchronous, parallel, distributed, time-aware systems Inadequate for design/requirements Can be part of solution, but only a part

Multiple Perspectives Any system has many desired properties Data structure invariants Flow of data between different components Timing requirements for computations Each property is inherently partial Many properties are best expressed as Declarative constraints NOT translatable into implementation

Design Conformance Object Referencing Relationships Timing Constraints Dataflow Interactions Implementation Check that implementation conforms to design properties

Future Scheduling and event delivery More precise referencing relationship analysis Completely characterize aliasing behavior More flexible memory management algorithms that preserve predictability More flexible regions Immediate deallocation Less restricted memory access constraints for real- time threads Design conformance for more control-oriented properties

Summary Real-time Java code coexists and interacts with standard Java code New complications (overhead + failure modes) Scoped memory checks Scoped memory sizes No-heap real-time threads Indirect priority inversions Attacked with program analysis Future: scheduling and timing