1. Productivity in Real-Time © Aonix Improving Abstraction, Encapsulation, and Performance within Mixed-Mode Real-Time Java Applications Kelvin Nilsen, CTO

2. Aonix© January 2007 Productivity in Real-Time 2 Real-Time Pyramid (Hierarchies and Layers) 10 μs 100 μ s 1 ms 10 ms 100 ms 1 s 10 s signaling sensing actuation coordination tactics strategy Perception reaction cognition Custom Hardware Hard Real-Time and/or Safety-Critical Java Soft Real-Time Java Traditional Non-Real-Time Java Credit: paper by David Bacon in ACM Queue, Feb. 2007

3. Aonix© January 2007 Productivity in Real-Time Domain of Lowest Level Software Demands for small footprint, high throughput, predictably low latency, long battery life “Hard Real-Time” means: –Proven compliance with all deadline constraints –Proven resource (memory, network bandwidth, other) availability –Requires Small and simple run-time environment, and Static behavior to enable analysis to prove reliable operation 3

4. Aonix© January 2007 Productivity in Real-Time Domain of Soft Real-Time Software tends to be larger, more complex, more dynamic Often incorporates many independently developed software modules, including COTS and open-source components Often includes dynamic reconfiguration (class loading), data replication and redistribution, and workload rebalancing in order to implement high availability and fault tolerance Usually defies static analysis 4

5. Aonix© January 2007 Productivity in Real-Time Premise One The domains of hard real-time software and soft real-time software are so different as to require very different: –Abstraction mechanisms –Development tools –Library standards –Even programming language semantics (e.g. tracing garbage collection vs. stack-memory allocation) 5

6. Aonix© January 2007 Productivity in Real-Time Premise Two As real-time systems grow in functionality, size, and complexity, it is increasingly important to enable: –Developers to select between hard real-time and soft real-time technologies for different parts of a system –Both hard real-time and soft real-time components to be efficiently and robustly combined in working systems –Strong separation of concerns between independently developed components to enable “blind” modular composition 6

7. Aonix© January 2007 Productivity in Real-Time Paper Focus This paper is not about proving that you can do real-time with Java –“Obviously, you can do both soft real-time and hard real- time with the Java language.” Deeper questions: Which approaches to soft and hard real-time with Java lead to: –The most cost savings during software development and maintenance –The most reliable and error free development –Greatest flexibility and fielded system longevity 7

8. Aonix© January 2007 Productivity in Real-Time The Java Concurrency Model Traditional Java programmers are told: –Make no assumption about implementation of thread priorities or time slicing –Make no assumption about synchronization queue orders –Make no assumption about priority inversion avoidance The Beauty: all of this “correctly written” Java software runs correctly if the Java VM implements “real-time scheduling” 8

9. Aonix© January 2007 Productivity in Real-Time Virtual Machine Abstractions Support Soft Real-Time Fixed-priority preemptive scheduling, with extended priority range Priority inheritance on all synchronization locks Maintain all thread queues in priority order Bounded-time implementations of interface invocations, instance-of, get-synchronization lock Real-time garbage collection 9

10. Aonix© January 2007 Productivity in Real-Time Contrast with RTSJ Soft real-time virtual machine does not support: –Scoped memory protocols –Deadline overrun detection –Cost overrun detection –Asynchronous transfer of control We consider these features to be quite difficult to use correctly, non-portable, and/or incompatible with off-the-shelf Java software 10

11. Aonix© January 2007 Productivity in Real-Time Virtual Machine Configuration Support Those responsible for deploying soft real- time systems need the ability to analyze software components empirically. The virtual machine reports “approximations” of: –CPU time spent by each thread –Cumulative CPU times consumed at each priority level –Memory allocation rates of running threads –CPU time required to perform garbage collection –Amount of memory live at end of each collection 11

12. Aonix© January 2007 Productivity in Real-Time Trace of Garbage Collection Pacing Preemption of GC by higher priority non-Java threads Preemption of GC by higher Priority Java threads 12

13. Aonix© January 2007 Productivity in Real-Time Configuration of the Pacing Agent Priority of garbage collection (GC) thread Lowest priority at which real-time application threads run –Heuristic: under transient overload, the garbage collector will steal additional CPU cycles below this priority Percentage of CPU time budgeted to real-time threads above the GC priority Percentage of CPU time budgeted to real-time threads below the GC priority Shortest relative deadline M of real-time threads at priority lower than the GC priority –The pacing agent divides the total GC effort into increments, taking no more than N% of each M time units 13

14. Aonix© January 2007 Productivity in Real-Time Robustness Under Normal Configuration Garbage collection can be preempted in approximately 100 µs Pacing heuristic attempts to predict the future based on recent history – this is not fool proof –Source code for pacing agent is supplied - customize if necessary to model “your favorite application” –If GC falls behind allocation pace, GC inherits priority of allocating thread Threads that have most demanding latency requirements avoid memory allocation and run at priority above GC –Will never experience unbounded priority inversion with GC 14

15. Aonix© January 2007 Productivity in Real-Time Demonstrated Relevance Telecommunications: fiber-optic switches, DSL broadband servers, television broadcast Industrial Automation: semiconductor manufacturing, oil exploration, power plants Aerospace: unmanned aircraft, satellites Defense: next-generation artillery, infantry PDA, unmanned autonomous land vehicles, Aegis warship 15

16. Aonix© January 2007 Productivity in Real-Time Combining Hard and Soft Real-Time For lowest level software, we recommend a small subset of RTSJ, patterned after the current work of JSR-302 (not discussed here) The challenge: –If you have an application that includes both low-level software and high-level software, what is the “best” way to combine these together? 16

17. Aonix© January 2007 Productivity in Real-Time Options Use Java for high-level software, and C for low-level software, connected with JNI Use Java for high-level software, and NoHeapRealtimeThread for low-level software, connected with wait-free queues Use the alternative protocol described here 17

18. Aonix© January 2007 Productivity in Real-Time Java and C with JNI Difficulties: –There is a run-time overhead associated with each call across the boundary between JNI and C –C code integrated with Java code compromises the Java security model Many Aonix customers report that the C/Java interface is the single most problematic source of errors in their embedded real-time systems 18

19. Aonix© January 2007 Productivity in Real-Time Java and NoHeapRealTimeThread NoHeapRealtimeThread and wait-free queues enable hard real-time determinism and cooperation between high- and low-level software However: –RTSJ Java runs slower and bigger than traditional Java –The wait-free queue abstraction introduces additional polling inefficiencies and handoff delays –Requires full trust regarding the other side of the queue 19

20. Aonix© January 2007 Productivity in Real-Time Cooperating HRT Components Hard Real-Time Execution Engine Traditional Java Virtual Machine 20

21. Aonix© January 2007 Productivity in Real-Time TraditionalJava Registry TraditionalJava TraditionalJava.publish(“deviceXYZ”, x) 21

22. Aonix© January 2007 Productivity in Real-Time TraditionalJava Registry “Hard Real-Time Domain” HardRealTimeDomain.lookup(“deviceXYZ”) 22

23. Aonix© January 2007 Productivity in Real-Time Contrast With Wait-Free Queues The semantic models are kept entirely separate –Libraries for hard real-time are different than libraries for soft real-time –No implementation/verification complexities because of mixing immortal, GC, and scoped memory Byte code verification of each world statically maintains referential integrity Either world can block waiting for communication from the other world, but always under full control of the hard real-time programmer –Hard real-time programmer is assumed more “trustworthy” 23

24. Aonix© January 2007 Productivity in Real-Time 24 All Java Solution Twice as Fast as C/Java! At ESC Silicon Valley 2007, Aonix demonstrated all Java application runs over twice as fast as hybrid solution comprised of Java and C Further benefit: all Java solution is easier to develop and maintain, offering superior separation of concerns 24

25. Aonix© January 2007 Productivity in Real-Time Summary Market adoption of Java for “mainstream” development has been driven largely by improvements in: –Reliability, modularity, scalability, portability, generality, flexibility, reusability, etc. If we intend to enable similar adoption of Java by the embedded real-time community, we must deliver similar benefits This paper describes progress towards this end 25