1. RTDroid: toward building dynamic real-time systems?
Lukasz Ziarek
University at Buffalo
Dagstuhl Seminar: Resource Bound Analysis

2. Collaborators Ethan Blanton – Fiji Systems Inc
Jan Vitek – Northeastern
Karthik Dantu – UB
Steve Ko - UB
Yin Yan (Ph.D. Student)
Many masters students

3. Motivations Open source project Well-defined APIs
Rich on-device peripherals
How can we write a real-time application on Android, while still leveraging Android’s existing functionality?

4. Cochlear Implant Device
Cochlear implant as a motivating example
An external device for audio recording and processing
Implanted electrodes stimulate the cochlea

5. Architecture of Cochlear Implant Application on Android
UI Activity
Audio Recorder
(Service)
Periodic Audio Recording Task
Volume Receiver
Audio Processor
(Service)
Button Listener
MsgHandler
Periodic Audio Processing Task
8 ms
UI Handler
Output Simulator
(Broadcast Receiver)

6. Sound Processing Latency
Duration of 12% of audio processing releases is longer than 8 ms.
Cochlear Implant: Audio Processing Duration

7. What Is Missing in Android?
Lack of real-time expressiveness
Can’t prioritize the execution of application components
Can’t specify real-time configuration
Lack of real-time communication
No priority awareness for message/intent delivery
No protection mechanism for cross-context communication
Lack of memory guarantees
Non Real-time components
Real-time components
UI Activity
Audio Recorder
(Service)
Periodic Audio Recording Task
Volume Receiver
Audio Processor
(Service)
Button Listener
MsgHandler
Periodic Audio Processing Task
UI Handler
Output Simulator
(Broadcast Receiver)

8. A View Of Android Applications No Real-time Expressiveness
FIFO Message Passing
Fair System Services
Application Framework
Constructs And APIs
System
Services
In order to see why Android is not predictable, we have to examine each layer in Android stack.
In bottom layer, it is well-known.
Many features are not predictable, for example ….
In nutshell, there is not predictability in kernel layer
In VM Layer, it is also well-known that it is not predictable.
In framework, our research show it is not predictable.
There layers are known no predictable,
The Android Linux Kernel is developed to achieving full CPU utilization, the predictability of the system were not taken into account.
VM: Dalvik was design for mobile devices and consider more about the memory optimization, battery lift and low frequent CPU.
Runtime
Unpredictable Garbage Collection
No Support for RT POSIX Extension
Bionic
Dalvik/ART
Fair Scheduling
Out of Memory Killer
Kernel Forced GC
Power Management
Linux with Android Modifications
Android Stack

9. Proposed Architectures for Real-time Android
Applications
RT Apps
Applications
RT Apps
Application Framework
Application Framework
Libraries
Core
Libraries
Core
Dalvik VM
Dalvik VM
Linux Kernel
Linux Real-time
Real-time Hypervisor
Completely Isolated

10. Proposed Architectures for Real-time Android
Applications
Applications
Application Framework
Application Framework
Libraries
Core RT
JVM
Libraries
Core
Dalvik VM
Extended Dalvik VM
Linux Real-time
Linux Real-time
Lower Latencies
Best Effort
Interactions Unclear ?

11. RTDroid Architecture Programing Model Validation Mechanisms
Applications
Application Framework
Redesigned Components
Redesigned Message Passing
Redesigned System Services
Runtime
Bionic
Extended Bionic
Real-time Garbage Collection
POSIX RT Extension
RTSJ Libraries
Dalvik/ART
Fiji MVM
Linux with Android Modifications
Priority Based Scheduling
PIP Locks
RT Linux or RTEMS
RTDroid Stack

12. Components and System Services

13. Redesigned Components
Application Framework
Constructs and APIs
Runtime System Services
RT Handler
RT AlarmManager
RTSJ
RT SensorManager
RTDroid
RT SoundManager

14. Why Looper and Handler? One of the central message-passing mechanisms.
All Android apps unknowingly use Looper-Handler.
Most of the system services use Looper-Handler.

15. Looper and Handler … Looper is a per-thread message loop.
Handler processes messages.
A thread uses Handler.send() to send a message.
Receiver Thread
Sender Thread
Looper
Message Queue
Handler
Handler Ref
(provides send()) …
Message
Message

16. What’s Wrong with Looper/Handler in a Real-time Context?
Message Queue
Looper …
RT msg
msg N
msg 1
Messages are processed FIFO or by client-specified time
Handler
No guarantee on
message delivery
RT client thread
Non-RT client thread
Non-RT client thread

17. RTLooper and RTHandler
Msg Queue
Msg Queue
RT Looper
(RT Thread)
Looper
(Non-RT Thread)
msg1
msg1 … …
msg n
msg n
RT Handler
Handler instance
Handler instance
RT client thread
Non-RT client thread
Non-RT client thread …

18. Android Sensor Manager APIs
Application
Enable/disable hardware
Accelerometer
Sensor Event
Listener
Configure sampling rate
Gyroscope
Subscribe to sensor events …
Magnetometer

19. How Does Android’s Sensor Manager Work?
Input
Event
HAL
Sensor
Service
Sensor
Manager
Sensor
Event
Listener
Sensor
Thread
Native
SensorManager
Sensor
Fusion
Sensor
Thread
Event
Queue
Kernel
System
Runtime
Framework
Application

20. Shared delivery path and unbound delivery time
What Happens In a Real-Time Context?
One application is listening on two sensors:
Accelerometer with higher priority
Gyroscope with lower priority
Application
Android Framework
Kernel
Accelerometer Data
Gyroscope Data
Shared delivery path and unbound delivery time

21. RT Sensor Manager Event-driven architecture
Prioritize the delivery procedure of sensor events
Multiple listeners with different sampling rates

22. Orientation Polling Thread
RT Sensor Manager
Listener 1
Listener 2
Listener 3
Listener 4
Delivery
Thread 1
Delivery
Thread 2
Delivery
Thread 3
Delivery
Thread 4
Listener’s rate and priority
Accel Events
Gyro Events
Orientation Event
The highest priority of its listeners
Sensor Fusion
Accel Polling Thread
Gyro Polling Thread
Orientation Polling Thread
/dev/input/
event6
/dev/input/
event1

23. Remaining Questions Multi-app execution
Real-time Android Programming model
Static Memory Guarantees

24. Multi-app Execution

25. Android: Application Isolation & Communication
One VM per application
Virtual Memory
Each application has its own PID, UID for isolation
App A
App B
System Server
App Components
App Components
ActivityManager
Activity Thread
ART
Activity Thread
ART
ServiceManager
Linux Kernel
Binder Driver

26. Multi-Tasking VM (MVM)
All applications run in a single VM. The VM runs each application in an isolation unit called a partition, but all applications run as a single process (the same memory space).
How does MVM provide isolation?
Separated Class loading (Fiji uses lightweight VMs)
Dynamically manage Memory boundaries
Statically Configured Heap Memory

27. Time Partitions in MVM MVM slice manager:
Fixed-length time slice for each partition
Partition initiation, suspension, resume, termination
Two approaches for thread scheduling:
Direct OS scheduling vs partition dispatcher with a thread pool

28. Communication in MVM Shared address space provides flexibility
Wait free pair transactions
One way communication channel over primitives
CAS and replicas
Priority Rollback Protocol
Inspired by transactional memory
Similar tracking structures
write buffers, undo log
The lock itself provides concurrency control

29. Programming Model

30. Can we write an App with RTDroid?
Applications
RT-Looper and RT-Handler
RT Alarm Manager
RT Sensor Manager
What else is missing?
Application Framework
Runtime
Extended Bionic
Fiji VM
RT Linux or RTEMS
Linux with Android Modifications

31. Android Linux Kernel with RT Patch
Programming Model
Android Apps
RT app
Android
Applications
RTDroid
Applications
Declarative real-time components
Real-time communication channel
Memory guarantees
region-based allocation
Design principles:
Retains a similar style of Android
Minimizes code complexity in application
Enables application validation with real-time configuration prior to execution
Android Framework
RTDroid Framework
Dalvik / ART
Fiji Java VM
Android Linux Kernel with RT Patch

32. Declarative Real-Time Components
<service name="pkg.ProcessingService” priority="79">
<memSizes persistent="1M” release="1M" />
<periodic-task name="procTask“ priority="79">
<release start="0ms" periodic="8ms" />
</periodic-task> …
</service>
Service
+ onStartCommand()
+ onBind()
+ onCreate()
+ onDestroy()
BroadReceiver
+ onReceive(…)
Real-time components
RT service
RT broadcast receiver
Periodic task
Application manifest
Priority
Timing parameters
Memory bounds
RealtimeReceiver
+ onClean()
RealtimeService
+ onPause()
PeriodicTask
+ onRelease()

33. Real-Time Communication Channels
<channel name=“msghandler” type=“msg- passing”>
<!-- message delivery and processing policies -->
</ channel>
Four types of channels
RT message passing channel
RT intent broadcasting channel
RT bulk data transfer channel
Cross-context channel
Communication policies
Message delivery order
Component invocation
Message constraints
<service name="pkg.RecordingService" priority=“78">
<intent-filter count="2" role=“publisher">
<action name="msghandler"/>
</intent-filter>
</service>
<service name=" pkg.ProcessingService " priority=“79">
<intent-filter count="2" role=“subscriber">
<action name="msghandler"/>
</intent-filter>
</service>

34. Memory Guarantees in RTDroid
Region-based memory management with constant cost for object allocation and reclamation.
Based on the concept of the scoped memory in RTSJ
The complexity of region management is hidden in our program model
The lifetime management of scope is associated with the lifetime of each application component.

35. Memory Guarantees in RTDroid
Immortal scope
Objects that have the same lifetime as the application
Persistent scope
Objects that have the same lifetime as its associated component
Release scope
Temporary objects for the execution of callback functions

36. Region-Based Memory Management in Cochlear Implant
Processing Service
Persistent
Release
Output
Receiver
Heap
Memory
(FijiVM)
Heap
Memory
(ART/Dalvik)
Android
Adaptor
UI Activities
Immortal Scope
RT Channels
RTDroid
Proxy
Real-time Runtime
in RTDroid
Android
Runtime

37. A Fully Dynamic System

38. Dynamic Real-Time Systems
Ability to install / remove / restart applications
Non real-time
Real-time
JIT as a service … for validation?
Validation of system wide constraints
Modification of an existing application
Adding / removing capabilities

39. System wide memory management
Can we manage memory more efficiently / predictably if we take a whole system view?
Hussein et al. showed for non RT

40. Validating Interactions
Session Types
Mechanisms for validating interactions
Communication protocols “match”
Participants are duals of one another
No notion of rate / resources
Partial failures
Error handling without restarting a session

41. Reasoning about power?
Power management is incredibly important for mobile
We currently do not do anything interesting
Lots of interesting work to explore
Energy types
Power modeling + power management

42. Validating Timeliness?
What is feasible to validate at runtime?
Can package static results with the app
Bytecode based WCET (Bates et al.)
Potential starting point?
JIT based specialization / validation?
Online Schedulability analysis?

43. Evaluation

44. Platform Categories RTEMS Soft Real-time Soft Real-time Desktop
Smartphone
Soft Real-time Desktop
Soft Real-time Embedded

45. Cochlear Implant Sensing Processing … Real-Time Processing
Non-Real-Time UI
Real-Time Processing …

46. Cochlear Implant

47. Evaluation on jPapabench

48. jPapaBench (UAV flight control)

49. Wind Farm Blade Monitoring

50. Wind Farm Blade Monitoring

51. Conclusion
RTDroid
A real-time mobile system with Android-like programming interfaces.
RT kernel + RT JVM + RT framework layer
RTDroid runs multi-apps in a single Multi-tasking VM with isolation.
RTDroid provides a real-time programming model with memory guarantees.

53. Porting jPapaBench into RTDroid
Fly-By-Wired (FBW)
Simulation
Autopilot
TestPPMTask
Handler
SimulatorFlight
ModelTaskHandler
Navigation
TaskHandler
SendDataTo
Autopolit
SimulatorIR
TaskHandler
AltitudeControl
TaskHandler
Data injection
SimulatorGPS
TaskHandler …
CheckFailsafe
TaskHandler
Data subscription
SensorManager
CheckMega128
ValuesTaskHandler
Stablization
TaskHandler

54. Base line performance on Base line performance on LEON3
Evaluation for Java Autopilot
RTEMS
The separation is caused by the interval between the time sensor data are injected into buffer in sensor manager, and the time the polling thread reads data.
Base line performance on
Nexus S
Base line performance on LEON3

55. RT SensorManager stress tests on LEON3
Evaluation for Java Autopilot
RTEMS
Message buffering dominates the overall latency,
Which means the interval between the time sensor
data are injected into buffer in sensor manager, and
the time the polling thread reads data.
The polling rate for IR sensor is bounded to 25ms. This is the largest interval in Polling.
We obtain a few latencies that are 30 ms in the listener extensive experiment.
the combination of the largest interval in polling with the preemption cost.
This overhead is mainly additional to additionally buffering costs
as the data is replicated for each priority level and the delivered to
corresponding handlers.
The overhead is proportional to the number
of low priority listeners and is the combination of the largest
interval in polling including the preemption cost.
There is little difference between 30 memory and 30 computational threads.
There is enough slack in the system for the GC to keep up
with memory allocations performed in the memory noise generating
threads.
RT SensorManager stress tests on LEON3

56. RTService Code Example
<service name=“.AutopilotService”
persistent=“1M” release=“1M”>
<rt-android:priority priority=90>
<SubService id=“stabilization-task”
release= “1M”>
<rt-android:priority priority=29>
<rt-android:periodic ms=“50”, ns=“0”>
</SubService>
<SubReceiver id=“sf-recvr” release=“1M”>
<intent-filter count=4>
<action name=“sim-status”>
<data size=“50K”
mimetype=“app/octet-stream”>
</itent-filter>
</SubReceiver>
</service>
Class AutopilotService extents RTService{
FlightModel fModel;
AutopilotModel aptModel;
public final SubReceiver sfRecvr = new
SubReceiver(){
pubic void onReciver(Content t, Intent i){
read intent msg; fModel.process(i); } };
public void onCreate(){
//flightModel, autopilotModel instantiation
stabTask = initSubService( new StabRunnable(aptModel, fModel));
registerRTSubReceiver(sfRecvr, ”sf-recvr”);

57. AutopilotService Persistent Scope (1M)
RTService Code Example
Class AutopilotService extents RTService{
FlightModel fModel;
AutopilotModel aptModel;
public final SubReceiver sfRecvr = new
SubReceiver(){
pubic void onReciver(Content t, Intent i){
read intent msg; fModel.process(i); } };
public void onCreate(){
//flightModel, autopilotModel instantiation
stabTask = initSubService( new StabRunnable(aptModel, fModel));
registerRTSubReceiver(sfRecvr, ”sf-recvr”);
AutopilotService Persistent Scope (1M)
AutopilotService
Release (1M)
stabTask
Release (1M)
sfRecvr
Release (1M)