Real-time Embedded Systems Complex RMS and deadline monotonic scheduling

D ESIGN U SING RMT: I SSUES STILL TO DO Integration with background (non-essential) tasks Handling aperiodic tasks (safety critical & essential) but: somewhat more complex is: Handling task interaction (both periodic & aperiodic ) and linking the whole lot together

S CHEDULING: SHARING RESOURCES Preemption Execution delayed by higher priority tasks (as we expect) Blocking (Priority inversion) Execution delayed by lower priority tasks Mutual Exclusion (Mutex) Sequenced access to a shared resource, Critical Section Exclusive execution while holding a 'lock'** ** of course actually a semaphore or equivalent

S CHEDULING AND T ASK I NTERACTION Task synchronisation mechanism: Binary Semaphores: Takes either value 0 or 1 Operations: wait(S) :- if S=1 then S:=0 {& carry on} else suspend; signal(S) :- S:=1; {& unsuspend} Sharing resources: wait (mutex) -- access resource {critical section} signal(mutex) wait & signal used for clarity could be P & V

P ROBLEM: P RIORITY I NVERSION Periodic tasks T1, T2, T3 have priorities (high, med, low) T1 & T3 share a resource guarded by semaphore S. T1 is blocked by T3 in the critical section and also by T2 which pre-empted T3 while it was in the critical section. T1 delay is potentially unbounded.

P RIORITY I NVERSION: S OLUTION 1 Make critical sections non-preemptible Only works well if critical sections are very short Now look at a real example

M ARS P ATHFINDER AND R OVER Landed on Martian surface July 4th, Had total system resets resulting in data loss VxWorks (Wind River) Real-Time Embedded Kernel Preemptive priority scheduling of threads Task Synchronization by Mutex, priorities assigned by relative urgency of tasks Management task: moves data for shared store "information bus" Short Period, High Priority Meteorological data task Long Period, Low Priority Communications task: Long task, Med Period, Med Priority

P RIORITY I NVERSION ON M ARS R OVER Single CPU controls spacecraft VME bus containing interface cards for: Radio, Camera, Interface to 1553 bus for.... (inherited from Cassini spacecraft) Hardware-specific schedule activity at 8 Hz rate Reference: David Wilner CTO of Wind River Systems, photos NASA Interrupt caused the Communications task to Be scheduled during the short time while the Management task was blocked waiting for the Meteorological task The Watchdog Timer on the data bus initiated a total System Reset to recover - losing a days work

P RIORITY I NVERSION ON M ARS R OVER Interrupt caused the Communications task to be scheduled during the short time while the Management task was blocked waiting for the Meteorological task. The Watchdog Timer on the data bus initiated total System Reset to recover thus losing a days work each time it happened Watchdog Timer System Reset Time Priority Management Communications Meteorology wait(S) signal(S) Intrpt signal(S)

H OW T HEY F IXED R OVER P RIORITY I NVERSION Rover's Trace/Log facility records system events (e.g. context switch, synchronizations, interrupts) JPL engineers took 18 hrs to reproduce the reset (on the ground replica) IPC (Inter Process Comms) mechanism uses VxWorks pipe () Tasks wait on one or more IPC "queues" for messages to arrive Tasks use select () to wait for message arrival Multiple queues when both high & lower priority messages are needed

H OW T HEY F IXED R OVER P RIORITY I NVERSION 2 Select mechanism creates a mutual exclusion semaphore to protect the "wait list" of file descriptors for devices using select (). e.g. pipe() VxWorks mutex object is initialized with Priority Inheritance parameter set OFF. Patching spacecraft software involves sending difference updates +whole bunch of validation and was successful. "Test what you fly and fly what you Test "

P RIORITY I NVERSION: S OLUTION 2 Priority Inheritance Protocol (PIP) Reference: Sha, Rajkumar & Lehoczky, "Priority Inheritance Protocols: An Approach to Real-Time Synchronisation", IEEE TOCS, Vol 39, No 9, Sept Task T runs at its assigned priority unless it is in a critical section and blocks higher priority tasks. If task T blocks higher priority tasks, T inherits the highest priority of jobs blocked by T. When T exits critical section it resumes entry priority

P RIORITY I NHERITANCE P ROTOCOL Priority Inheritance is transitive. If T3 blocks T2 AND T2 blocks T1 then T3 inherits the priority of T1 via T2. When a task performs signal operation at the end of a critical section, the highest priority blocked task is activated. i.e. Priority queued not FIFO

P RIORITY I NHERITANCE P ROTOCOL EXAMPLE Periodic tasks T1, T2, T3 have priorities (high, med,low) T1 and T3 share a resource guarded by semaphore S T1 is now only blocked for length of critical section

P RIORITY I NHERITANCE : B LOCKING C HAINS Here T1 is blocked for duration of TWO critical sections Time In general, under P.I.P. if there are m semaphores which can block T then T can be blocked at most m times. With n lower priority tasks than T; T can be blocked for at most the duration of one critical section in each. Priority T1 T2 T3 wait(S1) wait(S1);wait(S2) signal(S1) wait(S2)signal(S2)

P RIORITY I NVERSION: S OLUTION 3 - C EILING Priority Ceiling Protocol is an extension of P.I.P. Each semaphore is assigned a priority ceiling equal to the highest priority task which can use it A task T may only lock a semaphore S if T's priority is higher than all priority ceilings of all semaphores locked by jobs other than T Guarantees that if T preempts a critical section & starts executing a new critical section, the priority of the new section will be higher than inherited priorities of all the preempted critical sections

P RIORITY C EILING P ROTOCOL: PRINCIPLE T2 cannot lock S2 since T2's priority is less than the ceiling of S1 T1 is only blocked for duration of one critical section The cost is the extra blocking...

P RIORITY C EILING P ROTOCOL: THEOREM 3 At most one lower priority task can block a higher priority task under the priority ceiling protocol

T1 High priority uses S1, S2 T2 Med priority uses S2 T3 Low priority uses S1 P RIORITY C EILING P ROTOCOL: EXAMPLE T3T3 T3 is only active task. Takes S1 Requests S1 & gets blocked T3T3 Preempted by T2 T2T2 T3 runs & releases S1 T2 tries to take S2 - CP stops it T3T3 T1T1 Preempted by T1 T1T1 T1 runs and Takes S1 & S2 T1 ends & releases S1 & S2 T2T2 T2 runs & takes S2 T3T3 T2 ends & releases S2

P RIORITY C EILING : M UTUAL D EADLOCK Priority Ceiling Protocol prevents mutual deadlock T1:{..wait(S1); wait(s2).....signal(s2);signal(S1)..} T2:{..wait(S2); wait(s1).....signal(s1);signal(S2)..}

P RIORITY C EILING P ROTOCOL: THEOREM 4 Extend the exact schedulability test (based on theorem 2) to include interactions and blocking Theorem 4: Critical Zone Theorem +B For a set of periodic tasks with dependencies, if each task meets its first deadline when all tasks are started at the same time and all blocking is included, then the deadlines will always be met for any combination of start times

I MPLEMENTATION D ETAIL: P RIORITY I NHERITANCE Tasks suspended on semaphores must be queued in priority order wait(s) if task is to be blocked then increase priority of task holding the semaphore to that of the task calling wait i.e. Priority inheritance operation PLUS usual queue operation signal(s) reduce priority to value task had before it called the corresponding signal operation + usual activation or increment operation

I MPLEMENTATION D ETAIL: P RIORITY C EILING No semaphore queues. Ready list replaced by a priority ordered list of tasks which are either ready to run or blocked by the priority ceiling protocol. Task at the head of the list is running. Executive also maintains S_list, list of currently locked semaphores ordered by priority ceiling. i.e. a wait(s) operation can easily check whether its caller has higher priority than head of S_list. Each semaphore records the task which holds it (if any) and its ceiling.

D ESIGN U SING R ATE M ONOTONIC T HEORY Integration with background (non-essential) tasks Stability Handling aperiodic tasks (safety critical and essential) Sporadic Servers Periodic Task Inter-action Priority Ceiling

P ERIODIC T ASKS - I NCLUDING D EADLINES We had - - Task has execution time C and period P A set of periodic tasks is schedulable if every periodic task finishes its execution before the end of its period. The Period defines the Window during which the task must execute. ADD: Task may have Deadline before end of its Period. Now: A set of periodic tasks is schedulable if every task finishes its execution before its Deadline is reached. Now: The Window is narrower than the period. The Window is (Deadline - Start of Period) using absolute times

D EADLINE M ONOTONIC S CHEDULING This is called: DEADLINE MONOTONIC SCHEDULING It is still Optimal to assign higher priority to tasks with narrower windows* * Reference: Leung J. and Whitehead J. "On the complexity of Fixed-Priority Scheduling of Periodic Real-Time Tasks", Performance Evaluation(2), Deadline Monotonic Scheduling Strategy Assign each periodic task a priority which is inversely proportional to its Window task with shortest window gets highest priority Priority 1 / Window

D EADLINE M ONOTONIC S CHEDULING All of the theorems & techniques of RMT [inc. scheduling points etc.] may be used with the Period for RMT replaced by the Window for DMT [Deadline Monotonic] Remembering: Blocking [task interactions], Aperiodics, Sporadics etc. An Early deadline (Di < Pi) can also be modeled as a blocking time of Ei = Pi – Di thus increasing the tasks utilisation to (Ci + Bi + Ei) / Pi

H ARD, R EAL- T IME S YSTEMS CONCLUSIONS Rate (or Deadline) Monotonic Analysis separates concern of logical behaviour from timing behaviour Provides a sound analytical, engineering basis for design Can be used to investigate performance problems in existing systems Can extend to handle multi-processors

T ESTING - H ARD, R EAL- T IME S YSTEMS Structural Testing Must validate all assumptions made in the design and that design has been correctly implemented. In particular check: Periods [and Deadlines if shorter] Execution Times Blocking Times Event inter-arrival times Behavioural Testing Does system meet deadlines? =>Under overload? Does system meet response times?

These are all Real-Time Performance? can not guarantee Hardware will not fail can not guarantee Software is error-free can not guarantee Operating Conditions will not violate design limits Is it meaningful to talk about a Guarantee of Real-Time Performance? because: