1. Mike Holenderski, m.holenderski@tue.nl Synchronization 2IN60: Real-time Architectures (for automotive systems)

2. Mike Holenderski, m.holenderski@tue.nl Goals for this slide set Describe different dependencies which may exist between tasks Describe different methods for synchronizing dependent tasks, and explain how to apply them Describe the priority inversion and deadlock problems, and how to address them Explain how to implement periodic tasks using semaphores 2

3. Mike Holenderski, m.holenderski@tue.nl Outline Synchronization requirements – Mutual exclusion – Precedence constraints Mutexes – Problems: priority inversion & deadlock – Solutions: Priority Calling Protocol & Stack Resource Policy Semaphores – Example: implementing periodic tasks 3

4. Mike Holenderski, m.holenderski@tue.nl Synchronization problem Mutual exclusion: – A resource can be used by at most one task or ISR at a time Precedence constraints: – Tasks or ISRs must execute in a particular order Problem: Co-ordinate execution of a given concurrent program such that no erroneous interleavings are possible 4

5. Mike Holenderski, m.holenderski@tue.nl Example: mutual exclusion (disabling interrupts) Implement atomicity by disabling interrupts Problem: busy waiting inside of conversion with non-preemptive execution w.r.t. tasks and ISRs – Higher priority tasks not sharing resources are penalized – May lead to missed interrupts 5 int ATDReadChannel(...) {... OS_ENTER_CRITICAL(); “do the conversion”; OS_EXIT_CRITICAL();... } void Task1(void) {... ATDReadChannel(PAD13);... } void Task2(void) {... ATDReadChannel(PAD14);... }

6. Mike Holenderski, m.holenderski@tue.nl Example: mutual exclusion (disabling scheduler) Implement atomicity by disabling scheduler Problem: busy waiting inside of conversion with non-preemptive execution w.r.t. tasks – Higher priority tasks not sharing resources are penalized 6 int ATDReadChannel(...) {... OSSchedLock(); “do the conversion”; OSSchedUnlock();... } void Task1(void) {... ATDReadChannel(PAD13);... } void Task2(void) {... ATDReadChannel(PAD14);... }

7. Mike Holenderski, m.holenderski@tue.nl Mutual exclusion: spot the bug Implement atomicity by checking a global flag Problem: if Task1 has higher priority than Task2 and preempts Task2 after Task2 has set avail to false, then Task1 will busy-wait forever, preventing Task2 to set avail to true Solution: suspend Task1 when avail is false, allowing Task2 to set it to true 7 int avail = true; int ATDReadChannel(...) {... while (!avail) {} avail = false; “do the conversion”; avail = true;... } void Task1(void) {... ATDReadChannel(PAD13);... } void Task2(void) {... ATDReadChannel(PAD14);... }

8. Mike Holenderski, m.holenderski@tue.nl Precedence constraints: spot the bug Task2 senses the light. Every time that the value is larger than some threshold, Task1 toggles a led Problem: if Task1 has a higher priority than Task2, Task1 will busy-wait and Task2 will never run Solution: have Task1 suspend until x is true, or swap the task priorities 8 int x = false; int light; void Task1(void) while (!x) {} x = false; if (light > Threshold) { ToggleLed(LED_D24); } void Task2(void) light = ATDReadChannel(PAD14); x = true; }

9. Mike Holenderski, m.holenderski@tue.nl Synchronization primitives Primitives for synchronizing tasks and ISRs: check for a condition and wait until the condition is satisfied – Examples of conditions: a shared resource is available a signal from ISR has arrived another task has arrived 9

10. Mike Holenderski, m.holenderski@tue.nl Synchronization primitives Two options for waiting: suspend or busy-wait – Suspend: execute the check and suspension atomically – Busy waiting is only reasonable if preemption is allowed and we are waiting for a higher priority task or ISR, or it involves another piece of hardware (e.g. multi-core, sensors). 10

11. Mike Holenderski, m.holenderski@tue.nl Synchronization primitives Blocking due to mutual exclusion: Disable interrupts Disable scheduler – Mutexes – Semaphores Blocking due to precedence constraints: – Semaphores 11

12. Mike Holenderski, m.holenderski@tue.nl Outline Synchronization requirements – Mutual exclusion – Precedence constraints Mutexes – Problems: priority inversion & deadlock – Solutions: Priority Calling Protocol & Stack Resource Policy Semaphores – Example: implementing periodic tasks 12

13. Mike Holenderski, m.holenderski@tue.nl Mutex Mutex guards a shared resource, allowing only one task to access it at a time Tasks can acquire and release a mutex While a task is holding a mutex, no other task is able to acquire it 13 void Task(void) {... (* acquire mutex *); (* execute critical section *); (* release mutex *);... }

14. Mike Holenderski, m.holenderski@tue.nl Dining philosophers problem 14

15. Mike Holenderski, m.holenderski@tue.nl Deadlock problem Deadlock can occur when resources are acquired in nested fashion and there is a cyclic dependency 15 OS_EVENT* mutex1; OS_EVENT* mutex2; void Task1(void* pArg) {... OSMutexPend(mutex1, 0, &err); OSMutexPend(mutex2, 0, &err); (* a critical section *) OSMutexPost(mutex2); OSMutexPost(mutex1);... } void Task2(void* pArg) {... OSMutexPend(mutex2, 0, &err); OSMutexPend(mutex1, 0, &err); (* a critical section *) OSMutexPost(mutex1); OSMutexPost(mutex2);... }

16. Mike Holenderski, m.holenderski@tue.nl Example: deadlock Consider a system comprised of tasks  1 and  2 –  1 and  2 both use resources r 1 and r 2 –  1 first locks r 1 and subsequently r 2 –  2 first locks r 2 and subsequently r 1 Problem: deadlock may occur 16

17. Mike Holenderski, m.holenderski@tue.nl Avoiding deadlock Avoid nested critical sections – in principle, no Pend operations between Pend(m)…Post(m) Acquire all required resources at once (atomically) – Let Pend({a, b, c, …}) be an indivisible operation when there is a danger of deadlock, acquiring resources a, b, c, … atomically Use a fixed order of Pend() operations – Pend(m);Pend(n);.... in one process may deadlock with Pend(n);Pend(m);... in another process – Solution: acquire resources in a fixed order 17

18. Mike Holenderski, m.holenderski@tue.nl Dining philosophers problem 18 2 3 1 Forks should be picked up in the order: 1, 2, 3

19. Mike Holenderski, m.holenderski@tue.nl Avoiding deadlock Avoid nested critical sections – in principle, no Pend operations between Pend(m)…Post(m) Acquire all required resources at once (atomically) – Let Pend({a, b, c, …}) be an indivisible operation when there is a danger of deadlock, acquiring resources a, b, c, … atomically Use a fixed order of Pend() operations – Pend(m);Pend(n);.... in one process may deadlock with Pend(n);Pend(m);... in another process – Solution: acquire resources in a fixed order Use an appropriate Resource Access Protocol – E.g. Stack Resource Policy (SRP) In general: avoid cyclic dependencies! 19

20. Mike Holenderski, m.holenderski@tue.nl U(r)U(r) Priority inversion A low priority job obtains a resource; a high priority job waits on its release A middle priority job pre-empts the low priority job during resource access –the high priority job now waits on the middle priority job... and effectively has the low priority while waiting L(r)L(r) task  h task  l L(r)L(r) U(r)U(r) U(r)U(r) L(r)L(r) L(r)L(r)U(r)U(r) task  h task  l task  m priority inversion“unbounded” priority inversion

21. Mike Holenderski, m.holenderski@tue.nl Priority inversion A pair of alternating middle priority jobs can block the high priority job “indefinitely” –“unbounded” priority inversion: example: Mars rover; see article. Resource access protocol: –Guarantee mutual exclusion (e.g. mutexes) –Resolve other problems, such as priority inversion –At least bound the inversion time –Adhere to the task priorities as closely as possible 21

22. Mike Holenderski, m.holenderski@tue.nl Resource access protocols: Priority Calling Protocol Protocol – Each mutex has a priority, which is used when accessing the mutex: When the mutex is already acquired and a higher priority task attempts to acquire the mutex, then the priority of the task owning the mutex is raised to the mutex priority Mutex priority must be higher than any of the tasks competing for the mutex 22

23. Mike Holenderski, m.holenderski@tue.nl Resource access protocols: Priority Calling Protocol Properties – Avoids “unbounded” priority inversion – Suffers from deadlock! Used in μC/OS-II for implementing mutexes 23

24. Mike Holenderski, m.holenderski@tue.nl Priority Calling Protocol 24

25. Mike Holenderski, m.holenderski@tue.nl Priority Calling Protocol 25

26. Mike Holenderski, m.holenderski@tue.nl Resource access protocols: Stack Resource Policy (SRP) Protocol – Assign to each mutex a ceiling, equal (or higher) to the maximum priority of any task that can acquire it – Scheduler keeps track of the system ceiling, equal to the maximum ceiling of any mutex currently acquired by any task – A task can only start executing if it has the highest priority among ready tasks, and its priority is higher than the current system ceiling 26

27. Mike Holenderski, m.holenderski@tue.nl Resource access protocols: Stack Resource Policy (SRP) Properties – Avoids deadlock – Priority inversion limited to at most one critical section of a lower priority task – Simple implementation: Can reuse the ready queue (used by the scheduler) for storing the tasks blocked on a mutex 27

28. Mike Holenderski, m.holenderski@tue.nl Stack Resource Policy 28

29. Mike Holenderski, m.holenderski@tue.nl Stack Resource Policy 29 Task2 is not blocked while Task3 accesses resource :-)

30. Mike Holenderski, m.holenderski@tue.nl Mutexes in μC/OS-II Mutexes in μC/OS-II implement the Priority Calling Protocol Mutexes cannot be used by ISRs Mutex is created using OS_EVENT* m = OSMutexCreate(prio, &err); – Mutex is assigned a priority prio prio must be set to a priority higher than any task accessing mutex m 30

31. Mike Holenderski, m.holenderski@tue.nl Mutexes in μC/OS-II Tasks can acquire mutex m using OSMutexPend(m, timeout, &err); – If the m is available it is immediately acquired – If the m is already acquired, the task is suspended until another task releases m Tasks with a higher priority than prio will not be blocked by the lower priority tasks while they are using m – Since prio is higher than the priority of any task using m, at most one task can be suspended on m – If timeout > 0 and m does not become available within that time, an error is returned indicating that m could not be granted. If timeout is 0, then OSMutexPend() will wait indefinitely until m is available. –&err is a pointer to an integer where the error code is returned 31

32. Mike Holenderski, m.holenderski@tue.nl Mutexes in μC/OS-II Tasks can release mutex m using OSMutexPost(m) ; – If there is any other task waiting for m, that task is made ready and the scheduler is invoked 32

33. Mike Holenderski, m.holenderski@tue.nl Disabling the scheduler vs. mutexes in μC/OS-II void Task(void) {... OSSchedLock(); (* a critical section *) OsSchedUnlock();... } void main(void) {... } OS_EVENT* mutex; void Task(void) { INT8U err;... OSMutexPend(mutex, 0, &err); (* a critical section *) OSMutexPost(mutex);... } void main(void) { INT8U err;... mutex = OSMutexCreate(prio, &err);... } Disabling the scheduler Using mutexes 33

34. Mike Holenderski, m.holenderski@tue.nl Evaluation: disabling interrupts Pros: – Avoids deadlock and “unbounded” priority inversion – Simple implementation (low memory processor overhead) – Simple timing analysis Cons: – May lead to unnecessarily long blocking times In particular, for higher priority tasks that do not need the shared resource. – May lead to missed interrupts 34

35. Mike Holenderski, m.holenderski@tue.nl Evaluation: disabling the scheduler Pros: – Simple implementation (low memory processor overhead) – Simple timing analysis Cons: – May lead to unnecessarily long blocking times In particular, for higher priority tasks that do not need the shared resource. – If ISRs use shared resources, then atomicity is not guaranteed 35

36. Mike Holenderski, m.holenderski@tue.nl Evaluation: mutexes Pros: – Allow handling of interrupts Reduce processor blocking to just the administration of the mutual exclusion primitives – Higher priority tasks not sharing the resource guarded by the critical section are not affected Cons: – Not allowed in ISRs – Elaborate implementation Need to maintain the task suspended on a mutex (μC/OS-II) – Introduces possibility for deadlock Can be prevented with the right resource access protocol 36

37. Mike Holenderski, m.holenderski@tue.nl Outline Synchronization requirements – Mutual exclusion – Precedence constraints Mutexes – Problems: priority inversion & deadlock – Solutions: Priority Calling Protocol & Stack Resource Policy Semaphores – Example: implementing periodic tasks 37

38. Mike Holenderski, m.holenderski@tue.nl Semaphores Conceived by Edsger W. Dijkstra (TU/e, 1965) Semaphore s is an integer with initial value s  0 and atomic operations P(s) and V(s). – Task executing P(s) is said to “try to acquire semaphore s” – P(s) stands for “prolaag”, i.e. portmanteau for “probeer te verlagen” – Task executing V(s) is said to “release semaphore s” – V(s) stands for “verhoog” 38

39. Mike Holenderski, m.holenderski@tue.nl Semaphores The effect of these operations is defined as follows: – P(s): 0); s = s-1 > – V(s): (here we used “ ” to denote atomicity) await: a statement to indicate suspension until a condition is met The s>0 check and s = s-1 must be executed atomically A process that executes P(s) is suspended until s > 0 A process executing V(s) possibly releases suspended processes 39

40. Mike Holenderski, m.holenderski@tue.nl Semaphores General assumptions: – atomic execution of P and V If several processes simultaneously invoke a P or V operation on the same semaphore, the operations will occur sequentially in an arbitrary order. – arbitrary selection of process to proceed: If more than one process is waiting inside a P operation on the same semaphore and the semaphore becomes positive (because of the execution of a V), one of the waiting processes is selected arbitrarily to complete the P operation. 40

41. Mike Holenderski, m.holenderski@tue.nl Semaphores Specific assumptions for Real-Time Systems: – specific selection of process to proceed: … the process with the highest priority among the waiting processes is selected to complete the P operation. 41

42. Mike Holenderski, m.holenderski@tue.nl Example: synchronizing tasks using semaphores Mutual exclusion: – Assume a resource guarded by semaphore s – Each task sharing resource guarded by s contains: P(s) /* use resource */ V(s) – Initialization: s = 1 Precedence (enforced limitations on order, where the triggered task waits for a triggering task): – Assume synchronization based on s – Triggering task: V(s) – Triggered task: P(s) – Initialization: s = 0 42

43. Mike Holenderski, m.holenderski@tue.nl Semaphores in μC/OS-II Semaphores can be used by tasks and ISRs – With the exception of OSSemPend() (task only) Semaphore is created using OS_EVENT* s = OSSemCreate(value); – Semaphore is not assigned a priority –value is the initial value of s 43

44. Mike Holenderski, m.holenderski@tue.nl Semaphores in μC/OS-II Tasks can acquire one unit of semaphore s using OSSemPend(s, timeout, &err); – If the value of s > 0, the semaphore is decremented and task continues – If the value of s = 0, the task is suspended – Several tasks may be blocked on the same semaphore They will be released according to their priority (highest first) – If timeout > 0 and s was not granted before that, an error is returned –&err is a pointer to an integer where the error code is returned – Note: this is the P(s) operation 44

45. Mike Holenderski, m.holenderski@tue.nl Semaphores in μC/OS-II Tasks can release semaphore s using OSSemPost(s) ; – If there is any other task waiting for s, that task is made ready and the scheduler is invoked – Note: this is the V(s) operation 45

46. Mike Holenderski, m.holenderski@tue.nl Example: implementing periodic tasks Periodic task τ i is specified by: – priority i, phasing φ i, period T i Standard μC/OS-II does not support periodic tasks – These are added by the RELTEQ extension 46

47. Mike Holenderski, m.holenderski@tue.nl Periodic task implementation Two options: – Create a new task context upon each job arrival – Create a task context once, and let jobs be the iterations of a while loop. 47 void Task( void (*f)(void) ) {... while (true) { OSTaskWaitPeriod(); f (); } void OSTimeTick(void) {... if (* Task_i period expired *) { (* signal Task_i *) }... } void main(void) {... OSTaskCreatePeriodic(f i, T i, φ i, s, i);... } ✓

48. Mike Holenderski, m.holenderski@tue.nl void Task( void (*f)(void) ) { int k = 0; while (true) { now = OSTimeGet(); OSTimeDly(φi + k ∗ Ti - now); f (); k = k + 1; } void OSTimeTick(void) {... if (* Task_i period expired *) { (* signal Task_i *) }... } void main(void) {... OSTaskCreatePeriodic(f i, T i, φ i, s, i);... } Periodic task implementation Synchronize loop iterations using OSTimeDly() Suffers from large jitter – Due to interference between OSTimeGet() and OSTimeDly() 48 void Task( void (*f)(void) ) {... while (true) { OSTaskWaitPeriod(); f (); } void OSTimeTick(void) {... if (* Task_i period expired *) { (* signal Task_i *) }... } void main(void) {... OSTaskCreatePeriodic(f i, T i, φ i, s, i);... }

49. Mike Holenderski, m.holenderski@tue.nl OS_EVENT* Task1Sem; void Task(void (*f)(void)) { INT8U err; while (true) { OSSemPend(Task1Sem, 0, &err); f(); } void OSTimeTick(void) {... if (* Task1 period expired *) { OSSemPost(Task1Sem); }... } void main(void) {... Task1Sem = OSSemCreate(0);... } Periodic task implementation Synchronize loop iterations using semaphores 49 void Task( void (*f)(void) ) {... while (true) { OSTaskWaitPeriod(); f (); } void OSTimeTick(void) {... if (* Task_i period expired *) { (* signal Task_i *) }... } void main(void) {... OSTaskCreatePeriodic(f i, T i, φ i, s, i);... }

50. Mike Holenderski, m.holenderski@tue.nl Summary of mutual exclusion primitives 50 PrimitiveProsCons Disable interrupts Avoid deadlock Simple implementation Prevent interference from interrupts Higher priority tasks not sharing resources are penalized Interrupts can be missed Disable scheduler Avoid deadlock Simple implementation Allow interrupts Higher priority tasks not sharing resources are penalized Cannot guard resources shared with ISRs Mutex Allow interrupts Higher priority tasks not sharing resources are not penalized Avoid “unbounded” priority inversion Can lead to deadlock (depends on implementation) Cannot guard resources shared with ISRs Suspension not allowed in ISRs Semaphore Allow interrupts Higher priority tasks not sharing resources are not penalized Can lead to deadlock Cannot guard resources shared with ISRs Suspension not allowed in ISRs “Unbounded” priority inversion

51. Mike Holenderski, m.holenderski@tue.nl References Recommended reading: – [Burns] Ch. 5.1-5.4, 11.8, 11.9 – Mike Jones, “What really happened on Mars?”, 1997 http://research.microsoft.com/~mbj/mars_pathfinder/ mars_pathfinder.html http://research.microsoft.com/~mbj/mars_pathfinder/ mars_pathfinder.html – T. Baker, “Stack-based scheduling for realtime processes”, Journal of Real-Time Systems, vol 3, 1991 51