1. 1 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld Chapter 2 Mutual Exclusion using atomic registers: Basic Topics Version: November 2007

2. 2 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 A note on the use of these ppt slides: I am making these slides freely available to all (faculty, students, readers). They are in PowerPoint form so you can add, modify, and delete slides and slide content to suit your needs. They obviously represent a lot of work on my part. In return for use, I only ask the following:  That you mention their source, after all, I would like people to use my book!  That you note that they are adapted from (or perhaps identical to) my slides, and note my copyright of this material. Thanks and enjoy! Gadi Taubenfeld All material copyright 2007 Gadi Taubenfeld, All Rights Reserved A note on the use of these ppt slides: I am making these slides freely available to all (faculty, students, readers). They are in PowerPoint form so you can add, modify, and delete slides and slide content to suit your needs. They obviously represent a lot of work on my part. In return for use, I only ask the following:  That you mention their source, after all, I would like people to use my book!  That you note that they are adapted from (or perhaps identical to) my slides, and note my copyright of this material. Thanks and enjoy! Gadi Taubenfeld All material copyright 2007 Gadi Taubenfeld, All Rights Reserved Synchronization Algorithms and Concurrent Programming ISBN: 0131972596, 1 st edition To get the most updated version of these slides go to: http://www.faculty.idc.ac.il/gadi/book.htm

3. 3 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 2.1 Algorithms for Two Processes 2.2 Tournament Algorithms 2.3 A Fast Mutual Exclusion Algorithms 2.4 Starvation-free Algorithms 2.5 Tight Space bounds 2.6 Automatic Discovery of algorithms 2.1 Algorithms for Two Processes 2.2 Tournament Algorithms 2.3 A Fast Mutual Exclusion Algorithms 2.4 Starvation-free Algorithms 2.5 Tight Space bounds 2.6 Automatic Discovery of algorithms Chapter 2 Mutual Exclusion using atomic registers: Basic Topics

4. 4 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 remainder code entry code critical section exit code The problem is to design the entry and exit code in a way that guarantees that the mutual exclusion and deadlock-freedom properties are satisfied. The mutual exclusion problem

5. 5 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 The mutual exclusion problem  Mutual Exclusion: No two processes are in their critical sections at the same time.  Deadlock-freedom: If a process is trying to enter its critical section, then some process, not necessarily the same one, eventually enters its critical section.  Starvation-freedom: If a process is trying to enter its critical section, then this process must eventually enter its critical section.  Mutual Exclusion: No two processes are in their critical sections at the same time.  Deadlock-freedom: If a process is trying to enter its critical section, then some process, not necessarily the same one, eventually enters its critical section.  Starvation-freedom: If a process is trying to enter its critical section, then this process must eventually enter its critical section. entry code exit code critical section remainder

6. 6 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Question: true or false ?  A and B are deadlock-free  C is deadlock-free.  A and B are starvation-free  C is starvation-free.  A or B satisfies mutual exclusion  C satisfies mutual exclusion.  A is deadlock-free and B is starvation-free  C is starvation- free.  A is starvation-free and B is deadlock-free  C is starvation- free.  A and B are deadlock-free  C is deadlock-free.  A and B are starvation-free  C is starvation-free.  A or B satisfies mutual exclusion  C satisfies mutual exclusion.  A is deadlock-free and B is starvation-free  C is starvation- free.  A is starvation-free and B is deadlock-free  C is starvation- free. entry code of A exit code of A critical section remainder entry code of B exit code of B Algorithm C

7. 7 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Does it work? Atomic read/write registers Proposed solution 1 Thread 0 while (true} { remainder code while (turn = 1) {skip} critical section turn = 1 } Thread 1 while (true} { remainder code while (turn = 0) {skip} critical section turn = 0 } entry code exit code 0/1 turn mutual exclusion  deadlock-freedom

8. 8 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Proposed solution 2 Thread 0 flag[0] = true while (flag[1]) {skip} critical section flag[0] = false Thread 1 flag[1] = true while (flag[0]) {skip} critical section flag[1] = false false flag false 0 1 Does it work? mutual exclusion  deadlock-freedom

9. 9 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Proposed solution 3 Thread 0 while (flag[1]) {skip} flag[0] = true critical section flag[0] = false Thread 1 while (flag[0]) {skip} flag[1] = true critical section flag[1] = false false flag false 0 1 Does it work?  mutual exclusion Deadlock-freedom

10. 10 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Peterson’s algorithm Thread 0 flag[0] = true turn = 1 while (flag[1] and turn = 1) {skip} critical section flag[0] = false Thread 1 flag[1] = true turn = 0 while (flag[0] and turn = 0) {skip} critical section flag[1] = false false flag false 0 1 0/1 turn Section 2.1.1

11. 11 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 0/1 A variant of Peterson’s algorithm false flag false 0 1 turn Is it correct ? true 10 Thread 0 turn = 1 flag[0] = true while (flag[1] and turn = 1) {skip} critical section flag[0] = false Thread 1 turn = 0 flag[1] = true while (flag[0] and turn = 0) {skip} critical section flag[1] = false

12. 12 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Indicate contending b[i] := true Indicate contending b[i] := true Barrier turn := i Barrier turn := i Contention? b[i] = true ? Contention? b[i] = true ? critical section exit code b[i] = false ? exit code b[i] = false ? First to cross the barrier? turn = j ? First to cross the barrier? turn = j ? yes no / maybe no Schematic for Peterson’s algorithm

13. 13 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Properties of Peterson’s Solution  Satisfies mutual exclusion and starvation-freedom  Memory accesses are assumed to be atomic  Solution for two processes only

14. 14 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Question Replace the multi-writer register turn with two single-writer registers. What is new algorithm? Hint: use Solution #4 for the too-much-milk problem. Answer (Kessels’ Alg.) turn = 0  turn[0] = turn[1] turn = 1  turn[0]  turn[1] Using only single-writer registers Section 2.1.2

15. 15 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Solutions for Many Processes How can we use a two-process algorithm to construct an algorithm for many processes?

16. 16 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Tournament Algorithms 12345678 Section 2.2

17. 17 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Question Assume that the base two-process algorithm satisfies starvation-freedom. Does the tournament algorithm also satisfy starvation-freedom? Yes

18. 18 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Properties & complexity  Time complexity m Fast m Adaptive  Fairness m FIFO,...  Fault-tolerance  Local spinning  Space complexity  Communication primitives  Time complexity m Fast m Adaptive  Fairness m FIFO,...  Fault-tolerance  Local spinning  Space complexity  Communication primitives

19. 19 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Complexity Measures  Counting steps m process step complexity m process time complexity m contention-free time complexity  Counting time units m system response time m process response time  Counting communications m distributed shared memory m coherent cashing  Space complexity  Counting steps m process step complexity m process time complexity m contention-free time complexity  Counting time units m system response time m process response time  Counting communications m distributed shared memory m coherent cashing  Space complexity

20. 20 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Thread 0 flag[0] = true turn = 1 while (flag[1] and turn = 1) {skip} critical section flag[0] = false Thread 1 flag[1] = true turn = 0 while (flag[0] and turn = 0) {skip} critical section flag[1] = false What is the contention-free time complexity? 4 4

21. 21 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Thread 0 flag[0] = true turn = 1 while (flag[1] and turn = 1) {skip} critical section flag[0] = false Thread 1 flag[1] = true turn = 0 while (flag[0] and turn = 0) {skip} critical section flag[1] = false What is the process step complexity? Theorem: There is no two process mutual exclusion algorithm with an upper bound on the process time/step complexity. [1992] Proof: see Section 3.2.5. Theorem: There is no two process mutual exclusion algorithm with an upper bound on the process time/step complexity. [1992] Proof: see Section 3.2.5.

22. 22 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Show a run of Peterson’s algorithm where the system response time is 5 time units. What is the system response time? 123 45 Thread 0 flag[0] = true turn = 1 while (flag[1] and turn = 1) {skip} critical section flag[0] = false Thread 1 flag[1] = true turn = 0 while (flag[0] and turn = 0) {skip} critical section flag[1] = false

23. 23 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Fast Mutual exclusion Algorithm  Mutual exclusion and deadlock-freedom  Starvation of individual processes is possible  fast access m in the absence of contention, only 7 accesses to shared memory are needed  With contention m Even if only 2 processes contend, the winner may need to check all the 0(n) shared registers m System response time is of order n time units  n+2 shared registers are used  Mutual exclusion and deadlock-freedom  Starvation of individual processes is possible  fast access m in the absence of contention, only 7 accesses to shared memory are needed  With contention m Even if only 2 processes contend, the winner may need to check all the 0(n) shared registers m System response time is of order n time units  n+2 shared registers are used Section 2.3

24. 24 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007  At most n-1 can move right  At most n-1 can move down  At most 1 can win  In solo run  1 win Splitter right win down n 1n-1 right

25. 25 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 The code of the splitter The code of the splitter x = i if y = 1 then go right fi y = 1 if x  i then go down fi win x 0 y down n 1n-1 right

26. 26 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Notation await (condition) == while (not condition) do skip od

27. 27 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Fast Mutual exclusion Algorithm Fast Mutual exclusion Algorithm code of process i, i  {1,..., n} start: b[i] = tru e x = i if y  0 then b[i] = false await y = 0 goto start fi y = i if x  i then b[i] = false for j = 1 to n do await b[j] = false od if y  i then await y = 0 goto start fi fi critical section y = 0 b[i] = false 12n b false yx 0

28. 28 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Can we switch the order of the last two statements? start: b[i] = tru e x = i if y  0 then b[i] = false await y = 0 goto start fi y = i if x  i then b[i] = false for j = 1 to n do await b[j] = false od if y  i then await y = 0 goto start fi fi critical section b[i] = false y = 0 yx 0 What is the maximum number of processes that can be in their CS at the same time?

29. 29 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Can we replace y=0 in line 12 with... ? start: b[i] = tru e x = i if y  0 then b[i] = false await y = 0 goto start fi y = i if x  i then b[i] = false for j = 1 to n do await b[j] = false od if y  i then await y = 0 goto start fi fi critical section y = 0 replace with: if y = i then y = 0 fi b[i] = false Yes

30. 30 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Can we remove await y=0 in line 4 ? start: b[i] = tru e x = i if y  0 then b[i] = false await y = 0 goto start fi y = i if x  i then b[i] = false for j = 1 to n do await b[j] = false od if y  i then await y = 0 goto start fi fi critical section y = 0 b[i] = false No. Deadlock is possible remove ?

31. 31 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Can we remove await y=0 in line 9 ? start: b[i] = tru e x = i if y  0 then b[i] = false await y = 0 goto start fi y = i if x  i then b[i] = false for j = 1 to n do await b[j] = false od if y  i then await y = 0 goto start fi fi critical section y = 0 b[i] = false Yes remove ?

32. 32 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Indicate contending b[i] := true Indicate contending b[i] := true Barrier turn := i Barrier turn := i Contention? x  i ? Contention? x  i ? critical section exit code Last to cross the barrier? y = i ? Last to cross the barrier? y = i ? yes Contention? y = 0 ? Contention? y = 0 ? Continue only after it is guaranteed that no one can cross the barrier Continue only after it is guaranteed that no one can cross the barrier no Wait until CS is released yes The last to cross the barrier! no Schematic for the fast algorithm

33. 33 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 4.1 Basic Definitions (i.e., FIFO) 4.2 The Bakery Algorithm 4.3 The Black-White Bakery Algorithm Algorithm 4.1 Basic Definitions (i.e., FIFO) 4.2 The Bakery Algorithm 4.3 The Black-White Bakery Algorithm Algorithm Starvation-free Algorithms Section 2.4

34. 34 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 The mutual exclusion problem entry code exit code critical section remainder  Mutual Exclusion  Deadlock-freedom  Starvation-freedom  FIFO doorway waiting

35. 35 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 The Bakery Algorithm Section 2.4.2

36. 36 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 time The Bakery Algorithm 000000 doorway 12345n CS exit 1 1 22 22 1 1 0 2 2 0 3 3 2 2 0 4 4 waiting entry remainder Section 2.4.2

37. 37 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Implementation 1 Implementation 1 code of process i, i  {1,..., n} number[i] = 1 + max {number[j] | (1  j  n)} for j = 1 to n { await (number[j] = 0)  (number[j] > number[i]) } critical section number[i] = 0 1234n numberinteger 000000 Answer: No! can deadlock

38. 38 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 time Implementation 1: deadlock 000000 doorway 12345n CS exit 1 1 22 22 1 1 0 waiting entry remainder deadlock

39. 39 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Implementation 1 Implementation 1 code of process i, i  {1,..., n} number[i] = 1 + max {number[j] | (1  j  n)} for j = 1 to n { await (number[j] = 0)  (number[j] > number[i]) } critical section number[i] = 0 1234n numberinteger 000000 Answer: does not satisy mutual exclusion What if we replace > with  ?

40. 40 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 number[i] = 1 + max {number[j] | (1  j  n)} for j = 1 to n { await (number[j] = 0)  (number[j],j)  number[i],i) // lexicographical order } critical section number[i] = 0 1234n numberinteger 000000 Answer: does not satisfy mutual exclusion Implementation 2 Implementation 2 code of process i, i  {1,..., n}

41. 41 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 time Implementation 2: no mutual exclusion 00000 doorway 12345n CS exit 0 1 00 22 1 1 0 22 waiting entry remainder 122 0 2

42. 42 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 The Bakery Algorithm The Bakery Algorithm code of process i, i  {1,..., n} choosing[i] = tru e number[i] = 1 + max {number[j] | (1  j  n)} choosing[i] = false for j = 1 to n { await choosing[j] = false await (number[j] = 0)  (number[j],j)  (number[i],i) } critical section number[i] = 0 1234n choosingbits false numberinteger 000000 false

43. 43 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 time The Bakery Algorithm 000000 doorway 12345n CS exit 1 1 22 22 1 1 0 2 2 0 3 3 2 2 0 4 4 waiting entry remainder

44. 44 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Is the following version correct? Is the following version correct? code of process i, i  {1,..., n} choosing[i] = tru e number[i] = 1 + max {number[j] | (1  j  n)} choosing[i] = false for j = 1 to n { if i > j { await choosing[j] = false } await (number[j] = 0)  (number[j],j)  (number[i],i) } critical section number[i] = 0 1234n choosingbits false numberinteger 000000 false

45. 45 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 time No 00000 doorway 12345n CS exit 0 1 00 21 1 1 0 21 waiting entry remainder 121 0 1

46. 46 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Is the following version correct? Is the following version correct? code of process i, i  {1,..., n} number[i] = -1 number[i] = 1 + max {number[j] | (1  j  n), 0} for j = 1 to n { await number[j]  -1 await (number[j]  0)  (number[j],j)  (number[i],i) } critical section number[i] = 0 1234n choosingbits false numberinteger 000000 false Yes Can we replace  with = ?

47. 47 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Computing the Maximum #1 Computing the Maximum #1 code of process i, i  {1,..., n} Correct implementation local1 = 0 for local2 = 1 to n do local3 = number[local2] if local1 < local3 then local1 = local3 fi od number[i] = 1 + local1 1234n numberinteger 000000 number[i] = 1 + max {number[j] | (1  j  n)}

48. 48 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Computing the Maximum #2 Computing the Maximum #2 code of process i, i  {1,..., n} Is the following implementation correct ? local1 = i for local2 = 1 to n do if number[local1] < number[local2] then local1 = local2 fi od number[i] = 1 + number[local1] 1234n numberinteger 000000 number[i] = 1 + max {number[j] | (1  j  n)}

49. 49 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 time 0000 doorway 12345n CS exit 11 11 1 1 0 1 waiting entry remainder 00 ? 1 1 ? local1 2 1 No Passed process 1 Waiting for process 2

50. 50 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Computing the Maximum #3 Computing the Maximum #3 code of process i, i  {1,..., n} Is the following implementation correct ? local1 = i for local2 = 1 to n do if number[local1] number[local2] then local1 = local2 fi od number[i] = 1 + number[local1]  A difficult question !!! This is Problem 2.39

51. 51 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Properties of the Bakery Algorithm  Satisfies mutex & FIFO.  The size of number[i] is unbounded.  Safe registers: reads which are concurrent with writes may return arbitrary value.

52. 52 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 The Black-White Bakery Algorithm Section 2.4.3 Bounding the space of the Bakery Algorithm Bakery (FIFO, unbounded) The Black-White Bakery Algorithm FIFO Bounded space + one bit

53. 53 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 time The Black-White Bakery Algorithm 00000 doorway 12345n CS exit 0 1 00 22 1 1 0 2 2 0 1 2 2 0 2 waiting entry remainder 1202012 1 1 00 color bit

54. 54 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 The Black-White Bakery Algorithm 1234n choosing Data Structures mycolor number color bit bits {0,1,...,n} {black,white}

55. 55 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 The Black-White Bakery Algorithm The Black-White Bakery Algorithm code of process i, i  {1,..., n} choosing[i] = true mycolor[i] = color number[i] = 1 + max{number[j] | (1  j  n)  (mycolor[j] = mycolor[i])} choosing[i] = false for j = 0 to n do await choosing[j] = false if mycolor[j] = mycolor[i] then await (number[j] = 0)  (number[j],j)  (number[i],i)  (mycolor[j]  mycolor[i]) else await (number[j] = 0)  (mycolor[i]  color)  (mycolor[j] = mycolor[i]) fi od critical section if mycolor[i] = black then color = white else color = black fi number[i] = 0

56. 56 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 What happens if in the algorithm each process chooses its identifier instead of choosing 1 + max ? Would the algorithm be correct? What properties would it satisfy? Question Answer: Incorrect. Run process i first until it enters its CS; Now run a process with a smaller id until it also enters its CS.

57. 57 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Does it matter if we change the order of the last two statements (in the exit code) of the Black-White Bakery Algorithm? Question Answer: Yes, it matters.

58. 58 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Bakery Black-White Bakery Use single-writer bits only! Question Replace the multi-writer bit color with n single-writer bits. Answer: See page 57.

59. 59 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 A lower bound (Section 2.5.1) An Upper Bound (Section 2.5.2) Tight space bounds for mutual exclusion using atomic registers

60. 60 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 A Simple Observation Theorem: Any deadlock-free mutual exclusion algorithm for n processes using only SWMR registers must use at least n such registers. A very simple Lower Bound Proof: Before entering its critical section a process must write at least once … Question: Can we do better using MWMR registers ? Answer: No. (SWMR == Single Writer Multi Reader)

61. 61 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Tight Space Bounds Theorem: Any deadlock-free mutual exclusion algorithm for n processes must use at least n shared registers. Section 2.5.1 A Lower Bound (very difficult to prove!) Proof:  (see next few slides)

62. 62 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Definitions  run – a sequence of events  run x looks like run y to process p  process p is hidden in run x  process p covers register r in run x

63. 63 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Example run x looks like run y to process p run x p reads 5 from r q writes 6 to r p writes 7 to r q writes 8 to r p reads 8 from r run y p reads 5 from r p writes 7 to r q writes 6 to r q reads 6 from r q writes 8 to r p reads 8 from r 8 r 8 r q write 6 to r1 6 The values of the shared registers must also be the same

64. 64 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Example p reads 5 from r q reads 5 from r p writes 7 to r q writes 8 to r p reads 8 from r q writes 6 from r r writes must be overwritten before any other process has read the value written process p is hidden in run x

65. 65 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Example p reads 5 from r q reads 5 from r p writes 7 to r q writes 8 to r p reads 8 from r p writes 2 to r r writes must be overwritten before any other process has read the value written process p covers register r in run x p covers r at these three points

66. 66 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Lemma 2.17 (very simple) xy run x looks like run y to processes in P z P events only then, this is also a run Proof: By induction on the number of events in (z-x) …

67. 67 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Lemma 2.18 (very simple) If p is in its critical section in run z then p is not hidden in z. z p is in its critical section then, p is not hidden Proof: Assume to the contrary … by Lemma 2.17 …

68. 68 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Lemma 2.19 x all the processes are hidden all processes except may p are in their remainders y Then, for any p there exists y such that x looks like y to p Proof: By induction on the number of steps of processes other than p in x...

69. 69 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Lemma 2.19 : Example w reads 5 p reads 5 q reads 5 p writes 7 q writes 8 q reads 8 w writes 3 w reads 3 p write 9 p in its critical section p exits 5 r w is hidden  remove w w is in its remainder q is in its remainder important: q is still hidden !!! q is hidden  remove q p is in its remainder Formal proof: By induction on the number of steps of processes other than p in x...

70. 70 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Lemma 2.20 (simple) x all the processes are hidden only p takes steps z p covers unique register Then, for every p there exists z

71. 71 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Lemma 2.20: Proof x all the processes are hidden all processes except may p are in their remainders x looks like y to p p is in its critical section By Lemma 2.17 … … p is hidden … imp. by Lemma 2.18 p covers unique register z By Lemma 2.19: there exists y … By the deadlock-freedom assumption …

72. 72 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Lemma 2.21 (main lemma) x all the processes are in their remainder z only P take steps P are hidden and cover |P| distinct register Then, for every set of processes P there exists z Proof: By induction on k the size of P.

73. 73 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Lemma 2.21 (main lemma) Proof: By induction on k the size of P.  Base: k = 0 is trivial.  Induction hypothesis: Assume that it holds for k  0.  Step: We prove that it holds for k+1. m let P be a set of processes where |P|= k; m let p be a process not in P. m We show that: z only P+p take steps P+p are hidden and cover |P+p| distinct register

74. 74 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Lemma 2.21 x y1y1 W1W1... y2y2 yiyi yjyj W2W2 WiWi WjWj only P take steps P are hidden and cover |P| distinct registers By the induction hypothesis: z1z1 w1w1 z2z2 zizi zjzj w2w2 wiwi wjwj... each process in P takes one step first; all in their remainders ;... equals This completes the lower bound proof only p takes steps p covers a unique register w 1  W 1 (Lemma 2.20) P are hidden? Yes P cover W 1 ? Yes p covers w 1 ? Yes p is hidden? Maybe P are hidden? Yes P cover W j ? Yes p covers w j ? Yes p is hidden? Yes!

75. 75 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Question p reads 5 from r q reads 5 to r p writes 7 to r q writes 8 to r p reads 8 from r p writes 8 to r 8 r What breaks in the lower bound proof if we change the definition of “process p is hidden in run x” as follows: this write does not change the value of r!

76. 76 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Tight Space Bounds Theorem: There is a deadlock-free mutual exclusion algorithm for n processes that uses n shared bits. Section 2.5.2 An Upper Bound

77. 77 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 The One-Bit Algorithm The One-Bit Algorithm code of process i, i  {1,..., n} 1234n bbits false true false true false critical section false p 3 writes true p 3 writes false p n writes true p n writes false

78. 78 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 The One-Bit Algorithm The One-Bit Algorithm code of process i, i  {1,..., n} repeat b[i] = true; j = 1; while (b[i] = true) and (j < i) do if b[j] = true then b[i] = false; await b[j] =false fi j = j+1 od until b[i] = true for j = i+1 to n do await b[j] = false od critical section b[i] = false 1234n bbits false

79. 79 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Does the correctness of the One-bit algorithm depend on the order in which the bits are read in the loops ? Question

80. 80 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Properties of the One-Bit Algorithm Satisfies mutual exclusion and deadlock-freedom Starvation is possible It is not fast It is not symmetric It uses only n shared bits and hence it is space optimal Would it work with safe bits? Satisfies mutual exclusion and deadlock-freedom Starvation is possible It is not fast It is not symmetric It uses only n shared bits and hence it is space optimal Would it work with safe bits?

81. 81 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Question Does it follow from the lower bound that there is no solution when the number of processes is not known in advance or when the number is infinite? Computing with Infinitely Many Process Section 2.5.3 No. It follows that in such a case infinitely many registers are needed. A simple algorithm for infinitely many processes is presented in Section 2.5.3.

82. 82 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Computing with Infinitely Many Process: The algorithm Section 2.5.3 0000000000 owner 0000000000 other 0000000000 loser 0000000000 winner 12345678910 1 1 1111 process 4 runs alone … process 4 is the winner process 3 runs... 1 1 process 3 lost process 6 runs... 1 1 1 1 1 1 process 3 lost This is just one game, we need infinitely many … … … … …

83. 83 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Automatic Discovery of Algorithms How do we invent algorithms? Section 2.6 eureka This is one way  See next slide for another way... (maybe they are the same...)

84. 84 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Automatic Discovery Algorithm Generator Algorithm Verifier algorithms correct algorithms

85. 85 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 System Overview Algorithm Generator Algorithm Verifier User Interface algorithms verification results correct algorithms parameters (# of: processes, bits, lines of code)

86. 86 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Results Shared bits 2 3 4 5 6 Entry 6 7 4 6 4 5 6 5 Exit 1 Complex conditions Yes Starvation freedom Yes Tested algorithms 7,196,536,269 846,712,059 25,221,389 1,838,128,995 129,542,873 129,190,403 *900,000,000 *22,000,000 *70,000,000 Correct alg. 0 66 105 10 480 56 80 106 96 appx. running hours 216 39 0.4 47 1 12 0.4 1 User-defined parameters Results * This run was stopped after few solutions were found. Not all algorithms were tested.

87. 87 Chapter 2 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2007 Next Chapter In Chapter 3 we will consider more advanced solutions to the mutual exclusion problem using atomic registers. -- Gadi