1. Multiprocessor Synchronization Algorithms (20225241) Lecturer: Danny Hendler

2. 2 Grade structure 3 (theoretic) problem sets –Must submit at least 2 –Weight 30% of final grade (15% each) Take-home final exam –70% of final grade Electronic submissions only!!!

3. 3 Adminstrative details -Office: Alon 218 -Office hours: -Sundays, 2pm-4pm -Or schedule an appointment -Course site: http://www.cs.bgu.ac.il/~msa141/ (or simply access through my home page)

4. 4 Books Distributed Computing: Fundamentals, Simulations, and Advanced Topics, Hagit Attiya and Jennifer Welch, John Wiley and Sons The Art of Multiprocessor Programming Maurice Herlihy and Nir Shavit Synchronization Algorithms and Concurrent Programming, Gadi Taubenfeld, Pearson/Prentice Hall. Book site: http://www.faculty.idc.ac.il/gadi/book.htm

5. 55

6. 66 Moore’s law Exponential growth in computing power

7. 77 The Future of Computing  Speeding up uni-processors is harder and harder  Intel, Sun (RIP), AMD, IBM now focusing on “multi- core” architectures  Soon, most desktops will be multiprocessors How can we write correct and efficient algorithms for multiprocessors ?  Soon, most laptops will be multiprocessors  Soon, most cellular phonse will contain multiprocessors  Soon, most TVs will contain multiprocessors

8. 8 Distributed Systems A distributed system is a collection of individual computing devices that communicate with one another. E.g.: –VLSI chips –Shared-memory multiprocessor –Local-area network –The Internet

9. 9 One sequential model, MANY distributed models Shared-memory / Message-passing –Which operations are allowed (read, write, read- modify-write)? Synchronous, timing-conditions, asynchronous Are failures allowed? –Fail-stop, Byzantine failures, message omission Is the number of processes known?

10. 10 Motivating examples of Synchronization

11. 11 The Too-Much-Milk Problem TimeAliceBob 5:00Arrive Home 5:05Look in fridge; no milk 5:10Leave for grocery 5:15Arrive home 5:20Arrive at groceryLook in fridge; no milk 5:25Buy milkLeave for grocery 5:30Arrive home; put milk in fridge 5:40Buy milk 5:45Arrive home; too much milk! Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

12. 12 Solving the Too-Much-Milk Problem Required properties  Mutual Exclusion: Only one person checks&buys milk “at a time”  Progress: Someone always buys milk if it is needed Communication primitives  Leave a note (set a flag)  Remove a note (reset a flag)  Read a note (test the flag) Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

13. 13 Important Distinction  Safety Property –Nothing bad ever happens –Example: mutual exclusion  Liveness Property –Something good happens eventually –Example: progress  Safety Property –Nothing bad ever happens –Example: mutual exclusion  Liveness Property –Something good happens eventually –Example: progress Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

14. 14 Solution 1 Does it work? Alice if (no milk) { if (no note) { leave note buy milk remove note } Bob if (no milk) { if (no note) { leave note buy milk remove note } No, may end up with “two milk”.  mutual exclusion progress Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

15. 15 Solution 2 Alice leave note A if (no note B) { if (no milk) {buy milk} } remove note A Bob leave note B if (no note A) { if (no milk) {buy milk} } remove note B Using labeled notes Does it work? No, may end up with no milk. mutual exclusion  progress Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

16. 16 Solution 3 Does it work? Alice leave note A while (note B) {skip} if (no milk) {buy milk} remove note A Bob leave note B if (no note A) { if (no milk) {buy milk} } remove note B Only if we make a timing assumption! mutual exclusion  progress Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

17. 17 Is solution #3 a good solution? No  A timing assumption is needed!  Unfair to one of the processes  Alice is busy waiting – consuming CPU cycles without accomplishing useful work  A timing assumption is needed!  Unfair to one of the processes  Alice is busy waiting – consuming CPU cycles without accomplishing useful work Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

18. 18 Solution 4 Using 4 labeled notes A1A2B2B1 the fridge’s door Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

19. 19 Alice leave A1 if B2 {leave A2} else {remove A2} while B1 and ((A2 and B2) or (no A2 and no B2)) {skip} if (no milk) {buy milk} remove A1 Bob leave B1 if (no A2) {leave B2} else {remove B2} while A1 and ((A2 and no B2) or (no A2 and B2)) {skip} if (no milk) {buy milk} remove B1 Solution 4 A correct, fair, symmetric algorithm! Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

20. 20 Solution 4 Notations color: there is a note A1A2B2 Alice’s turn B2 dotted line: there is no note solid line without color: we do not care if there is a note who will buy milk ? XX Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

21. 21 A1A2B2 Alice’s turn A1A2B2B1 Alice’s turn A2B2B1 Bob’s turn A1A2B2B1 Alice’s turn A1A2B2B1 Bob’s turn A1A2B2B1 Bob’s turn Solution 4 B2 XX XX Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

22. 22 A variant of Solution 4 The order of the first two statements is replaced Is it correct ? No, may end up with two milk. Alice if B2 {leave A2} else {remove A2} leave A1 while B1 and ((A2 and B2) or (no A2 and no B2)) {skip} if (no milk) {buy milk} remove A1 Bob if (no A2) {leave B2} else {remove B2} leave B1 while A1 and ((A2 and no B2) or (no A2 and B2)) {skip} if (no milk) {buy milk} remove B1 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006 B2 B1A1

23. 23 A variant of Solution 4 The order of the first two statements is replaced Is it correct ? No, may end up with two milk.  mutual exclusion progress Alice if B2 {leave A2} else {remove A2} leave A1 while B1 and ((A2 and B2) or (no A2 and no B2)) {skip} if (no milk) {buy milk} remove A1 Bob if (no A2) {leave B2} else {remove B2} leave B1 while A1 and ((A2 and no B2) or (no A2 and B2)) {skip} if (no milk) {buy milk} remove B1 MILK Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006 B2 B1A1

24. 24 The coordinated attack problem Blue army wins if both blue camps attack simultaneously Design an algorithm to ensure that both blue camps attack simultaneously Enemy 12 The problem is due to Jim Gray (1977) Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

25. 25 The coordinated attack problem Communication is done by sending messengers across valley Messengers may be lost or captured by the enemy  Enemy 12 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006 We seek a deterministic solution

26. 26 The coordinated attack problem Theorem: There is no algorithm that solves this problem! Proof (informal): Assume to the contrary that such an algorithm exists. Let P be an algorithm that solves it with the fewest # of messages, when no message is lost. P should work even when the last messenger is captured. So P should work even if the last messenger is never sent. So there is a correct algorithm P’ that sends one message less. A contradiction.  The model is too weak! Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006