1. Multiprocessor Synchronization Algorithms (20225241)
Lecturer: Danny Hendler

2. Electronic submissions only!!!
Grade structure
3 (theoretic) problem sets
Up to 30% of grade
Take-home final exam
At least 70% of grade
Electronic submissions only!!!

3. Adminstrative details
Office: Alon 218
Office hours:
Mondays, 2pm-4pm
Or make an appointment
Course site: (or simply enter through my home page)

4. Books
Distributed Computing: Fundamentals, Simulations, and Advanced Topics, Hagit Attiya and Jennifer Welch, John Wiley and Sons
Synchronization Algorithms and Concurrent Programming, Gadi Taubenfeld, Pearson/Prentice Hall. Book site:

5. 5

6. Moore’s law Exponential growth in computing power6

7. How can we write correct and efficient algorithms for multiprocessors?
The Future of Computing
Speeding up uni-processors is harder and harder
Intel, Sun, AMD, IBM now focusing on “multi- core” architectures
Soon, most computers will be multiprocessors
How can we write correct and efficient algorithms for multiprocessors? 7

8. Distributed Systems The Internet
A distributed system is a collection of individual computing devices that communicate with one another. E.g.:
VLSI chips
Shared-memory multiprocessor
Local-area network
The Internet

9. One sequential model, MANY distributed models
Shared-memory / Message-passing
Which operations are allowed (read, write, read-modify-write)?
Synchronous, timing-conditions, asynchronous
Are failures allowed?
Fail-stop, Byzantine failures, message omission
Is the number of processes known?

10. Types of distributed systems
Multi-core computers
The Internet
Peer-to-peer systems
Grid computers
Wireless networks
Sensor networks
Ad-hoc networks …

11. Motivating examples of Synchronization

12. The Too-Much-Milk Problem
Time
Alice
Bob
5:00
Arrive Home
5:05
Look in fridge; no milk
5:10
Leave for grocery
5:15
Arrive home
5:20
Arrive at grocery
5:25
Buy milk
5:30
Arrive home; put milk in fridge
5:40
5:45
Arrive home; too much milk!
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

13. Solving the Too-Much-Milk Problem
Required properties
Mutual Exclusion: Only one person buys milk “at a time”
Progress: Someone always buys milk if it is needed
Communication primitives
Leave a note (set a flag)
Remove a note (reset a flag)
Read a note (test the flag)
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

14. Important Distinction
Safety Property
Nothing bad ever happens
Example: mutual exclusion
Liveness Property
Something good happens eventually
Example: progress
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

15. Solution 1 Alice if (no milk) { if (no note) { leave Note buy milk
remove note }
Bob
Does it work?
No, may end up with “two milk”.
mutual exclusion
progress
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

16. Solution 2 Using labeled notes Alice leave note A if (no note B) {
if (no milk) {buy milk} }
remove note A
Bob
leave note B
if (no note A) {
remove note B
No, may end up with no milk.
mutual exclusion
progress
Does it work?
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

17. Solution 3 Alice leave note A while (note B) {skip}
if (no milk) {buy milk}
remove note A
Bob
leave note B
if (no note A) { }
remove note B
Does it work?
mutual exclusion
progress
Only if we make a timing assumption!
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

18. Solution 4 Using 4 labeled notes A1 A2 B2 B1 the fridge’s door
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

19. Solution 4 A correct, fair, symmetric algorithm! Alice leave A1
if B2 {leave A2} else {remove A2}
while B1 and ((A2 and B2) or
(no A2 and no B2))
{skip}
if (no milk) {buy milk}
remove A1
Bob
leave B1
if (no A2) {leave B2} else {remove B2}
while A1 and ((A2 and no B2) or
(no A2 and B2))
remove B1
A correct, fair, symmetric algorithm!
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

20. Solution 4 A1 A2 X B2 X B2 A1 A2 B2 B1 A1 A2 B2 B1 Alice’s turn
These should be interpreted as follows: if the participant (Alice/Bob) evaluates the waiting condition and finds out that the condition depicted in the box is met – it can safely buy milk if there is none.
Bob’s turn
Bob’s turn
Bob’s turn
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

21. A variant of Solution 4 The order of the first two statements is replaced
Is it correct ?
Alice
if B2 {leave A2} else {remove A2}
leave A1
while B1 and ((A2 and B2) or
(no A2 and no B2))
{skip}
if (no milk) {buy milk}
remove A1
Bob
if (no A2) {leave B2} else {remove B2}
leave B1
while A1 and ((A2 and no B2) or
(no A2 and B2))
remove B1
No, may end up with two milk. B2 A1 B2 B2 B2 B2 B1
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

22. A variant of Solution 4 The order of the first two statements is replaced
Is it correct ?
Alice
if B2 {leave A2} else {remove A2}
leave A1
while B1 and ((A2 and B2) or
(no A2 and no B2))
{skip}
if (no milk) {buy milk}
remove A1
Bob
if (no A2) {leave B2} else {remove B2}
leave B1
while A1 and ((A2 and no B2) or
(no A2 and B2))
remove B1
MILK
No, may end up with two milk. B2 A1 B2 B2 B2 B1 B2
mutual exclusion
progress
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

23. The coordinated attack problem
Blue army wins if both blue camps attack simultaneously
Design an algorithm to ensure that both blue camps attack
simultaneously
Enemy 1 2
The problem is due to by Jim Gray (1977)
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

24. The coordinated attack problem
Communication is done by sending messengers across valley
Messengers may be lost or captured by the enemy 
Enemy 1 2
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006

25. The coordinated attack problem
Theorem: There is no algorithm that solves this problem !
Proof:
Assume to the contrary that such an algorithm exits.
Let P be an algorithm that solves it with the fewest # of messages, when no message is lost.
P should work even when the last messenger is captured.
So P should work even if the last messenger is never sent.
But this is a new algorithm with one less message.
A contradiction.
 The model is too weak!
Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2006