1. Mutual Exclusion
A condition in which there is a set of processes, only one of which is able to access a given resource or perform a given function at any time

2. Centralized Systems Mutual exclusion via: Test & set Semaphores
Messages
Monitors

3. Distributed Mutual Exclusion
Assume there is agreement on how a resource is identified
Pass identifier with requests
Create an algorithm to allow a process to obtain exclusive access to a resource

4. Distributed Mutual Exclusion
Centralized Algorithm
Token Ring Algorithm
Distributed Algorithm
Decentralized Algorithm

5. Centralized algorithm
Mimic single processor system
One process elected as coordinator
request(R) C
Request resource
Wait for response
Receive grant
access resource
Release resource
grant(R) P
release(R)

6. Centralized algorithm
If another process claimed resource:
Coordinator does not reply until release
Maintain queue
Service requests in FIFO order P2
Queue
request(R)
request(R) P1 C P2
grant(R)
request(R) P0
grant(R) P1
release(R)

7. Centralized algorithm
Benefits
Fair
All requests processed in order
Easy to implement, understand, verify
Problems
Process cannot distinguish being blocked from a dead coordinator
Centralized server can be a bottleneck

8. Token Ring algorithm Assume known group of processes
Some ordering can be imposed on group
Construct logical ring in software
Process communicates with neighbor P0 P1 P2 P3 P4 P5

9. Token Ring algorithm Initialization Token circulates around ring
Process 0 gets token for resource R
Token circulates around ring
From Pi to P(i+1)mod N
When process acquires token
Checks to see if it needs to enter critical section
If no, send token to neighbor
If yes, access resource
Hold token until done P0 P1 P2 P3 P4 P5
token(R)

10. Token Ring algorithm Only one process at a time has token
Mutual exclusion guaranteed
Order well-defined
Starvation cannot occur
If token is lost (e.g. process died)
It will have to be regenerated
Does not guarantee FIFO order

11. Ricart & Agrawala algorithm
Distributed algorithm using reliable multicast and logical clocks
Process wants to enter critical section:
Compose message containing:
Identifier (machine ID, process ID)
Name of resource
Timestamp (totally-ordered Lamport)
Send request to all processes in group
Wait until everyone gives permission
Enter critical section / use resource

12. Ricart & Agrawala algorithm
When process receives request:
If receiver not interested:
Send OK to sender
If receiver is in critical section
Do not reply; add request to queue
If receiver just sent a request as well:
Compare timestamps: received & sent messages
Earliest wins
If receiver is loser, send OK
If receiver is winner, do not reply, queue
When done with critical section
Send OK to all queued requests

13. Ricart & Agrawala algorithm
N points of failure
A lot of messaging traffic
Demonstrates that a fully distributed algorithm is possible

14. Lamport’s Mutual Exclusion
Each process maintains request queue
Contains mutual exclusion requests
Requesting critical section:
Process Pi sends request(i, Ti) to all nodes
Places request on its own queue
When a process Pj receives a request, it returns a timestamped ack
Lamport time

15. Lamport’s Mutual Exclusion
Entering critical section (accessing resource):
Pi received a message (ack or release) from every other process with a timestamp larger than Ti
Pi’s request has the earliest timestamp in its queue
Difference from Ricart-Agrawala:
Everyone responds … always - no hold-back
Process decides to go based on whether its request is the earliest in its queue

16. Lamport’s Mutual Exclusion
Releasing critical section:
Remove request from its own queue
Send a timestamped release message
When a process receives a release message
Removes request for that process from its queue
This may cause its own entry have the earliest timestamp in the queue, enabling it to access the critical section

17. Characteristics of Decentralized Algorithms
No machine has complete information about the system state
Machines make decisions based only on local information
Failure of one machine does not ruin the algorithm
Three is no implicit assumption that a global clock exists

18. Decentralized Algorithm
Based on the Distributed Hash Table (DHT) system structure previously introduced
Peer-to-peer
Object names are hashed to find the successor node that will store them
Here, we assume that n replicas of each object are stored

19. Mutual Exclusion Algorithms
Non-token based:
A site/process can enter a critical section when an
assertion (condition) becomes true.
Algorithm should ensure that the assertion will be true
in only one site/process.
Token based:
A unique token (a known, unique message) is shared
among cooperating sites/processes.
Possessor of the token has access to critical section.
Need to take care of conditions such as loss of token,
crash of token holder, possibility of multiple tokens, etc.

20. General System Model
At any instant, a site may have several requests for critical section (CS), queued up, and serviced one at a time.
Site States: Requesting CS, executing CS, idle (neither requesting nor executing CS).
Requesting CS: blocked until granted access, cannot make additional requests for CS.
Executing CS: using the CS.
Idle: action is outside the site. In token-based approaches, idle site can have the token.

21. Mutual Exclusion: Requirements
Freedom from deadlocks: two or more sites should not endlessly wait on conditions/messages that never become true/arrive.
Freedom from starvation: No indefinite waiting.
Fairness: Order of execution of CS follows the order of the requests for CS. (equal priority).
Fault tolerance: recognize “faults”, reorganize, continue. (e.g., loss of token).

22. Performance Number of messages per CS invocation: should be minimized.
Synchronization delay, i.e., time between the leaving of CS by a site and the entry of CS by the next one: should be minimized.
Response time: time interval between request messages transmissions and exit of CS.
System throughput, i.e., rate at which system executes requests for CS: should be maximized.
If sd is synchronization delay, E the average CS execution time: system throughput = 1 / (sd + E).

23. Performance metrics Next site Last site enters CS exits CS Time
Synchronization
delay
Messages
sent
Enter CS
CS Request
arrives
Exit CS
Time E
Response Time

24. Performance ... Low and High Load: Best and Worst Case:
Low load: No more than one request at a given point in time.
High load: Always a pending mutual exclusion request at a site.
Best and Worst Case:
Best Case (low loads): Round-trip message delay + Execution time. 2T + E.
Worst case (high loads).
Message traffic: low at low loads, high at high loads.
Average performance: when load conditions fluctuate widely.

25. Simple Solution Control site: grants permission for CS execution.
A site sends REQUEST message to control site.
Controller grants access one by one.
Synchronization delay: 2T -> A site release CS by sending message to controller and controller sends permission to another site.
System throughput: 1/(2T + E). If synchronization delay is reduced to T, throughput doubles.
Controller becomes a bottleneck, congestion can occur.

26. Non-token Based Algorithms
Notations:
Si: Site I
Ri: Request set, containing the ids of all Sis from which permission must be received before accessing CS.
Non-token based approaches use time stamps to order requests for CS.
Smaller time stamps get priority over larger ones.
Lamport’s Algorithm
Ri = {S1, S2, …, Sn}, i.e., all sites.
Request queue: maintained at each Si. Ordered by time stamps.
Assumption: message delivered in FIFO.

27. Lamport’s Algorithm Requesting CS: Executing CS: Releasing CS:
Send REQUEST(tsi, i). (tsi,i): Request time stamp. Place REQUEST in request_queuei.
On receiving the message; sj sends time-stamped REPLY message to si. Si’s request placed in request_queuej.
Executing CS:
Si has received a message with time stamp larger than (tsi,i) from all other sites.
Si’s request is the top most one in request_queuei.
Releasing CS:
Exiting CS: send a time stamped RELEASE message to all sites in its request set.
Receiving RELEASE message: Sj removes Si’s request from its queue.

28. Lamport’s Algorithm… Performance. Optimization
3(N-1) messages per CS invocation. (N - 1) REQUEST, (N - 1) REPLY, (N - 1) RELEASE messages.
Synchronization delay: T
Optimization
Suppress reply messages. (e.g.,) Sj receives a REQUEST message from Si after sending its own REQUEST message with time stamp higher than that of Si’s. Do NOT send REPLY message.
Messages reduced to between 2(N-1) and 3(N-1).

29. Lamport’s Algorithm: Example
Step 1:
(2,1) S1 S2
(1,2) S3
Step 2: S1
(1,2) (2,1)
S2 enters CS S2
(1,2) (2,1) S3
(1,2) (2,1)

30. Lamport’s: Example… Step 3: (1,2) (2,1) S1 S2 leaves CS S2 (1,2) (2,1)
S1 enters CS S2
(2,1)
(1,2) (2,1) S3
(1,2) (2,1)
(2,1)

31. Ricart-Agrawala Algorithm
Requesting critical section
Si sends time stamped REQUEST message
Sj sends REPLY to Si, if
Sj is not requesting nor executing CS
If Sj is requesting CS and Si’s time stamp is smaller than its own request.
Request is deferred otherwise.
Executing CS: after it has received REPLY from all sites in its request set.
Releasing CS: Send REPLY to all deferred requests. i.e., a site’s REPLY messages are blocked only by sites with smaller time stamps

32. Ricart-Agrawala: Performance
2(N-1) messages per CS execution. (N-1) REQUEST + (N-1) REPLY.
Synchronization delay: T.
Optimization:
When Si receives REPLY message from Sj -> authorization to access CS till
Sj sends a REQUEST message and Si sends a REPLY message.
Access CS repeatedly till then.
A site requests permission from dynamically varying set of sites: 0 to 2(N-1) messages.

33. Ricart-Agrawala: Example
Step 1: S1
(2,1) S2
(1,2) S3
Step 2: S1
S2 enters CS S2
(2,1) S3

34. Ricart-Agrawala: Example…
Step 3: S1
S1 enters CS S2
(2,1)
S2 leaves CS S3

35. Distributed Deadlock

36. Distributed Deadlock Problem definition
Permanent blocking of a set of processes that either compete for system resources or communicate with each other
No node has complete and up-to-date knowledge of the entire distributed system
Message transfers between processes take unpredictable delays

37. Distributed Deadlock
A distributed system consists of a number of sites connected by a network. Each site maintains some of the resources of the system.
Processes with a global unique identifier run on the distributed system. They make resource request to a controller. There is one controller per site.
If the resource is local, the process makes a request of the local controller.
The controller at each site could maintain a WFG on the process request s that it know about it. This is the local WFG.
However, each site’s WFG could be cycle free and yet the distributed system could be blocked. This is called Global Deadlock.

38. System Model :
System has only reusable resources
Only exclusive access to resources
Only one copy of each resource
States of a process: running or blocked
Running state: process has all the resources
Blocked state: waiting on one or more resource

39. Types of Deadlocks Resource Deadlocks
A process needs multiple resources for an activity.
Use AND Condition
AND Condition : Deadlock occurs if each process in a set request resources
held by another process in the same set, and it must receive
all the requested resources to move further.
Communication Deadlocks
Processes wait to communicate with other processes in a set.
Use Or Condition: Each process in the set is waiting on another process’s message, and no process in the set initiates a message until it receives a message for which it is waiting.

40. Graph Models
Nodes of a graph are processes. Edges of a graph the pending requests or assignment of resources.
Wait-for Graphs (WFG): P1 -> P2 implies P1 is waiting for a resource from P2.
Transaction-wait-for Graphs (TWF): WFG in databases.
Deadlock: directed cycle in the graph.
Cycle example: P1 P2

41. Graph Models
Wait-for Graphs (WFG): P1 -> P2 implies P1 is waiting for a resource from P2. R1 P1 R2 P2

42. Deadlock in resource allocation
Conditions for deadlock in resource allocation
Mutual exclusion: The resource can be used by only one process at a time
Hold and wait: A process holds a resource while waiting for other resources
No preemption: A process cannot be preempted to free up the resource
Circular wait: A closed cycle of processes is formed, where each process holds one or more resources needed by the next process in the cycle
Strategies
Prevent the formation of a circular wait
Detect the potential or the actual occurrence of a circular wait
Types of algorithms
Deadlock prevention
Deadlock avoidance
Deadlock detection
Special issues in distributed systems
Resources are distributed across many sites
The control processes that control access to resources do not have complete, up-to-date information on the global state of the system

43. Deadlock in Resource Allocation: Deadlock handling strategies
Deadlock prevention
1.a Prevent the circular-wait condition by defining a linear ordering of resource types
A process can be assigned resources only according to the linear ordering
Disadvantages
Resources cannot be requested in the order that are needed
Resources will be longer than necessary
1.b Prevent the hold-and-wait condition by requiring the process to acquire all needed resources before starting execution
Inefficient use of resources
Reduced concurrency
Process can become deadlocked during the initial resource acquisition
Future needs of a process cannot be always predicted

44. Deadlock prevention (cont.)
1.c Use of time-stamps
Example: Use time-stamps for transactions to a database – each transaction has the time-stamp of its creation
The circular wait condition is avoided by comparing time-stamps: strict ordering of transactions is obtained, the transaction with an earlier time-stamp always wins
“Wait-die” method
if [ e (T2) < e (T1) ]
halt_T2 (‘wait’);
else
kill_T2 (‘die’);
“Wound-wait” method
kill_T1 (‘wound’);

45. Wait-die Vs. Wound-wait
If an old process wants a resource held by a young process, the old one will wait.
If a young process wants a resource held by an old process, the young process will be killed.
Observation: The young process, after being killed, will then start up again, and be killed again. This cycle may go on many times before the old one release the resource.
Once we are assuming the existence of transactions, we can do something that had previously been forbidden: take resources away from running processes.
When a conflict arises, instead of killing the process making the request, we can kill the resource owner. Without transactions, killing a process might have severe consequences. With transactions, these effects will vanish magically when the transaction dies.
Wound-wait: (we allow preemption & ancestor worship)
If an old process wants a resource held by a young process, the old one will preempt the young process -- wounded and killed, restarts and wait.
If a young process wants a resource held by an old process, the young process will wait.

46. (2) Deadlock avoidance
Decision made dynamically, before allocating a resource, the resulting global system state is checked - if safe, allow allocation
Disadvantages
Every site has to maintain global state of system (extensive overhead in storage and communication)
Different sites may determine (concurrently) that state is safe, but global state may be unsafe: verification for safe global state by different sites must be mutually exclusive
Large overhead to check for every allocation (distributed system may have large number of processes and resources)
Conclusion: Deadlock avoidance impractical in distributed systems

47. (3) Deadlock Detection Principle of operation
Detection of a cycle in WFG proceeds concurrently with normal operation
Requirements for the deadlock detection and resolution algorithms
Detection
The algorithm must detect all existing deadlock in finite time
The algorithm should not report non-existent (phantom) deadlock
Resolution (recovery)
All existing wait-for dependencies in WFG must be removed, i.e. roll-back one or more processes that are deadlocked and give their resources to other blocked processes
Observation
Deadlock detection is the most popular strategy for handling deadlocks in distributed systems

48. Algorithms for Distributed Deadlock Detection
3) Deadlock Detection (cont..)
Control for distributed deadlock detection can be:
Centralized
Distributed
Hierarchical
 a.1 Centralized deadlock detection algorithms
A central control site constructs the global WFG and searches for cycles
Control site maintain WFG continuously (with every assignment) or when running deadlock detection (and asking all sites for WFG updates)
Disadvantages: single point of failure and congestion
 a.2 The completely centralized algorithm
All sites request resources and release resources by sending corresponding messages to control site
Control site updates WFG for each request/release
For every new request edge added to WFG, control site checks WFG for deadlock
Alternative: each site maintain its WFG and update control site periodically or on request

49. 3) Deadlock Detection (cont.)
Deadlock in resource allocation: Algorithms for distributed deadlock detection
  3) Deadlock Detection (cont.)
b. Hierarchical deadlock detection algorithms
Sites organized in a tree structure with one site at the root of the tree
Each node (except for leaf nodes) has information about the dependent nodes
Deadlock is detected by the node that is the common ancestor of all sites which have resource allocations in conflict
Deadlock is detected at the lowest level

50. 3) Deadlock Detection (cont.)
Deadlock in resource allocation: Algorithms for distributed deadlock detection (cont.)
3) Deadlock Detection (cont.)
c. Distributed deadlock detection algorithms
Principles
All sites responsible for detecting a global deadlock
Global state graph distributed over many sites: several of them participate in detection
Detection initiated when a process suspected to be deadlocked
Advantages: No single point of failure, no congestion
Disadvantages: Difficult to implement (no shared memory)
Types of algorithms
Path-pushing algorithms
Each node builds a WFG based on local info & info from other sites
Detect and resolves local deadlocks
Transmits to other sites deadlock info in form of (waiting path)
Edge-chasing algorithms
Special messages (probes) sent along edges of WFG to detect a cycle
When blocked process receives probe, resends it on its outgoing edges of WFG
When a process receives a probe it initiated, declares deadlock