1. Ch11 Distributed Agreement

2. Outline Distributed Agreement Adversaries Byzantine Agreement Impossibility of Consensus Randomized Distributed Agreement Exponential Time Shared Memory Consensus

3. Outline Distributed Agreement Adversaries Byzantine Agreement Impossibility of Consensus Randomized Distributed Agreement Exponential Time Shared Memory Consensus

4. Distributed Agreement Adversaries When proving correctness or analyzing an algorithm, it is convenient to assume that: the inputs, the failure times, failure behaviors any system variables are under the control of an adversary who, intuitively, makes as much difficulty for the algorithm as possible Worst case analysis: worst case choices of the adversary

5. Outline Distributed Agreement Adversaries Byzantine Agreement Impossibility of Consensus Randomized Distributed Agreement Exponential Time Shared Memory Consensus

6. Distributed Agreement The Agreement or the Consensus Problem: Assume P = {p 1,…,p M } is the set of all the processors in the system Some processors in P are faulty Let F be the set of all faulty processors in P Every processor p in P has a value p.Val The requirement: devise a distributed algorithm that lets each processor p computes a value p.A such that when the execution of this distributed algorithm terminates, the following two conditions hold: 1. (agreement value) For every pair of processors p and q, q.A = p.A 2. The agreement value is a function of the initial values {p.Val} of non-faulty processors

7. Outline Why Distributed Agreement is an interesting problem? Processor p is the leader Processor p has the right to enter the critical section Processor p has failed

8. Distributed Agreement Byzantine Agreement: Assumptions A failed processor can send arbitrary messages A non-failed processor always responds to a message within T seconds When a processor receives a message, it can reliably determine the sender of that message

9. Distributed Agreement Byzantine Agreement: The Byzantine Generals Problem: “ENEMY” the Sultan’s army Byz_A1 Byz_A3 Byz_A2 G2 G1G3 Some byzantine generals are “corrupted” “Non-corrupted” generals knew that they will be victorious only if they attack simultaneously Loyal generals must find a consensus to attack or to retreat Byz_A4 G4

10. Distributed Agreement Byzantine Agreement: The Byzantine Generals Problem (basic idea) Each general has to make a decision based on the opinions it gets from the other generals All loyal generals must make the same decision If all loyal generals get the same set of opinions for making the decision, then all loyal generals can achieve a consensus using the same procedure to decide How can we ensure that all loyal generals get the same set of opinions ?

11. Distributed Agreement Byzantine Agreement: The Byzantine Generals Problem (basic idea) To ensure that each loyal general gets the same set of values, it is sufficient that each loyal general uses the same value V j for every other general G j in order to decide The Byzantine Generals Problem is then reduced to agreement by generals on the value sent by a particular general: a commanding general Formally, we must have: 1. If the sender p s is loyal and sends the value V s, the loyal generals will decide that the value sent is V s 2. If the sender p s is treacherous, the loyal general will agree on the same value This problem is known as the interactive consistency problem

12. Distributed Agreement Byzantine Agreement: The Byzantine Generals Problem (continued) Assuming that each general can reliably broadcast its opinion, the loyal generals can reach an agreement! How and under which conditions?

13. Distributed Agreement The Byzantine Generals Problem (continued) Question 1 assuming that there is a reliable protocol for broadcast, is it possible to reach an agreement with three generals with one disloyal ? The answer is NO! Why? L1 C L2L1 C L2 disloyal retreat attack retreat attack retreat attack

14. Distributed Agreement The Byzantine Generals Problem (continued) Question 2 assuming that there is a reliable protocol for broadcast, is it possible to reach an agreement with four generals with one disloyal ? The answer is Yes! Why? Two cases are in order: The commanding general is disloyal The commanding general is loyal

15. Distributed Agreement The Byzantine Generals Problem (continued) Justification of the answer to Question 2 CL1L2L3 attack retreat By the end of the second round, L1 has 2 attack and 1 retreat L2 has 2 attack and 1 retreat L3 has 2 attack and 1 retreat Disloyal commanding General:C Each Lieutenant obeys the majority Round 1 Round 2

16. Distributed Agreement The Byzantine Generals Problem (continued) Justification of the answer to Question 2 CL1L2L3 attack retreat By the end of the second round, L1 has 2 attack and 1 retreat L2 has 2 attack and 1 retreat L3 has 3 attack and 1 retreat Disloyal Lieutenant: L3 Each Lieutenant obeys the majority: Each Loyal General decides “attack” Round 1 Round 2

17. Distributed Agreement The Byzantine Generals Problem (continued) Theorem: Assuming a synchronous system with M processors,of which up to t can be faulty, the loyal generals can reach a consensus only if M  3t+1 The algorithm to solve the Byzantine Generals Problem is parameterized by k the maximum number of disloyal generals This algorithm is BG(k)

18. Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea: The algorithm works in rounds of messages exchange. C L1L2L M-1 L(C)=the set of Lieutenants for C; size of L(C) = M-1 Round 1 If no message is sent to a Lieutenant that Lieutenant takes “retreat” as the default value L(C)

19. Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea: p1p1 L1L2L M-2 L(p 1 :C)=the set of Lieutenants for p 1 with respect to C’s opinion; size of L(p 1 :C) = M-2 Round 2 Every processor p 1 in L(C) acts as the commanding General p 1 sends M-2 messages p 1 receives M-2 messages p 1.v(2) := majority(V) where V = {p 1.v(1)}  {p 1.R q (2) : q in L(p 1 :C)} L(p 1 )

20. Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea: p2p2 L1L2L M-3 L(p 2 :p 1 )=the set of Lieutenants for p 2 with respect to p1’s opinion; size of L(p 2 :p 1 ) = M-3 Round 3 Every processor p 2 in L(p 1 ), for each p 1 in L(C), acts as the commanding General p 2 sends M-3 messages p 2 receives M-3 messages p 2.vr(3,p 1 ) := majority(V) where V = {p 2.vr(2,p 1 )}  {p 2.R q (3,p 1 ) : q in L(p 2 :p 1 )} L(p 2 )

21. Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea: p3p3 L1L2L M-3 L(p 3 :p 2 )=the set of Lieutenants for p 3 with respect to p 2 ’s opinion on...; size of L(p 3 ) = M-4 Round 4 Every processor p 3 in L(p 2 ), for each p 2 in L(p 1 ), p 1 in L(C), acts as the commanding General p 3 sends M-4 messages p 3 receives M-4 messages p 3.vr(4,p 2 ) := majority(V) where V = {p 3.vr(3,p 2 )}  {p 2.R q (4,p 2 ) : q in L(p 3 :p 2 )} L(p 3 )

22. Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea: p i-1 L1L2L M-i L(p i-1 :p i-2 )=the set of Lieutenants for p i-1 with respect to p i-2 ’s opinion on...; size of L(p i-1 :p i-2 ) = M-i Round i Every processor p i-1 in L(p i-2 ), for each p i-2 in L(p i-3 ), …, p 2 in L(p 1 ), p 1 in L(C), acts as the commanding General p i-1 sends M-i messages p i-1 receives M-i messages p i-1.vr(i,p i-2 ) := majority(V) where V = {p i-1.vr(i-1,p i-2 )}  {p i-1.R q (i,p i-2 ) : q in L(p i-1 :p i-2 )} L(p i-1 )

23. Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea: pkpk L1L2L M-k-1 L(p k :p k-1 )=the set of Lieutenants for p k with respect to p k-1 ’s opinion on...; size of L(p k ) = M-k-1 Round k+1, BG(0) Every processor p k in L(p k-1 ), for each p k-1 in L(p k-2 ), …, p 2 in L(p 1 ), p 1 in L(C), acts as the commanding General P k sends M-k-1 messages p k receives M-k-1 messages p k.vr(k+1,p k-1 ) := majority(V) where V = {p k.vr(k,p k-1 )}  {p k.R q (k+1,p k-1 ) : q in L(p k )} L(p k ) For each p k-1, p k decides p k.vr(k+1,p k-1 )

24. Distributed Agreement The Byzantine Generals Problem (continued) The BG(k) idea: pkpk L1L2L M-k-1 Round k+1 L(p k ) p k decides p k.vr(k+1,p k-1 ) Processor p k decides using (1+M-k-1) opinions So, if M=3k+1, then we have that p k decides using 2k+1 opinions Since at most k processors can be faulty, it follows that all non faulty processors make the same decision

25. Distributed Agreement The Byzantine Generals Problem (continued) The BG(k): BG(0): 1. The commanding sends its value to all the other n-1 processors 2. Each processor uses the value it receives from the commanding or uses the default value

26. Distributed Agreement The Byzantine Generals Problem (continued) The BG(k): BG(k), k>0: 1. The commanding sends its value to all the other n-1 processors 2. Let v p be the value the processor p receives from the commanding general, or the default value if no value is received. Processor p acts as the commanding in BG(k-1) to send the value v p to each of the other M-2 processors. 3. For each processor p, let v q be the value received from processor q (q  p). Processor p uses the majority({v}  {v q : q in L(C)}) where v is the value processor p received from the commanding general

27. Distributed Agreement The Byzantine Generals Problem (continued) The BG(k): number of messages sent Following the presentation gave above, one can see that the number of messages sent is proportional to (M-1)(M-2)(M-3)…(M-k-1) Since k can be (M-1)/3, it follows that the number of messages is O(M k )

28. Outline Distributed Agreement Adversaries Byzantine Agreement Impossibility of Consensus Randomized Distributed Agreement Exponential Time Shared Memory Consensus

29. Distributed Agreement The Byzantine Generals Problem (continued) Impossibility result If the system is asynchronous (no bound on the relative speeds of processors or the communication delays), then it can be shown (Fisher, Lynch, Paterson 1985) that agreement is impossible if even one processor can fail, and even if the failure is a crash failure

30. Outline Distributed Agreement Adversaries Byzantine Agreement Impossibility of Consensus Randomized Distributed Agreement Exponential Time Shared Memory Consensus

31. Distributed Agreement The Randomized Distributed Agreement Randomization: processors can flip coin Assumptions: The system consists of N processors, of which up to t can be faulty Processors communicate by using shared registers The shared registers are non-faulty The accesses to the shared register are sequentially consistent Atomic reads and writes of the contents of the registers The system is asynchronous

32. Outline Distributed Agreement Adversaries Byzantine Agreement Impossibility of Consensus Randomized Distributed Agreement Exponential Time Shared Memory Consensus

33. Distributed Agreement The Randomized Distributed Agreement The Exponential Time Shared Memory Consensus: The Naïve algorithm: Assume : the system is synchronous each processor p has a initial value V p to prefer The idea (algorithm for processor p): while I have not yet decided do 1. Read the set {V i } of values of every other processors 2. If for all i, V i = V p then decide V p else V p := coin_flip() end

34. Distributed Agreement The Randomized Distributed Agreement The Exponential Time Shared Memory Consensus: Transformation of the Naïve synchronous algorithm into an asynchronous algorithm Idea 1: “simulate” the synchronous algorithm: add a round variable at each processor The naïve algorithm becomes

35. Distributed Agreement The Randomized Distributed Agreement The Exponential Time Shared Memory Consensus: Idea 1 continue (algorithm for processor p): while I have not yet decided do 1. Read the set {V i } of values of every other processors 2. If for all i, V i = V p and p.round = I.round then decide V p else V p := coin_flip(); p.round := pround+1 end Problem : some processors can fail; a fail processor may not increment its round variable when it executed

36. Distributed Agreement The Randomized Distributed Agreement The Exponential Time Shared Memory Consensus: Idea 2: At any moment, the set of processors can be regarded as consisting of FP: the set of the largest round value; LP: the other processors If (  p,q in FP: V p = V q ) and eventually (  s in LP: V s = V q, q in FP ) then one can decides on V q, q in FP How can we achieve the eventually part ?

37. Distributed Agreement The Randomized Distributed Agreement The Exponential Time Shared Memory Consensus: Idea 2(continued): How can we achieve the eventually part ? Intuitively, the idea is to make the slower processors prefer to accept the value of faster processors.

38. Distributed Agreement The Randomized Distributed Agreement The Exponential Time Shared Memory Consensus: The algorithm :Variable used V[1..M] shared array of records, one per each processor V[i].value : the preferred decision of processor i; V[i].round : execution round of processor i; Local_V[1..M] local copy of V[1..M] Leaders : the processors that have the largest round values in round Local_V

39. Distributed Agreement The Randomized Distributed Agreement The Exponential Time Shared Memory Consensus: The algorithm :Functions used leader_set(Local_V) : returns the set of leaders Flip() : randomly returns either 0 or 1.

40. Distributed Agreement The Randomized Distributed Agreement The Exponential Time Shared Memory Consensus: The algorithm :Initially V[i].value := NIL; V[i].round :=0; /* not necessary */ Local_V[i].value := NIL; Local_V[i].round := 0; leaders := empty

41. Distributed Agreement The Randomized Distributed Agreement The Exponential Time Shared Memory Consensus: The algorithm : SM_Consensus(self, preference) (V[self].value, V[self].round) := (preference,1) while I have not made a decision do read V into Local_V; leaders := leader_set(Local_V); if (self  leaders) and (  i : Local_V[i].value  Local_V[self] : Local_V[i].round < Local_V[self].round -1) then decide(V[self].value) elseif (  i,j in leaders: Local_V[i].value = Local_V[j].value) then (V[self].value, V[self].round) := (V[i].value, V[i].round) for a i in leaders elseif V[self].value  NIL then (V[self].value, V[self].round):=(NIL, V[self].round) else (V[self].value, V[self].round) := (Flip(), round+1)

42. Distributed Agreement The Randomized Distributed Agreement The Exponential Time Shared Memory Consensus: The probability that all leaders choose the same value: O(2 -N ) The expected number of rounds: O(2 N )