1. DISTRIBUTED SYSTEMS II FAULT-TOLERANT AGREEMENT Prof Philippas Tsigas Distributed Computing and Systems Research Group

2. Copyright © George Coulouris, Jean Dollimore, Tim Kindberg 2001 email: authors@cdk2.net This material is made available for private study and for direct use by individual teachers. It may not be included in any product or employed in any service without the written permission of the authors. Viewing: These slides must be viewed in slide show mode. Teaching material based on Distributed Systems: Concepts and Design, Edition 3, Addison-Wesley 2001. Distributed Systems Course Coordination and Agreement 11.5 Consensus and Related problems

3. Agreement  All processes start with an initial value from some set V  Every process has to decide on a value in V such that: –Agreement: no two non-faulty processes decide on different values –Validity: if all processes start with the same value v, then no non- faulty process decides on a value different from v –Termination: all non-faulty processes decide within finite time 3

4. 4 The one general problem (Trivial!)  Battlefield G Troops

5. 5 The two general problem: messengers Blue armyRed army Blue G Red G Enemy

6. 6  Blue and red army must attack at same time  Blue and red generals synchronize through messengers  Messengers (messages) can be lost Rules:

7. 7 How Many Messages Do We Need? BGRG attack at 9am assume blue starts... Is this enough??

8. 8 How Many Messages Do We Need? BGRG attack at 9am assume blue starts... Is this enough?? ack (red goes at 9am)

9. 9 How Many Messages Do We Need? BGRG attack at 9am assume blue starts... Is this enough?? ack (red goes at 9am) got ack

10. 10 Stated problem is Impossible!  Theorem: There is no protocol that uses a finite number of messages that solves the two-generals problem (as stated here)  Proof: Consier the shortest such protocol(execution) –Consider last message –Protocol must work if last message never arrives –So don’t send it –But, now you have a shorter protocol(execution)

11. 11 Stated problem is Impossible!  Theorem: There is no protocol that uses a finite number of messages that solves the two-generals problem (as stated here) Alternatives??

12. 12 Probabilistic Approach?  Send as many messages as possible, hope one gets through... BGRG attack at 9am assume blue starts... attack at 9am

13. 13 Eventual Commit  Eventually both sides attack... BGRG attack ASAP assume blue starts... on my way! retransmits

14. 14 2-Phase Eventual Commit  Eventually both sides attack... BGRG ready to attack? assume blue starts... yes, at your disposal attack ASAP ack retransmits phase 1 phase 2

15. 15 Chalmers surrounded by army units Armies have to attack simultaneously in order to conquer Chalmers Communication between generals by means of messengers Some generals of the armies are traitors

16. The Byzantine agreement problem  One process(the source or commander) starts with a binary value  Each of the remaining processes (the lieutenants) has to decide on a binary value such that: Agreement: all non-faulty processes agree on the same value Validity: if the source is non-faulty, then all non-faulty processes agree on the initial value of the source Termination: all processes decide within finite time  So if the source is faulty, the non-faulty processes can agree on any value  It is irrelevant on what value a faulty process decides 16

17. Byzantine Empire

18. Conditions for a solution for Byzantine faults Number of processes: n Maximum number of possibly failing processes: f Necessary and sufficient condition for a solution to Byzantine agreement: f<n/3 Minimal number of rounds in a deterministic solution: f+1  There exist randomized solutions with a lower expected number of rounds 18

19. Senario 1 19

20. Senario 2 20

21. The Byzantine agreement problem (general formulation)  All processes starts with a binary value  They have to decide on a binary value such that: Agreement: all non-faulty processes agree on the same value Validity: Input value Termination: All correct processes decide within finite time It is irrelevant on what value a faulty process decides 21

22. Impossibility of 1-resilient 3-processor Agreement 22 A:V A =0 B:V B =0 C:V C =0 A´:V A´= 1 B´:V B´ =1 C´:V C´ =1 E1E1

23. Impossibility of 1-resilient 3-processor Agreement 23 A:V A =0 B:V B =0 C:V C =0 A´:V A´= 1 B´:V B´ =1 C´:V C´ =1 E0E0

24. Impossibility of 1-resilient 3-processor Agreement 24 A:V A =0 B:V B =0 C:V C =0 A´:V A´= 1 B´:V B´ =1 C´:V C´ =1 E1E1

25. Impossibility of 1-resilient 3-processor Agreement 25 A:V A =0 B:V B =0 C:V C =0 A´:V A´= 1 B´:V B´ =1 C´:V C´ =1 E2E2

26. Proof In E 0 A and B decide 0 In E 1 B´ and C´ decide 1 In E 2 C´ has to decide 1 and A has to decide 0, contradiction! 26

27. t-resilient algorithm requiring n 2 27 P1, P4,…P2,…P3,… P1, P2, P3, P4,...,Pn processors P´1= P´2= P´3= Distribute them Evenly to 3 logical Processes: P’1, P’2, P’3

28. t-resilient algorithm requiring n 2 28 P1, P4,…P2,…P3,… P1, P2, P3, P4,...,Pn processors P´1= P´2= P´3= |P´1|, |P´2|, |P´3| < t

29. t-resilient algorithm requiring n 2 29 P1, P4,…P2,…P3,… P1, P2, P3, P4,...,Pn processors P´1= P´2= P´3= In the system with 3 processes (P´1, P´2 and P´3) if one is faulty at most of the initial system are going to be faulty.

30. Proof  Run the solution to the problem (sysyem with n processes) at the 3 process system.  Ask P’1, P’2 and P’3 to simulate their respective substem and decide what the processes of its subsystem have decided.  Contradiction! This is a solution to a 3 processes system with one byzantine process. 30

31. o For a system with at most f processes crashing, the algorithm proceeds in f+1 rounds (with timeout), using basic multicast. o Values r i : the set of proposed values known to P i at the beginning of round r. o Initially Values 0 i = {} ; Values 1 i = {v i } for round = 1 to f+1 do multicast (Values r i – Values r-1 i ) Values r+1 i  Values r i for each V j received Values r+1 i = Values r+1 i  V j end d i = minimum(Values f+2 i ) Consensus in a Synchronous System with process crashing

32. Proof of Correctness Proof by contradiction.  Assume that two processes differ in their final set of values.  Assume that p i possesses a value v that p j does not possess.  A third process, p k, sent v to p i, and crashed before sending v to p j.  Any process sending v in the previous round must have crashed; otherwise, both p k and p j should have received v.  Proceeding in this way, we infer at least one crash in each of the preceding rounds.  But we have assumed at most f crashes can occur and there are f+1 rounds  contradiction.

33. Byzantine agreem. with authentication and Synchrony Every message carries a signature The signature of a loyal general cannot be forged Alteration of the contents of a signed message can be detected Every (loyal) general can verify the signature of any other (loyal) general Any number f of traitors can be allowed Commander is process 0 Structure of message from (and signed by) the commander, and subsequently signed and sent by lieutenants Li1, Li2,…: (v : s0 : si1: … : sik) Every lieutenant maintains a set of orders V Some choice function on V for deciding (e.g., majority, minimum) 33

34.  Algorithm in commander: send(v: s0)to every lieutenant –Algorithm in every lieutenant Li : upon receipt of (v : s0: si1: …. : sik) do if (v not in V) then V := V union {v} if (k < f) then for(j in {1,2,…,n-1} \{i,i1,…,ik}) do send(v: s0: si1: … : sik: i) to Lj If (Li will not receive any more messages) then decide(choice(V)) 34