1. EEC 693/793 Special Topics in Electrical Engineering Secure and Dependable Computing Lecture 15 Wenbing Zhao Department of Electrical and Computer Engineering Cleveland State University wenbing@ieee.org

2. 2 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Outline Reminder: Project Progress Report due tonight Review –Lamport timestamp –Vector timestamp Byzantine general problem –By Leslie Lamport, Robert Shostak, & Marshall Pease –http://berkeley.intel-research.net/maniatis/p382- lamport.pdf Practical Byzantine fault tolerance –By Miguel Castro and Barbara Liskov, OSDI’99 –http://www.pmg.csail.mit.edu/papers/osdi99.pdf

3. 3 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Lamport Clock A Lamport logical clock is a monotonically increasing software counter Each process P i keeps its own logical clock C i to apply Lamport timestamps to events To capture the happens-before relation →, processes must do the following: –Before each event at P i : C i := C i +1 –When P i sends a message m, it piggybacks t = C i –When P j receives (m,t): C j := max(C j,t) + 1 e → e’  C(e) < C(e’)

4. 4 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Lamport Clock: An Example

5. 5 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Vector Timestamps Lamport timestamps do not guarantee if C(a)< C(b) then a indeed happened before b We need vector timestamps for that –Each process P i has an array V i [1..n], where V i [j] denotes the number of events that process P i knows have taken place at process P j –When P i sends a message m, it adds 1 to V i [i], and sends V i along with m as vector timestamp vt(m) –When P j receives m from P i with vt(m), it updates each V j [k] to max(V j [k],vt(m)[k]), and increments V j [j] by 1

6. 6 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Vector Timestamps V i (a) < V j (b)  (if and only if) − V i (a)[ k ] <= V j (b)[ k ] for every k, and − V i (a)[ m ] < V j (b)[ m ] for some m

7. 7 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Implementing Causal Ordering Using Vector Timestamps When a process p j receives a message from p i, it delivery the message only if the following two conditions are met: –It has delivered all earlier messages sent by p i –It has delivered any message that process p i had delivered at the time it multicast the message vt(m)[i] = V j [i] + 1 vt(m)[k] ≤ V j [k] for k  i Delivery Condition at P j for m from P i

8. 8 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Exercise Label the events using Lamport Timestamps and show the timestamp piggybacked with each message P1 P2 P3 a b c d e f m1 m2 m3

9. 9 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Exercise Label the events using Vector Timestamps and show the timestamp piggybacked with each message P1 P2 P3 a b c d e f m1 m2 m3 Assuming the only type of events is sending/receiving of message, and each process increment its own counter on sending, can p2 received both m1 and m3, if we want to ensure causal delivery, can we deliver m3 ahead of m1? Why?

10. 10 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao The Byzantine Generals Problem Abstract model of a computer system that may have faulty components Faulty components may send conflicting information to different parts of the system Scenario where Byzantine Generals must reach agreement in the presence of traitors

11. 11 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao The Byzantine Generals Scenario Byzantine Army Division General Byzantine Army Division General Byzantine Army Division General Byzantine Army Division General Enemy City Commanding General Lieutenant General Traitorous General

12. 12 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Byzantine Generals Problem A commanding general must send an order to his n-1 lieutenants such that –IC1. All loyal lieutenants obey the same order –IC2. If the commanding general is loyal, then every loyal lieutenant obeys the order he sends IC1 = Agreement clause IC2 = Validity clause IC1 and IC2 are called interactive consistency Conditions

13. 13 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Byzantine Agreement Protocol Assumption: –Every message that is sent is delivered correctly Traitors cannot interfere with messages they do not sent –The receiver of a message knows who sent it Traitors cannot spoof messages –The absence of a message can be detected Traitors cannot prevent an agreement by not sending => Synchronous system + no spoofing

14. 14 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Byzantine Agreement Protocol (f=1) Round 1: the commander sends a value to each of the lieutenants Round 2: each of the lieutenants sends the value it received to its peers At the end of round 2, each lieutenant check to see if there is a majority opinion (attack or retreat). We have a solution if there is Question is: how many generals needed to tolerate f number of traitors?

15. 15 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Unsolvable Situations – N=3, f=1 Commander lieutenant Commander lieutenant Attack Retreat Attack He said Retreat

16. 16 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Byzantine Agreement Protocol (f=1) Commander lieutenant Retreat Attack He said Retreat lieutenant Attack He said Retreat He said Attack If there are f traitors, then there must be at least 3f + 1 total generals for IC1 and IC2 to hold

17. 17 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Byzantine Agreement Protocol (f=1) Under out assumption, if message digest is used and assuming the digest cannot be forged, we need only N=2f+1 to tolerate f traitors –The commander still can send different information to different lieutenant, but a lieutenant cannot lie about what the commander has told him In asynchronous systems, N=2f+1 is not sufficient –We have to stop after collecting f+1 input because the f faulty traitor could simply refrain from sending –Unfortunately there might be f inputs from traitors

18. 18 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Introduction to BFT Paper The growing reliance of industry and government on online information services Malicious attacks become more serious and successful More software errors due to increased size and complexity of software This paper presents “practical” algorithm for state machine replication that works in asynchronous systems like the Internet

19. 19 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Assumptions Asynchronous distributed system The network may fail to deliver, delay, duplicate or deliver them out of order Faulty nodes may behave arbitrarily Independent node failures The adversary cannot delay correct nodes indefinitely All messages are cryptographically signed by their sender and these signatures cannot be subverted by the adversary

20. 20 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Service Properties A (deterministic) service is replicated among ≥ 3f+1 processors. Resilient to ≤ f failures Safety: All replicas guaranteed to process the same requests in the same order Liveness: Clients eventually receive replies to their requests

21. 21 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Optimal Resiliency Imagine non-faulty processors trying to agree upon a piece of data by telling each other what they believe the data to be A non-faulty processor must be sure about a piece of data before it can proceed f replicas may refuse to send messages, so each processor must be ready to proceed after having received ( n-1)-f messages –Total of n-1 other replicas

22. 22 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Optimal Resiliency But what if f of the ( n-1)-f messages come from faulty replicas? To avoid confusion, the majority of messages must come from non-faulty nodes, i.e, ( n-f-1)/2 ≥ f => Need a total of ≥3f+1 replicas

23. 23 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao BFT Algorithm in a Nutshell Client Primary Backup f + 1 Match (OK)

24. 24 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Replicas and Views R0 R1 R2 R|R-1| Set of replicas (R): |R| ≥ 3f + 1 For view v: primary p is assigned such that p= v mod |R| R0 View 0 R0 R1 View 1 ………

25. 25 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Safeguards If the client does not receive replies soon enough, it broadcasts the request to all replicas If the request has already been processed, the replicas simply re-send the reply (replicas remember the last reply message they sent to each client) If the primary does not multicast the request to the group, it will eventually be suspected to be faulty by enough replicas to cause a view change

26. 26 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Normal Case Operation Client Primary {REQUEST, o, t, c} o – Operation t – Timestamp c - Client Timestamps are totally ordered such that later requests have higher timestamps than earlier ones

27. 27 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Normal Case Operation Primary p receives a client request m, it starts a three-phase protocol Three phases are: pre-prepare, prepare, commit

28. 28 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Pre-Prepare Phase v – view number n – sequence number d – digest of the message D(m) m – message Primary Backup, m>, m>

29. 29 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Prepare Phase A backup accepts the PRE-PREPARE message only if: –The signatures are valid and the digest matches m –It is in view v –It has not accepted a PRE-PREPARE for the same v and n –Sequence number is within accepted bounds

30. 30 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Prepare Phase If backup i accepts the pre-prepare message it enters prepare phase by multicasting to all other replicas and adds both messages to its log Otherwise does nothing Replica (including primary) accepts prepare message and adds them to its log, provided that –Signatures are correct –View numbers match the current view –Sequence number is within accepted bounds

31. 31 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Prepare Phase At replica i, prepared (m, v, n, i) = true, iff 2f PREPARE from different backups (not including replica i) that match the pre-prepare When prepared = true, replica i multicasts to other replicas

32. 32 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Agreement Achieved If primary is non-faulty then all 2f+1 non-faulty replicas agree on the sequence number If primary is faulty –Either ≥f+1 non-faulty replicas (majority) agree on some other sequence and the rest realize that the primary is faulty –Or, all non-faulty replicas will suspect the primary is faulty When a faulty primary is replaced, the minority of confused non-faulty replicas are brought up to date up by the majority

33. 33 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Commit Phase Replicas accept commit messages and insert them in their log provided signatures are same Define committed and committed-local predicates as –Committed (m, v, n) = true, iff prepared (m, v, n, i) is true for all i in some set of f+1 non- faulty replicas –Committed-local (m, v, n, i) = true iff the replica has accepted 2f+1 commit message from different replicas that match the pre- prepare for m If Committed-local (m,v,n,i) is true for some non- faulty replica i, then committed (m,v,n) is true

34. 34 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Commit Phase Replica i executes the operation requested by m after committed-local (m, v, n, i) = true and i’s state reflects the sequential execution of all requests with lower sequence numbers The PRE-PREPARE and PREPARE phases of the protocol ensure agreement on the total order of requests within a view The PREPARE and COMMIT phases ensure total ordering across views

35. 35 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Normal Operation Reply All replicas sends the reply, directly to the client v = current view number t = timestamp of the corresponding request i = replica number r = result of executing the requested operation c = client id Client waits for f+1 replies with valid signatures from different replicas, and with same t and r, before accepting the result r

36. 36 Spring 2007EEC693: Secure & Dependable ComputingWenbing Zhao Normal Case Operation: Summery C Primary: 0 2 1 Faulty: 3 Request Pre-prepare Prepare Commit Reply X