1. Giovanni Chierico | May 2012 | Дубна Consistency in a distributed world

2. Giovanni Chierico | May 2012 | Дубна My goals Quick Review Distributed Transactions A Different Approach CAP Theorem Eventually Consistent Best Practices 2

3. Giovanni Chierico | May 2012 | Дубна Quick Review 3 The Good Concurrency Consistency Integrity The Bad Locks The really ugly Failures large scale systems?

4. Giovanni Chierico | May 2012 | Дубна Distributed System 4 Still the same problems Concurrency Consistency Integrity But more things that can fail

5. Giovanni Chierico | May 2012 | Дубна Two Generals’ Problem 5 G1G2 E Rules 1.Single General attack ⇒ Defeat 2.Double attack ⇒ Victory 3.Unreliable communication 1.G1: Let’s attack at 18:45 2.G2: confirmation 3.G1: confirmation 4.…

6. Giovanni Chierico | May 2012 | Дубна Two-Phase Commit (2PC) 6 Goal: all commit or all rollback 1.Prepare Phase Initiator asks other nodes to promise to commit or rollback, even if there’s a failure If any node cannot prepare ⇒ rollback 2.Commit Phase Initiator commits and asks others to do the same

7. Giovanni Chierico | May 2012 | Дубна Two-Phase Commit (2PC) 7 Prepare Phase: promise to commit or roll-back 1.Record operation in the “REDO” logs so that it can either commit or rollback regardless of failures 2.Place a “read lock” on the modified tables 3.Flag the transaction as “in-doubt” Non-failure case 1.Coordinator will ask to either commit or rollback 2.Remove the locks Failure Transaction will remain “in-doubt” and resources are inaccessible

8. Giovanni Chierico | May 2012 | Дубна Two-Phase Commit (2PC) Pros ACID Transparent (abstraction doesn’t “leak”) Cons When it works Expensive Read Locks When it fails Leaves Locks 8 “Better” Solutions Exist More Complex More expensive Consensus (Paxos) Google’s “Chubby”

9. Giovanni Chierico | May 2012 | Дубна A Different Approach 9 Your Coffee Shop Doesn’t Use Two-Phase Commit Employees: Baristas (B) and Cashiers (C) Process 1.You order to C 2.C writes your name on cup and puts it in a queue 3.You pay to C 4.B eventually prepares coffee and calls your name 5.You pick up your coffee Asynchronous Pros: less locking ⇒ more efficient use of resources Cons: A whole set of different problems …

10. Giovanni Chierico | May 2012 | Дубна Asynchronous Problems 10 Correlation: orders might be fulfilled not in the order they are queued ⇒ correlation identifier Exception Handling: cannot be easily “abstracted” Write-off: coffee made but you can’t pay Retry: coffee was not good (idempotent receivers) Compensating action: coffee machines breaks Optimistic “Happy Day” Approach Pessimistic Approach: Escrow Company Prepare: debit money Rollback: credit money back Commit: do nothing

11. Giovanni Chierico | May 2012 | Дубна Compensating Action Not as “simple” as in the ACID world Some things cannot be compensated for Shifts burden From infrastructure (declarative) To the client (ad-hoc solutions) Less “economy of scale” Monitoring, Control,... 11

12. Giovanni Chierico | May 2012 | Дубна Conversation Pattern 12 Async Sync Half-sync Half-async Half-sync Half-async

13. Giovanni Chierico | May 2012 | Дубна CAP Theorem Consistency Availability Partition-tolerance All 3 are desirable Can have any 2 but not 3 ACID Vs BASE Basically Available, Soft State, Eventually Consistent 13 Distributed Transactions

14. Giovanni Chierico | May 2012 | Дубна Eventually Consistent 14 Partitions are a given in larger systems Relax availability Might refuse to write Relax consistency Accept a write but this is not reflected in subsequent reads Strong: after update any read will get the updated value Weak: inconsistency window Eventual: if not further updates all the read will “eventually” get the update value. Inconsistency Window = f(delays, load, #replicas, …) http://www.allthingsdistributed.com/2008/12/eventually_consistent.html

15. Giovanni Chierico | May 2012 | Дубна Eventually Consistent Causal C: A updates, tells B, B reads updated value Read-yours-writes C special case of previous Session Consistency C as long the session exists RYWC is guaranteed Monotonic Read C Once you see a value you never see a previous version Monotonic Write C Serialise Writes Amazon’s Dynamo “has brought all of these properties under explicit control of the application architecture”, “allow the application service owner […] to make the trade-offs between consistency, durability, availability, and performance at a certain cost point" 15

16. Giovanni Chierico | May 2012 | Дубна Scalability Best Practices Avoid Distributed Transactions “we allow absolutely no client-side or distributed transactions of any kind - no two-phase commit.” Decouple Functions Asynchronously Messages and queues Move Processing To Asynchronous Flows Execution latency Vs User latency Scale for peak Vs Scale for average 16 EBay

17. Giovanni Chierico | May 2012 | Дубна Lesson Learned Large distributed systems often have different needs and requirements To maximise “business value” we might need to relax some constraints Problems are often “wicked” and the best solution depends on a lot of details and dependencies 17

18. Giovanni Chierico | May 2012 | Дубна Q&A 18

19. Giovanni Chierico | May 2012 | Дубна спасибо Globe of Science and Innovation, CERN 19

20. Giovanni Chierico | May 2012 | Дубна 20