1. EE324 INTRO. TO DISTRIBUTED SYSTEMS LECTURE 13 TRANSACTIONS

2. Midterm  Midterm grading will take about a week and a half.  Assignment 3 will be out.  Thursday there will be a in-class session to prepare you for the assignment.

3. Last lecture  Distributed mutex

4. Lamport’s Shared Priority Queue  Each process i locally maintains Qi (its own version of the priority Q)  To execute critical section, you must have replies from all other processes AND your request must be at the front of Qi  When you have all replies:  All other processes are aware of your request (because the request happens before response)  You are aware of any earlier requests (assume messages from the same process are not reordered)

5. Lamport’s Shared Priority Queue  To enter critical section at process i :Stamp your request with the current time T  Add request to Qi  Broadcast REQUEST(T) to all processes  Wait for all replies and for T to reach front of Qi  To leave  Pop head of Qi, Broadcast RELEASE to all processes  On receipt of REQUEST(T’) from process j: Add T’ to Qi  If waiting for REPLY from j for an earlier request T, wait until j replies to you  Otherwise REPLY  On receipt of RELEASEPop head of Qi

6. Shared priority queue Node1: timeAction 40(start) 41Recv 42Reply to Node2: timeAction 11(start) 12Recv 13Reply to Node3: timeAction 14(start) 15Request Q:

7. Shared priority queue Node1: timeAction 40(start) 41Recv 42Reply to Node2: timeAction 11(start) 12Recv 13Reply to Node3: timeAction 14(start) 15Request 43Recv reply 1 44Recv reply 2 45Run critical section Q:

8. Shared priority queue Node1: timeAction 40(start) 41Recv 42Reply to 43Requet Node2: timeAction 11(start) 16Recv 17Reply to 18Request Node3: timeAction 14(start) 15Request 43Recv reply 1 44Recv reply 2 45Run critical section 46Recv Reply 48Recv Q:, Q:,, Q:,

9. Shared priority queue Node1: timeAction 40(start) 41Recv 42Reply to 43Request Node2: timeAction 11(start) 16Recv 17Reply to 18Request 50Recv reply from 1 51Recv Delay reply because is my earlier request which 1 didn’t reply Node3: timeAction 14(start) 15Request 43Recv reply 1 44Recv reply 2 45Run critical section 46Recv 47Reply to 1 48Recv 49Reply to 2 Q:, Q:,,

10. Shared priority queue Node1: timeAction 40(start) 41Recv 42Reply to 43Request Recv Reply to 1 Node2: timeAction 11(start) 16Recv 17Reply to 18Request 50Recv reply from 3 51Recv Recv reply from 1 Node3: timeAction 14(start) 15Request 43Recv reply 1 44Recv reply 2 45Run critical section 46Recv 47Reply to 1 48Recv 49Reply to 2 Q:, Q:,,

11. Shared Queue approach  Everyone eventually sees the same ordering  Ordered by Lamport’s clock.  Disadvantages: Very unreliable  Any process failure halts progress  3(N-1) messages per entry/exit  Advantages: Fair, Short synchronization delay

12. Lamport’s Shared Priority Queue  Advantages:  Fair  Short synchronization delay  Disadvantages:  Very unreliable (Any process failure halts progress)  3(N-1) messages per entry/exit

13. Today  We want to look at distributed transactions, but first we need to understand transactions in a single machine.

14. Today's Lecture  Reading CDK5 16.2~.4  Transaction basics  Locking and deadlock in transactions 14

15. Transactions  A group of operations often represent a unit of “work”.  Fundamental abstraction to group operations into a single unit of work  begin: begins the transaction  commit: attempts to complete the transaction  rollback / abort: aborts the transaction

16. Transactions  A transaction is a sequence of server operations that is guaranteed by the server to be atomic in the presence of multiple clients and server cr ashes.  Free from interference by operations being performed on behalf of other co ncurrent clients  Either all of the operations must be completed successfully or they must have no effect at all in the presence of server crashes 16

17. Transactions – The ACID Properties  The four desirable properties for reliable handling of concurren t transactions. (The alternative definition of transactions.)  Atomicity: “All or Nothing”  Consistency: Each transaction, if executed by itself, maintains the correctn ess of the database.  Isolation (Serializability): each transaction runs as if alone  Durability: once a transaction is done, it stays done. Cannot be undone. 17

18. Bank Operations 18 deposit(amount) deposit amount in the account withdraw(amount) withdraw amount from the account getBalance() -> amount return the balance of the account setBalance(amount) set the balance of the account to amount Operations of the Account interface bool xfer(Account src, Account dest, long x) { Transaction t = begin(); if (src.getBalance() >= x) { src.setBalance(src.getBalance() – x); dest.setBalance(dest.getBalance() + x); return t.commit(); } t.abort(); return FALSE; } A client’s banking transaction

19. The transactional model  Applications are coded in a stylized way: begin transaction Perform a series of read, update operations Terminate by commit or abort.  Terminology  The application is the transaction manager  The data manager is presented with operations from concurrently activ e transactions  It schedules them in an interleaved but serializable order 19

20. Transaction and Data Managers Transactions read update read update transactions are stateful: transaction “knows” about database contents and updates Data (and Lock) Managers 20

21. Transaction life histories  openTransaction()  trans;  starts a new transaction and delivers a unique TID trans. This identifier will be used in the other ope rations in the transaction.  closeTransaction(trans)  (commit, abort);  ends a transaction: a commit return value indicates that the transaction has committed; an abort retu rn value indicates that it has aborted.  abortTransaction(trans);  aborts the transaction. 21 SuccessfulAborted by clientAborted by server openTransaction operation server aborts transaction operation operation ERROR reported to client closeTransactionabortTransaction

22. Transactional Execution Log  As the transaction runs, it creates a history of its actions. Suppose we were to write down the sequence of operations it performs.  Data manager does this, one by one  This yields a “schedule”  Operations and order they executed  Can infer order in which transactions ran  Scheduling is called “concurrency control” 22

23. Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 © Pearson Education 2012 Figure 16.5 The lost update problem TransactionT: balance = b.getBalance(); b.setBalance(balance*1.1); a.withdraw(balance/10) TransactionU: balance = b.getBalance(); b.setBalance(balance*1.1); c.withdraw(balance/10) balance = b.getBalance(); $200 balance = b.getBalance(); $200 b.setBalance(balance*1.1); $220 b.setBalance(balance*1.1); $220 a.withdraw(balance/10) $80 c.withdraw(balance/10) $280

24. Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5 © Pearson Education 2012 Figure 16.6 The inconsistent retrievals problem TransactionV: a.withdraw(100) b.deposit(100) TransactionW : aBranch.branchTotal() a.withdraw(100); $100 total = a.getBalance() $100 total = total+b.getBalance() $300 total = total+c.getBalance() b.deposit(100) $300

25. Concurrency control  Motivation: without concurrency control, we have lost updates, inconsistent retrievals, etc.  Concurrency control schemes are designed to allow two or more transacti ons to be executed correctly while maintaining serial equivalence  Serial Equivalence is correctness criterion Schedule produced by concurrency control scheme should be equivalent to a seri al schedule in which transactions are executed one after the other  Schemes: locking, optimistic concurrency control, time-stamp based concur rency control 25

26. Serially Equivalent Interleaving  Means that effect of the interleaved execution is indistinguishable from s ome possible serial execution of the committed transactions  For example: T1 and T2 are interleaved but it “looks like” T2 ran before T 1  Idea is that transactions can be coded to be correct if run in isolation, an d yet will run correctly when executed concurrently (and hence gain a sp eedup) 26

27. Need for serially equivalent interleaving Data manager interleaves operations to improve concurrency DB: R 1 (X) R 2 (X) W 2 (X) R 1 (Y) W 1 (X) W 2 (Y) commit 1 commit 2 27 T 1 : R 1 (X) R 1 (Y) W 1 (X) commit 1 T 2 : R 2 (X) W 2 (X) W 2 (Y) commit 2

28. Need for serially equivalent interleaving Problem: transactions may “interfere”. Here, T 2 changes x, henc e T 1 should have either run first (read and write) or after (reading the changed value). Unsafe! Not serially equivalent DB: R 1 (X) R 2 (X) W 2 (X) R 1 (Y) W 1 (X) W 2 (Y) commit 2 commit 1 T 1 : R 1 (X) R 1 (Y) W 1 (X) commit 1 T 2 : R 2 (X) W 2 (X) W 2 (Y) commit 2 28

29. Serially equivalent interleaving Data manager interleaves operations to improve concurrency but schedules the m so that it looks as if one transaction ran at a time. This schedule “looks” like T 2 ran first. DB: R 2 (X) W 2 (X) R 1 (X) W 1 (X) W 2 (Y) R 1 (Y) commit 2 commit 1 29 T 1 : R 1 (X) R 1 (Y) W 1 (X) commit 1 T 2 : R 2 (X) W 2 (X) W 2 (Y) commit 2

30. Conflicting operations  A pair of operations conflicts when their combined effect depends on the ordering.  Read and write operation conflict rules Operations of different transactions ConflictReason read NoBecause the effect of a pair of read operations does not depend on the order in which they are executed readwriteYesBecause the effect of aread and awrite operation depends on the order of their execution write YesBecause the effect of a pair ofwrite operations depends on the order of their execution

31. Serial equivalence property  For two transactions to be serially equivalent, it is necessary and sufficient that all pairs of conflicting operations of the two transactions be executed in the same order at all of the objects they both access.

32. Recovery from abort  Servers must record all the effects of committed transactions and non o f the effects of aborted transactions.  Aborted transactions can cause “dirty reads” and “premature writes”.

33. 33 A dirty read when transaction T aborts TransactionT: a.getBalance() a.setBalance(balance + 10) TransactionU: a.getBalance() a.setBalance(balance + 20) balance = a.getBalance()$100 a.setBalance(balance + 10)$110 balance = a.getBalance()$110 a.setBalance(balance + 20) $130 commit transaction abort transaction uses result of uncommitted transaction!

34. Today's Lecture  Transaction basics  Locking and deadlock 34

35. Schemes for Concurrency control  Locking  Server attempts to gain an exclusive ‘lock’ that is about to be used by one o f its operations in a transaction.  Can use different lock types (read/write for example)  Two-phase locking  Optimistic concurrency control  Time-stamp based concurrency control 35

36. What about the locks?  Unlike other kinds of distributed systems, transactional systems t ypically lock the data they access  They obtain these locks as they run:  Before accessing “x” get a lock on “x”  Usually we assume that the application knows enough to get the right kind of lock. It is not good to get a read lock if you’ll later need to update the object  In clever applications, one lock will often cover many objects 36

37. Locking rule  Suppose that transaction T will access object x.  We need to know that first, T gets a lock that “covers” x  What does coverage entail?  We need to know that if any other transaction T’ tries to access x it will attempt to get the same lock 37

38. Examples of lock coverage  We could have one lock per object  … or one lock for the whole database (a global lock)  … or one lock for a category of objects  In a tree, we could have one lock for the whole tree associated with the root  In a table we could have one lock for row, or one for each column, or one for the w hole table  All transactions must use the same rules!  And if you will update the object, the lock must be a “write” lock, not a “ read” lock 38

39. Global lock?  Only let one transaction run at a time  Poor solution  Performance issues. bool xfer(Account src, Account dest, long x) { lock(); if (src.getBalance() >= x) { src.setBalance(src.getBalance() – x); dest.setBalance(dest.getBalance() + x); unlock(); return TRUE; } unlock(); return FALSE; }

40. Per-Object Locking  Other transactions can execute concurrently, as long as they don’t read or write the src or dest accounts bool xfer(Account src, Account dest, long x) { lock(src); if (src.getBalance() >= x) { src.setBalance(src.getBalance() – x); unlock(src); lock(dest); dest.setBalance(dest.getBalance() + x); unlock(dest); return TRUE; } unlock(src); return FALSE; } See any problem?

41. Read/Write locks  We can use different type of locks to increase concurrency.  Read/write locks. Need to respect the conflict rule.

42. 42 Read/Write locks: Lock compatibility For one objectLock requested readwrite Lock already set noneOK readOKwait writewait Operation Conflict rules: 1. If a transaction T has already performed a read operation on a particular object, then a concurrent transaction U must not write that object until T commits or aborts 2. If a transaction T has already performed a read operation on a particular object, then a concurrent transaction U must not read or write that object until T commits or aborts

43. 43 Strict Two-Phase Locking  Strict two-phase locking. Automatically release all locks upon commit or abort.

44. Why does strict 2PL imply serializability?  Suppose that T’ will perform an operation that conflicts with an operation that T has done:  T’ will update data item X that T read or updated  T updated item Y and T’ will read or update it  T must have had a lock on X/Y that conflicts with the lock that T’ wants  T won’t release it until it commits or aborts  So T’ will wait until T commits or aborts 44

45. 45 Use of locks in strict two-phase locking 1. When an operation accesses an object within a transaction: (a)If the object is not already locked, it is locked and the operation proceeds. (b)If the object has a conflicting lock set by another transaction, the transaction m ust wait until it is unlocked. (c)If the object has a non-conflicting lock set by another transaction, the lock is s hared and the operation proceeds. (d)If the object has already been locked in the same transaction, the lock will be p romoted if necessary and the operation proceeds. (Where promotion is prevent ed by a conflicting lock, rule (b) is used.) Lock promotion: getting a more exclusive lock (e.g., read  write lock) 2. When a transaction is committed or aborted, the server unlocks all objects it locked for the transaction.

46. 46 Deadlock with write locks TransactionT U OperationsLocksOperationsLocks a.deposit(100); write lockA b.deposit(200) write lockB b.withdraw(100) waits forU’sa.withdraw(200);waits forT’s lock onB A

47. Dealing with Deadlock in two-phase locking  Deadlock prevention  Acquire all needed locks in a single atomic operation  Acquire locks in a particular order  Often impractical in practice: transactions may not know which lock they may need in the future 47

48. Dealing with Deadlock in two-phase locking  Deadlock detection  Keep graph of locks held. Check for cycles periodically or each time an edge is added  Cycles can be eliminated by aborting transactions  Timeouts (“ignoring”)  Aborting transactions when time expires  Most transactions are short. Long-lived ones are probably deadlocked, so abort and retry. 48

49. 49 Deadlock detection: The wait-for graph B A Waits for Held by T U U T Waits for

50. 50 Timeouts Transaction TTransaction U OperationsLocksOperationsLocks a.deposit(100); write lock A b.deposit(200) write lock B b.withdraw(100) waits for U ’s a.withdraw(200); waits for T’s lock onB A (timeout elapses) T’s lock onA becomes vulnerable, unlockA, abort T a.withdraw(200); write locksA unlockA, B