1. Gray& Reuter: Locking 6-7: 1 Concurrency Control Jim Gray Microsoft, Gray @ Microsoft.com Andreas Reuter International University, Andreas.Reuter@i-u.de 9:00 11:00 1:30 3:30 7:00 Overview Faults Tolerance T Models Party TP mons Lock Theory Lock Techniq Queues Workflow Log ResMgr CICS & Inet Adv TM Cyberbrick Files &Buffers COM+ Corba Replication Party B-tree Access Paths Groupware Benchmark MonTueWedThurFri

2. Gray& Reuter: Locking 6-7: 2 Concurrency Control: Outline Why lock (isolation)Why lock (isolation) When to lock (2ø locking, 1°, 2°,3° isolationWhen to lock (2ø locking, 1°, 2°,3° isolation What to lock (granularity)What to lock (granularity) How to lockHow to lock Exotics (optimistic, field calls, escrow)Exotics (optimistic, field calls, escrow) DeadlockDeadlock PerformancePerformance

3. Gray& Reuter: Locking 6-7: 3 Why Lock? Need isolation (the "I" of ACID):Need isolation (the "I" of ACID): Give each transaction the illusion that there are no concurrent updatesGive each transaction the illusion that there are no concurrent updates Hide concurrency anomalies.Hide concurrency anomalies. Do it automaticallyDo it automatically –(system does not know transaction semantics) Goal:Goal: –Although there is concurrency in system execution is equivalent to some serial execution of the system –Not deterministic outcome, just a consistent transformation

4. Gray& Reuter: Locking 6-7: 4 The Essentials Transactions Conflict if Reads And Writes overlapTransactions Conflict if Reads And Writes overlap More formally: Transaction T has Read Set: R(T) Write Set: W(T)More formally: Transaction T has Read Set: R(T) Write Set: W(T) T1 and T2 conflict IFF W(T2) & (R(T1) U W(T1)) Ø OrW(T1) & (R(T2) U W(T2)) ØT1 and T2 conflict IFF W(T2) & (R(T1) U W(T1)) Ø OrW(T1) & (R(T2) U W(T2)) Ø If they conflict, delay one until the other finishesIf they conflict, delay one until the other finishes

5. Gray& Reuter: Locking 6-7: 5 Laws Of Concurrency Control First Law of Concurrency Control Concurrent execution should not cause application programs to malfunction.First Law of Concurrency Control Concurrent execution should not cause application programs to malfunction. Second Law of Concurrency Control Concurrent execution should not have lower throughput or much higher response times than serial execution.Second Law of Concurrency Control Concurrent execution should not have lower throughput or much higher response times than serial execution.

6. Gray& Reuter: Locking 6-7: 6 Concurrency Control: Outline Why lock (isolation)Why lock (isolation) When to lock (2ø locking, 1°, 2°,3° isolationWhen to lock (2ø locking, 1°, 2°,3° isolation What to lock (granularity)What to lock (granularity) How to lockHow to lock Exotics (optimistic, field calls, escrow)Exotics (optimistic, field calls, escrow) DeadlockDeadlock PerformancePerformance

7. Gray& Reuter: Locking 6-7: 7 Formal Model: Transactions and Serial(izble) Histories State is a set of name value pairs: DB = { }State is a set of name value pairs: DB = { } Actions are defined on state: means Ti reads value of entity e means Ti writes value of entity eActions are defined on state: means Ti reads value of entity e means Ti writes value of entity e Each Transaction is a sequence of actions: Ti = | i = 1,...,ni>Each Transaction is a sequence of actions: Ti = | i = 1,...,ni> Want to run a set of transactions: T = {Ti | i= 1,...,n}Want to run a set of transactions: T = {Ti | i= 1,...,n} A History is any sequence S such that each and every Ti is a subsequence of SA History is any sequence S such that each and every Ti is a subsequence of S A history is Serial if it is of the form: Ti,Tj,.....,Tz (i.e. one transaction at a time).A history is Serial if it is of the form: Ti,Tj,.....,Tz (i.e. one transaction at a time). A history is serializable (isolated) if it is equivalent to a serial history.A history is serializable (isolated) if it is equivalent to a serial history.

8. Gray& Reuter: Locking 6-7: 8 Formal Mode: Execution History Equivalence Define the following permutation subgroup: commutes with (for all i, j) commutes with (iff i j and ei ej)Define the following permutation subgroup: commutes with (for all i, j) commutes with (iff i j and ei ej) Si is Equivalent to Sj if it can be permuted to SjSi is Equivalent to Sj if it can be permuted to Sj Alternative Definition (inputs and outputs):Alternative Definition (inputs and outputs): Define the Dependency set of history S D(S): { |,..., is a subsequence of S and ai = write or aj = write } (note: some initial and terminal dependencies are also needed)Define the Dependency set of history S D(S): { |,..., is a subsequence of S and ai = write or aj = write } (note: some initial and terminal dependencies are also needed) Two Schedules are equivalent iff they have same dependenciesTwo Schedules are equivalent iff they have same dependencies

9. Gray& Reuter: Locking 6-7: 9 T1 T2 W W T1 T2 R W T1 T2 R W Transaction Dependency Relations Shows data flow among transactions T1 READ e,1 T1 WRITE e,2 T1 WRITE e,2 T2 WRITE e,2 T2 READ e,2 T2 WRITE e,3Shows data flow among transactions T1 READ e,1 T1 WRITE e,2 T1 WRITE e,2 T2 WRITE e,2 T2 READ e,2 T2 WRITE e,3

10. Gray& Reuter: Locking 6-7: 10 The Three Bad Transaction Dependencies Locks are often used to prevent these dependencies. T1 T2 Lost Update T2 READ e,1 T1 WRITE e,2 T2 WRITE e,3 T1 T2 Dirty Read T2 WRITE e,2 T1 READ e,2 T2 WRITE e,3 T1 T2 UnRepeatable Read T1 READ e,1 T2 WRITE e,2 T1 READ e,2 T2 T1 OK

11. Gray& Reuter: Locking 6-7: 11 Dependencies Show Dataflows Among Transactions T1 T2 T3 T4 T5 T6 T1 T2 T3 T4 T5 T6 Two histories are equivalent iff they have the same dependencies We want only histories equivalent to a serial history. If T2 depends on data from T1, then T2 ran after T1. If T4 depends on data from T3, then T4 ran after T3. This is a wormhole (in time): T4 ran after T4. T3-> T4 -> T6 ->T3 ->T4 Cycles in the dependency graph are bad.

12. Gray& Reuter: Locking 6-7: 12 Locks Cover Actions Introduce three new actions: LOCK [READ | WRITE] UNLOCK Lock Covers Lock Covers If is at or after the lock step AND No intervening unlock AND a is WRITE OR a' IS READ (write covers write and read) t slock o1 t xlock o2 t read o1 t read o3 t write o1 t write o2 t read o2 t unlock o1 t unlock o2 t unlock o3 Not Covered

13. Gray& Reuter: Locking 6-7: 13 Well Formed and Two-Phased Transactions Transaction T is well formed if: All actions of T are covered by a lock Lock Act Unlock Transaction T is 2-phase if: No unlock precedes a lock in T. (i.e. A T has a LOCK phase and an UNLOCK phase) GROWShrink Basic rules:Basic rules: –Lock everything you read/write –Hold locks to end-of- transaction

14. Gray& Reuter: Locking 6-7: 14 Lock Commutativity commutes with commutes with (if i j, and (ei ej)) (if i j, and (ei ej)) commutes with (if i j) commutes with (if i j)

15. Gray& Reuter: Locking 6-7: 15 LOCKS define LEGAL HISTORIES History is legal if: Don't grant incompatible locks to two at once. If T1 covers e with a WRITE lock, No T2 covers e at that point in the history (if T1 T2).History is legal if: Don't grant incompatible locks to two at once. If T1 covers e with a WRITE lock, No T2 covers e at that point in the history (if T1 T2). Legal & Serial T1 SLOCKA T1 XLOCKB T1 READA T1 WRITEB T1 UNLOCKA T1 UNLOCKB T2 SLOCKA T2 READA T2 XLOCKB T2 WRITEB T2 WRITEB T2 UNLOCKA T2 UNLOCKB T1 Begin T1 Slock A T1 Xlock B T1 Read A T1 Write B T1 Commit T2 Begin T2 Slock A T2 Read A T2 Xlock B T2 Write B T2 Rollback T2 SLOCKA T1 SLOCKA T2 READA T2 XLOCKB T2 WRITEB T2 WRITEB T2 UNLOCKA T2 UNLOCKB T1 XLOCKB T1 READA T1 WRITEB T1 UNLOCKA T1 UNLOCKB T2 Begin T2 Slock A T2 Read A T2 Xlock B T2 Write B T2 Rollback T1 Begin T1 Slock A T1 Xlock B T1 Read A T1 Write B T1 Commit Legal & Not Serial T1 SLOCKA T1 XLOCKB T2 SLOCKA T2 READA T2 XLOCKB T2 WRITEB T2 WRITEB T2 UNLOCKA T2 UNLOCKB T1 READA T1 WRITEB T1 UNLOCKA T1 UNLOCKB T Begin T Slock A T Xlock B T Read A T Write B T Commit T' Begin T' Slock A T' Read A T' Xlock B T' Write B T' Rollback NotLegal & Not Serial

16. Gray& Reuter: Locking 6-7: 16 SERIALIZABILITY THEOREM (Wormhole Theorem) 1. If all transactions are well formed and 2 1. If all transactions are well formed and 2 then all legal histories are serializable. 2. If T1 is not well formed or not 2 2. If T1 is not well formed or not 2 then there is a T2 such that T1 and T2 have a legal but not serializable history except for trivial cases.

17. Gray& Reuter: Locking 6-7: 17 WF & 2 => No Wormholes Consider first unlock in schedule Consider first unlock in schedule Claim: Ti is a firstClaim: Ti is a first –There is no Tj >> Ti for all j I (Tj before Ti) Suppose not ( suppose Tj >> Ti ):Suppose not ( suppose Tj >> Ti ): –Then Tj accesses some e2 –Then Tj unlocks e2 (Tj is WF) –Then Ti locks e2 (Ti is WF) –Then Ti reads or writes e2 –So Tj unlock is before Ti unlock (2 ) => <= Contradiction proves Ti is a first.Contradiction proves Ti is a first.

18. Gray& Reuter: Locking 6-7: 18 Proof of Wormhole Theorem If no wormhole, then equivalent to serial Induction: If no cycles, then there is a last transaction Permute it to end of history (this will preserve dependencies). So, equivalent history. By induction remaining history is equivalent to a serial history. OriginalT4 Moved T3 Moved T2 Moved = Serial T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4

19. Gray& Reuter: Locking 6-7: 19 Proof of serializability theorem 2.1. Not WF = > nonserial T1 not well formed: not covered by a lock. Construct WF & 2 T2 = Construct WF & 2 T2 =,,, >, > Now the history H: <...,,,,,,,...> Is legal, but not equivalent to serial T1 is "after" T2 and also "before" T2. T1 T2 e e e

20. Gray& Reuter: Locking 6-7: 20 Proof of serializability theorem 2.1. Not 2 = > nonserial T1 NOT 2 means...... <T1, LOCK, e2 Construct WF & 2 T2: Construct WF & 2 T2:,, > Now the history H: Now the history H: <...,,,, >,<T1,LOCK, e2,...> Is legal, but not equivalent to a serial history T1 is "after" T2 and also "before" T2 T2 T1 e1 e2

21. Gray& Reuter: Locking 6-7: 21 Restatement of serializability theorem Lock everything transaction accessesLock everything transaction accesses Do not lock after unlock.Do not lock after unlock. Backout may have to undo a unlock ( = lock).Backout may have to undo a unlock ( = lock). So do not release locks prior to commit ø 1So do not release locks prior to commit ø 1 Keep exclusive locks (write locks) to commit ø 2Keep exclusive locks (write locks) to commit ø 2

22. Gray& Reuter: Locking 6-7: 22 Serializability Theorems Wormhole Theorem : A history is isolated if, and only if, it has no wormhole transactions.Wormhole Theorem : A history is isolated if, and only if, it has no wormhole transactions. Locking Theorem: If all transactions are well-formed and two-phase, then any legal history will be isolated.Locking Theorem: If all transactions are well-formed and two-phase, then any legal history will be isolated. Locking Theorem (converse): If a transaction is not well-formed or is not two-phase, then it is possible to write another transaction, such that the resulting pair is a wormhole.Locking Theorem (converse): If a transaction is not well-formed or is not two-phase, then it is possible to write another transaction, such that the resulting pair is a wormhole. Rollback Theorem: An update transaction that does an UNLOCK and then a ROLLBACK is not two-phase.Rollback Theorem: An update transaction that does an UNLOCK and then a ROLLBACK is not two-phase.

23. Gray& Reuter: Locking 6-7: 23 Concurrency Control: Outline Why lock (isolation)Why lock (isolation) When to lock (2ø locking, 1°, 2°,3° isolationWhen to lock (2ø locking, 1°, 2°,3° isolation What to lock (granularity)What to lock (granularity) How to lockHow to lock Exotics (optimistic, field calls, escrow)Exotics (optimistic, field calls, escrow) DeadlockDeadlock PerformancePerformance

24. Gray& Reuter: Locking 6-7: 24 Isolation Levels = Degrees of { Isolation | Consistency } 0 :transaction gets short xlocks for writes (well formed writes not 2Ø, no read locks) 1 :transaction gets no read locks (well formed and 2 Ø writes,) 2 :transaction releases read locks right after read (well formed with respect to reads but not 2 Ø with respect to reads) 3 :well formed and 2Ø (= Serializable by previous theorems!!) Transaction backout prohibits 0.

25. Gray& Reuter: Locking 6-7: 25 What Do Systems Do? Most non SQL systems support 2 Most non SQL systems support 2 Most SQL systems default to 3 and allow forms of 1, 2 Most SQL systems default to 3 and allow forms of 1, 2 e.g.:NonStop SQL: 1 = BROWSE 2 ~ STABLE READ DB2:2 ~ CURSOR STABILITY 3 ~ REPEATABLE READ SQL Standard 1 = READ UNCOMMITTED 2 = READ COMMITTED 2.99 = REPEATABLE READ 3 = SERIALIZABLE 3 = SERIALIZABLE

26. Gray& Reuter: Locking 6-7: 26 Isolation Levels Theorem If others lock 1° or 2° and I lock 3° then I get 3° (serializable). Any other trans is before me or after me. BUT DB may be corrupted by them. DB may be corrupted by them.

27. Gray& Reuter: Locking 6-7: 27 Comparison of Isolation Levels Protection Provided Lets others run at higher isolation 0 and no lost updates 1+ No dirty reads 2 + Repeatable reads Committed dataWrites visible immediately Writes visible at eot Same Dirty dataYou don't overwrite dirty data 0 and others do not overwrite your dirty data 0, 1, and you don't read dirty data 0,1,2 and others don't dirty data you read Lock protocolSet shortSet long 1 and set short1 and set long exclusive locks on data you write exclusive locks on data you write share locks on data you read share locks on data you read Trans structureWell-formed Wrt writes Well-formed wrt writes Two-phase wrt writes Well-formed And Two-phasewrt writes Well-formed and Two-phase ConcurrencyGreatest: only set short write locks Great: only wait for write locks Medium: hold few read locks Lowest: any data touched Locked toeot IssueDegree 0Degree 1 Degree 2Degree 3 Common nameChaos Isolated Serializable Repeatable reads Browse read uncommited Cursor stability read committed Rollback supported

28. Gray& Reuter: Locking 6-7: 28 Comparison of Isolation Levels Issue Degree 0 Degree 1 Degree 2 Degree 3 OverheadLeast: short W locks Small: Only write locks Medium: Set R&W but short R Most: Set long R&W Rollback UNDO may cascade Cant rollback Can Undo incomplete transactions sameSame System Recovery Dangerous, Updates may be lost and violate 3° Apply log in 1° order orderSamesame DependenciesNone W W W W W W W R W W W R R W

29. Gray& Reuter: Locking 6-7: 29 Concurrency Control: Outline Why lock (isolation)Why lock (isolation) When to lock (2ø locking, 1°, 2°,3° isolationWhen to lock (2ø locking, 1°, 2°,3° isolation What to lock (granularity)What to lock (granularity) How to lockHow to lock Exotics (optimistic, field calls, escrow)Exotics (optimistic, field calls, escrow) DeadlockDeadlock PerformancePerformance

30. Gray& Reuter: Locking 6-7: 30 The Phantom Detail If I try to read hair = "red" and eyes = "blue" and get not found, what gets locked? No records have been accessed so no records get lockedIf I try to read hair = "red" and eyes = "blue" and get not found, what gets locked? No records have been accessed so no records get locked If I delete a record, what gets locked?(the record is gone)If I delete a record, what gets locked?(the record is gone) These are cases of phantom records.These are cases of phantom records. Predicate locks solve this problem (see below)Predicate locks solve this problem (see below) Page Locks (done right) can solve this problem lock the red hair page and the blue eye page, prevents others red hair and blue eye inserts & updatesPage Locks (done right) can solve this problem lock the red hair page and the blue eye page, prevents others red hair and blue eye inserts & updates High volume TP systems use esoteric locking mechanisms:High volume TP systems use esoteric locking mechanisms: Key Range Locks: to protect b-trees Hole Locks: to protect space for uncommitted deletes

31. Gray& Reuter: Locking 6-7: 31 Predicate Locks Read and write sets can be defined by predicates (e.g. Where clauses in SQL statements) When a transaction accesses a set for the first time, 1. Automatically capture the predicate 2. Do set intersection with predicates of others. 3. Delay this transaction if conflict with others. Problems with predicate locks: 1.Set intersection = predicate satisfiability is NP complete (slow). 2.Hard to capture predicates 3. Pessimistic:Jim locks eye = blue Andreas lockshair=red Predicate says conflict, but DB may not have blue eyed red haired person.

32. Gray& Reuter: Locking 6-7: 32 Precision Locks: Lazy Predicate Locks Precision locks solve problems 1 & 3: Check returned records against predicates on each read/write Example: Andreas can't insert/read blue eyes Jim can't insert/read red hair. check records as they go by

33. Gray& Reuter: Locking 6-7: 33 Granularity Of Locks An Engineering Solution To Predicate Locks Can lock whole DB, whole file, or just one key value. Size of lock is called granule. Idea: Pick a fixed set of predicates They form a lattice under and, or This can be represented as a graph Lock the nodes in this graph Simple example:

34. Gray& Reuter: Locking 6-7: 34 Lock Granularity Batch wants to lock whole DB Interactive wants to lock records How can we allow both granularities? Intention mode locks on coarse granules Simple compatibility matrix Compatibility Matrix Mode Intent ShareeXclusive I+ -- S- +- X- --

35. Gray& Reuter: Locking 6-7: 35 Lock Granularity: refined intent modes Intent mode locks say locks being set at finer granularity If only reading at finer granularity then I compatible with S. Introduce IS: intend to set fine S locks IX: intend to set fine S or X locks SIX: S + IX

36. Gray& Reuter: Locking 6-7: 36 Granularity ExampleNotes: T3 is waiting T2 has all of file 3 locked shared Most of file 2 locked shared (Fine granularity) T1 has record locks in file-1 and file-2 Rules: Lock root to leaf If set X,S below get IX or IS above On a DAG (Directed Acyclic Graph) Get ONE IS,IS,...,S path for reads Get ALL IX,IX,...,X paths for a write DATABASE FILE-1 FILE-2 FILE-3 KEY-A T1:IX, T2:IS T3:S T1:X T1:S, T2:S T1:IXT1:IX, T2:IS T2:S T1:XT2:S T1:IXT2:IS T1:IX T1:X T3:S T1:IX T1:S T2:IS T2:S

37. Gray& Reuter: Locking 6-7: 37 Update Mode Locks (minor tangent) Most common form of deadlock T1 READ A(lock A shared) T2 READ A(lock A shared) T1 UPDATE A(lock A exclusive, wait for T2) T2 UPDATE A(deadlock A exclusive, wait for T1) So introduce update mode lock: Compatibility Matrix ISIXS SIXUX IS+++ +-- IX++- --- S+-+ - - - SIX+-- --- U--+ --- X--- --- U compatible with S so updators do not hurt readers. If certain to update record then get x mode lock right away.

38. Gray& Reuter: Locking 6-7: 38 Escalation If transaction gets too many locks, system probably guessed wrong about granularity Convert fine grain locks to one coarse one Example: 1000 record locks on table t becomes 1 file lock on table t. In some contexts, DE-Escalation is best: Get course grained locks. Remember fine grain resource names. On callback: request fine-grained locks and de-escalate coarse lock

39. Gray& Reuter: Locking 6-7: 39 Lock Conversion If requested lock already held in one mode, new mode is: max (old, requested) X SIX U IX S IS

40. Gray& Reuter: Locking 6-7: 40 Key Range Locking (for Phantoms) Suppose operations are: Read(key); /* return current value*/ Write(key, value) ; /* set key's value*/ Delete(key); /* delete key + value */ Read_Next(key); /* returns next key + val */ Insert between X and Y must test to see that No one else cares that [X,Y] was empty, but is now full no other concurrent trans did a Read_Next("X");).

41. Gray& Reuter: Locking 6-7: 41 Key Range Locking (a solution) Prev-Key Key-Range locking. Fixed ranges [A,B), [B,C),...., [Z, ) (this is easy) (this is easy) Dynamic ranges: If X, Y, Z are in list....[X, Y), [Y,Z), [Z, )....[X, Y), [Y,Z), [Z, ) are the lock ranges. List changes as list changes. are the lock ranges. List changes as list changes. Ranges named by first key in range. Lock a range when operating on the range Insert and delete Lock 3 ranges [X. Y), [X, Z), [Y, Z) Dual Is next key locking. Best solution is not published.

42. Gray& Reuter: Locking 6-7: 42 DAG Locking In general predicate locks and key-range locks form a DAG not just a tree: a lock can have many parents. Blue-eye key range Blonde-hair key range. Hierarchical locks work for this. Read locks any path Writes lock all paths. Hair Index Auburn Black Blonde Brunette Platinum Red White Yellow Eye Index Black Blue Brown Green Gray Red Famous People T : IX, T' : IX T : IX T : X T' : S Blonde Hair Red Hair Blue Eyes Don Marilyn Monroe T : X

43. Gray& Reuter: Locking 6-7: 43 Concurrency Control: Outline Why lock (isolation)Why lock (isolation) When to lock (2ø locking, 1°, 2°,3° isolationWhen to lock (2ø locking, 1°, 2°,3° isolation What to lock (granularity)What to lock (granularity) How to lock Exotics (optimistic, field calls, escrow)Exotics (optimistic, field calls, escrow) DeadlockDeadlock PerformancePerformance

44. Gray& Reuter: Locking 6-7: 44 Basic Synchronization Primitives (hardware) boolean CompareAndSwap(long * cell, long old, long new) { if ( *cell == old) { *cell = *new; return TRUE:} { *old = * cell; return FALSE} } value = Load Locked(long * cell); other stuff; boolean = StoreConditional(*cell, value); /* store conditional fails if cell has changed */ Can use these operators to implement: Shared storage managers Queues Semaphores (latches)

45. Gray& Reuter: Locking 6-7: 45 Lock Interface (API) (name,- name of resource LOCK (name,- name of resource,mode- S, X, SIX, IX, IS, U,duration- instant, short, long,wait)- no, timeout,yes (name - name of resource UNLOCK (name - name of resource,clear)- decrement count to zero or not.,clear)- decrement count to zero or not. Locks must count: if lock twice and unlock once, lock kept Lock state must be saved (somehow) at transaction save points (allow rollback) commit (allow restart to reacquire) generally, save is implicit (recompute them)

46. Gray& Reuter: Locking 6-7: 46 LOCK NAME HASH LINK SEMAPHORE MODE WAITS? QUEUE HEADER FROM HASH TABLE NEXT IN HASH CHAIN NEXT IN QUEUE HEADER MODE HELD MODE DESIRED GRANTED? DURATION COUNT TRANSACTION GRANTED GROUP WAITING GROUP list of locks of transaction T (from Trans control block) next lock of T Most locks are free most of the time so only allocate space for a lock if busy hash table points at lock names Lock queue: Notes: this is a very busy one, most queues = 1 holder transaction lock queue helps at commit (want to free all locks of T) semaphores cover hash chain, lock queue (not shown below)

47. Gray& Reuter: Locking 6-7: 47 LOCK Control Flow Hash Name & Search for Name Not Found: Allocate & Format Header and Body Exit Requestor Already Granted To Requestor? Yes: Requested Mode Compatible With Other Granted? Yes: Grant, Bump Count, Exit No: Bump Count, Set Desire, Wait No:Allocate Queue Element at End Anyone Waiting? Yes: Mark Waiting, Wait No: Compatible With Grantees? Yes: Grant No: Wait Exit

48. Gray& Reuter: Locking 6-7: 48 UNLOCK Control Flow Hash Name & Find Name Find Requestor In Queue Decrement Count If Count > 0 Then Exit -- No Change Dequeue Requestor If Queue Empty ThenDeallocate Header Exit For Each Waiting Conversion If Compatible With Granted Group Then Mark Granted & Wakeup If No More Conversions Waiting Then For Each Waiter (in FIFO order) If Compatible With Granted Then Mark Granted & Wakeup Else Exit Exit

49. Gray& Reuter: Locking 6-7: 49 Concurrency Control: Outline Why lock (isolation)Why lock (isolation) When to lock (2ø locking, 1°, 2°,3° isolationWhen to lock (2ø locking, 1°, 2°,3° isolation What to lock (granularity)What to lock (granularity) How to lockHow to lock Exotics (optimistic, field calls, escrow) DeadlockDeadlock PerformancePerformance

50. Gray& Reuter: Locking 6-7: 50 Optimistic Locking Idea: hope no one else changes data Timestamp: keep timestamp with each object defer all updates at ø1 commit:lock objects if timestamps original then abort else release read locks at ø2 commit: apply deferred updates Value:keep original value of each object at ø1 commit:lock objects if values original then abort else release read locks at ø2 commit: apply deferred updates

51. Gray& Reuter: Locking 6-7: 51 Optimistic Locking: Field Calls Field call: if predicate then update predicates:single variable query updates:SQL update clause (single variable update) Assumption: if predicate true, transform applies good for hot spots original idea in IMS/FP MSDBS. at call: test predicate if false give diagnostic else record call in log at commit ø1:lock test predicate if false then abort (restart) at commit ø2:apply update unlock Examples: no predicate, just an update, no contention: updatetransactions set count := count + 1 where branch = :home_branch; typical predicate: update inventory set quantity = quantity - :delta where part_no = :part and quantity > :delta;

52. Gray& Reuter: Locking 6-7: 52 Escrow Locking Schemes that hold locks at end of Ø1 fail to solve hot spot problem for distributed system since ø1 to Ø2 transition may take time Idea: leave value fuzzy, store minimum and maximum values Poly-values of (Warren Montgomery's MIT thesis) Escrow locking of (Pat ONeil) Example: I order 100 widgets: quantity was:[1000,1000] now is:[900, 1000] I commitnow is:[900, 900] No locks are held Works only for commutative ops

53. Gray& Reuter: Locking 6-7: 53 Versioning vs Locking (Oracles Approach) Records & Objects have values over time: R: [T0..T1): V0, [T1…T2) V1, [T2….now) V3. Value time Ti is transaction commit timestamp. Writes set conventional locks and transaction sees its own updates Transaction makes read request at time Tx, return Value as of that time Value as of time statement began (Oracle Consistent Read) Value as of time transaction began (Snapshot Isolation) (= Oracle Serializable) There is a whole neat theory here waiting to be worked out: see A Critique of SQL Isolation Levels, SIGMOD 95.

54. Gray& Reuter: Locking 6-7: 54 Tradeoffs Deferred updates are confusing for user (application).Deferred updates are confusing for user (application). Value is good if object small (record)Value is good if object small (record) Timestamp is good if object large (file)Timestamp is good if object large (file) Field calls more general than eitherField calls more general than either Field calls great for SQL systems (automatic)Field calls great for SQL systems (automatic) Field call commutativity assumption is prone to errorField call commutativity assumption is prone to error

55. Gray& Reuter: Locking 6-7: 55 Weird Lock Options Available To Application Read past locks: caller wants to skip locked items e.g.,: get next free item in a queue, Notify locks: caller waits for object to change e.g.: wait for nonempty queue or trigger on data change Bounce locks: caller never waits (timeout = 0) example: want any free record. Read thru locks: wants to see all data, even uncommitted updates example: ad hoc query Adaptive locking: holder of lock notified that others wait holder releases/changes lock access

56. Gray& Reuter: Locking 6-7: 56 Concurrency Control: Outline Why lock (isolation)Why lock (isolation) When to lock (2ø locking, 1°, 2°,3° isolationWhen to lock (2ø locking, 1°, 2°,3° isolation What to lock (granularity)What to lock (granularity) How to lockHow to lock Exotics (optimistic, field calls, escrow)Exotics (optimistic, field calls, escrow) Deadlock PerformancePerformance

57. Gray& Reuter: Locking 6-7: 57 Deadlock Detection Deadlock: a cycle in the wait-for graph Kinds of waits: database locks terminal/device storage session server Correct detection must get complete graph Not likely, so always fall back on timeout Model of deadlock shows: waits are rare deadlocks are rare 2 (very very rare) virtually all cycles are length 2 so do depth-first search eitheras soon as you wait ORafter a timeout

58. Gray& Reuter: Locking 6-7: 58 How To Find a Deadlock Call each object manager getting his part of wait-for graph Do depth-first search (transitive closure) of graph If find cycle: pick minimum cost victim Reference: Obermarck and Beri

59. Gray& Reuter: Locking 6-7: 59 Model of Deadlock (1) R:number of objects (locks) r:objects locked per transaction N+1: Concurrent Transactions ASSUME Transaction is: LOCK r lock steps, then commit Uniform distribution exclusive locks only Nr << R r R Nxr/2 Probability a request waits: Prob a transaction waits:

60. Gray& Reuter: Locking 6-7: 60 Model of Deadlock (2) R:number of objects (locks) r:objects locked per transaction N+1: Concurrent Transactions Probability of cycle length 2 + length 3 +... Prob transaction deadlocks PD: assumes all cycles of length 2 System deadlock rate, N+1 times higher Conclusions: control transaction size and duration limit multiprogramming

61. Gray& Reuter: Locking 6-7: 61 Common Performance Bugs Convoys on semaphores or high-traffic locks Log semaphore is hotspot Sequential insert is hotspot Lock manager costs too much A good number: 300 instructions for lock+unlock (no wait case)) = 1500 clocks on a P6/200 file or page granularity locking causes hotspot for small files

62. Gray& Reuter: Locking 6-7: 62 Concurrency Control: Outline Why lock (isolation) When to lock (2ø locking, 1°, 2°,3° isolation What to lock (granularity) How to lock Exotics (optimistic, field calls, escrow) Deadlock Performance