1. cs4432recovery1 CS4432: Database Systems II Database Consistency and Violations?

2. cs4432recovery2 Transactions, etc. Crash recovery Chapter 17 Concurrency control Chapter 18 Transaction processing Chapter 19

3. cs4432recovery3 Integrity or correctness of data ? We would like data in our database to be “accurate” ( “correct” ) at all times. EMP How DBMS decides if data is consistent? Name White Green Gray Age 52 3421 1

4. cs4432recovery4 Integrity or consistency constraints Utilize predicates data must satisfy Examples: - x is key of relation R - x  y holds in R - Domain(x) = {Red, Blue, Green} - no employee should make more than twice the average salary

5. cs4432recovery5 Definitions: Consistent state: satisfies all constraints Consistent DB: DB in consistent state

6. cs4432recovery6 Such constraints may not capture “full correctness” Example 1 : Transaction constraints When salary is updated, new salary > old salary When account record is deleted, balance = 0

7. cs4432recovery7 Example 2 Database should reflect real world DB Reality Constraints ( as we use here) may not capture “full correctness”

8. cs4432recovery8  in any case, continue with constraints... Observation: DB cannot be consistent always Example: Constraint : a 1 + a 2 +…. a n = TOT Action: Deposit $100 in a 2 : a 2  a 2 + 100 TOT  TOT + 100

9. cs4432recovery9 a 2 TOT.... 50.... 1000.... 150.... 1000.... 150.... 1100 Example: a 1 + a 2 +…. a n = TOT ( constraint ) Deposit $100 in a 2 : a 2  a 2 + 100 TOT  TOT + 100

10. cs4432recovery10 Transaction: a collection of actions that preserve consistency Consistent DBConsistent DB’ T

11. cs4432recovery11 Big assumption: If T starts with consistent state AND T executes in isolation  T leaves consistent state

12. cs4432recovery12 Correctness (informally) If we stop running transaction(s), DB left consistent Each transaction sees a consistent DB

13. cs4432recovery13 How can constraints be violated? Transaction bug DBMS bug Hardware failure e.g., disk crash alters balance of account Data sharing e.g.: T1: give 10% raise to programmers T2: change programmers  systems analysts

14. cs4432recovery14 Will not consider: How to write correct transactions How to write correct DBMS system Constraint checking & repair

15. cs4432recovery15 How can we prevent/fix violations? Chapter 17: due to failures only Chapter 18: due to data sharing only Chapter 19: due to failures and sharing

16. cs4432recovery16 Chapter 17: Recovery First : Failure Model

17. cs4432recovery17 Events Desired Undesired Expected Unexpected

18. cs4432recovery18 Our failure model processor memory disk CPU M D

19. cs4432recovery19 Desired events: see product manuals…. Undesired expected events: System crash - memory lost - cpu halts, resets Undesired Unexpected: Everything else! that’s it!!

20. cs4432recovery20 Examples: Disk data is lost Memory lost without CPU halt CPU implodes wiping out universe…. You name it … Undesired Unexpected: Everything else!

21. cs4432recovery21 Is this model reasonable? Approach: Add low level checks + redundancy to increase probability that model holds E.g., Replicate disk storage (stable store) Memory parity CPU checks

22. cs4432recovery22 Operations re Storage Hierarchy: Input (x): block containing x  memory Output (x): block containing x  disk Read (x,t): do input(x) if necessary t  value of x in block Write (x,t): do input(x) if necessary value of x in block  t Memory Disk x x

23. cs4432recovery23 Key problem : Unfinished transaction Example:Constraint: A=B T 1 : A  A  2 B  B  2

24. cs4432recovery24 T 1 :Read (A,t); t  t  2 Write (A,t); Read (B,t); t  t  2 Write (B,t); Output-to-Disk (A); Output-to-Disk (B); A: 8 B: 8 A: 8 B: 8 memory disk 16 failure!

25. cs4432recovery25 Need atomicity: – execute all actions of a transaction or none at all

26. cs4432recovery26 One solution: undo logging (immediate modification) A la Hansel and Gretel recording their navigation through forest via bread crumbs … Must have durable undo logging !!!

27. cs4432recovery27 T 1 :Read (A,t); t  t  2 A=B Write (A,t); Read (B,t); t  t  2 Write (B,t); Output-to-disk (A); Output-to-disk (B); A:8 B:8 A:8 B:8 memory disk log Undo logging (Immediate modification) 16 16 16

28. cs4432recovery28 One “complication” : First disk, then log Log is first written in memory Not written to disk on every action memory DB Log A: 8 16 B: 8 16 Log: A: 8 B: 8 16 BAD STATE # 1

29. cs4432recovery29 One “complication” : first log, then disk. Log is first written in memory Not written to disk on every action memory DB Log A: 8 16 B: 8 16 Log: A: 8 B: 8 16 BAD STATE # 2...

30. cs4432recovery30 Undo logging rules (1)For every action generate undo log record (containing old value) (2) Before x is modified on disk, log records pertaining to x must be on disk (write ahead logging) (3) Before commit is written to log on disk, all writes of transaction must be reflected on disk

31. cs4432recovery31 Recovery rules: Undo logging (1) Let S = set of transactions with in log, but no (or ) record in log (2) For each in log, in reverse order (latest  earliest) do: - write old value from log back to disk: - if Ti  S then - write (X, v) - output (X) (3) For each Ti  S do - write to log

32. cs4432recovery32 What if failure during recovery? No problem!  Undo idempotent

33. cs4432recovery33 To discuss next : Redo logging Undo/redo logging, why both? Checkpoints

34. cs4432recovery34 Redo logging (deferred modification) T 1: Read(A,t); t t  2; write (A,t); Read(B,t); t t  2; write (B,t); Output(A); Output(B) A: 8 B: 8 A: 8 B: 8 memory DB LOG 16 output 16

35. cs4432recovery35 Redo logging rules (1) For every action, generate redo log record (containing new value) (2) Before X is modified on disk (DB), all log records for transaction that modified X (including commit) must be on disk (3) Flush log at commit

36. cs4432recovery36 (1) Let S = set of transactions with in log (2) For each in log, in forward order (earliest  latest) do: - if Ti  S then Write(X, v) Output(X) optional Recovery rules: Redo logging

37. cs4432recovery37 Recovery is very, very SLOW ! Redo log: FirstT1 wrote A,BLast RecordCommitted a year agoRecord (1 year ago)--> STILL, Need to redo after crash!!... Crash

38. cs4432recovery38 Solution: Checkpoint (simple version) Periodically: (1) Do not accept new transactions (2) Wait until all transactions finish (3) Flush all log records to disk (log) (4) Flush all buffers to disk (DB) (do not discard buffers) (5) Write “checkpoint” record on disk (log) (6) Resume transaction processing

39. cs4432recovery39 Example: what to do at recovery? Redo log (disk): Checkpoint Crash...

40. cs4432recovery40 Key drawbacks: Undo logging: Too many disk IOs Redo logging: need to keep all modified blocks inmemory until commit

41. cs4432recovery41 Solution: undo/redo logging! Combine Undo/Redo Logging, namely: Update  page X

42. cs4432recovery42 Rules Page X can be flushed before or after Ti commit Log record flushed before corresponding updated page (WAL) Flush at commit (log only)

43. cs4432recovery43 Non-quiescent checkpoint L O G for undodirty buffer pool pages flushed Start-ckpt active TR: Ti,T2,... end ckpt...

44. cs4432recovery44 Examples what to do at recovery time? no T1 commit L O G T 1,- a... Ckpt T 1... Ckpt end... T1-bT1-b  Undo T 1 (undo a,b)

45. cs4432recovery45 Example LOGLOG... T1aT1a T1bT1b T1cT1c T 1 cmt... ckpt- end ckpt-s T 1  Redo T1: (redo b,c)

46. cs4432recovery46 Recovery process: Backwards pass (end of log  latest checkpoint start) –construct set S of committed transactions –undo actions of transactions not in S Undo pending transactions –follow undo chains for transactions in (checkpoint active list) - S Forward pass (latest checkpoint start  end of log) –redo actions of S transactions backward pass forward pass start check- point

47. cs4432recovery47 Summary Consistency of data One source of problems: failures - Logging - Redundancy Another source of problems: Data Sharing..... next