1. Advanced Database Systems: DBS CB, 2nd Edition
System Failure & Recovery
Ch. 17

2. Outline Background Failure Modes and Failure Model Storage Hierarchy
Undo Logging
Redo Logging
Undo/Redo Logging
Protecting Against Media Failures

3. Background 3 3 3

4. Background Integrity or correctness of data:
Would like data to be “accurate” or “correct” at all times
Integrity or consistency constraints:
Predicates data must satisfy any constraints
x is key of relation R (x is unique)
x  y holds in R (equal x implies equal y)
Domain(x) = {Red, Blue, Green}
a is valid index for attribute x of R
no employee should make more than twice the average salary
Consistent state: satisfies all constraints
Consistent DB: DB in consistent state

5. Background
Constraints (as we use here) may not capture “full correctness”
Example 1: Transaction constraints
When salary is updated,
new salary > old salary
When account record is deleted,
balance = 0
Note: could be “emulated” by simple constraints, e.g.,
Example 2: a1 + a2 +…. an = TOT (constraint)
Deposit $100 in a2: a2  a
TOT  TOT + 100
Acct # ….
balance
deleted?
Account

6. Background
Transaction: collection of actions that preserve consistency . . . a2 50
150
150 . . .
TOT
1000
1000
1100
Constraint violation
Consistent DB
Consistent DB’ T

7. Background
Assumption: If T starts with consistent state + T executes in isolation  T leaves consistent state
Correctness (informally)
If we stop running transactions, DB left consistent
Each transaction sees a consistent DB
How can constraints be violated?
Transaction bug
DBMS bug
Hardware failure, e.g., disk crash alters balance of account
Data sharing, e.g.: T1: give 10% raise to programmers and T2: change programmers  systems analysts

8. Failure Modes and Failure Model8 8 8

9. Failure Modes and Failure Model
Events for DB needs to recover:
Wrong data entry
Disk failure
Fire / earthquake / ….
Systems crashes
Software errors
Power failures
Failure Model:
Events Desired
Undesired Expected
Unexpected

10. Failure Modes and Failure Model
Our failure Model:
Desired Events: see product manual
Undesired expected events: system crash, memory lost, CPU halt, resets, etc.
Undesired unexpected events: everything else! such as disk data is lost, etc.
CPU M D
processor
memory
disk

11. Failure Modes and Failure Model
Is this model reasonable?
Approach: Add low level checks + redundancy to increase the probability that the model holds
E.g., Replicate disk storage (stable store)
Memory parity
CPU checks

12. Storage Hierarchy 12 12 12

13. Storage Hierarchy Operations:
Input (x): block containing x on disk  memory
Output (x): block containing x in memory  disk
Read (x, t): do input(x) if necessary (input block from disk into memory then extract x from memory into variable t in memory) t  value of x in block
Write (x, t): do input(t) if necessary (write t from memory into x in memory) value of x in block  t
Memory Disk x

14. Storage Hierarchy Constraint with unfinished transaction
Example: Constraint: A=B
T1: A  A  2
B  B  2
T1: Read (A,t); t  t2
Write (A,t);
Read (B,t); t  t2
Write (B,t);
Output (A);
Output (B);
failure!
Constraint violation as A ≠ B)
A: 8
B: 8
memory
disk 16 16

15. Undo Logging 15 15 15

16. Undo Logging
Needs atomicity: execute all actions of a transaction or none at all
One solution: Undo logging (immediate modification)
T1: Read (A,t); t  t A=B
Write (A,t);
Read (B,t); t  t2
Write (B,t);
Output (A);
Output (B);
A:8
B:8 16
<T1, start>
<T1, A, 8> 16
<T1, B, 8>
<T1, commit> 16
memory
disk
log

17. Undo Logging Undo Logging – One “complication”
Log is first written in memory
Not written to disk on every action
A: 8 16
B: 8 16
Log:
<T1,start>
<T1, A, 8>
<T1, B, 8>
A: 8
B: 8 16
BAD STATE
# 1
(if memory is lost)
memory DB
Log

18. Undo Logging
Undo Logging – Write the commit record on the Log before commit
A: 8 16
B: 8 16
Log:
<T1,start>
<T1, A, 8>
<T1, B, 8>
<T1, commit>
A: 8
B: 8 16
BAD STATE
# 2
(tx committed but
DB is not accurate)
... DB
memory
Log

19. Undo Logging Undo Logging rules
For every action generate undo log record (containing old value)
Before x is modified on disk, log records pertaining to x must be on disk (Write Ahead Logging: WAL)
Before commit record is flushed to log, all writes of transaction must be reflected on disk

20. Undo Logging Undo Logging – Recovery rules
For every Ti with <Ti, start> in log:
If <Ti, commit> or <Ti, abort> in log, do nothing
Else For all <Ti, X, v> in log: // v is an old value for x
write (X, v) // write v from memory to X in memory
output (X ) // write block containing x from memory to disk
Write <Ti, abort> to log
IS THIS CORRECT??

21. Undo Logging Undo Logging – Recovery rules
Let S = set of transactions with <Ti, start> in log, but no <Ti, commit> (or <Ti, abort>) record in log
For each <Ti, X, v> in log, in reverse order (latest  earliest) do:
if Ti  S then - write (X, v) // write original value of x to disk
- output (X)
For each Ti  S do
- write <Ti, abort> to log

22. Undo Logging Undo Logging – What if failure happens during recovery?
No problem!  Undo is idempotent

23. Undo Logging Undo Logging – Checkpoint
In principal, once tx has its COMMIT log record written to disk, the undo log records of that tx are not needed during recovery
However, for other txs executing in parallel at that time; we can’t ignore log records prior to a COMMIT  solution is a checkpoint:
Stop accepting new transactions
Wait till all current active txs commit or abort and have written a COMMIT or ABORT record in the log
Flush the log to disk, and flush all DB buffers to disk as well
Write a log record <CKPT>, and flush the log again
Resume accepting new transactions

24. Undo Logging Undo Logging – Nonquiescent Checkpoint
A problem with regular checkpoint is it effectively halts the system from processing new transaction for a while!
A more advanced approach is Nonquiescent checkpoint:
Write a log record <START CKPT(T1,…,Tk)>, and flush to the log. The (T1, …,Tk) transactions are the current active transactions that did not commit/abort yet
Wait till (T1, …,Tk) commit or abort, but do not prohibit new transactions from starting
When all (T1, …,Tk) have completed, write a log record <END CKPT> and flush the log

25. Redo Logging 25 25 25

26. Redo Logging
Redo Logging (deferred modification)
T1: Read(A,t); t t2; write (A,t);
Read(B,t); t t2; write (B,t);
Output(A); Output(B) 16
<T1, start>
<T1, A, 16>
<T1, B, 16>
<T1, commit>
A: 8
B: 8
output 16
A: 8
B: 8 ……
<T1, end> DB
memory
LOG

27. Redo Logging Redo Logging Rules
For every action, generate redo log record (containing new value)
Before X is modified on disk (DB), all log records for transaction that modified X (including commit) must be on disk
Flush log at commit
Write END record after DB updates are flushed to disk

28. Redo Logging Redo Logging – Recovery Rules
For every Ti with <Ti, commit> in log:
For all <Ti, X, v> in log: // v is the new value for x
Write(X, v) // write v from memory into x in memory
Output(X) // write x from memory to disk
IS THIS CORRECT??

29. Redo Logging Redo Logging – Recovery Rules
Let S = set of transactions with <Ti, commit> (and no <Ti, end>) in log
For each <Ti, X, v> in log, in forward order (earliest  latest) do:
if Ti  S then Write(X, v) // write v from memory to x in memory
Output(X) // write x from memory to disk
For each Ti  S, write <Ti, end>

30. Redo Logging Redo Logging - Combining <Ti, end> Records
Want to delay DB flushes for hot objects
Actions:
write X
output X
Say X is branch balance:
T1: ... update X...
T2: ... update X...
T3: ... update X...
T4: ... update X...
combined <end> (checkpoint)

31. Redo Logging Redo Logging - Checkpoint
Unique problem is DB changes made to a committed tx can be copied to disk much later than the commit time
We can’t limit our concerns to active transactions when we decide to create a CKPT (regardless if it is quiescent or nonquiescent);
Between start and end CKPT we need to write to disk all DB elements (dirty buffers) that has been modified by committed transactions
On the other hand, we can complete the CKPT w/o waiting for the active txs to commit or abort

32. Redo Logging Redo Logging – Nonquiescent Checkpoint
Write a log record <START CKPT(T1,…,Tk)>, and flush to the log. The (T1, …,Tk) transactions are the current active transactions that did not commit/abort yet
Write to disk all DB elements that were written to buffers (dirty buffers) that are not written yet to disk by transactions that had already committed when the <START CKPT...> record was written to the log
Write an <END CKPT> record and flush to the log

33. Undo/Redo Logging 33 33 33

34. Undo/Redo Logging Undo/Redo Logging – Key drawbacks:
Undo logging: cannot bring backup DB copies up to date
Redo logging: need to keep all modified blocks in memory until commit
Solution: undo/redo logging!
Update  <Ti, Xid, New X val, Old X val> page X
Undo/Redo Rules:
Page X can be flushed before or after Ti commit
Log record flushed before corresponding updated page (WAL)
Flush at commit (log only)

35. Undo/Redo Logging Undo/Redo Logging – Nonquiescent checkpoint
Write a log record <START CKPT(T1,…,Tk)>, and flush to the log. The (T1, …,Tk) transactions are the current active transactions that did not commit/abort yet, and flush the log
Write to disk all the DB dirty buffers; unlike redo logging, we flush all dirty buffers and not only those written by committed transactions
Write an <END CKPT> record and flush to the log L O G
Start-ckpt
active TR:
T1,T2,...
end
ckpt
...
...
...
...
Dirty buffers
Pool pages
flushed
for
Undo

36. Undo/Redo Logging Undo/Redo Logging – Nonquiescent checkpointa
...
Ckpt T1
end
T1- b
No T1
commit
 Undo T1: (undo a,b) L O G
... T1 a b c
cmt
ckpt-
end
ckpt-s
T1 commit
 Redo T1: (redo b,c)

37. Undo/Redo Logging Undo/Redo Logging – Recovery Process:
Backwards pass (end of log  latest valid checkpoint start)
construct set S of committed transactions
undo actions of transactions not in S
Undo pending transactions
follow undo chains for transactions in (checkpoint active list) - S
Forward pass (latest checkpoint start  end of log)
redo actions of S transactions
backward pass
forward pass
start
check-
point

38. Protecting Against Media Failures38 38 38

39. Protecting Against Media failures
Media failure (loss of non-volatile storage)
Solution: Make copies of data!
A: 16

40. Protecting Against Media failures
Example 1: Triple modular redundancy
Keep 3 copies on separate disks
Output(X) --> three outputs
Input(X) --> three inputs + vote
Example 2: Redundant writes, Single reads
Keep N copies on separate disks
Output(X) --> N outputs
Input(X) --> Input one copy if ok, done
- else try another one
 Assumes bad data can be detected

41. Protecting Against Media failures
Example 3: DB Dump + Log
If active database is lost:
restore active database from backup
bring up-to-date using redo entries in log
backup
database
active
log

42. Protecting Against Media failures
When can Log be discarded
Active tx at Ckpt
check-
point db
dump
last
needed
undo
not needed for
media recovery
not needed for undo
after system failure
redo after system failure
log
time

43. END