1. Quiz 2 Review Slides

2. Quiz Topics (Lectures 12-21)
Transactions
Serializability (Conflict vs View)
2PL (Strict / Rigorous, Deadlocks)
OCC
Granularity of locking (IX/IS locks)
Snapshot isolation
Distributed Transactions / 2PC
Durability / Recovery (ARIES)
Basic protocol
CLRs
Physiological logging
Distributed Query Processing
Partitioning
Distributed Join Algorithms
NoSQL/Non-Database Data Processing Systems
Dynamo
Spark
Big Table
Kafka/Kafka Streaming

3. Conflict Serializability
A schedule is conflict serializable if it is possible to swap non-conflicting operations to derive a serial schedule.
For all pairs of conflicting operations {O1 in T1, O2 in T2} either
O1 always precedes O2, or
O2 always precedes O1.

4. Precedence Graph Given transactions Ti and Tj,
Create an edge from TiTj if:
Ti reads/writes some A before Tj writes A, or
Ti writes some A before Tj reads A
If there are cycles in this graph, schedule is not conflict serializable

5. Example Is this schedule conflict serializable? T3 T2 T1 T1 T2 T3 RARB WA WB
COMMIT

6. Two Phase Locking Protocol
Before every read, acquire a shared lock
Before every write, acquire an exclusive lock (or "upgrade") a shared to an exclusive lock
Release locks:
After all locks are acquired (standard protocol)
Write locks after transaction commits (strict 2PL)
Only after the transaction commits (rigorous 2PL)
Strict & Rigorous ensures cascadeless-ness
Rigorous ensures that commit order = serialization order

7. OCC
Transactions execute, with updates affecting local copies of the data
Build a list of data items that were read or written
At end of transaction, validate; guarantee for a committing transaction Tj that:
Tj read all of the (conflicting) writes of every earlier transaction, and
Tj completed all of it writes after the writes of every earlier transaction.
Validation rules ensure these conditions hold; vary depending on whether doing serial or parallel validation

8. Granularity of Locking
Idea: allow locks at different granularites:
I.e., Table
Pages
Tuples
Challenge: how to allow a transaction T1 to hold e.g., a X table lock that prevents other transactions from reading e.g., tuples without T2 locking all tuples
Soln: Intention locks: T2 locks tables and pages with intention locks before reading / writing tuples
E.g., to lock a tuple Y in page P in Table Z in X mode, T2 first needs to hold Intention X (IX) locks on P and Z.
IX(Z) means ”T2 intends to update something contained in Z", while X(Y) means "I update/read ALL OF Y"

10. Log Based Recovery Recall: database state is memory + disk pages
Add a log managed via write ahead logging
Write what we plan to do, before we do it, so that:
We can UNDO dirty data written by uncommitted/aborting transactions
We can REDO committed actions not flushed to disk at commit time
Q: do you always need to write UNDO/REDO state? Q:

11. ARIES/Recovery Review example in lec 15 PPT
Make sure you understand details of CLRs
What does “repeating history” mean? Why is it important?
Why does UNDO have to be logical?

12. Distributed QP Look at lecture 17 slides Partitioning Strategies:
Hash/Range/Round Robin
What are tradeoffs?
Join strategies:
If tables are partitioned properly, just run local joins
Also, if one table is replicated, no need to join
Otherwise, several options:
Collect all tables at one node
Inferior except in extreme cases, i.e., very small tables
Re-partition one or both tables
Depending on initial partitioning
Replicate (smaller) table on all nodes
What are tradeoffs? When is one preferred over the other?

13. Two-phase commit Coordinator Suboordinate PREPARE(T)
FW(PREPARE, T,locks,…)
VOTE(T,YES/NO)
Prepared suboorindate’s wait to hear outcome from C
Recover into prepared state and ask C for outcome
If all yes
FW(COMMIT)
Else
FW(ABORT)
COMMIT/ABORT(T)
S can forget about transaction at this point --
C must remember outcome
until all S’s ack
FW(COMMIT/ABORT)
ACK
W(DONE), once all S’s ACK
C can forget about transaction at this point

14. Two-phase commit Understand why different log records are necessary
What happens if failures occur at different points
Presumed abort/presumed commit variants

15. “NoSQL”
Relax consistency guarantees, provide less than full SQL for various reasons:
High availability (Dynamo)
Simplicity
Scalability

16. Dynamo Key ideas: Replicated, highly available key-value store
Consistent hashing (ring-based) replication
No central coordinator
Quorum writes ensure atomicity of single key-writes
Sloppy quorum favors availability over consistency (atomicity) in the presence of failures
Vector clocks allow nodes to detect conflicting concurrent writes to versions

17. Dynamo Recap (Lec 19 Slides)
Key ideas:
Get/put KV-store w/ single key atomicity
All data replicated on N nodes
Data stored in “ring” representing space of hash values from say, 02128
“Overlay network”: Nodes are not actually in a physical ring, but are just machines on the Internet
Each node occupies one (or multiple) random locations on ring
N=3 k
A key hashed to location k is stored on N successors in ring

18. Spark Distributed “dataflow” language
Programs operate on partitions of data in parallel
Lineage graph tracks dependencies, enables recovery via replay (instead of writing all intermediate data to disk)
Scheduling done in stages, with stage boundaries between wide dependencies
Lineage Graph Example

19. Big Table 4D table Row, column family, column name, time -> value
Table range partitioned by key into a number of “tablets” (horizontal partitions)
Each node responsible for a set of tablets
Updates logged to GFS, written into memory memtables
memtables periodically flushed to GFS as SSTables
(Fig 1)

20. Metadata tablets used used to find tablets of interest

21. Kafka/Kafka Streams
Kafka: messaging service the delivers messages to subscribers in a scalable & efficient way
Kafka Streams: stream processing API with high- level stream DSL on top of Kafka