1. Consistency Guarantees and Snapshot isolation Marcos Aguilera, Mahesh Balakrishnan, Rama Kotla, Vijayan Prabhakaran, Doug Terry MSR Silicon Valley

2. Goals Develop a cloud storage system featuring 1.multiple consistency levels – requires one API to learn, one system to administer – handles diversity of requirements within and across applications 2.read-write transactions – with snapshot isolation – on replicated and partitioned data 3.consistency-based SLAs

3. Geo-Replication primary secondaries datacenter remote secondaries remote datacenter Read Write

4. Client API – Get (key) – Put (key, object) – BeginTx (consistency) – EndTx () – BeginSession (consistency) – EndSession () Puts/ Gets Transaction Session

5. Transaction Properties Conventional transaction model – BeginTx … EndTx Atomic updates to multiple objects Multi-object reads from snapshots Even across partitions

6. Partitioned Data for Scalability Data partitioned by key range Each partition has its own primary and secondary servers Key-rangePrimarySecondaries A-F S1 S2, S4 G-P S2 S4, S5 Q-Z S3 S1, S4, S5

7. Write Operations Writes performed at primary server(s) – May have different primaries for different objects Propagate to secondary servers eventually – Any gossip or anti-entropy protocol will do Have a commit timestamp, i.e. global order – And deterministic outcomes No write conflicts => All replicas converge towards a mutually consistent state

8. Versioned Data Store Store version history for each object Can perform writes as soon as commit timestamp is known – need not perform writes in commit order Can eventually prune old versions time Object A Object B V1 V2 V3V4

9. Per-Replica State Datastore = set of High-time = timestamp of latest received write transaction – Assumes transactions are received in order – May receive periodic null transactions Low-time = timestamp of most recent discarded object version

10. Read Operations Single-key Gets go to one server Multi-partition transactions may read from multiple servers Server(s) selected based on desired consistency – E.g. read from nearby server when possible Alternative: Broadcast operation to all servers – Take first response that is consistent enough

11. Read-Only Transactions Transaction assigned a read timestamp Read from snapshot at that time – See all write transactions committed before this time, and only those writes Consistency guarantee places constraints on read timestamp

12. Reads on Versioned Data Store Allows reads at any timestamp – Without placing constraints on write propagation Assuming no future transaction could be assigned a commit timestamp before the read timestamp time Object A Object B V1 V2 V3V4 Read timestamp

13. Selecting Read Timestamp GuaranteeRead timestamp Strong Consistencynow (or time of last committed write) Eventual Consistencyany time Consistent Prefixany time Bounded Stalenessany time within bound Monotonic Readsany time later or equal to that of previous read transaction in this session Read My Writesany time later or equal to that of previous write transaction in this session assuming in-order delivery of writes

14. Acceptable Read Timestamps time 0 BeginTx strong read-my-writes monotonic bounded causal eventual read timestamp

15. Selecting Read Timestamp time node A low high node B low high node C low high Read timestamp

16. Read-Write Transactions Transaction assigned a read timestamp and a commit timestamp Use optimistic concurrency control – Old read timestamps increase the chance of abort Read from snapshot at read timestamp – With selected consistency guarantee Batch writes until commit – No undo needed Validate transaction at commit timestamp

17. Transaction Lifetime Get(x) … Put(x, value) Session Transaction Select read timestamp and perform Get Buffer Put Get commit timestamp, validate, and perform Puts time

18. Committing Write Transactions Snapshot isolation => Check that no object being written has a version between the transaction’s read timestamp and commit timestamp Serializability => Check that no object being read or written has a version between the transaction’s read timestamp and commit timestamp