TRUST Spring Conference, April 2-3, 2008 Write Markers for Probabilistic Quorum Systems Michael Merideth, Carnegie Mellon University Michael Reiter, University of North Carolina

4/3/08 Michael Merideth2 Replication via Quorum Systems Replicated data – Server becomes n replicas Server Clients

4/3/08 Michael Merideth3 Replicas Replicated data – Server becomes n replicas Clients issue read and write operations – Involve quorums (subsets) of replicas High availability – Yet, no writes lost, forged, or corrupted Clients Replication via Quorum Systems

4/3/08 Michael Merideth4 Types of Servers (in Examples) bowling ball ice creamfishany value non-faultyfaulty

4/3/08 Michael Merideth5 Types of Clients (in Examples) non-faultyfaulty

4/3/08 Michael Merideth6 Write Operation Client wants to write “ice cream” to system

4/3/08 Michael Merideth7 Write Operation Client submits write to write quorum

4/3/08 Michael Merideth8 Write Operation Complete Positive responses from quorum means write complete

4/3/08 Michael Merideth9 Write Operation Complete

4/3/08 Michael Merideth10 Read Operation Client queries read quorum for values

4/3/08 Michael Merideth11 Read Operation Determines read value based on votes (responses) from entire quorum (Chooses “ice cream”)

4/3/08 Michael Merideth12 Write Markers Concept Write marker: additional data (written with value) that identifies write quorum – Verified by clients during read Improves properties of probabilistic quorum systems – Tolerate more faults and use smaller quorums

4/3/08 Michael Merideth13 Outline Strict, Byzantine quorum systems Probabilistic, Byzantine quorum systems Benefits of write markers Idea for implementation

4/3/08 Michael Merideth14 Byzantine Quorum System [malkhi & reiter 98] Byzantine (arbitrary) faults – Faulty nodes may lie – Faulty clients and servers may collude b faulty servers – Identity of faulty nodes unknown by non-faulty nodes

4/3/08 Michael Merideth15 Write Operation Write quorum may contain faulty servers

4/3/08 Michael Merideth16 Write Operation Complete

4/3/08 Michael Merideth17 Read Operation Faulty servers may fabricate value

4/3/08 Michael Merideth18 Stale Values Stale (logically older) values are detectible

4/3/08 Michael Merideth19 Conflicting Values Faulty servers may also fabricate conflicting (logically concurrent) values – E.g., same timestamp Here “fish” conflicts with “ice cream” – But ice cream has more votes

4/3/08 Michael Merideth20 More Conflicting Values Non-faulty servers may also return conflicting values For example, in single-round write protocols – Such protocols are desirable for efficiency – Client may (perhaps unknowingly) submit a write that is conflicting

4/3/08 Michael Merideth21 Conflicting Write Same as normal write

4/3/08 Michael Merideth22 Conflicting Write Incomplete Accepted by non-faulty servers that have not accepted (conflicting) value Write does not complete

4/3/08 Michael Merideth23 Which Value is Correct? “Ice cream” was complete – … therefore is correct “Fish” was incomplete – … therefore should be ignored But ice cream and fish get equal votes Client uncertain ?

4/3/08 Michael Merideth24 Conflicting Values: Problematic Must outvote conflicting replicas Thus, many potentially conflicting replicas implies ability to tolerate (relatively) few faults ?

4/3/08 Michael Merideth25 Impact of Conflicting Replicas QuorumConflictFaultsProtocols Opaque < n/5 (least) e.g., Q/U Masking< n/4 e.g., Fleet, PASIS Dissemin- ation < n/3 (most) e.g., BFT, HQ ?

4/3/08 Michael Merideth26 Choice of Quorums Important Choices of read quorum and both write quorums led to problem – Other choices lead to correct answer ?

4/3/08 Michael Merideth27 Choice of Quorums Important Choices of read quorum and both write quorums led to problem – Other choices lead to correct answer

4/3/08 Michael Merideth28 Idea: Select Quorums at Random In fact, correct answer in expectation (in this example) – If quorums chosen uniformly at random (an access strategy)

4/3/08 Michael Merideth29 Probabilistic Quorum Systems [malkhi, reiter, wool, wright 01] Weakening intersection property to hold only with high probability – Provides better availability – Tolerates more faults Bounds error probability – Probability that quorums chosen according to access strategy yield incorrect (or uncertain) result

4/3/08 Michael Merideth30 Probabilistic Opaque Quorum Systems [merideth & reiter 07] Generalize access strategy – Quorums chosen from access sets – Access sets are chosen according to access strategy Tolerate Byzantine clients for all probabilistic quorum systems – Enforce access strategy

4/3/08 Michael Merideth31 Probabilistic Quorum Systems Reduce number of conflicting values in expectation – Therefore, tolerate more faults (with some bounded probability of error) Conflicting Faults StrictProb. Opaque < n/5 (fewest) Masking< n/4 Dissemination< n/3 < n/3.15 < n/2.62 < n (most)

4/3/08 Michael Merideth32 Reduce conflicting replicas further? Yes (for probabilistic masking and opaque quorum systems) – Write markers

4/3/08 Michael Merideth33 Write Markers Recall, – Write operations write values – Read operations poll replicas for values Write marker – Additional data (written with value) that identifies the write quorum (or access set) that was used – Client accepts vote (during read) only if replica was part of write quorum (or access set)

4/3/08 Michael Merideth34 Write Operations with Write Markers Create write marker for quorum

4/3/08 Michael Merideth35 Write Operation Complete

4/3/08 Michael Merideth36 Conflicting Write with Write Markers Same as normal write

4/3/08 Michael Merideth37 Conflicting Write Incomplete Accepted by non-faulty servers that have not accepted (conflicting) value

4/3/08 Michael Merideth38 Which Value is Correct? “Ice cream” was complete – … therefore is correct “Fish” was incomplete – … therefore should be ignored

4/3/08 Michael Merideth39 Which Value is Correct? Faulty client can only vote for “triangle” Faulty client cannot vote for “star”

4/3/08 Michael Merideth40 Benefit of Write Markers Faulty servers cannot vote for conflicting value unless they are part of write Due to probabilistic access strategy, faulty server not always part of write Thus, fewer conflicting servers to outvote in expectation

4/3/08 Michael Merideth41 Benefits of Write Markers Conflicting Faults StrictProb. Write- markers Opaque < n/5 (fewest) Masking< n/4 Dissemination< n/3 < n/3.15 < n/2.62 < n (most) < n/2.62 < n/2 < n (most) Tolerate more faults

4/3/08 Michael Merideth42 Benefits of Write Markers Tolerate more faults Use smaller quorums – See paper

4/3/08 Michael Merideth43 Example with Benign Clients For writes: clients choose access sets uniformly at random – Then encode and, e.g., digitally sign their choices (i.e., create a write marker) For reads: clients verify write marker

4/3/08 Michael Merideth44 Write Markers with Byzantine Clients Faulty clients: – Cannot be trusted to follow access strategy – May intentionally choose quorums that maximize conflicting values Constrain clients [merideth&reiter 07] – Even faulty clients follow access strategy – Avoids additional communication on critical path – Choice is verified by servers as (pseudo) random Treat choice as write marker – Modify protocol so that clients also verify choice

4/3/08 Michael Merideth45 Protocol Intuition Servers provide pseudorandom sequence of access sets per client – Threshold signature from servers …

4/3/08 Michael Merideth46 Servers provide pseudorandom sequence of access sets per client – Threshold signature from servers For each operation, client locally chooses next access set in sequence; servers verify choice Protocol Intuition …

4/3/08 Michael Merideth47 Protocol Intuition … Servers provide pseudorandom sequence of access sets per client – Threshold signature from servers For each operation, client locally chooses next access set in sequence; servers verify choice

4/3/08 Michael Merideth48 Misuse by Faulty Client What if faulty client: – Skips ahead to “better” access set? – Waits to perform operation until advantageous? In either case, access set no longer random …

4/3/08 Michael Merideth49 Defending Against Misuse Exponential increase in cost to use later access sets – Client puzzle (requires solution) Correct value propagates in background [c.f. malkhi et al. 03] Sequence becomes invalid as system progresses – Must obtain new sequence …

4/3/08 Michael Merideth50 Write Markers Mechanism Use client puzzle – Servers already verify solution Have clients verify as well – Treat solution and access set as write marker – Return during read operations Provides mechanism for write markers …

4/3/08 Michael Merideth51 Conclusion Write markers provide benefits for probabilistic quorum systems – Reduce number of faulty servers that can vote for conflicting value in expectation – Increase number of faults that can be tolerated Opaque: up to n/2.62 (probabilistic: n/3.15; strict: n/5) Masking: up to n/2 (probabilistic: n/2.62; strict: n/4) – Allow for smaller quorums in some cases For more information: – Write Markers for Probabilistic Quorum Systems. Michael G. Merideth and Michael K. Reiter. CMU Technical Report: CMU-ISR

4/3/08 Michael Merideth52 Questions?

4/3/08 Michael Merideth53

4/3/08 Michael Merideth54

4/3/08 Michael Merideth55

4/3/08 Michael Merideth56

4/3/08 Michael Merideth57