CHUBBY and PAXOS Sergio Bernales 1Dennis Kafura – CS5204 – Operating Systems

chubby and paxos Overview Chubby  Why?  How?  More precise how Paxos  Why?  How?  More precise why Dennis Kafura – CS5204 – Operating Systems2

chubby and paxos 10 – 30,000 Clients Different client services Who is in charge? Guarantee it? What if you go down? i.e. DNS Server Dennis Kafura – CS5204 – Operating Systems3

chubby and paxos Service Requirements Easy for client developers (of the variety kind) Reliable and Available Master is clear –hint, hint Paxos Thousands of clients, tiny things Growing clients = growing transactions Dennis Kafura – CS5204 – Operating Systems4

chubby and paxos Chubby Coarse-grained lock service Caching Event Notification Access Control Lists /ls/foo/wombat/pouch Dennis Kafura – CS5204 – Operating Systems5

chubby and paxos System Structure Dennis Kafura – CS5204 – Operating Systems6 Cache: data, metadata, absent file, handles (invalidated when data is changed)

chubby and paxos Files, Directories, Handles /ls/foo/wombat/pouch ACL  Controls: reading, writing, ACL changes  ACL authentication in RPC Meta-data of node (64 bits entries) Dennis Kafura – CS5204 – Operating Systems7 Instance Number Content generation number Lock generation ACL generation

chubby and paxos Sequencer Sequence numbers provided only for interactionc that use locks Request sequencer  Name of lock  Acquisition mode (exclusive, shared)  Lock generation number Lock delay Dennis Kafura – CS5204 – Operating Systems8

chubby and paxos API (1 of 2) Open(/ls/foo/wombat/pouch )* Close()  Poison() GetContentAndStats()  getStat()  ReadDir() SetContents()  Compare and swap Delete() Acquire(), TryAcquire(), Release() Dennis Kafura – CS5204 – Operating Systems9

chubby and paxos API (2 of 2) GetSequencer() SetSequencer() CheckSequencer() Dennis Kafura – CS5204 – Operating Systems10

chubby and paxos Sessions Keeps locks active Data consistent Survives failures KeepAlives  Most calls are KeepAlives Dennis Kafura – CS5204 – Operating Systems11

chubby and paxos Events Subcribe to  Contents modified  Child node added  Chubby master fail  Handle has become invalid  Lock acquired  Conflicting lock request Dennis Kafura – CS5204 – Operating Systems12

chubby and paxos Efficiency Atomic operations No current directories No last access times Proxies KeepAlive replies used to transmit events and cache invalidations Dennis Kafura – CS5204 – Operating Systems13

chubby and paxos Back to the DNS Server Chubby caches Chubby batches Don’t use as fileserver Do use for config files, elect masters Dennis Kafura – CS5204 – Operating Systems14

chubby and paxos Master Fail - Over Dennis Kafura – CS5204 – Operating Systems15

chubby and paxos Recover 1. Pick a new client epoch number(sent by clients) 2. Master responds to master location requests 3. Rebuilds session memory strucuture 4. KeepAlives allowed 5. Notifies clients of failure 6. Master waits sessions syncs 7. Fully operational 8. While operating, handles are verified 9. Ephemeral files removed Dennis Kafura – CS5204 – Operating Systems16

chubby and paxos Intro Paxos A consensus algorithm 1. Only a value that has been proposed may be chosen 2. Only a single value is chosen 3. Unless value actually chosen, process don’t learn about it If someone disagrees, no one has agreed Dennis Kafura – CS5204 – Operating Systems17

chubby and paxos Paxos Proposers, Acceptors, Learners Multiples of each System assumes failure Data never corrupted Dennis Kafura – CS5204 – Operating Systems18

chubby and paxos Choosing a Value P1- An acceptor must accept the first proposal it receives Problem  Several values proposed, none chosen  Possible to accept multiple A solution: number proposals Dennis Kafura – CS5204 – Operating Systems19

chubby and paxos Choosing a Value P2- If a proposal with value v is chosen, then every higher numbered proposal that is chosen has value v P2a- If a proposal with value v is chosen, then every higher accepted by any proposer has value v P2b- If a proposal with value v is chosen, then every higher accepted by any proposer has value v Dennis Kafura – CS5204 – Operating Systems20

chubby and paxos Choosing a Value P2c- For any v and n, if a proposal with value v and number n is issued, then there is a set S consisting of a majority of acceptors such that either (a) no acceptor in S has accepted any proposal numbered less than n, or (b) v is the value of the highest-numbered proposal among all proposals numbered less than n accepted by the acceptors in S P1a- An acceptor can accept a proposal, iff if has not reponded to a prepare request having a number greater than n Dennis Kafura – CS5204 – Operating Systems21

chubby and paxos Algorithm Phase 1 a) Proposer selects proposal number n and sends prepare request to majority of acceptors b) If acceptor receives request number n greater than that of any other prepare request it has accepted, it sends back promise not to accept any more and also it’s last n Phase 2 a) Proposer receives prepare request form majority of acceptors, then sends the proposal number n plus the actual value v b) If acceptor receives accept request for proposal n, accepts it only if it has not responded to another prepare request. Dennis Kafura – CS5204 – Operating Systems22

chubby and paxos Learn a Chosen Value Chosen value propagated  All acceptors send messages to learners  Distinguished learner gets the message, passes it on  Distinguished learnerS  Learner ask acceptors Distinguished Acceptors Dennis Kafura – CS5204 – Operating Systems23