Presentation is loading. Please wait.

Presentation is loading. Please wait.

GLOBAL STATES AND CHECKPOINTS CS 2711. 2 Distributed Checkpoints and Rollback Recovery Fault tolerance is achieved by periodically using stable storage.

Similar presentations


Presentation on theme: "GLOBAL STATES AND CHECKPOINTS CS 2711. 2 Distributed Checkpoints and Rollback Recovery Fault tolerance is achieved by periodically using stable storage."— Presentation transcript:

1 GLOBAL STATES AND CHECKPOINTS CS 2711

2 2 Distributed Checkpoints and Rollback Recovery Fault tolerance is achieved by periodically using stable storage to save the processes’ states during the failure-free execution. Upon a failure, a failed process rolls back from one of its saved states, thereby reducing the amount of lost computation. Each of the saved states is called a checkpoint

3 CS 2713 Checkpoint based Recovery Uncoordinated checkpointing: Each process takes its checkpoints independently Coordinated checkpointing: Process coordinate their checkpoints in order to save a system-wide consistent state. Communication-induced checkpointing: It forces each process to take checkpoints based on information piggybacked on the application messages it receives from other processes.

4 CS 2714 Domino effect: example P0P0 P1P1 P2P2 m0m0 m1m1 m2m2 m3m3 m4m4 m5m5 m7m7 m6m6 Recovery Line Domino Effect: Cascaded rollback which causes the system to roll back to too far in the computation (even to the beginning), in spite of all the checkpoints

5 Global State Chandy and Lamport—TOCS 1985 Global state of a distributed system – Local state of each process – Messages sent but not received Many applications need the state of the system – Failure recovery, distributed deadlock detection – Detect stable properties. Problem: how can you figure out the state of a distributed system? – Each process is independent – Network does not have any processing power. Distributed snapshot: a consistent global state CS 2715

6 Global State a)A consistent cut b)An inconsistent cut CS 2716

7 Distributed Snapshot Algorithm Assume each process communicates with another process using unidirectional FIFO point- to-point channels (e.g, TCP connections) Any process can initiate the algorithm – Checkpoint local state – Send MARKER on every outgoing channel On receiving a first marker on a channel: – Process checkpoints local state and – Send markers on all outgoing channels, and save messages on all other channels. On receiving subsequent marker on a channel: – stop saving messages for that channel – Saved messages are the state of the channel CS 2717

8 Distributed Snapshot A process finishes when – It receives a marker on each incoming channel and processes them all – State: local state plus state of all channels – Send state to initiator Any process can initiate snapshot – Multiple snapshots may be in progress Each is distinguished by tagging the marker with the initiator ID (and sequence number) CS 2718

9 Snapshot Algorithm Example a)Organization of a process and channels for a distributed snapshot CS 2719

10 Snapshot Algorithm Example b)Process Q receives a marker for first time and records local state c)Q records all incoming message d)Q receives a marker for its incoming channel and finishes recording the state of the incoming channel CS 27110

11 CS 27111 Execution Example p q Sq0Sq0 Sq1Sq1 Sq2Sq2 Sq3Sq3 Sp0Sp0 Sp1Sp1 Sp2Sp2 Sp3Sp3 m1m1 m2m2 m3m3

12 CS 27112 Execution Example p q Sq0Sq0 Sq1Sq1 Sq2Sq2 Sq3Sq3 Sp0Sp0 Sp1Sp1 Sp2Sp2 Sp3Sp3 m1m1 m2m2 m3m3 q records state as S q 1, sends marker to p

13 CS 27113 Execution Example p q Sq0Sq0 Sq1Sq1 Sq2Sq2 Sq3Sq3 Sp0Sp0 Sp1Sp1 Sp2Sp2 Sp3Sp3 m1m1 m2m2 m3m3 p records state as S p 2, channel state as empty

14 CS 27114 Execution Example p q Sq0Sq0 Sq1Sq1 Sq2Sq2 Sq3Sq3 Sp0Sp0 Sp1Sp1 Sp2Sp2 Sp3Sp3 m1m1 m2m2 m3m3 q records channel state as m 3

15 CS 27115 Execution Example p q Sq0Sq0 Sq1Sq1 Sq2Sq2 Sq3Sq3 Sp0Sp0 Sp1Sp1 Sp2Sp2 Sp3Sp3 m1m1 m2m2 m3m3 Recorded Global State = ((S p 2, S q 1 ), (0,m 3 ) )

16 Take home Message (Snapshot and global states) General solution for global state detection. Causality based detection of stable properties. Simple efficient protocol, uses Markers and FIFO properties. Don’t forget Channel States. Foundation for Distributed Checkpointing and Rollback Recovery CS 27116


Download ppt "GLOBAL STATES AND CHECKPOINTS CS 2711. 2 Distributed Checkpoints and Rollback Recovery Fault tolerance is achieved by periodically using stable storage."

Similar presentations


Ads by Google