Presentation is loading. Please wait.

Presentation is loading. Please wait.

Global States and Checkpoints

Similar presentations

Presentation on theme: "Global States and Checkpoints"— Presentation transcript:

1 Global States and Checkpoints
CS 271

2 Distributed Checkpoints and Rollback Recovery
Fault tolerance is achieved by periodically using stable storage to save the processes’ states during the failure-free execution. Upon a failure, a failed process rolls back from one of its saved states, thereby reducing the amount of lost computation. Each of the saved states is called a checkpoint CS 271

3 Checkpoint based Recovery
Uncoordinated checkpointing: Each process takes its checkpoints independently Coordinated checkpointing: Process coordinate their checkpoints in order to save a system-wide consistent state. Communication-induced checkpointing: It forces each process to take checkpoints based on information piggybacked on the application messages it receives from other processes. CS 271

4 Domino effect: example
Recovery Line P0 m7 m2 m5 m0 m3 P1 m6 m4 m1 P2 Domino Effect: Cascaded rollback which causes the system to roll back to too far in the computation (even to the beginning), in spite of all the checkpoints CS 271

5 Global State Chandy and Lamport—TOCS 1985
Global state of a distributed system Local state of each process Messages sent but not received Many applications need the state of the system Failure recovery, distributed deadlock detection Detect stable properties. Problem: how can you figure out the state of a distributed system? Each process is independent Network does not have any processing power. Distributed snapshot: a consistent global state CS 271

6 Global State A consistent cut An inconsistent cut CS 271

7 Distributed Snapshot Algorithm
Assume each process communicates with another process using unidirectional FIFO point-to-point channels (e.g, TCP connections) Any process can initiate the algorithm Checkpoint local state Send MARKER on every outgoing channel On receiving a first marker on a channel: Process checkpoints local state and Send markers on all outgoing channels, and save messages on all other channels. On receiving subsequent marker on a channel: stop saving messages for that channel Saved messages are the state of the channel CS 271

8 Distributed Snapshot A process finishes when
It receives a marker on each incoming channel and processes them all State: local state plus state of all channels Send state to initiator Any process can initiate snapshot Multiple snapshots may be in progress Each is distinguished by tagging the marker with the initiator ID (and sequence number) CS 271

9 Snapshot Algorithm Example
Organization of a process and channels for a distributed snapshot CS 271

10 Snapshot Algorithm Example
Process Q receives a marker for first time and records local state Q records all incoming message Q receives a marker for its incoming channel and finishes recording the state of the incoming channel CS 271

11 Execution Example Sp0 Sp1 Sp2 Sp3 p m1 m2 m3 q Sq0 Sq1 Sq2 Sq3 CS 271

12 Execution Example q records state as Sq1 , sends marker to p Sp0 Sp1
CS 271

13 Execution Example p records state as Sp2, channel state as empty Sp0
q Sq0 Sq1 Sq2 Sq3 CS 271

14 Execution Example q records channel state as m3 Sp0 Sp1 Sp2 Sp3 p m1
Sq0 Sq1 Sq2 Sq3 CS 271

15 Execution Example Recorded Global State = ((Sp2, Sq1), (0,m3) ) Sp0
CS 271

16 Take home Message (Snapshot and global states)
General solution for global state detection. Causality based detection of stable properties. Simple efficient protocol, uses Markers and FIFO properties. Don’t forget Channel States. Foundation for Distributed Checkpointing and Rollback Recovery CS 271

Download ppt "Global States and Checkpoints"

Similar presentations

Ads by Google