CS4231 Parallel and Distributed Algorithms AY 2006/2007 Semester 2 Lecture 10 Instructor: Haifeng YU

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 22 Review of Last Lecture System/Failure ModelConsensus Protocol Ver 0: No node or link failuresTrivial: all-to-all broadcast Ver 1: Node crash failures; Channels are reliable; Synchronous; (f+1)-round protocol can tolerate f crash failures Ver 2: No node failures; Channels may drop messages (the coordinated attack problem) Impossible without error Randomized algorithm with 1/r error prob Ver 3: Node crash failures; Channels are reliable; Asynchronous; Impossible (the FLP theorem) Ver 4: Node Byzantine failures; Channels are reliable; Synchronous; (the Byzantine Generals problem) If n ≤ 3f, impossible. If n ≥ 4f + 1, we have a (2f+2)- round protocol.

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 23 Today’s Roadmap  Chapter 18 “Self-Stabilization”  Formalizing the notion of self-stabilization  A toy problem: “Rotating Privilege on A Ring”  The very first self-stabilization algorithm  More for theoretical interest  A practical problem: “Self-Stabilizing Spanning Tree Construction”  Very useful in multicast (BitTorrent-style data streaming)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 24 parent value of these two nodes no longer valid Motivation for Self-Stabilization  (Motivation on the book is not very practical)  Distributed systems can get into illegal state due to  Topology changes  Failures / reboots  Malicious processes  Generally called “faults” A multicast tree: Each node records who its parent and children are

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 25 Motivation for Self-Stabilization  Distributed systems can get into illegal state due to  Topology changes  Failures / reboots  Malicious processes  Generally called “faults” Mobile ad hoc networks: Maintaining the shortest route back to sink A A should now have an improved route back to sink

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 26 Defining Self-Stabilization  The state (i.e., data state in all processes) of a distributed system is either legal or illegal  Definition based on application semantics  The code on each process is assumed to be correct all the time  A distributed algorithm is self-stabilizing if  Starting from any (legal or illegal) state, the protocol will eventually reach a legal state if there are no more faults  Once the system is in a legal state, it will only transit to other legal states unless there are faults  Intuitively, will always recover from faults and once recovered, will stay recovered forever  Self-stabilizing algorithm typically runs in background and never stops

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 27 The Rotating Privilege Problem  A ring of n processes, each process can only communicate with neighbors  There is a privilege in the system  At any time, only one node may have the privilege (you can think of this as a token)  The node with the privilege may for example, have exclusive access to some resource  The privilege needs to “rotate” among the nodes so that each node has a chance

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 28 The Rotating Privilege Algorithm  Each process i has a local integer variable V_i  0  V_i  k where k is some constant no smaller than n Example: n = 5 and k = 12

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester Red process’s action: Retrieve value L of my clockwise neighbor; Let V be my value; if (L == V) { // I have the privilege // complete whatever I want to do; V = (V+1) % k; } Each process executes each action repeatedly – we will assume each action happens instantaneously (for this algorithm only) Green process’s action: Retrieve value L of my clockwise neighbor; Let V be my value; if (L != V) { // I have the privilege // complete whatever I want to do; V = L; }

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 211 What’s Interesting about the Algorithm  This problem is mainly for theoretical interests  What is interesting about it:  Regardless of the initial values of the processes, eventually the system will get into a legal state and stay in legal states  Self-stabilizing!

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 212 Legal States  We say that a process makes a “move” if it has the privilege and changes its value  System in legal state if exactly one machine can make a move  Easy to prove that in any state, at least one machine can make move  Lemma: The following are legal states and are the only legal state  All n values same OR  Only two different values forming two consecutive bands, and one band starts from the red process  To prove these are the only legal states, consider the value V of the red process and the value L of its clockwise neighbor  Case I: V=L  Case II: V  L

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 213 Legal States  Legal States  Theorem: If the system is in a legal state, then it will stay in legal states  Our assumption on instantaneous actions will simplify this proof  We can consider actions one by one

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 214 Illegal States  Legal States  Lemma: Let P be a green process, and let Q be P’s clockwise neighbor. If Q makes i moves, then P can make at most i+1 move.  Lemma: Let Q be the red process. If Q makes i moves, then system-wide there can be at most the following number of moves:  Lemma: Let Q be the red process, and consider a sequence of n^2 moves in the system. Q makes at least one move in the sequence

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 215 Illegal States  Legal States  Lemma 1: Regardless of the starting state, the system eventually reach a state T where the red process has a different value from all other process (though the system may not stay in such states)  Proof: Let Q be the red process. If in the starting state Q has the same value as some other process, then there must be an integer j (0  j  k-1) that is not the value of any process. Q will eventually take j as its value.  (It takes Q any most n moves to do so.)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 216 Illegal States  Legal States  Lemma 2: If the system is in a state T where the red process has a different value from all other process, then the system will eventually each a state where all processes have the same value (though the system make not stay in such states)  Theorem: Regardless of the initial states of the system, the system will eventually reach a legal state.  Proof: From Lemma 1 and Lemma 2.

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 217 Self Stabilizing Spanning Tree Algorithm  Given n processes connected by an undirected graph and one special process P1, construct a spanning tree rooted at P1  Not all processes can communicated with all processes directly  A very useful / practical algorithm  Can also be used to compute shortest path P1

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 218 Self Stabilizing Spanning Tree Algorithm  Each process maintains two variables  parent: Who my parent is  dist: My distance to root  Runs in the background  parent and dist are continuously updated  At any given point of time, the values of the two variables can be wrong  Due to “faults” such as topology change resulted from node movement P1

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 219 Self Stabilizing Spanning Tree Algorithm (3, P5) (9, P3) P1 P2 P3 P4 P5 P6 P7 P8 (8, P4) (6, P7) (5, P7) (1, P1) (2, P3) Red values are initially incorrect values; Green values are values that have become correct (0, P8)  On P1 (executed periodically):  dist = 0; parent = -1;  On all other processes (executed periodically):  Retrieve dist from all neighbors  Set my own dist = 1 + (the smallest dist received)  Set my own parent = my neighbor with the smallest dist (tie break if needed)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 220 Self Stabilizing Spanning Tree Algorithm  On P1 (executed periodically):  dist = 0; parent = -1;  On all other processes (executed periodically):  Retrieve dist from all neighbors  Set my own dist = 1 + (the smallest dist received)  Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (9, P3) P1 P2 P3 P4 P5 P6 P7 P8 (8, P4) (6, P7) (5, P7) (1, P1) (2, P3) Red values are initially incorrect values; Green values are values that have become correct (0, P8)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 221 Self Stabilizing Spanning Tree Algorithm  On P1 (executed periodically):  dist = 0; parent = -1;  On all other processes (executed periodically):  Retrieve dist from all neighbors  Set my own dist = 1 + (the smallest dist received)  Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (1, P1) P1 P2 P3 P4 P5 P6 P7 P8 (1, P1) (6, P7) (5, P7) (1, P1) Red values are initially incorrect values; Green values are values that have become correct (0, P8)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 222 Self Stabilizing Spanning Tree Algorithm  On P1 (executed periodically):  dist = 0; parent = -1;  On all other processes (executed periodically):  Retrieve dist from all neighbors  Set my own dist = 1 + (the smallest dist received)  Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (1, P1) P1 P2 P3 P4 P5 P6 P7 P8 (1, P1) (2, P6) (1, P7) (1, P1) Red values are initially incorrect values; Green values are values that have become correct (0, P8)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 223 Self Stabilizing Spanning Tree Algorithm  On P1 (executed periodically):  dist = 0; parent = -1;  On all other processes (executed periodically):  Retrieve dist from all neighbors  Set my own dist = 1 + (the smallest dist received)  Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (1, P1) P1 P2 P3 P4 P5 P6 P7 P8 (1, P1) (2, P6) (1, P7) (1, P1) Red values are initially incorrect values; Green values are values that have become correct (2, P4)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 224 Self Stabilizing Spanning Tree Algorithm  On P1 (executed periodically):  dist = 0; parent = -1;  On all other processes (executed periodically):  Retrieve dist from all neighbors  Set my own dist = 1 + (the smallest dist received)  Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (1, P1) P1 P2 P3 P4 P5 P6 P7 P8 (1, P1) (2, P6) (2, P5) (1, P1) Red values are initially incorrect values; Green values are values that have become correct (2, P4)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 225 Self Stabilizing Spanning Tree Algorithm  On P1 (executed periodically):  dist = 0; parent = -1;  On all other processes (executed periodically):  Retrieve dist from all neighbors  Set my own dist = 1 + (the smallest dist received)  Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (1, P1) P1 P2 P3 P4 P5 P6 P7 P8 (1, P1) (2, P6) (2, P5) (3, P4) (1, P1) Red values are initially incorrect values; Green values are values that have become correct (3, P4)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 226 Correctness Proof  Define a phase to be the minimum time period where each process has executed its code at least once (called “has taken an action”)  Some process may execute its code more than once  The definition of a phase here is different from a round in synchronous systems !  Let A_i to be the length of the shortest path from process i to the root, let dist_i to be the value of dist on process i  dist_i is not allowed to be negative

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 227 Correctness Proof  Lemma: At the end of phase 1, dist_1 = 0 and dist_i  1 for any i  2  Lemma: At the end of phase 2,  Any process i whose A_i = 0, we have dist_i = 0;  Any process i whose A_i = 1, we have dist_i = 1;  Any process i whose A_i  2, we have dist_i  2;

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 228 Correctness Proof  Lemma: At the end of phase r,  Any process i whose A_i  r-1, we have dist_i = A_i;  Any process i whose A_i  r, we have dist_i  r;  Prove by induction: assume the lemma holds at phase r, now consider phase r+1, we need to prove  Any process i whose A_i  r-1, we have dist_i = A_i;  Any process i whose A_i = r, we have dist_i = A_i;  Any process i whose A_i  r+1, we have dist_i  r+1;

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 229 Correctness Proof  Consider all t actions taken during phase r+1  We will use an induction on t  This proof is tricky if this is your first self-stabilization proof  A process may take multiple actions in a phase !  Processes may take actions in parallel – cannot assume a serialization of all actions !  The proof technique is typical for proving self-stabilization  Step 1: Prove that the t actions will not roll back what is already achieved so far (no backward move)  Step 2: Prove that at some point, each node will achieve more (forward move)  Step 3: Prove that the t actions will not roll back the effects of the forward move (no backward move after the forward move)

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 230 A_i  r - 1 A_i = r A_i  r - 1 …… for nodes with already know: phase r want to show: phase r+1 A_i  r-1 dist_i = A_i A_i = r dist_i  r dist_i = A_i A_i  r+1dist_i  rdist_i  r+1 Step 1: The t actions will not change the green conditions Proof: Induction on t and consider action (t+1) by some process. (Cannot assume action (t+1) happens after the t actions.) Regardless of what values the process draws from its neighbors, the action will not end up violating the condition.

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 231 A_i  r - 1 A_i = r A_i  r - 1 …… for nodes with already know: phase r want to show: phase r+1 A_i  r-1 dist_i = A_i A_i = r dist_i  r dist_i = A_i A_i  r+1dist_i  rdist_i  r+1 Step 1: The t actions will not change the green conditions satisfied at the beginning of phase r Proof (continued): True because a level A_i process only have neighbors from level A_i – 1, A_i, and A_i + 1.

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 232 A_i  r - 1 A_i = r A_i  r - 1 …… for nodes with already know: phase r want to show: phase r+1 A_i  r-1 dist_i = A_i A_i = r dist_i  r dist_i = A_i A_i  r+1dist_i  rdist_i  r+1 Step 2: For each process, at some point during phase r+1, it will satisfy the red conditions Proof: By definition of a phase, each process will take at least one action during phase r+1

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 233 A_i  r - 1 A_i = r A_i  r - 1 …… for nodes with already know: phase r want to show: phase r+1 A_i  r-1 dist_i = A_i A_i = r dist_i  r dist_i = A_i A_i  r+1dist_i  rdist_i  r+1 Step 3: For each process, after it first satisfies the red condition, it will continue to satisfy the red condition for the remainder of the phase Proof: Trivial – but do need to enumerate three cases

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 234 Correctness Proof  Theorem: After H rounds, A_i = dist_i on all processes  H being the length of the shortest path from the most far away process to the root  Theorem: After H rounds, the dist and parent values on all processes are correct  Proof: Each process has a single parent pointer except the root. So the graph has n nodes and n-1 edges. Each process has a path to the root, thus the graph is connected.

CS4231 Parallel and Distributed Algorithms AY2006/2007 Semester 235 Homework Assignment  For the “rotating privilege algorithm”: Consider a ring with 4 processes, and let k = 2. In other words, each process may have a value of 0 or 1. Construct a scenario where the algorithm will not stabilize  Think about: For the self-stabilizing tree algorithm, assume now that the system is synchronous, prove that the algorithm is self-stabilizing.