1. CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS
Set 5: Synchronous LE in Rings
CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS
CSCE 668
Spring 2014
Prof. Jennifer Welch

2. Leader Election in Synchronous Rings
Here is a simple algorithm; assumes ids are nonnegative integers
Group rounds into phases, each phase containing n rounds; start counting at 0
In phase i, the processor with id i, if there is one, sends a message around the ring and is elected.
Set 5: Synchronous LE in Rings
CSCE 668

3. Example of Simple Synchronous Algorithm
n = 4, the smallest id is 7.
In phases 0 through 6 (corresponding to rounds 1 through 28), no message is ever sent.
At beginning of phase 7 (round 29), processor with id 7 sends message which is forwarded around the ring.
Relies on synchrony and knowing n
Set 5: Synchronous LE in Rings
CSCE 668

4. Analysis of Simple Algorithm
Correctness: Easy to see.
Message complexity: O(n), which is optimal
Time complexity: O(n*m), where m is the smallest id in the ring.
not bounded by any function of n, undesirable
Set 5: Synchronous LE in Rings
CSCE 668

5. Another Synchronous Algorithm
Works in a slightly weaker model than the previous synchronous algorithm:
processors might not all start at same round; a processor either wakes up spontaneously or when it first gets a message
uniform (does not rely on knowing n)
Set 5: Synchronous LE in Rings
CSCE 668

6. Another Synchronous Algorithm
A processor that wakes up spontaneously is active; sends its id in a fast message (one edge per round)
A processor that wakes up when receiving a msg is relay; never in the competition
A fast message carrying id m becomes slow if it reaches an active processor; starts traveling at one edge per 2m rounds
A processor only forwards a msg whose id is smaller than any id it has previously sent
If a processor gets its own id back, it elects itself
Set 5: Synchronous LE in Rings
CSCE 668

7. Analysis of Synchronous Algorithm
Correctness: convince yourself that the active processor with smallest id is elected.
Message complexity: Winner's msg is the fastest. While it traverses the ring, other msgs are slower, so they are overtaken and stopped before too many messages are sent.
Quantify these ideas next…
Set 5: Synchronous LE in Rings
CSCE 668

8. Message Complexity Divide msgs into three kinds:
fast msgs
slow msgs sent while the leader's msg is fast
slow msgs sent while the leader's msg is slow
Next, count the number of each type of msg.
Set 5: Synchronous LE in Rings
CSCE 668

9. Number of Type 1 Messages
Show that no processor forwards more than one fast msg:
Suppose pi forwards pj's fast msg and pk's fast msg. When pk's fast msg arrives at pj :
either pj has already sent its fast msg, so pk's msg becomes slow before reaching pi, or
pj has not already sent its fast msg, so it never will
Thus number of type 1 msgs is n. pk pj pi
Set 5: Synchronous LE in Rings
CSCE 668

10. Number of Type 2 Messages
(slow sent while leader's msg is fast)
Leader's msg is fast for at most n rounds
by then it would have returned to leader
Slow msg i is forwarded n/2i times in n rounds
Maximum number of msgs is when ids are small as possible (0 to n-1 and leader is 0)
Number of type 2 msgs is at most
∑ (n/2i) ≤ n
n-1
i=1
Set 5: Synchronous LE in Rings
CSCE 668

11. Number of Type 3 Messages
(slow msgs sent while leader's is slow)
Maximum number of rounds during which leader's msg is slow is n*2L (L is leader's id).
No msgs are sent once leader's msg has returned to leader
Slow msg i is forwarded n*2L/2i times during n*2L rounds.
Worst case is when ids are L to L + n-1
Number of type 3 msgs is at most
∑(n*2L/2i) ≤ 2n
L+n-1
i=L
Set 5: Synchronous LE in Rings
CSCE 668

12. Total Number of Messages
We showed
number of type 1 msgs is at most n
number of type 2 msgs is at most n
number of type 3 msgs is at most 2n
Thus total number of msgs is at most 4n = O(n).
Set 5: Synchronous LE in Rings
CSCE 668

13. Time Complexity of Synchronous Algorithm
Running time is O(n 2x), where x is smallest id.
Even worse than previous algorithm, which was O(n x)
Both algorithms have two potentially undesirable properties:
rely on numeric values of ids to count
number of rounds depends on minimum id, which may bear no relationship to n
Next result shows that to obtain linear msg complexity, an algorithm must rely on numeric values of the ids.
Set 5: Synchronous LE in Rings
CSCE 668

14. Comparison-Based LE Algorithms
We will show that if nodes cannot rely on the numeric values of the ids, then any synchronous LE algorithm has message complexity (n log n).
How do we formalize "not relying on numeric values of ids"?
Need a definition of "comparison-based", similar to that for sorting algs.
Set 5: Synchronous LE in Rings
CSCE 668

15. Definition of Comparison-Based
First note that in the synchronous model, the behavior of an LE algorithm is totally determined by the distribution of the ids. Denote execution on ring R by exec(R).
An LE algorithm is comparison-based if, in any two "order-equivalent" rings R1 and R2, "matching" processors pi in R1 and pj in R2 have "similar" behaviors in exec(R1) and exec(R2).
Set 5: Synchronous LE in Rings
CSCE 668

16. Definition of Order-Equivalent
Rings R1 = (x1,x2,…,xn) and R2 = (y1,y2,…,yn) are order-equivalent if
xi < xj if and only if yi < yj
Example: x3 y2 y1 x4 x2 y3 x1 y4 x5 y5
Set 5: Synchronous LE in Rings
CSCE 668

17. Definition of Matching Processors
Processors pi in R1 and pj in R2 are matching if they are the same distance from the processor with minimum id.
Example: p0 (with id 18) in R1 and p1 (with id 5) in R2, since p0 is 2 hops from p3 (with min id 12) and p1 is 2 hops from p4 (with min id 2)
Set 5: Synchronous LE in Rings
CSCE 668

18. Definition of Similar Behaviors
Two processors have similar behaviors in two executions if, in every round
one processor sends a message to the left (right) if and only if the other one does
content of message is irrelevant
one processor is elected if and only if the other one is
Set 5: Synchronous LE in Rings
CSCE 668

19. Synchronous Lower Bound
Theorem (3.18): For every n ≥ 8 that is a power of 2, there is a ring of size n on which any synchronous comparison-based algorithm sends (n log n) msgs.
Proof: For each n, construct a highly symmetric ring Sn on which every comparison-based algorithm takes lots of messages.
Symmetry means many processors have order- equivalent neighborhoods and thus behave similarly (including sending msgs)
Set 5: Synchronous LE in Rings
CSCE 668

20. Proof Strategy Define highly symmetric ring Sn
Show that the number of "active" rounds when any c-b alg runs on Sn is at least n/8 (L. 3.20)
Show that in the k-th active round, at least
n/(2(2k+1)) msgs are sent (L. 3.21)
Do some arithmetic to show that
∑(n/(2(2k+1))) = (n log n)
n/8
k=1
Set 5: Synchronous LE in Rings
CSCE 668

21. Active Rounds A round is active if at least one processor sends a msg
A proc. in a synchronous algorithm can potentially learn something even in an inactive round
cf. the simple synchronous alg
But in a comparison-based alg., a proc. can't learn about the order pattern of its ring in an inactive round (intuition)
Note that the k-th active round might be much later than the k-th round
Set 5: Synchronous LE in Rings
CSCE 668

22. A Highly Symmetric Ring Sn
Let pi's id be rev(i) , the integer whose binary representation using log n bits is the reverse of i's binary rep. Example:
For technical reasons, then multiply id by n+1 and add n
00  00
01  10 4 2 14 p0 p1 p2 19 3 p3 1 9
11  11
10  01
Set 5: Synchronous LE in Rings
CSCE 668

23. Why Lots of Active Rounds
Lemma 3.20: Number of active rounds, T, in exec(Sn) is
at least n/8.
Proof: Suppose T < n/8. Let pi be the elected leader.
Number of T-neighborhoods order-equivalent to pi's is at least n/(2(2T+1)) (Lemma 3.19, to be shown)
Since n ≥ 8 and T < n/8, algebra shows
n/(2(2T+1)) > 1.
So there is pj  pi whose T-neighborhood is order- equivalent to pi 's.
Then pj is also elected (Lemma 3.17, to be shown), contradiction.
Set 5: Synchronous LE in Rings
CSCE 668

24. Why Enough Msgs Sent in Each Active Round
Lemma 3.21: At least n/(2(2k+1)) msgs are sent in kth active round.
Proof: Since the round is active, at least one proc., say pi, sends a msg.
There are at least n/(2(2k+1)) procs. whose k- neighborhood is order-equivalent to pi's (Lemma 3.19, to be shown)
Each of those procs. sends a msg in kth active round, since pi does (Lemma 3.17, to be shown).
Set 5: Synchronous LE in Rings
CSCE 668

25. Why So Many O-E Neighborhoods
Lemma 3.19: Every k-nbrhood of Sn has at least
n/(2(2k+1)) order-equivalent k-nbrhoods.
Proof Sketch: Let N be a k-nbrhood (sequence of 2k+1 ids in the ring).
Let j be smallest power of 2 larger than 2k+1.
Break Sn into n/j segments of length j, with one segment encompassing N.
Claim: Each segment is order-equivalent to N.
Set 5: Synchronous LE in Rings
CSCE 668

26. Why Similar Behavior in Order-Equivalent Neighborhoods
Intuition is that two nodes in order-equivalent neighborhoods in the same ring will behave similarly, at least until differentiating information from beyond those neighborhoods has reached them.
Yet the definition of comparison-based algorithm only requires similar behavior for matching processors in different (albeit order-equivalent) rings.
Set 5: Synchronous LE in Rings
CSCE 668

27. Order-Equivalent Means Similar
Lemma 3.17: If pi and pj have order-equivalent k- nbrhoods, then pi and pj have similar behaviors through k-th active round.
Proof Sketch: Construct another ring Sn' such that
pi in Sn behaves same as pj in Sn'
pj in Sn' behaves similarly to pj in Sn
Set 5: Synchronous LE in Rings
CSCE 668

28. Order-Equivalent Means Similar
In more detail, construct Sn' such that
pi's k-nbrhood in Sn equals pj's in Sn'
so behavior is same through k-th active round
Sn' and Sn are order-equivalent
pj in Sn' is matching to pj in Sn
so behavior is similar through k-th active round
Can be done since ids in Sn are spaced (multiplied by n+1, plus n)
So pi and pj have similar behavior in Sn through k- th active round
Set 5: Synchronous LE in Rings
CSCE 668

29. Anonymous Rings Revisited
Leader election is impossible in anonymous rings
No way to break symmetry
No deterministic algorithm which works in every execution
Another way to break symmetry, which works in some (but not all) executions is to use randomization.
Set 5: Synchronous LE in Rings
CSCE 668

30. Randomized Algorithm
In each computation step, the processor receives an additional input to its state transition function, a random number.
Set 5: Synchronous LE in Rings
CSCE 668

31. Revised LE Problem Definition
Weakened problem definition compared to original:
At most one leader is elected in every state of every admissible execution
same as previous definition
At least one leader is elected "with high probability".
weaker than previous definition
But what does "with high probability" mean?
Set 5: Synchronous LE in Rings
CSCE 668

32. Randomized LE Algorithm
Assume synchronous model
Initially:
set id to 1 with probability 1 - 1/n and to 2 with probability 1/n
send id to left
When msg M is received:
if M contains n ids then
if id is unique maximum in M then elect self
else not elected
else append id to M and send to left
Set 5: Synchronous LE in Rings
CSCE 668

33. Analysis of Randomized LE Alg.
Uses O(n2) msgs
There is never more than one leader
Sometimes there is no leader
leader is only elected if there is exactly one processor that sets its id to 2
How often is there no leader, i.e., what is the probability?
Need some more definitions…
Set 5: Synchronous LE in Rings
CSCE 668

34. Random Choices and Probabilities
Since system is synchronous, an admissible execution of the algorithm is determined solely by the initial random choices.
Call this collection random choices
RC = <r0, r1, …, rn-1>
where each ri is either 1 or 2.
Let exec(RC) be the resulting execution.
Definition: For any predicate P on executions, Pr[P] is the probability of
{RC : exec(RC) satisfies P}
I.e., the proportion of random choices resulting in an execution that satisfies P.
Set 5: Synchronous LE in Rings
CSCE 668

35. Probability of Electing a Leader
Let P be the predicate "there is a leader".
Pr[P] = probability RC contains exactly one 2
Set 5: Synchronous LE in Rings
CSCE 668

36. Improving the Probability of Electing a Leader
If procs. notice there is no leader, they can try again.
Each iteration of the basic algorithm is a phase.
Keep trying until success.
Random choices that define an execution consist of an infinite sequence of 1's and 2's, one for each proc.
It is possible that algorithm doesn't terminate.
Set 5: Synchronous LE in Rings
CSCE 668

37. Probability of Not Terminating
Probability of terminating in a particular phase is (1 - 1/n)n-1
Probability of not terminating in a particular phase is 1 - (1 - 1/n)n-1
Probability of not terminating in k phases is
(1 - (1 - 1/n)n-1)k since phases are independent.
Last expression goes to 0 as k increases.
Set 5: Synchronous LE in Rings
CSCE 668

38. Expected Number of Phases
Definition: Expected value of a random variable T is
E[T] =  k Pr[T = k]
Let T be number of phases until termination.
Pr[T = k]
= Pr[first k-1 phases fail & kth succeeds]
= (1 - (1 - 1/n)n-1)k-1 (1 - 1/n)n-1
= (1 - p)k-1 p , where p = (1 - 1/n)n-1
This is a geometric random variable with expected value p-1 < e.
So expected number of phases is < 3. k
Set 5: Synchronous LE in Rings
CSCE 668