1. 1 EquiCast Scalable Multicast with Selfish Users Idit Keidar, Roie Melamed, Ariel Orda Technion

2. 2 Users Are Typically Selfish Not willing to share upload bandwidth Not willing to share upload bandwidth Need incentives in order to contribute their available resources Need incentives in order to contribute their available resources What's in it for me? Share with thy neighbor

3. 3 The Problem Long session of single-source multicast Long session of single-source multicast –Push-data: software updates, file distribution Peer-to-peer dissemination Peer-to-peer dissemination Selfish users Selfish users

4. 4 Additional Design Goals Low overhead Low overhead –Independent of network size (scalability) Fairness Fairness –Equally distribute multicast load among all nodes Scalable latency, increasing at most like Scalable latency, increasing at most like O(log N), where N is the number of nodes O(log N), where N is the number of nodes

5. 5 The Game Model the system as a non-cooperative game Model the system as a non-cooperative game The players: N nodes The players: N nodes Choose strategies that minimize their selfish cost Choose strategies that minimize their selfish cost Dominating strategy: yields better cost regardless of other users’ actions Dominating strategy: yields better cost regardless of other users’ actions

6. 6 The Cost Function for Node Ni Source’s strategy N1’s strategy Number of packets Ni received Why this function? Number of packets Ni sent Total number of data packets

7. 7 Architecture: K-Regular Random Graph N2 N1 N4 N3 N6 N5 1) Each node’s degree is k 2) The diameter is O(log N) 3) All nodes were created equal: the expected distance between a given node and a random node in the overlay equals the average distance between a pair of nodes in the overlay

8. 8 The Source’s Protocol S N2 N1 N4 N3 N6 N5 P1-P10 P11-P20 P21-P30 Round #1Round #2Round #3

9. 9 Gossip-Based Multicast Data is disseminated through gossip over the overlay’s links Data is disseminated through gossip over the overlay’s links

10. 10 Phase I: Gossip N2 N1 N4 N3 N6 N5 I have packets P1, P3, P8, and P9 I have packets P1, P4, P6, and P7

11. 11 Phase II: Process Gossip, Send Requests N2 N1 N4 N3 N6 N5 Send me packets P1, P3, and P9 Send me packets P4, P6, and P7

12. 12 Phase III: Send Data N2 N1 N4 N3 N6 N5 Packets P1, P3, and P9 Packets P4, P6, and P7

13. 13 If All Nodes Cooperate [Melamed and Keidar, NCA 2004] Fairness: all nodes send the same number of packets Fairness: all nodes send the same number of packets –Due to overlay’s properties and S’s random injections Pair-wise fairness: help each neighbor as much as it helps me Pair-wise fairness: help each neighbor as much as it helps me –Due to overlay’s properties and S’s random injections Low overhead: no redundant packets sent Low overhead: no redundant packets sent How do we achieve these nice properties with selfish users? How do we achieve these nice properties with selfish users?

14. 14 In Selfish Environments… Monitor neighbors' contribution Monitor neighbors' contribution –Compare against expected throughput Disconnect from neighbors that do not deliver the goods Disconnect from neighbors that do not deliver the goods –Cannot be on a per- round basis, due to randomness N1 sent me 17 packets N2 sent me 21 packets …

15. 15 Monitoring Neighbors Allow slack in imbalance in order to account for randomness Allow slack in imbalance in order to account for randomness Define lower bound L on allowed imbalance Define lower bound L on allowed imbalance In each round, for each neighbor n: In each round, for each neighbor n: –neighbor_balance[n] += received messages - expected throughput –if (neighbor_balance[n] < L) "n is in trouble!" What does it mean? Should I disconnect from n?

16. 16 When Node n is in Trouble A cooperative neighbor n has a non-zero probability of having an overdraft of L A cooperative neighbor n has a non-zero probability of having an overdraft of L –Disconnecting from cooperative nodes is not a good idea A bail-out option: n can ask S to send w data packets to a given neighbor on behalf of n A bail-out option: n can ask S to send w data packets to a given neighbor on behalf of n –n must send w empty packets to S –w ≤ expected throughput –Such packets accepted only for neighbors with a sub- par balance

17. 17 Problem with Asking S to Help Too lenient towards nodes with a negative balance Too lenient towards nodes with a negative balance –No penalty for having a low balance –Nodes not motivated to have positive balance A selfish node may let its balance drop without the risk of being disconnected A selfish node may let its balance drop without the risk of being disconnected Over-use of this mechanism overloads S Over-use of this mechanism overloads S

18. 18 Negative Balance Fines A neighbor sends an additional fine packet for every round with a negative balance A neighbor sends an additional fine packet for every round with a negative balance Does not affect the sender’s balance Does not affect the sender’s balance –Motivates nodes to have non-negative balance as much as possible –Thus, minimizes requests from S

19. 19 The Monitoring Mechanism In each round, for each neighbor n: In each round, for each neighbor n: –neighbor_balance[n] += received data messages - expected throughput –if (neighbor_balance[n] < L) if received w data packets from S on behalf of n then neighbor_balance[n]+= w if received w data packets from S on behalf of n then neighbor_balance[n]+= w –if (neighbor_balance[n] < L  (neighbor_balance[n] < 0  n did not send a fine packet)) disconnect from N disconnect from N ask S for an emulated neighbor ask S for an emulated neighbor What does emulated mean?

20. 20 An Emulated Node In case of a disconnection, S can emulate a selfish rational EquiCast node e In case of a disconnection, S can emulate a selfish rational EquiCast node e e’s interface is identical to the interface of an EquiCast node except… e’s interface is identical to the interface of an EquiCast node except… –n’s balance with respect to e is initialized to L –In each round, n must send a fine packet to e Hence, a node prefers to maintain a connection with a non-emulated node over an emulated one Hence, a node prefers to maintain a connection with a non-emulated node over an emulated one

21. 21 The Need for L – Lower Bound on Balance If we have a per-round fine, why do we need L? If we have a per-round fine, why do we need L? For high data throughput and low overhead, fines are small For high data throughput and low overhead, fines are small –E.g., 1 fine packet per 50 data packets Without L, a selfish node would send only fine packets Without L, a selfish node would send only fine packets

22. 22 Upper Bound on Balance Each node selfishly chooses its own upper bound on its balance wrt its neighbors Each node selfishly chooses its own upper bound on its balance wrt its neighbors Parameter H Parameter H

23. 23 Throughput and Goodput Expected incoming rate per link per round: Expected incoming rate per link per round: –1 gossip –1 request –½ fine –Tens of data packets (goodput) May come from source instead of neighbor May come from source instead of neighbor } overhead

24. 24 Proof of Cooperation

25. 25 Notations and Model k – the graph’s degree k – the graph’s degree P – the total number of multicast packets (distributed exponentially) P – the total number of multicast packets (distributed exponentially) p – the per-round number of multicast packets p – the per-round number of multicast packets R – the total number of rounds (P/p) R – the total number of rounds (P/p) H – The maximal allowed balance (chosen by the node) H – The maximal allowed balance (chosen by the node) Each node has an upload bandwidth of at most p+kc packets per round Each node has an upload bandwidth of at most p+kc packets per round (k^2 – k)(c-3)<p (k^2 – k)(c-3)<p k^2(c – 2)–2k<p k^2(c – 2)–2k<p

26. 26 Protocol-obedient strategy (POS) Choose your own H Choose your own H Maintain connections only with nodes whose identities are received from S Maintain connections only with nodes whose identities are received from S –choose any subset of them in each round Run the protocol's code as is for each of these connections Run the protocol's code as is for each of these connections

27. 27 k-POS A POS in which a node maintains exactly k connections in each round throughout the entire multicast session is called a k-POS A POS in which a node maintains exactly k connections in each round throughout the entire multicast session is called a k-POS

28. 28 Lemma A k-POS strictly dominates every POS in which n maintains connections with j nodes, where j≠k A k-POS strictly dominates every POS in which n maintains connections with j nodes, where j≠k Rationale: Rationale: –If j<k, then the cost is infinite Due to bandwidth limitations (see paper) Due to bandwidth limitations (see paper) –If j>k, then the overhead is larger Maintaining connections with k nodes suffices in order to receive all the multicast packets Maintaining connections with k nodes suffices in order to receive all the multicast packets

29. 29 Theorem 1 If all nodes choose strongly dominating strategies* out of the set of POSs, then If all nodes choose strongly dominating strategies* out of the set of POSs, then –every node maintains connections with its initial k neighbors –every node receives all the multicast packets * Strongly dominating strategy: assume nothing on what others do (except POS)

30. 30 Theorem 1 – Proof Sketch Each node chooses a k-POS in order to minimize its cost Each node chooses a k-POS in order to minimize its cost A node prefers to maintain a connection with a non-emulated node over an emulated one A node prefers to maintain a connection with a non-emulated node over an emulated one –Each node maintains connections with its initial k neighbors and receives from them (and from S on behalf of them) all the multicast packets

31. 31 Theorem 2: Unilateral Defection from the Protocol Does not Help If all the nodes, except for one (rational) node n, choose a strategy out of the set of possible POSs, then n also chooses a POS If all the nodes, except for one (rational) node n, choose a strategy out of the set of possible POSs, then n also chooses a POS

32. 32 Theorem 2 – Proof Sketch n does not communicate with nodes whose identities were not received from S, since all other nodes choose POSs n does not communicate with nodes whose identities were not received from S, since all other nodes choose POSs Since all of n’s neighbors choose POSs, in order to achieve a finite cost n must maintain these k connections Since all of n’s neighbors choose POSs, in order to achieve a finite cost n must maintain these k connections –I.e., n chooses a K-POS

33. 33 Theorem 3 If If –all of a node n’s initial k neighbors are rational and choose POSs and –n cannot locate an identity of a node that does not choose a POS, then … [same good things as before] then … [same good things as before]

34. 34 Theorem 4: Choosing H Each node chooses a non-negative H parameter with each of its neighbors Each node chooses a non-negative H parameter with each of its neighbors

35. 35 Proof (i) Since n maintains the connection with n’, the sending rate to n’ does not affect the data receiving rate from n’, and hence n can minimize its sending rate to n’ in order to minimize its cost Since n maintains the connection with n’, the sending rate to n’ does not affect the data receiving rate from n’, and hence n can minimize its sending rate to n’ in order to minimize its cost

36. 36 Proof (ii) The per-round overhead incurred by maintaining the connection with n’ consists of: data overhead, gossip/request packets, and penalty packets The per-round overhead incurred by maintaining the connection with n’ consists of: data overhead, gossip/request packets, and penalty packets The gossip/request overhead is fixed The gossip/request overhead is fixed –Hence, n tries to minimize the data and penalty overheads

37. 37 Proof (iii) The per-round data overhead is bounded from below by p/k + L/R = p/k The per-round data overhead is bounded from below by p/k + L/R = p/k The per-round penalty overhead is the percentage of rounds in which the balance is negative The per-round penalty overhead is the percentage of rounds in which the balance is negative Since the session duration is exponentially distributed and unknown to n, a zero balance at all rounds minimized the overhead Since the session duration is exponentially distributed and unknown to n, a zero balance at all rounds minimized the overhead

38. 38 Conclusions - Algorithm Combination of Combination of –Fine for negative balance; –Help from source at balance L; and –Emulated nodes achieves cooperation and high throughput Emulated nodes are available as a safety net, but will never be used if all nodes are rational Emulated nodes are available as a safety net, but will never be used if all nodes are rational

39. 39 Conclusions – Proof POS captures “reasonable” behavior POS captures “reasonable” behavior If all users run POSs If all users run POSs –cooperating is a strongly dominating strategy –everything works great Unilateral defection from POS (hacking the protocol's code) does not help Unilateral defection from POS (hacking the protocol's code) does not help

40. 40 Related Work

41. 41 Incentive-Based Solutions Two approaches: Two approaches: –A user receives a service only if it contributes upload bandwidth or disk space for some other users Examples: Samsara, BitTorrent, Ngan et al. Examples: Samsara, BitTorrent, Ngan et al. –The quality of service experienced by a node is proportional to its contribution to the system Examples: GIA, Habib et al., Feldman et al. Examples: GIA, Habib et al., Feldman et al.

42. 42 Incentive-Based Content Distribution BitTorrent, Avalanche BitTorrent, Avalanche –No scalable dissemination of single stream –Assumes altruism Why does it work? Why does it work? –“ –“How to cheat BitTorrent and why nobody does” [Hales, Patarin] Will nobody cheat forever? Will nobody cheat forever?

43. 43 Incentive-Based Multicast [Ngan et al.] Detection of selfish nodes Detection of selfish nodes Periodic reconstruction of multicast trees that exclude previously misbehaving nodes Periodic reconstruction of multicast trees that exclude previously misbehaving nodes Drawbacks: Drawbacks: –Not all users are selfish –High overhead: When the group size is 2000 nodes, each node sends nearly 400 control messages every two minutes, in addition to data messages

44. 44 Related Work: Summary First p2p incentive-based service that First p2p incentive-based service that –Uses a game theory approach –Proves that cooperation is dominating strategy –Proves that solution “works” in selfish environments

45. 45 Dynamic Setting DEC (Dynamic EquiCast)

46. 46 DEC (Dynamic EquiCast) DEC is very similar to EquiCast DEC is very similar to EquiCast Changes compared to EquiCast: Changes compared to EquiCast: –The overlay –The cost function –A protocol to initialize the balances upon a join or leave operation

47. 47 The Overlay We use a k-regular overlay that supports join and leave operations We use a k-regular overlay that supports join and leave operations –E.g., [Law and Siu, 2003], composed of k/2 Hamiltonian cycles –For proving cooperation, we only need the regularity property

48. 48 The Cost Function Obtained from EquiCast’s cost function Obtained from EquiCast’s cost function Requires each node to receive t*p data packets, where t is the number of rounds in which the node receives the service, instead of all the data packets Requires each node to receive t*p data packets, where t is the number of rounds in which the node receives the service, instead of all the data packets

49. 49 A Join Operation N2N1 N1.my_balance[N2]=10 N1.neighbor_balance[N2]=-5 N2.my_balance[N1]=-5 N2.neighbor_balance[N1]=10 N

50. 50 A Join Operation N2N1 N1.my_balance[N]=10 N1.neighbor_balance[N]=-5 N2.my_balance[N]=-5 N2.neighbor_balance[N]=10 N S Send N N.neighbor_balance[N1]+ N.neighbor_balance[N2]= 10+(-5)=5 data packets WHY??? N.my_balance[N1]=-5 N.neighbor_balance[N1]=10 N.my_balance[N2]=10 N.neighbor_balance[N2]=-5