2. Computer Science ROMA: Reliable Overlay Multicast with Loosely Coupled TCP Connections Gu-In Kwon and John Byers Computer Science Dept. Boston University

3. Computer Science IP Multicast CMU Berkeley MIT UCSD routers end systems multicast flow  Highly efficient  Challenges  Congestion Control  Reliability http://esm.cs.cmu.edu/Sigcomm2001/SigcommTalk.ppt

4. Computer Science Overlay Multicast CMU2 UCSD MIT1 MIT2 Overlay Tree Berkeley CMU1 MIT1 MIT2 CMU1 CMU2 CMU Berkeley MIT UCSD http://esm.cs.cmu.edu/Sigcomm2001/SigcommTalk.ppt Traditional Performance Metrics Stretch Relative Delay Penalty Stress # of identical packet over a physical link

5. Computer Science  Quick deployment  ISP’s reluctant to turn on IP Multicast  All multicast state in end systems  Routers maintain per-group state in IP Multicast  Congestion control easier on “unicast” end to end connections Potential Benefits over IP Multicast http://esm.cs.cmu.edu/Sigcomm2001/SigcommTalk.ppt MIT1 MIT2 CMU1 CMU2 CMU Berkeley MIT UCSD

6. Computer Science Past Work on Overlay Multicast  Yoid, Narada, Scattercast, Overcast, NICE, ALMI, RMX, PRM, ZIGZAG, OMNI …….  Target Application  Non-reliable Streaming Application  Design Goal  Low overhead on tree construction.  Optimize the performance metrics.  Stretch and Stress

7. Computer Science Reliable Content Delivery  Claim:  TCP on each overlay link is sufficient for reliable delivery. A B C 5Mbps 2Mbps Available Bandwidth

8. Computer Science Store-and-forward approach  Back-pressure mechanism  If the application buffer is full, ask the parent to reduce the sending rate to avoid the buffer overflow.  ALMI(USITS’01) and MCC(NGC’02). A BC D E F S 1Mbps 10Mbps 6Mbps 8Mbps Single rate Multicast Congestion Control 1Mbps 10Mbps 1Mbps

9. Computer Science ROMA Contributions  ROMA  Reliable Overlay Multicast Architecture.  Multirate.  TCP on each overlay link.  Forward-when-feasible.  Digital Fountain Approach.  Performance evaluation of the chains of TCP.  Loosely coupled TCP connections.  Conventional wisdom on overlay network is not correct.  Conventional wisdom: Increased latency and loss rate will reduce the performance of overlay node comparing to the direct unicast.

10. Computer Science Forward-when-feasible  Digital Fountain Approach. A sender encodes n packets of original content into an unbounded set of encoding packets. A receiver can reconstruct the original content by receiving any n distinct encoding packets.

11. Computer Science Overview ROMA A B C 5Mbps 2Mbps

12. Computer Science Candidate Architectures  Limited Buffer Space solution  Back-pressure mechanism  Unlimited Buffer Space Solution  Use a disk as an extra buffer.  Limitations.  A separate application buffer for each downstream.  Substantial complexity to support I/O access.  The overlay cannot be adaptively reconfigured. Memory Incoming TCP Outgoing TCP

13. Computer Science Adaptive reconfiguration  Adaptive reconfiguration of overlay network.  Reconfigure when congestion or failures of intermediate nodes occur. A BC D 10Mbps 5Mbps X 1Mbps Memory Shapeshifter BCMR’02

14. Computer Science Overlay Node Implementation Transport Layer Application Layer Incoming TCP Outgoing TCP Application Layer Buffer

15. Computer Science Modeling Chains of TCP Connections

16. Computer Science Chains of TCP Connections  Loosely coupled TCP connections.  An upstream TCP connection may or may not affect the performance of a downstream TCP.  A downstream connection never affects the performance of an upstream connection.

17. Computer Science Modeling Chains of TCP flows I  When the downstream transfer rate is slower than the upstream transfer rate.  The application layer buffer will grow without bound.  Behaves like a normal TCP driven by an application that always has data to send. A B C 5Mbps2Mbps

18. Computer Science Modeling Chains of TCP flows II  When the downstream transfer rate is faster than the upstream transfer rate.  B will periodically drain the application level buffer.  The throughput to C is limited to that of the upstream rate into B. A B C 2Mbps5Mbps

19. Computer Science Expected throughput on ROMA S 12i RTT_1 P_1 RTT_2RTT_i P_2P_i Local network condition limits the throughput of overlay node. OR the upstream connection limits the throughput of overlay node.

20. Computer Science Other Measurement Studies  S. Savage et al.  The end-to-end effects of Internet path selection, ACM SIGCOMM 1999.  There often exists detour route with lower aggregate loss rate and shorter round-trip time than IP rounte.  D. Andersen et al.  Resilient Overlay Networks, SOSP 2001.  Use detour route both to improve performance and to route around faults in the overlay. Conventional wisdom on overlay network follows this model. Increased latency and loss rate will reduce the performance of overlay node comparing to the direct unicast.

21. Computer Science Example Conservative Formula Our Formula A C9 Mbps A B C9Mbps22.2Mbps

22. Computer Science Experiments  PlanetLab  160 machines hosted by 65 sites.  Linux.  We use University hosts in the U.S.  Abilene.

23. Computer Science Multirate Reliable Multicast Slow link does not impact the performance either at upstream nodes, or at nodes in other regions of the tree Effect of Link Stress

24. Computer Science Multirate Overlay Multicast Effect of Link Stress

25. Computer Science Throughput Improvement BU UIUC GT 26.4ms 0.0049% 54.7Mbps 15.8ms 0.0295% 37.2Mbps 33ms 0.0236% 19.9Mbps

26. Computer Science How to construct the overlay tree  Construct the single-source widest-path tree.  Maximize the minimum per-hop available bandwidth to every destination.  Simple variant of Dijkstra’s algorithm.  Weight on each hop is the available bandwidth.  Select the unvisited node with the widest path from the source.  Path width is measured by the minimum of the weights on the path.

27. Computer Science Maximizing Overall Throughput

28. Computer Science Throughput Advantage

29. Computer Science Conclusion  ROMA  New architecture for reliable distribution of large content across an overlay network using TCP.  Multiple-rate reception.  Minimal amount of resources at the application layer.  Provides ability to adaptively reconfigure the topology.  Provides ability to speed up downloads with collaborative peer-to-peer transfers. [BCMR’02]  Analysis of chains of loosely coupled TCP  TCP chains offer an opportunity to increase performance.