1. Peer-Assisted Content Distribution Networks: Techniques and Challenges Pei Cao Stanford University

2. Traditional Intra-Provider Content Distribution Networks National Center Regional Center Branch... Users...

3. Peer-to-Peer Content Distribution National Center Regional Center Branch... Users...

4. P2P vs CDN P2P: P2P: –No infrastructure cost –Supply grows linearly with demand –Simple distributed, randomized algorithms –No QoS CDN: CDN: –Initial infrastructure cost –Centralized scheduling algorithms –Network efficiency –Capable of supporting QoS

5. Combine P2P with CDN? Use P2P to complement CDN Use P2P to complement CDN –P2P reduces load on the CDN, covers areas where CDN is not installed –Must be able to control, or “shape”, P2P traffic Use CDN to complement P2P Use CDN to complement P2P –CDN steps in when peer-based distribution is falling short, enabling QoS –Must be able to detect when peers won’t meet the delivery time guarantee

6. Outline Review of BitTorrent Review of BitTorrent Traffic-shaping BitTorrent: biased neighbor selection Traffic-shaping BitTorrent: biased neighbor selection QoS in BitTorrent: delivery time prediction QoS in BitTorrent: delivery time prediction

7. BitTorrent File Sharing Network Goal: replicate K chunks of data among N nodes Form neighbor connection graph Form neighbor connection graph Neighbors exchange data Neighbors exchange data

8. BitTorrent: Neighbor Selection Tracker file.torrent 1 Seed Whole file A 5 2 3 4

9. BitTorrent: Piece Replication Tracker file.torrent 1 Seed Whole file A 5 3

10. BitTorrent: Piece Replication Algorithms “Tit-for-tat” (choking/unchoking): “Tit-for-tat” (choking/unchoking): –Each peer only uploads to 7 other peers at a time –6 of these are chosen based on amount of data received from the neighbor in the last 20 seconds –The last one is chosen randomly, with a 75% bias toward newcomers (Local) Rarest-first replication: (Local) Rarest-first replication: –When peer 3 unchokes peer A, A selects which piece to download

11. Analysis of BitTorrent Conclusion from modeling studies: BitTorrent is nearly optimal in idealized, homogeneous networks Conclusion from modeling studies: BitTorrent is nearly optimal in idealized, homogeneous networks –Demonstrated by simulation studies –Confirmed by theoretical modeling studies Intuition: in a random graph, Intuition: in a random graph, Prob(Peer A’s content is a subset of Peer B’s) ≤ 50%

12. Traffic-Shaping BitTorrent

13. Random Neighbor Graph Existing studies all assume random neighbor selection Existing studies all assume random neighbor selection –BitTorrent no longer optimal if nodes in the same ISP only connect to each other Random neighbor selection  high cross- ISP traffic Random neighbor selection  high cross- ISP traffic

14. Difficulty in Traffic-Shaping P2P Applications ISPs: ISPs: –Different links have different monetary costs –Prefer “clustering” of traffic P2P Applications: P2P Applications: –No knowledge of underlying ISP topology –Use randomized algorithms that don’t do well under clustering Current solution: throttling  users suffer Current solution: throttling  users suffer

15. A Network-Friendly BitTorrent? ISPs inform BitTorrent of its link preferences ISPs inform BitTorrent of its link preferences Algorithm of BitTorrent is adjusted such that both users and ISPs benefit Algorithm of BitTorrent is adjusted such that both users and ISPs benefit Example: Biased Neighbor Selection Example: Biased Neighbor Selection –Works when cost function is transitive

16. Biased Neighbor Selection Idea: of N neighbors, choose N-k from peers in the same ISP, and choose k randomly from peers outside the ISP Idea: of N neighbors, choose N-k from peers in the same ISP, and choose k randomly from peers outside the ISP ISP

17. Implementing Biased Neighbor Selection By Tracker By Tracker –Need ISP affiliations of peers Peer to AS maps Peer to AS maps Public IP address ranges from ISPs Public IP address ranges from ISPs Special “X-” HTTP header Special “X-” HTTP header By traffic shaping devices By traffic shaping devices –Intercept “peer  tracker” messages and manipulate responses –No need to change tracker or client

18. Evaluation Methodology Event-driven simulator Event-driven simulator –Use actual client and tracker codes as much as possible –Calculate bandwidth contention, assume perfect fair- share from TCP Network settings Network settings –14 ISPs, each with 50 peers, 100Kb/s upload, 1Mb/s download –Seed node, 400Kb/s upload –Optional “university” nodes (1Mb/s upload) –Optional ISP bottleneck to other ISPs

19. Limitation of Throttling

20. Throttling: Cross-ISP Traffic Redundancy: Average # of times a data chunk enters the ISP

21. Biased Neighbor Selection: Download Times

22. Biased Neighbor Selection: Cross- ISP Traffic

23. Importance of Rarest-First Replication Random piece replication performs badly Random piece replication performs badly –Increases download time by 84% - 150% –Increase traffic redundancy from 3 to 14 Biased neighbors + Rarest-First  More uniform progress of peers Biased neighbors + Rarest-First  More uniform progress of peers

24. Presence of External High- Bandwidth Peers Biased neighbor selection alone: Biased neighbor selection alone: –Average download time same as regular BitTorrent –Cross-ISP traffic increases as # of “university” peers increase Result of tit-for-tat Result of tit-for-tat Biased neighbor selection + Throttling: Biased neighbor selection + Throttling: –Download time only increases by 12% Most neighbors do not cross the bottleneck Most neighbors do not cross the bottleneck –Traffic redundancy (i.e. cross-ISP traffic) same as the scenario without “university” peers

25. Comparison with Simple Clustering Gateway peer: only one peer connects to the peers outside the ISP, all other peers only connect to peers inside the ISP Gateway peer: only one peer connects to the peers outside the ISP, all other peers only connect to peers inside the ISP –Gateway peer must have high bandwidth It is the “seed” for this ISP It is the “seed” for this ISP –Ends up benefiting peers in other ISPs

26. Combining Biased Neighbor Selection with Caches Under random neighbor selection Under random neighbor selection –bandwidth requirement of cache is high Under biased neighbor selection Under biased neighbor selection –bandwidth needed from the cache is reduced by an order of magnitude

27. Conclusions By choosing neighbors well, BitTorrent can achieve high peer performance without increasing ISP cost By choosing neighbors well, BitTorrent can achieve high peer performance without increasing ISP cost –Biased neighbor selection: choose initial set of neighbors well –Can be combined with throttling and caching  BitTorrent’s algorithm can be shaped!

28. Delivery Time Prediction

29. Motivation Provide delivery time guarantee under P2P+CDN Provide delivery time guarantee under P2P+CDN What contributes to delivery time of a download via BitTorrent? What contributes to delivery time of a download via BitTorrent? –From simulations: seed bandwidth and even replication of blocks –Missing: node join/leave dynamics, TCP effects, etc.

30. Side-by-Side Live Experiments Two clients, running on the same machine, starting at the same time, downloading the same Two clients, running on the same machine, starting at the same time, downloading the same 13 experiments from Apr-May 2006 13 experiments from Apr-May 2006 File sizes: 700MB ~ 1.4GB File sizes: 700MB ~ 1.4GB Network size: 1100 ~ 2100 peers Network size: 1100 ~ 2100 peers Duration: 10 hrs ~ 2 days Duration: 10 hrs ~ 2 days

31. Results from Experiments Effective download rate: 10 ~ 30KB/s Effective download rate: 10 ~ 30KB/s Speed difference between the two peers: 3% ~ 82% Speed difference between the two peers: 3% ~ 82% What made the slower peer slow? What made the slower peer slow?

32. Suspicion #1: Slower Neighbors? Calculate unweighted average of observed throughput at application level Calculate unweighted average of observed throughput at application level –R 1 : average from all neighbors –R 2 : average from neighbors uploading >250KB of data –R 3 : average from neighbors uploading >2.5MB of data Low correlation between download-time ratio and neighbor-speed ratio Low correlation between download-time ratio and neighbor-speed ratio –0.57 for R 1, 0.43 for R 2, 0.47 for R 3 –Faster neighbors corresponds to slower downloads in 3 experiments

33. Suspicion #2: Fewer Neighbors Uploading to the Peer? Slot analysis: calculate download concurrency Slot analysis: calculate download concurrency –Maximum number of neighbors: 35 –Neighbors come and go  align neighbors into 35 slots –Calculate time-average of number of concurrent slots with neighbors uploading Upload concurrency varies from 7 to 11 Upload concurrency varies from 7 to 11 –Explains one of the download-time/neighbor-speed reversal case –But doesn’t explain the two others

34. “Close” Neighbors 90% of data downloaded from 1-4% of neighbors 90% of data downloaded from 1-4% of neighbors Let F(p) and G(p) be the number of neighbors that provides p of data to peers F and G, then Let F(p) and G(p) be the number of neighbors that provides p of data to peers F and G, then F(p) > G(p)  peer F is slower than G F(p) > G(p)  peer F is slower than G –This holds for p = 90%, 75%, and 50%

35. What makes a neighbor close? Not related to speed, or order of connection to peer, or order of unchoking by peer Not related to speed, or order of connection to peer, or order of unchoking by peer

36. Cost of Departure of a Close Neighbor Departure cost: if one close neighbor leaves, calculate the time until the earliest next close neighbor Departure cost: if one close neighbor leaves, calculate the time until the earliest next close neighbor The average departure cost: 30 min The average departure cost: 30 min  The convergence time of the tit-for-tat algorithm is slow

37. Why Do Close Neighbors Leave Five possible reasons Five possible reasons –A: Random disconnect –B: Finished downloading –C: Peer broke off the relationship –D: Neighbor broke off the relationship Results: B is most common, followed by C/D, then A Results: B is most common, followed by C/D, then A

38. Conclusions Content delivery time in BitTorrent is determined by: Content delivery time in BitTorrent is determined by: –Neighbor upload speed –Stability of neighbor relationship Disruption of the pairing leads to long delivery time Disruption of the pairing leads to long delivery time Neighbors may leave due to random disconnection, completion of download, or finding faster neighbors Neighbors may leave due to random disconnection, completion of download, or finding faster neighbors

39. Using CDN to Complement P2P Use nodes CDN as high-speed specially managed seeds Use nodes CDN as high-speed specially managed seeds Seeds are called to help whenever a node loses a close neighbor Seeds are called to help whenever a node loses a close neighbor

40. Summary A way to shape BitTorrent traffic A way to shape BitTorrent traffic Predicting BitTorrent performance by monitoring close peer relationship Predicting BitTorrent performance by monitoring close peer relationship

41. Related Work Many modeling studies of BitTorrent Many modeling studies of BitTorrent Simulation studies Simulation studies Measurements of real torrents Measurements of real torrents

42. Ongoing Work Live experiments with biased neighbor selections Live experiments with biased neighbor selections A k-regular graph algorithm with faster convergence A k-regular graph algorithm with faster convergence Prototype implementation of “P2P+CDN” Prototype implementation of “P2P+CDN”