1. An Analysis of the P2P Traffic Characteristics on File Transfers Between Prefectures and Between Autonomous Systems in the Winny Network Nov. 1, 20101 Takashi Koizumi, Masahiro Yoshida, Satoshi Ohzahata, Konosuke Kawashima Tokyo University of Agriculture and Technology, The University of Tokyo, The University of Electro-Communications, APCC2010 1) 3)3) 2)2) 3)3) 2)2)

2. Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results Analysis between prefectures Analysis between ASs Conclusion Nov. 1, 20102APCC2010

3. Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results Analysis between prefectures Analysis between ASs Conclusion Nov. 1, 20103APCC2010

4. Introduction P2P file sharing network generates huge traffic. it has an enormous impact to backbone network. however, the technology of P2P is useful. VOD, Video streaming, etc... Analyzing P2P traffic is important. Problems: many ISPs do not make public report on their traffic data. difficulty of identifying file transfers from where did peer download files ? Nov. 1, 20104APCC2010 P2P network

5. Introduction Analysis methodology identify the file transfers between peers in Winny network. Analyze the characteristics of the traffic matrix. focus on between ASs and between Japanese Prefectures. Winny network one of the most popular P2P file sharing applications in Japan. it generates 20% of traffic in one ISP in 2008. Reveal the factor of increasing traffic from P2P file-sharing. Nov. 1, 20105APCC2010

6. Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, 20106APCC2010

7. File distribution protocol in Winny Nov. 1, 20107 Feature of Winny network - unstructured P2P network. - layered structure. - diffuse metadata of files (KEY) - 2 types of query. Feature of Winny network - unstructured P2P network. - layered structure. - diffuse metadata of files (KEY) - 2 types of query. APCC2010 KEY information file name file location(IP address, service port) file hash value lifetime of KEY (TTL) etc... (1)File uploading peer diffuses KEYs to other peers. (2)File searching peer generates search queries to find KEYs. (3)Download the file from the location of file uploading peer included in the KEY. File uploading peer File searching peer

8. Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, 20108APCC2010

9. Analyze time series of file KEY. focus on TTL and the file location in the KEY. 1500 < TTL < 2000. file location is same with the KEY having peer. Peer A Definition of the file uploading peers Nov. 1, 20109 Crawler search query query hit APCC2010 Peer A is file uploading peer : File : File KEY 1500 < TTL < 2000 file location is Peer A. TTL should be smaller than 2000 seconds to exclude spam KEYs. TTL of the KEY in file uploading peer is set to be greater than 1500 seconds.

10. Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, 201010APCC2010

11. Definition of the file transfer between peers process of the file transfer. changes of a KEY having peer to a file uploading peer. Nov. 1, 2010APCC201011 Peer APeer B Peer A is the file uploading peer Peer B is the KEY having peer Diffusion of KEY Peer B becomes a file uploading peer Peer B file transfer to Peer B from Peer A : File : File KEY Download demand Peer A with continuous crawling

12. Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, 201012APCC2010

13. Measurement environment Winny Crawling System collects KEYs. crawl more than 100,000 peers within 10 minutes. Measurement period Aug. 4, 2009 01:00 ~(12 hours) Nov. 1, 2010APCC201013 6 crawlers DB server Management server

14. Measurement environment Types of measured file. 15 popular files (selected in 1 hour crawling). books, movies and compressed files. average file size is 245MB Nov. 1, 2010APCC201014 6 crawlers DB server Management server

15. Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, 201015APCC2010

16. Measurement results Analysis of the file transfer characteristics focus on between prefectures and between ASs. Origin-Destination file transfer matrix. measurement summary about 220,000 peers, 280 ASs. about 30,000 instances of the file transfers between 33,000 peers Nov. 1, 201016 results are for 47 prefectures and top 50 ASs with number of file transfers about 15 popular files. APCC2010

17. Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, 201017APCC2010

18. File transfer traffic matrix between prefectures src prefectures on Y-axis, dst prefectures on X-axis large portion of file transfers are generated in few prefectures. Nov. 1, 201018APCC2010 Destination Prefectures Source Prefectures Tokyo(dst)Osaka(dst) Tokyo(src) Osaka(src) % highlow

19. File transfer traffic matrix between prefectures the sum of elements is 100% for each row. similar traffic distribution in every prefectures. Nov. 1, 201019 Destination Prefectures APCC2010 Source Prefectures within-prefectures highlow src and dst is same prefecture

20. Comparison of Winny and telephone traffic upper: Winny below: telephone traffic (in 2007) traffic within-prefectures Winny: 2% telephone traffic: 70% similar prefectures are selected as destinations. Tokyo, Osaka, etc. Nov. 1, 2010APCC201020 TokyoOsaka

21. Comparison of Winny and Internet upper: Winny below: Internet (in 2008)* traffic within-prefectures. Winny: 2% Internet: 3% few locality of traffic. distribution patterns are similar. Nov. 1, 201021 * Kenjiro Cho, Kensuke Fukuda, Hiroshi Esaki, Akira Kato, Jun Murai, Where does all the traffic go? – observing trends in Japanese residential traffic –, International Workshop on Protocols for Future, Large-Scale and Diverse Network Transports, 2009. APCC2010

22. Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, 201022APCC2010

23. File transfer traffic matrix between ASs src ASs on Y-axis, dst ASs on X-axis large portion of file transfers is generated by few ASs. 20 ASs generated 85% of total number of file transfers. Nov. 1, 201023 Source ASs Destination ASs APCC2010 % highlow

24. File transfer traffic matrix between ASs the sum of elements is 100% for each row. traffic distribution is similar in each AS. very poor locality of AS network in Winny. Nov. 1, 201024 Source ASs Destination ASs APCC2010 within-AS highlow

25. Outline Introduction File distribution protocol in Winny Analysis method Definition of the file uploading peers Definition of the file transfer between peers Measurement environment Measurement results analysis between prefectures analysis between ASs Conclusion Nov. 1, 201025APCC2010

26. Conclusion Analyzed traffic characteristics in Winny network. many file transfers are generated by few prefectures or ASs. since there is no mechanism to consider IP network, many file transfers are not within-prefecture or within-ASs. Future work long term measurement. analysis and measurement of other P2P networks (BitTorrent, Share, etc.). Nov. 1, 201026APCC2010

27. THANK YOU VERY MUCH FOR YOUR KIND ATTENTION. Nov. 1, 2010APCC201027

28. Percentage of file transfer in prefectures Nov. 1, 201028 top 10 prefectures accounted for 70% of the number of transfers. there were few file transfers within prefectures. APCC2010 Prefecture

29. Analysis of file transfer behavior of peers 3 cases of file transfers. Nov. 1, 2010APCC201029 From within the same prefecture (files within-prefecture) From a different prefecture (files did not exist in destination prefecture) From a different prefecture (although files did exist in destination prefecture) Prefecture A Prefecture B Prefecture A Prefecture BPrefecture A

30. Analysis of file transfer behavior of peers Nov. 1, 201030 93% of file transfers are from different prefectures although files did exist in same prefecture. it leads to increase the traffic between prefecture. APCC2010 the sum of inbound file transfer is 100% in each prefectures.

31. Percentage of file transfer in ASs Nov. 1, 201031 within-AS file transfers: 9%. 85 % of file transfers are generated by top 20 AS. result shows only top 50 ASs with number of file transfer. APCC2010 AS

32. Percentage of file transfer in ASs Nov. 1, 201032 sorted with times of file transfer result shows only top 50 ASs with number of file transfer. APCC2010 AS

33. Analysis of file transfer behavior of peers Nov. 1, 201033 the sum of inbound file transfer is 100% in each ASs. even if the file exists in same AS, most file transfers are from different ASs. it leads to increase of the AS traffic. APCC2010

34. Discussion Problem Small local region ASs communicate with large ASs. network latency and increasing traffic Considering the strategies both caching and peering between ASs. Choosing the optimal location of cache servers Building local region IXes. Nov. 1, 2010APCC201034

35. Discussion Problem in Winny network. peer selecting algorithm difficult for unstructured P2P networks. no central management servers. caching mechanism contribute file distribution in a Winny network. cannot effectively reduce the traffic. optimal caching and neighbor selection is important. geographical cost and IP network topology network characteristics of peers and files Nov. 1, 2010APCC201035