Internet Inter-Domain Traffic Craig Labovitz, Scott Iekel-Johnson, Danny McPherson, Jon Oberheide, Farnam Jahanian Presented by: Kaushik Choudhary

Outline Introduction Data Collection Methodology ASN Traffic Analysis Application Traffic Analysis Internet Size Estimates Conclusion

Introduction “The Internet has changed dramatically over the last five years” – Cliché. Internet traffic has gone over the roof due to:

The changes Content providers (like Google) build their own global backbones. Cable internet service providers (ISPs) offer wholesale national transit. Transit ISPs offer content distribution networks (CDNs).

The changes Transition from: Fig 1: Traditional Internet logical topology.

The changes to: Fig 2: Emerging new Internet logical topology.

What is new in this paper Most studies about traffic have focused on BGP route advertisements, DNS probing, industry surveys, private CDN statistics etc. In this paper, the authors studied over 3000 peering edge routers of 110 participating internet providers over two years.

Outline Introduction Data Collection Methodology ASN Traffic Analysis Application Traffic Analysis Internet Size Estimates Conclusion

Data Collection Methodology Data collected by probes of a commercial security and traffic monitoring platform instrumenting BGP edge routers. Each probe exports traffic flow statistics that includes traffic per BGP AS, ASPath, network and transport layer protocols, countries, etc. independently to form different datasets.

Data Collection Methodology Fig 3: Netflow network probes.

Challenges in Data Collection Commercial privacy concerns. Misconfiguration of probes (resulting in wild fluctuations in daily traffic). Decommissioning of edge routers! Decommissioning of older probes and addition of new ones!

Need for Data Aggregation Wild fluctuations in the data. Heterogeneity of the providers. Ratios and percentages were consistent despite the fluctuations.

Data Aggregation The probes calculated average traffic volume every five minutes for all members of all datasets throughout every 24 hour period. They also calculated the average volume of total inter-domain network traffic. Finally, the daily traffic volume per item and network total were used to calculate the daily percentage.

Data Aggregation

Data Defects and Validation The probes did not detect traffic over peering adjacencies between enterprise business partners or in cases of similar agreements. The data collected was validated through private discussions with large content providers, transit ISPs and regional networks.

Outline Introduction Data Collection Methodology ASN Traffic Analysis Application Traffic Analysis Internet Size Estimates Conclusion

ASN Traffic Analysis RankProviderPercentage 1ISP A5.77 2ISP B4.55 3ISP C3.35 4ISP D3.2 5ISP E2.6 6ISP F2.77 7ISP G2.24 8ISP H1.82 9ISP I ISP J1.23 Table 1: Traffic contributors by weighted average percentage in July 2007.

ASN Traffic Analysis RankProviderPercentage 1ISP A9.41 2ISP B5.7 3Google5.2 4ISP F5.0 5ISP H3.22 6Comcast3.12 7ISP D3.08 8ISP E2.32 9ISP C ISP G1.89 Table 2: Traffic contributors by weighted average percentage in July 2009.

Trends Data from July 2007 is compliant with textbook diagrams of internet topology. Changes in commercial policy and traffic engineering have drastically impacted shares in Internet inter-domain traffic.

Trends RankProviderPercentage 1Google4.04 2ISP A3.74 3ISP F2.86 4Comcast1.94 5ISP K1.60 6ISP B1.36 7ISP H1.21 8ISP L0.66 9Microsoft Akamai0.06 Table 3: Providers with most significant inter-domain traffic share growth in

Google Trends Google, a content provider, now rivals global transit networks and enjoyed the highest growth. Providers and the data collected suggest that Google’s huge growth may be ascribed to the acquisition of Youtube.

Google Trends Fig 4: Google inter-domain traffic contribution.

Comcast Trends Consumer Content Fig 5: Comcast inter-domain traffic contribution.

Comcast Trends Majority of the traffic growth in Comcast came from transit traffic. What did Comcast do? – Consolidated regional backbones into a single nationwide network. – Rolled out a consumer product called “triple play” (voice, video, data). – Began offering wholesale transit, cellular backhaul and IP video distribution!

Outline Introduction Data Collection Methodology ASN Traffic Analysis Application Traffic Analysis Internet Size Estimates Conclusion

Application Traffic Analysis Applications could be identified from TCP/UDP port numbers but: – Applications could use non-standard ports. – Port-based classification only accounts the control traffic and not the often random port numbers associated with subsequent data transfer. The authors obtained validation data from five cooperating provider deployments in Asia, Europe and North America.

Largest Applications RankApplication Change 1Web Video VPN News P2P Games SSH DNS FTP Other Unclassified Table 4: Top applications by weighted average percentage.

Largest Applications AveragePercentage Web52.12 Video VPN0.24 News0.07 P2P18.32 Games0.52 SSHN/A DNSN/A FTP0.16 Other20.54 Unclassified5.51 Table 5: Average application breakdown in July 2009 across five consumer providers.

Application Traffic Changes Fig 6: Distribution of weighted average percentage of traffic from well known ports (July).

Application Traffic Changes TCP and UDP combined account for more than 95% of all inter-domain traffic. In July 2007, 52 ports contributed 60% of traffic. In July 2009, 25 ports contributed 60% of traffic. – What’s happening here?

Application Traffic Changes Internet application traffic is getting migrated to a smaller set of ports and protocols. Microsoft migrated all Xbox Live traffic to use port 80 on June 16, One reason for this is that majority of firewalls allow HTTP traffic. The other reason for this trend is the dominance of

Applications Exhibiting Growth Much of the growth in web data is due to video (Youtube uses progressive downloading). Growth in video traffic is due to the introduction of services like Hulu, Youtube, Veoh, etc.

Applications Exhibiting Growth Fig 7: Change in weighted average percentage of video protocols. Obama!

Applications Exhibiting Decline P2P dropped by 2.8% between Possible reasons – Migration to tunnelled overlays. – Use of P2P encryption. – Stealthier P2P clients and algorithms. Network operators however, suggest P2P traffic may have migrated to alternatives like direct download and streaming video.

Outline Introduction Data Collection Methodology ASN Traffic Analysis Application Traffic Analysis Internet Size Estimates Conclusion

Internet Size Estimates Fig 8: Independent inter-domain traffic volumes vs calculated aggregate ASN share..

Internet Size Estimates

Outline Introduction Data Collection Methodology ASN Traffic Analysis Application Traffic Analysis Internet Size Estimates Conclusion

This paper is the first longitudinal study of Internet inter-domain traffic. It identifies a significant ongoing evolution of provider interconnection strategies from a hierarchical to a more densely connected model. Most internet content is migrating to a relatively small number of hosting, cloud and content providers.

Conclusion Google is the largest contributor to Internet traffic growth. Majority of inter-domain traffic has migrated to a relatively small number of ports and protocols. As of July 2009, the inter-domain traffic peaks exceeded 39 Tbps and were growing at 44.5% annually.

Questions?