Evolution 0.9: The Evolution of the U.S. Peering Ecosystem Gigabit Peering Forum VII Herndon, VA September 9, 2003 William B. Norton Co-Founder & Chief Technical Liaison Equinix, Inc.

Internet Researcher Four Years of Research Document Internet Operations Practices – Interconnect/Peering/IXes/etc. Research Process – Focus with Community and –Write White Paper version M.m –Walk throughs & Feedback –Fix/Update Paper 200 Walkthroughs later – a document reflecting community Internet Operations knowledge So far 6 White Papers

Internet Operations White Papers 1)“Interconnection Strategies for ISPs” 2)“Internet Service Providers and Peering” 3)“A Business Case for Peering” 4)“The Art of Peering: The Peering Playbook” 5)“The Peering Simulation Game” 6)“Do ATM-based Internet Exchanges Make Sense Anymore?” Freely available. Send to New white Paper: The Evolution of the U.S. Peering Ecosystem, Agenda…

Agenda The Evolution of the Internet Peering Ecosystem –The Internet Peering Ecosystem Environment –Each Player, their Motivations, their Behavior Four Major Events, led to … Three Major Evolutions in the Peering Ecosystem

The Evolution of the U.S. Peering Ecosystem Act I:

The Internet Peering Ecosystem The Environment The Players –Their role in the Peering Ecosystem –Their behavior in the Peering Ecosystem –Their relationship to others in the Peering Ecosystem The Evolution of the Peering Ecosystem

The Internet Peering Ecosystem Many “Internet Regions” (usually defined by geopolitical boundaries) Each with its own Peering Ecosystem consisting of –Tier 1 ISPs –Tier 2 ISPs –Content Players and Enterprises

Interconnect Regions Large Internet Regions have multiple geographical areas where ISPs peer

Player: Tier 1 ISPs Def: A Tier 1 ISP is an ISP that has access to the entire Regional Internet routing table solely through Peering Relationships (I.e. doesn’t buy transit from anyone).

Tier 1 Peering Full Mesh Peering among Tier 1 ISPs within each Interconnect Region

Some Tier 1 ISPs in the U.S.

Tier 1 Peering Across the U.S. 8 Interconnect Regions

Tier 1 Peering Motivations Generally not interested to peer with anyone else 1)They already get the traffic for free 2)They may be foregoing revenue (they could sell transit) 3)They don’t want to give away that which differentiates them

Player: Tier 2 ISP Def: A Tier 2 ISP is an ISP that purchases (and therefore resells) transit within an Internet Region.

Tier 2 ISP Peering Motivations 1)Reduce transit costs 2)Improve Performance 3)Greater control over routing

Tier 2 Peering Sparse Mesh Peering in each Interconnect Region

Tier 2 ISPs Widely varying characteristics and behaviors Scope: Global, National, Regional, Local Underfunded to well funded Food fights to solid relationships within the community Big Peoples Table and Little Peoples Table perhaps a bad generalization

The Content Provider and Enterprise Player Always buy transit Focus on Content Focus on Service Few Network Engineers SLAs valuable

The Internet Peering Ecosystem

Peering Inclinations and Peering Policies Def: A Peering Inclination is a predilection towards or against peering as demonstrated by Peering behavior in a Peering Ecosystem Def: A Peering Policy documents and defines the prerequisites to peering.

Four types of Peering Inclination Open Peering – Anyone, Anywhere Selective Peering – Some Requirements Restrictive Peering – No No Peering – No, prefer buying transit

The Evolution of the U.S. Peering Ecosystem Act II:

Evolution of the U.S. Peering Ecosystem Four key events leading to drastic disruption in the Peering Ecosystem: 1)The 1999/2000 Telecom Collapse 2)The growth of the Used Equipment Market 3)The Upstream Provider for the Cable Companies went bankrupt 4)Peer-to-Peer file sharing systems (like Kazaa) grow in popularity, traffic grows exponentially between Access Providers (1.5MB MP3  700MB AVI files) Leads to 3 Major Evolutions

Evolution #1 – Cable Companies are Peering 40% of traffic Kazaa 3-5 Gbps of transit traffic-> Gbps of peering potential ! Significant because 1)Volume 2)Open Peering 3)Kazaa Effect

U.S. Cable Companies Eyeballs

AOL

Content Tier 2 ISPs Tier 1 ISPs Transit $ Thickness represents In-Group Peering

Content Evolution #1 – Cable Companies Peering Tier 2 ISPs CableCos Tier 1 ISPs Peering

Evolution #2 – Network Savvy Large Scale Content Companies get into Peering To reduce transit costs To improve the end user experience They need to move out of bankrupt colo anyway Significant because 1)Volume of traffic is huge 2)Content Providers have Open Peering 3)Leading players pave the way

Content Tier 2 ISPs Tier 1 ISPs Transit $ Thickness represents In-Group Peering

Content Evolution #2 – Large Scale Network Savvy Content Peers Tier 2 ISPs LSNSC Tier 1 ISPs

Other Broadband Players Significant, but… Not Open Peers so disincentive to peer So less volume and less disruptive

Evolution #3 – Cable Companies Freely Peer with Content Companies Content literally on the Cable Company Network Lowest possible latency from content to eyeballs Internet Gaming, Broadband Streaming, etc.

Content Tier 2 ISPs Tier 1 ISPs Transit $ Thickness represents In-Group Peering

Content Evolution #1 – Cable Companies Peering Tier 2 ISPs CableCos Tier 1 ISPs

Content Evolution #2 – Large Scale Network Savvy Content Peers Tier 2 ISPs LSNSC Tier 1 ISPs

Content Evolution #3 – Cable Peers with Large Scale Network Savvy Content Tier 2 ISPs LSNSC Tier 1 ISPs CableCos

International Dynamics – Separate White Paper JPU.S. U.S. Tier 1 ISPs are Tier 2 ISPs in Japan Internet Region Japan Tier 1 ISPs are Tier 2 ISPs in the U.S. Internet Region

Summary 2003 Peering Ecosystem Evolving NoPeeringOpenSelectiveRestrictive Lg.Content Tier2 ISPs CableCos RBOCs Tier 1 ISPs Foreign ISPs