1. 1 Internet Topology COS 461: Computer Networks Spring 2007 (MW 1:30-2:50 in Friend 004) Jennifer Rexford Teaching Assistant: Ioannis Avramopoulos http://www.cs.princeton.edu/courses/archive/spring07/cos461/

2. 2 Returning the Midterm Exam Exam scoring break down –Average: 90 –High: 99 –Median: low 90s See the course Web site –Exam –Answer key

3. 3 Goals of Today’s Lecture IP routers –Interface cards –Switching fabric –Route processor Router-level topology with a single network –Points of Presence (PoPs) –Backbone and enterprise network topologies AS-level topology of the Internet –Autonomous System (AS) numbers –Tier-1 ISPs, regional providers, and stub ASes –Business relationships between ASes

4. 4 IP Routers

5. 5 Inside a High-End Router Switching Fabric Processor Line card

6. 6 Router Physical Layout Juniper T series Cisco 12000 Crossbar Linecards

7. 7 Line Cards (Interface Cards, Adaptors) Interfacing –Physical link –Switching fabric Packet handling –Packet forwarding –Decrement time-to-live –Buffer management –Link scheduling –Packet filtering –Rate limiting –Packet marking –Measurement to/from link to/from switch lookup Receive Transmit

8. 8 Switching Fabric Deliver packet inside the router –From incoming interface to outgoing interface –A small network in and of itself Must operate very quickly –Multiple packets going to same outgoing interface –Switch scheduling to match inputs to outputs Implementation techniques –Bus, crossbar, interconnection network, … –Running at a faster speed (e.g., 2X) than links –Dividing variable-length packets into cells

9. 9 Packet Switching R1 Link 1 Link 2 Link 3 Link 4 Link 1, ingressLink 1, egress Link 2, ingressLink 2, egress Link 3, ingressLink 3, egress Link 4, ingressLink 4, egress Choose Egress Choose Egress Choose Egress Choose Egress “4”

10. 10 Router Processor So-called “Loopback” interface –IP address of the CPU on the router Control-plane software –Implementation of the routing protocols –Creation of forwarding table for the line cards Interface to network administrators –Command-line interface for configuration –Transmission of measurement statistics Handling of special data packets –Packets with IP options enabled –Packets with expired Time-To-Live field

11. 11 Error Reporting Examples of errors a router may see –Router doesn’t know where to forward a packet –Packet’s time-to-live field expires Router doesn’t really need to respond –Best effort means never having to say you’re sorry –So, IP could conceivably just silently drop packets But, silent failures are really hard to diagnose –IP includes basic feedback about network problems –Internet Control Message Protocol (ICMP)

12. 12 Internet Control Message Protocol ICMP runs on top of IP –In parallel to TCP and UDP –Though still viewed as an integral part of IP Diagnostics –Triggered when an IP packet encounters a problem  E.g., time exceeded or destination unreachable –ICMP packet sent back to the source IP address  Includes the error information (e.g., type and code)  … and an excerpt of the original data packet for identification –Source host receives the ICMP packet  And inspects the except of the packet (e.g., protocol and ports)  … to identify which socket should receive the error

13. 13 Example: Time Exceeded hostDNS... host DNS... router host 1.2.3.7 8.9.10.11 5.6.7.156 Host sends an IP packet –Each router decrements the time-to-live field If time-to-live field reaches 0 –Router generates an ICMP message –Sends a “time exceeded” message back to the source Time exceeded

14. 14 Ping: Echo and Reply ICMP includes a simple “echo” function –Sending node sends an ICMP “echo” message –Receiving node sends an ICMP “echo reply” Ping tool –Tests the connectivity with a remote host –… by sending regularly spaced echo commands –… and measuring the delay until receiving the reply Pinging a host –“ping www.cs.princeton.edu” or “ping 12.157.34.212” –Used to test if a machine is reachable and alive –(However, some nodes have ICMP disabled…  )

15. 15 Router-level topology of a network

16. 16 Intra-AS Topology Node: router Edge: link

17. 17 Hub-and-Spoke Topology Single hub node –Common in enterprise networks –Main location and satellite sites –Simple design and trivial routing Problems –Single point of failure –Bandwidth limitations –High delay between sites –Costs to backhaul to hub

18. 18 Princeton Example Hub-and-spoke –Four hub routers and many spokes Hub routers –Outside world (e.g., AT&T, USLEC, …) –Dorms –Academic and administrative buildings –Servers

19. 19 Simple Alternatives to Hub-and-Spoke Dual hub-and-spoke –Higher reliability –Higher cost –Good building block Levels of hierarchy –Reduce backhaul cost –Aggregate the bandwidth –Shorter site-to-site delay …

20. 20 Backbone Networks Backbone networks –Multiple Points-of-Presence (PoPs) –Lots of communication between PoPs –Accommodate traffic demands and limit delay

21. 21 Abilene Internet2 Backbone

22. 22 Points-of-Presence (PoPs) Inter-PoP links –Long distances –High bandwidth Intra-PoP links –Short cables between racks or floors –Aggregated bandwidth Links to other networks –Wide range of media and bandwidth Intra-PoP Other networks Inter-PoP

23. 23 Where to Locate Nodes and Links Placing Points-of-Presence (PoPs) –Large population of potential customers –Other providers or exchange points –Cost and availability of real-estate –Mostly in major metropolitan areas (“NFL cities”) Placing links between PoPs –Already fiber in the ground –Needed to limit propagation delay –Needed to handle the traffic load

24. 24 AS-level topology of the Internet

25. 25 Internet Routing Architecture Divided into Autonomous Systems –Distinct regions of administrative control –Routers/links managed by a single “institution” –Service provider, company, university, … Hierarchy of Autonomous Systems –Large, tier-1 provider with a nationwide backbone –Medium-sized regional provider with smaller backbone –Small network run by a single company or university Interaction between Autonomous Systems –Internal topology is not shared between ASes –… but, neighboring ASes interact to coordinate routing

26. 26 Autonomous System Numbers AS Numbers are 16 bit values. Level 3: 1 MIT: 3 Harvard: 11 Yale: 29 Princeton: 88 AT&T: 7018, 6341, 5074, … UUNET: 701, 702, 284, 12199, … Sprint: 1239, 1240, 6211, 6242, … … Currently just over 20,000 in use.

27. 27 AS Topology Node: Autonomous System Edge: Two ASes that connect to each other 1 2 3 4 5 6 7

28. 28 What is an Edge, Really? Edge in the AS graph –At least one connection between two ASes –Some destinations reached from one AS via the other AS 1 AS 2 d Exchange Point AS 1 AS 2 d AS 3

29. 29 Interdomain Paths 1 2 3 4 5 6 7 Client Web server Path: 6, 5, 4, 3, 2, 1

30. 30 Business Relationships Neighboring ASes have business contracts –How much traffic to carry –Which destinations to reach –How much money to pay Common business relationships –Customer-provider  E.g., Princeton is a customer of AT&T  E.g., MIT is a customer of Level 3 –Peer-peer  E.g., Princeton is a peer of Patriot Media  E.g., AT&T is a peer of Sprint

31. 31 Customer-Provider Relationship Customer needs to be reachable from everyone –Provider tells all neighbors how to reach the customer Customer does not want to provide transit service –Customer does not let its providers route through it d d provider customer provider Traffic to the customerTraffic from the customer advertisements traffic

32. 32 Customer Connecting to a Provider Provider 1 access link 2 access links Provider 2 access routers Provider 2 access PoPs

33. 33 Multi-Homing: Two or More Providers Motivations for multi-homing –Extra reliability, survive single ISP failure –Financial leverage through competition –Better performance by selecting better path –Gaming the 95 th -percentile billing model Provider 1 Provider 2

34. 34 Princeton Example Internet: customer of AT&T and USLEC Research universities/labs: customer of Internet2 Local non-profits: provider for several non-profits AT&T USLEC Internet2

35. 35 Peer-Peer Relationship Peers exchange traffic between customers –AS exports only customer routes to a peer –AS exports a peer’s routes only to its customers –Often the relationship is settlement-free (i.e., no $$$) peer Traffic to/from the peer and its customers d advertisements traffic

36. 36 AS Structure: Tier-1 Providers Tier-1 provider –Has no upstream provider of its own –Typically has a national or international backbone –UUNET, Sprint, AT&T, Level 3, … Top of the Internet hierarchy of 12-20 ASes –Full peer-peer connections between tier-1 providers

37. 37 Efficient Early-Exit Routing Diverse peering locations –Both costs, and middle Comparable capacity at all peering points –Can handle even load Consistent routes –Same destinations advertised at all points –Same AS path length for a destination at all points Customer A Customer B multiple peering points Provider A Provider B Early-exit routing

38. 38 AS Structure: Other ASes Tier-2 providers –Provide transit service to downstream customers –… but, need at least one provider of their own –Typically have national or regional scope –E.g., Minnesota Regional Network –Includes a few thousand of the ASes Stub ASes –Do not provide transit service to others –Connect to one or more upstream providers –Includes vast majority (e.g., 85-90%) of the ASes

39. 39 Characteristics of the AS Graph AS graph structure –High variability in node degree (“power law”) –A few very highly-connected ASes –Many ASes have only a few connections 1 10 100 1000 CCDF 1 0.1 0.01 0.001 AS degree All ASes have 1 or more neighbors Very few have degree >= 100

40. 40 Characteristics of AS Paths AS path may be longer than shortest AS path Router path may be longer than shortest path s d 3 AS hops, 7 router hops 2 AS hops, 8 router hops

41. 41 Shared Risks Co-location facilities (“co-lo hotels”) –Places ISPs meet to connect to each other –… and co-locate their routers, and share space & power –E.g., 32 Avenue of the Americas in NYC Shared links –Fiber is sometimes leased by one institution to another –Multiple fibers run through the same conduits –… and run through the same tunnels, bridges, etc. Difficult to identify and accounts for these risks –Not visible in network-layer measurements –E.g., traceroute does not tell you links in the same ditch

42. 42 Conclusions Internet topology –Network inside a router –Network inside an AS –Network connecting ASes Coming up: two-tiered routing system –Intradomain routing –Interdomain routing Reading – 4.2, 4.3.3, 4.3.4