1. Nick Feamster Georgia Tech
Advanced Routing
Nick Feamster Georgia Tech

2. Tutorial Outline Topology BGP IS-IS Business relationships
BGP/MPLS VPNs

3. Internet Routing Overview
Autonomous Systems (ASes)
Abilene
Comcast
Georgia
Tech
AT&T
Cogent
Today: Intradomain (i.e., “intra-AS”) routing
Monday: Interdomain routing

4. Today: Routing Inside an AS
Intra-AS topology
Nodes and edges
Example: Abilene
Intradomain routing protocols
Distance Vector
Split-horizon/Poison-reverse
Example: RIP
Link State
Example: OSPF, ISIS

5. Topology Design Where to place “nodes”? Where to place “edges”?
Typically in dense population centers
Close to other providers (easier interconnection)
Close to other customers (cheaper backhaul)
Note: A “node” may in fact be a group of routers, located in a single city. Called a “Point-of-Presence” (PoP)
Where to place “edges”?
Often constrained by location of fiber

6. Node Clusters: Point-of-Presence (PoP)
A “cluster” of routers in a single physical location
Inter-PoP links
Long distances
High bandwidth
Intra-PoP links
Cables between racks or floors
Aggregated bandwidth
PoP

7. Example: Abilene Network Topology

8. Another Example Backbone

9. Problem: Routing
Routing: the process by which nodes discover where to forward traffic so that it reaches a certain node
Within an AS: there are two “styles”
Distance vector: iterative, asynchronous, distributed
Link State: global information, centralized algorithm

10. Forwarding vs. Routing Forwarding: data plane Routing: control plane
Directing a data packet to an outgoing link
Individual router using a forwarding table
Routing: control plane
Computing paths the packets will follow
Routers talking amongst themselves
Individual router creating a forwarding table

11. Distance-Vector Routingx y z 1 2 y x z 1 2 5 x y z 1 5 x y z 5 2
Routers send routing table copies to neighbors
Routers compute costs to destination based on shortest available path
Based on Bellman-Ford Algorithm
dx(y) = minv{ c(x,v) + dv(y) }
Solution to this equation is x’s forwarding table

12. Distance Vector Algorithm
Each node:
Iterative, asynchronous: each local iteration caused by:
Local link cost change
Distance vector update message from neighbor
Distributed:
Each node notifies neighbors only when its DV changes
Neighbors then notify their neighbors if necessary
wait for (change in local link cost or message from neighbor)
recompute estimates
if DV to any destination has changed, notify neighbors

13. Good News Travels Quicklyx y z 1 3 2 y x z 1 2 5 x y z 1 3 2 x y z 1 3 2
When costs decrease, network converges quickly

14. Problem: Bad News Travels Slowlyx y z 60 50 5 2 3 y x z 1 2 50 60 x y z 60 50 5 2 7
Note also that there is a forwarding loop between y and z.

15. This continues… 60 1 2 50 Question: How long does this continue?x y z 60 50 5 2 3 y x z 1 2 50 60 x y z 60 50 5 2 7
Question: How long does this continue?
Answer: Until z’s path cost to x via y is greater than 50.

16. “Solution”: Poison Reversex y z 1 X 2 y 1 x y z 1 3 2 2 x y z 1 3 2 x z 5
If z routes through y to get to x, z advertises infinite cost for x to y
Does poison reverse always work?

17. Does Poison Reverse Always Work?x z 1 3 50 60 w

18. Routing Information Protocol (RIP)
Distance vector protocol
Nodes send distance vectors every 30 seconds
… or, when an update causes a change in routing
Link costs in RIP
All links have cost 1
Valid distances of 1 through 15
… with 16 representing infinity
Small “infinity”  smaller “counting to infinity” problem

19. Link-State Routing Keep track of the state of incident links
Whether the link is up or down
The cost on the link
Broadcast the link state
Every router has a complete view of the graph
Compute Dijkstra’s algorithm
Examples:
Open Shortest Path First (OSPF)
Intermediate System – Intermediate System (IS-IS)

20. Link-State Routing Idea: distribute a network map
Each node performs shortest path (SPF) computation between itself and all other nodes
Initialization step
Add costs of immediate neighbors, D(v), else infinite
Flood costs c(u,v) to neighbors, N
For some D(w) that is not in N
D(v) = min( c(u,w) + D(w), D(v) )

21. Detecting Topology Changes
Beaconing
Periodic “hello” messages in both directions
Detect a failure after a few missed “hellos”
Performance trade-offs
Detection speed
Overhead on link bandwidth and CPU
Likelihood of false detection
“hello”

22. Broadcasting the Link State
Flooding
Node sends link-state information out its links
The next node sends out all of its links except the one where the information arrived X A X A C B D C B D
(a)
(b) X A X A C B D C B D
(c)
(d)

23. Broadcasting the Link State
Reliable flooding
Ensure all nodes receive the latestlink-state information
Challenges
Packet loss
Out-of-order arrival
Solutions
Acknowledgments and retransmissions
Sequence numbers
Time-to-live for each packet

24. When to Initiate Flooding
Topology change
Link or node failure
Link or node recovery
Configuration change
Link cost change
Periodically
Refresh the link-state information
Typically (say) 30 minutes
Corrects for possible corruption of the data

25. Scaling Link-State Routing
Message overhead
Suppose a link fails. How many LSAs will be flooded to each router in the network?
Two routers send LSA to A adjacent routers
Each of A routers sends to A adjacent routers …
Suppose a router fails. How many LSAs will be generated?
Each of A adjacent routers originates an LSA …

26. Scaling Link-State Routing
Two scaling problems
Message overhead: Flooding link-state packets
Computation: Running Dijkstra’s shortest-path algorithm
Introducing hierarchy through “areas”
Area 0
area
border
router

27. Link-State vs. Distance-Vector
Convergence
DV has count-to-infinity
DV often converges slowly (minutes)
DV has timing dependences
Link-state: O(n2) algorithm requires O(nE) messages
Robustness
Route calculations a bit more robust under link-state
DV algorithms can advertise incorrect least-cost paths
In DV, errors can propagate (nodes use each others tables)
Bandwidth Consumption for Messages
Messages flooded in link state

28. Open Shortest Paths First (OSPF)
Area 0
Key Feature: hierarchy
Network’s routers divided into areas
Backbone area is area 0
Area 0 routers perform SPF computation
All inter-area traffic travles through Area 0 routers (“border routers”)

29. Another Example: IS-IS
Originally: ISO Connectionless Network Protocol
CLNP: ISO equivalent to IP for datagram delivery services
ISO or RFC 1142
Later: Integrated or Dual IS-IS (RFC 1195)
IS-IS adapted for IP
Doesn’t use IP to carry routing messages
OSPF more widely used in enterprise, IS-IS in large service providers

30. Hierarchical Routing in IS-IS
Backbone
Area
Area
Level-1
Routing
Level-1
Routing
Level-2
Routing
Like OSPF, 2-level routing hierarchy
Within an area: level-1
Between areas: level-2
Level 1-2 Routers: Level-2 routers may also participate in L1 routing

31. ISIS on the Wire…

32. IS-IS Configuration on Abilene (atlang)
lo0 {
unit 0 { ….
family iso {
address ; }
isis {
level 2 wide-metrics-only;
/* OC192 to WASHng */
interface so-0/0/0.0 {
level 2 metric 846;
level 1 disable;
ISO Address Configured on Loopback Interface
Only Level 2 IS-IS in Abilene

33. Interdomain Routing Today’s interdomain routing protocol: BGP
BGP route attributes
Usage
Problems
Business relationships
See (Chapter ) for good coverage of this topic.

34. Internet Routing The Internet
Abilene
Georgia
Tech
Comcast
AT&T
Cogent
Large-scale: Thousands of autonomous networks
Self-interest: Independent economic and performance objectives
But, must cooperate for global connectivity

35. Internet Business Model (Simplified)
Provider
Preferences implemented with local preference manipulation
Free to use
Pay to use
Peer
Get paid to use
Customer
Destination
Customer/Provider: One AS pays another for reachability to some set of destinations
“Settlement-free” Peering: Bartering. Two ASes exchange routes with one another.

36. Relationship #1: Customer-Provider
Filtering
Routes from customer: to everyone
Routes from provider: only to customers
From other destinations
To the customer
From the customer
To other destinations
providers
providers
advertisements
traffic
customer
customer

37. Relationship #2: Peering
Filtering
Routes from peer: only to customers
No routes from other peers or providers
advertisements
peer
peer
traffic
customer
customer

38. The Business Game and Depeering
Cooperative competition (brinksmanship)
Much more desirable to have your peer’s customers
Much nicer to get paid for transit
Peering “tiffs” are relatively common
31 Jul 2005: Level 3 Notifies Cogent of intent to disconnect.
16 Aug 2005: Cogent begins massive sales effort and
mentions a 15 Sept. expected depeering date.
31 Aug 2005: Level 3 Notifies Cogent again of intent to
disconnect (according to Level 3)
5 Oct :50 UTC: Level 3 disconnects Cogent. Mass
hysteria ensues up to, and including policymakers in
Washington, D.C.
7 Oct 2005: Level 3 reconnects Cogent
During the “outage”, Level 3 and Cogent’s singly homed customers could not reach each other. (~ 4% of the Internet’s prefixes were isolated from each other)

39. Depeering Continued Resolution…
…but not before an attempt to steal customers!
As of 5:30 am EDT, October 5th, Level(3) terminated peering with
Cogent without cause (as permitted under its peering agreement with
Cogent) even though both Cogent and Level(3) remained in full
compliance with the previously existing interconnection agreement.
Cogent has left the peering circuits open in the hope that Level(3)
will change its mind and allow traffic to be exchanged between our
networks. We are extending a special offering to single homed Level 3 customers.
Cogent will offer any Level 3 customer, who is single homed to the
Level 3 network on the date of this notice, one year of full Internet
transit free of charge at the same bandwidth currently being supplied
by Level 3. Cogent will provide this connectivity in over 1,000
locations throughout North America and Europe.

40. Internet Routing Protocol: BGP
Autonomous Systems (ASes)
Route Advertisement
Destination Next-hop AS Path
/16
174… 2637
Session
Traffic
Diagram of routing table is very confusing because it’s not pointing to anything
Green arrow shorter, and too thick… green is a msg
More intuition about how the system actually works.
Don’t say “interdomain”
DESTINATION-BASED Routing
Tables look like a set of possible routes and a rankings over these routes
(pop up a simplified table fragment)

41. Question: What’s the difference between IGP and iBGP?
Two Flavors of BGP
iBGP
eBGP
External BGP (eBGP): exchanging routes between ASes
Internal BGP (iBGP): disseminating routes to external destinations among the routers within an AS
Question: What’s the difference between IGP and iBGP?

42. Example BGP Routing Table
The full routing table
> show ip bgp
Network Next Hop Metric LocPrf Weight Path
*>i i
*>i i
*>i / i
* i / i
> show ip bgp
BGP routing table entry for /16
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Not advertised to any peer
from ( )
Origin IGP, metric 0, localpref 150, valid, internal, best
Community: 10578: :950
Last update: Sat Jan 14 04:45:
Specific entry. Can do longest prefix lookup:
Prefix
AS path
Next-hop

43. Routing Attributes and Route Selection
BGP routes have the following attributes, on which the route selection process is based:
Local preference: numerical value assigned by routing policy. Higher values are more preferred.
AS path length: number of AS-level hops in the path
Multiple exit discriminator (“MED”): allows one AS to specify that one exit point is more preferred than another. Lower values are more preferred.
eBGP over iBGP
Shortest IGP path cost to next hop: implements “hot potato” routing
Router ID tiebreak: arbitrary tiebreak, since only a single “best” route can be selected

44. Other BGP Attributes
Next-hop:
Next-hop:
iBGP
Next-hop: IP address to send packets en route to destination. (Question: How to ensure that the next-hop IP address is reachable?)
Community value: Semantically meaningless. Used for passing around “signals” and labelling routes. More in a bit.

45. Local Preference Control over outbound traffic
Higher local pref
Primary
Destination
Backup
Lower local pref
Control over outbound traffic
Not transitive across ASes
Coarse hammer to implement route preference
Useful for preferring routes from one AS over another (e.g., primary-backup semantics)

46. Communities and Local Preference
Primary
Destination
Backup
“Backup” Community
Customer expresses provider that a link is a backup
Affords some control over inbound traffic
More on multihoming, traffic engineering in Lecture 7

47. AS Path Length
Traffic
Destination
Among routes with highest local preference, select route with shortest AS path length
Shortest AS path != shortest path, for any interpretation of “shortest path”

48. AS Path Length Hack: Prepending
Traffic
AS 2
AS 3
AS Path: “1”
AS Path: “1 1”
AS 1 D
Attempt to control inbound traffic
Make AS path length look artificially longer
How well does this work in practice vs. e.g., hacks on longest-prefix match?

49. Multiple Exit Discriminator (MED)
Dest.
Traffic
San Francisco
New York
MED: 20
MED: 10 I
Los Angeles
Mechanism for AS to control how traffic enters, given multiple possible entry points.

50. Hot-Potato Routing Prefer route with shorter IGP path cost to next-hop
Idea: traffic leaves AS as quickly as possible
Dest.
New York
Atlanta
Traffic
Common practice: Set IGP weights in accordance with propagation delay (e.g., miles, etc.) 10 5 I
Washington, DC

51. Problems with Hot-Potato Routing
Small changes in IGP weights can cause large traffic shifts
Dest.
San Fran
New York
Traffic
Question: Cost of sub-optimal exit vs. cost of large traffic shifts 11 10 5 I LA

52. MPLS Overview Main idea: Virtual circuit
Packets forwarded based only on circuit identifier
Source 1
Destination
Source 2
Router can forward traffic to the same destination on different interfaces/paths.

53. Circuit Abstraction: Label SwappingD A 2 1
Tag Out New 3 A 2 D
Label-switched paths (LSPs): Paths are “named” by the label at the path’s entry point
At each hop, label determines:
Outgoing interface
New label to attach
Label distribution protocol: responsible for disseminating signalling information

54. Layer 3 Virtual Private Networks
Private communications over a public network
A set of sites that are allowed to communicate with each other
Defined by a set of administrative policies
determine both connectivity and QoS among sites
established by VPN customers
One way to implement: BGP/MPLS VPN mechanisms (RFC 2547)

55. Building Private Networks
Separate physical network
Good security properties
Expensive!
Secure VPNs
Encryption of entire network stack between endpoints
Layer 2 Tunneling Protocol (L2TP)
“PPP over IP”
No encryption
Layer 3 VPNs
Privacy and interconnectivity (not confidentiality, integrity, etc.)

56. Layer 2 vs. Layer 3 VPNs
Layer 2 VPNs can carry traffic for many different protocols, whereas Layer 3 is “IP only”
More complicated to provision a Layer 2 VPN
Layer 3 VPNs: potentially more flexibility, fewer configuration headaches

57. Layer 3 BGP/MPLS VPNs
VPN A/Site 1
VPN A/Site 2
VPN A/Site 3
VPN B/Site 2
VPN B/Site 1
VPN B/Site 3
CEA1
CEB3
CEA3
CEB2
CEA2
CE1B1
CE2B1
PE1
PE2
PE3 P1 P2 P3
10.1/16
10.2/16
10.3/16
10.4/16
BGP to exchange routes
MPLS to forward traffic
Isolation: Multiple logical networks over a single, shared physical infrastructure
Tunneling: Keeping routes out of the core

58. High-Level Overview of Operation
IP packets arrive at PE
Destination IP address is looked up in forwarding table
Datagram sent to customer’s network using tunneling (i.e., an MPLS label-switched path)

59. BGP/MPLS VPN key components
Forwarding in the core: MPLS
Distributing routes between PEs: BGP
Isolation: Keeping different VPNs from routing traffic over one another
Constrained distribution of routing information
Multiple “virtual” forwarding tables
Unique addresses: VPN-IP4 Address extension

60. Layer 3 VPNs “Vanilla” Layer 3 VPNs: All customer routes in the core
Site 1
Site 2
CORE
IBGP
EBGP
BGP/MPLS VPNs: BGP between PEs; MPLS in the core
Site 1
LDP
LDP
LDP
Site 2 P
MPLS CORE P PE PE

61. Problems Introduced by Layer 3 VPNs
Overlapping address space in forwarding table
Solution: Virtual routing and forwarding table (“VRF”)
Overlapping address space in BGP routes
Solution: “Route distinguisher” byte VPN-specific identifier prepended to each IP address
Typically, one route distinguisher per VPN
New VPN-IP address family
Routes carried with multi-protocol BGP
Filtering routes from routes not at that site
Route target: basically a special BGP community value

62. Virtual Routing and Forwarding
Separate tables per customer at each router
Customer 1
/24
/24 RD: Green
Customer 1
Customer 2
/24
Customer 2
/24 RD: Blue

63. Routing: Constraining Distribution
Performed by Service Provider using route filtering based on BGP Extended Community attribute
BGP Community is attached by ingress PE route filtering based on BGP Community is performed by egress PE
Site 2
BGP
Static route, RIP, etc.
RD: /24 Route target: Green Next-hop: A
Site 1 A
/24
Site 3

64. BGP/MPLS VPN Routing in Cisco IOS
Customer A
Customer B
ip vrf Customer_A rd 100: route-target export 100: route-target import 100:1000 ! ip vrf Customer_B rd 100:120 route-target export 100: route-target import 100:2000

65. Forwarding
PE and P routers have BGP next-hop reachability through the backbone IGP
Labels are distributed through LDP (hop-by-hop) corresponding to BGP Next-Hops
Two-Label Stack is used for packet forwarding
Top label indicates Next-Hop (interior label)
Second level label indicates outgoing interface or VRF (exterior label)
Corresponds to LSP of BGP next-hop (PE)
Corresponds to VRF/interface at exit
Layer 2 Header
Label 1
Label 2
IP Datagram

66. Forwarding in BGP/MPLS VPNs
Step 1: Packet arrives at incoming interface
Site VRF determines BGP next-hop and Label #2
Label 2
IP Datagram
Step 2: BGP next-hop lookup, add corresponding LSP (also at site VRF)
Label 1
Label 2
IP Datagram

67. Scalability Problems
Lots of customers leads to explosion of routing tables
How to ensure that no single router needs to carry state for all customers?

68. Other Uses for MPLS/Tunneling
Reducing state in network core
Internal routers no longer need paths for every destination
Traffic engineering
Can shift traffic based on virtual circuits, not just destination prefixes

69. Open Research Questions
Static configuration analysis for enforcing isolation and other security policies
Easier, in some sense, since security (reachability) policies are likely easier to encode