1. COS 561: Advanced Computer Networks
BGP Instability
Jennifer Rexford
Fall 2016 (TTh 3:00-4:20 in CS 105)
COS 561: Advanced Computer Networks

2. Holding the Internet Together
Distributed cooperation for resource allocation
BGP: what end-to-end paths to take (for ~55K ASes)
TCP: what rate to send over each path (for ~3.4B hosts)
AS 2
AS 1
AS 3
AS 4

3. What Problem is BGP Solving?
The Stable Paths Problem

4. What Problem Does Routing Solve?
Most do shortest-path routing
Shortest hop count
Distance vector routing (e.g., RIP)
Shortest path as sum of link weights
Link-state routing (e.g., OSPF and IS-IS)
Policy makes BGP is more complicated
An AS might not tell a neighbor about a path
E.g., Sprint can’t reach Verizon through AT&T
An AS might prefer one path over a shorter one
E.g., ISP prefers to send traffic through a customer

5. Could Use A Simulation Model
Simulate the message passing
Advertisements and withdrawals
Message format
Timers
Simulate the routing policy on each session
Filter certain route advertisements
Manipulate the attributes of others
Simulate the decision process
Each router applying all the steps per prefix
Feasible, but tedious and ill-suited for formal arguments

6. Stable Paths Problem Instance
Node
BGP-speaking router
Node 0 is destination
Edge
BGP adjacency
Permitted paths
Set of routes to 0 at each node
Ranking of the paths
2 1 0
2 0 2 5 2
4 2 0
4 3 0 4 1 3
3 0 1
1 3 0
1 0
most preferred …
least preferred

7. Solution to a Stable Paths Problem2
Solution
Path assignment per node
Can be the “null” path
If node u has path uwP
{u,w} is an edge in the graph
Node w is assigned path wP
Each node is assigned
The highest ranked path
… consistent with the assignment of its neighbors
2 1 0
2 0 5 2
4 2 0
4 3 0 4 1 3
3 0
1 3 0
1 0 1
A solution need not represent
a shortest path tree, or
a spanning tree.

8. Translating a Real Configuration into SPP
Permitted paths at a node
Composition of “export” policies at other nodes
Ranking of paths at a node
“Import” policies at the node
Rank in terms of BGP decision process (i.e., local preference, AS path length, origin type, MED, …)
2 1 0
2 0
Node 0 exports
route to node 2
Node 2 exports
“2 1 0” but not “2 0” 2 5
Node 1 exports
“1 0” to node 2

9. An SPP May Have Multiple Solutions
First solution 1 2
1 2 0
1 0
2 1 0
2 0 1 2
1 2 0
1 0
2 1 0
2 0
Second solution
1 2 0
1 0 1 2
2 1 0
2 0

10. An SPP May Have No Solution
2 1 0
2 0 2 4
3 2 0
3 0
1 3 0
1 0 1 3 3

11. Stable System Unstable After Failure
2 1 0
2 0
Becomes a BAD GADGET if link
(4, 0) goes down. 2
4 0
4 2 0
4 3 0
BGP is not robust :
it is not guaranteed
to recover from
network failures. 4 3 1
3 0
1 3 0
1 0

12. Strawman Solution Doesn’t Work
Create a global Internet routing registry
Store the AS-level graph and all routing policies
Store all routing policies
But, ASes may be unwilling to divulge
Check for conflicting policies
Analyze the global system and identify conflicts
Contact the affected ASes to resolve them
But, checking is an NP-complete problem
… and, a safe system may be unsafe after failure
Goal: sufficient condition for convergence with local control

13. BGP Protocol

14. Two Kinds of Routing Protocols
Link State
Vectoring
Topology information is flooded within the routing domain
Best end-to-end paths are computed locally at each router.
Best end-to-end paths determine next-hops.
Based on minimizing some notion of distance
Works only if policy is shared and uniform
Examples: OSPF, IS-IS
Each router knows little about network topology
Only best next-hops are chosen by each router for each destination.
Best end-to-end paths result from composition of all next-hop choices
Does not require any notion of distance
Does not require uniform policies at all routers
Examples: RIP, BGP

15. AS Numbers (ASNs) Level 3: 1 MIT: 3 Harvard: 11 Yale: 29 Princeton: 88
ASes represent units of routing policy
Level 3: 1
MIT: 3
Harvard: 11
Yale: 29
Princeton: 88
AT&T: 7018, 6341, 5074, …
Verizon: 701, 702, 284, 12199, …
Sprint: 1239, 1240, 6211, 6242, … …
Currently around 55,000 in use.

16. Interdomain Routing Path: 6, 5, 4, 3, 2, 1 4 3 5 2 7 6 1 Web server
Client

17. Border Gateway Protocol
ASes exchange info about who they can reach
IP prefix: block of destination IP addresses
AS path: sequence of ASes along the path
Policies configured by the AS’s operator
Path selection: which of the paths to use?
Path export: which neighbors to tell? 3
“ /24:
path (2,1)”
“ /24:
path (1)” 2 1
data traffic
data traffic

18. BGP Session Operation Establish session on TCP port 179 Exchange all
AS1
BGP session
Exchange all
active routes
AS2
While connection
is ALIVE exchange
route UPDATE messages
Exchange incremental
updates

19. Incremental Protocol A node learns multiple paths to destination
Stores all of the routes in a routing table
Applies policy to select a single active route
… and may advertise the route to its neighbors
Incremental updates
Announcement
Upon selecting a new active route, add node id to path
… and (optionally) advertise to each neighbor
Withdrawal
If the active route is no longer available
… send a withdrawal message to the neighbors

20. BGP Route Destination prefix (e.g., 128.112.0.0/16)
Route attributes, including
AS path (e.g., “ ”)
Next-hop IP address (e.g., )
AS 7018
AT&T
AS 88
AS 11
Yale
Princeton
/16
AS path = 88
Next Hop =
/16
AS path =
Next Hop =

21. ASPATH Attribute AS 1129 AS 1755 AS 12654 AS 1239 AS7018 AS 3549 AS 88
/16
AS Path =
Global Access
AS 1755
/16
AS Path =
/16
AS Path =
Ebone
AS 12654
AS 1239
Sprint
RIPE NCC
RIS project
/16
AS Path =
AS7018
/16
AS Path =
/16
AS Path = 88
AT&T
AS 3549
AS 88
/16
AS Path =
Princeton
Global Crossing
/16
Prefix Originated

22. BGP Path Selection Simplest case Example
Shortest AS path
Arbitrary tie break
Example
Three-hop AS path preferred over a five-hop AS path
AS prefers path through Global Crossing
But, BGP is not limited to shortest-path routing
Policy-based routing
AS 1129
Global Access
/16
AS Path =
AS 12654
RIPE NCC
RIS project
/16
AS Path =
AS 3549
Global Crossing

23. BGP Routing Changes

24. Causes of BGP Routing Changes
Topology changes
Equipment going up or down
Deployment of new routers or sessions
BGP session failures
Due to equipment failures, maintenance, etc.
Or, due to congestion on the physical path
Changes in routing policy
Reconfiguration of preferences
Reconfiguration of route filters
Persistent protocol oscillation
More on this next class!

25. BGP Session Failure BGP runs over TCP Detecting a failure
BGP only sends updates when changes occur
TCP doesn’t detect lost connectivity on its own
Detecting a failure
Keep-alive: 60 seconds
Hold timer: 180 seconds
Reacting to a failure
Discard all routes learned from neighbor
Send updates for any routes that change
AS1
AS2

26. Routing Change: Before and After
(1,0)
(2,0)
(2,0)
(1,2,0) 1 1 2 2
(3,1,0)
(3,2,0) 3 3

27. Routing Change: Path Exploration
AS 1
Delete the route (1,0)
Switch to next route (1,2,0)
Send route (1,2,0) to AS 3
AS 3
Sees (1,2,0) replace (1,0)
Compares to route (2,0)
Switches to using AS 2
(2,0)
(1,2,0) 1 2
(3,2,0) 3

28. Routing Change: Path Exploration
Initial situation
Destination 0 is alive
All ASes use direct path
When destination dies
All ASes lose direct path
All switch to longer paths
Eventually withdrawn
E.g., AS 2
(2,0)  (2,1,0)
(2,1,0)  (2,3,0)
(2,3,0)  (2,1,3,0)
(2,1,3,0)  null
(2,0)
(2,1,0)
(2,3,0)
(2,1,3,0)
(1,0)
(1,2,0)
(1,3,0) 1 2 3
(3,0)
(3,1,0)
(3,2,0)

29. Discussion of BGP Instability Paper