1. Jennifer Rexford Fall 2010 (TTh 1:30-2:50 in COS 302) COS 561: Advanced Computer Networks http://www.cs.princeton.edu/courses/archive/fall10/cos561/ Stub Traffic Engineering

2. Why Connect to Multiple Providers? Reliability –Reduced fate sharing –Survive ISP failure Performance –Multiple paths –Select the best Financial –Leverage through competition –Game 95 th -percentile billing model Provider 1 Provider 2 Stub: an Autonomous System that does not provide transit

3. Outline Outbound traffic –Shortest-path routing –Primary and backup providers –Load balancing over multiple links Inbound traffic –Primary and backup providers –Selective advertising –BGP communities Looking forward –Control by routers vs. end hosts –Exposing greater path diversity –Stability of selfish routing

4. Controlling Outbound Traffic

5. Outbound Traffic: Pick a BGP Route Easier to control than inbound traffic –IP routing is destination based –Sender determines where the packets go Control only by selecting the next hop –Border router can pick the next-hop AS –Cannot control selection of the entire path Provider 1 Provider 2 “(2, 7, 8, 4)” “(1, 3, 4)”

6. Outbound Traffic: Shortest AS Path No import policy on border router –Pick route with shortest AS path –Arbitrary tie break (e.g., router-id) Performance? –Shortest path is not necessarily best –Propagation delay or congestion Load balancing? –Could lead to uneven split in traffic –E.g., one provider with shorter paths –E.g., too many ties with a skewed tie-break s d

7. Outbound Traffic: Primary and Backup Single policy for all prefixes –High local-pref for session to primary provider –Low load-pref for session to backup provider Outcome of BGP decision process –Choose the primary provider whenever possible –Use the backup provider when necessary But… –What if you want to balance traffic load? –What if you want to select better paths?

8. Outbound Traffic: Load Balancing Selectively use each provider –Assign local-pref across destination prefixes –Change the local-pref assignments over time Useful inputs to load balancing –End-to-end path performance data  E.g., active measurements along each path –Outbound traffic statistics per destination prefix  E.g., packet monitors or router-level support –Link capacity to each provider –Billing model of each provider

9. Outbound Traffic: What Kind of Probing? Lots of options –HTTP transfer –UDP traffic –TCP traffic –Traceroute –Ping Pros and cons for each –Accuracy –Overhead –Dropped by routers –Sets off intrusion detection systems How to monitor the “paths not taken”?

10. Outbound Traffic: How Often to Change? Stub ASes have no BGP customers –So, routing changes do not trigger BGP updates TCP flows that switch paths –Out-of-order packets during transition –Change in round-trip-time (RTT) Impact on the providers –Uncertainty in the offered load –Interaction with their own traffic engineering? Impact on other end users –Good: move traffic off of congested paths –Bad: potential oscillation as other stub ASes adapt?

11. Controlling Inbound Traffic

12. Inbound Traffic: Influencing Others Harder to control than outbound traffic –IP routing is destination based –Sender determines where the packets go Control only by influencing others’ decisions –Static configuration of the providers –BGP route attributes sent by the stub –Selective advertising of destination prefixes Provider 1 Provider 2

13. Inbound Traffic: Primary and Backup Ask your provider to be a backup –Provider violates “prefer customer” policy –… by assigning lower local-pref to customer –Backup link is only used if the primary link fails Provider 1 Provider 2 12.34.158.0/24 local-pref=90 local-pref=100 traffic

14. Inbound Traffic: AS Prepending Make one path look longer –Advertise short path one way –… and longer path another –In the hope of influencing choices –But, how much prepending to do? Provider 1 Provider 2 “12.34.158.024: (3)” “12.34.158.024: (3, 3, 3)”

15. Inbound Traffic: Prepending Fail Example where prepending doesn’t work –Customer does prepending of AS path –Provider has a “prefer customer” policy –Provider 2 prefers the longer path Provider 1 Provider 2 “12.34.158.024: (3)” “12.34.158.024: (3, 3, 3)” “12.34.158.024: (1, 3)”

16. Inbound Traffic: Programming Your Provider Better to have selective control over provider –Tell the provider whether to prefer your route –… on a per-prefix basis, with changes over time Enables adaptive load balancing –Without asking provider to reconfigure policy Provider 1 Provider 2 12.34.158.0/24

17. Inbound Traffic: BGP Communities Provider and customer agree on a “tag” –One tag mean “primary” and the other “backup” –Customer includes tags in BGP advertisements –Provider sets local preference based on tags BGP community attribute –Opaque attribute with no real meaning  Two numbers: usually AS number and arbitrary number –Sprint example (http://www.sprint.net/policy/bgp.html)  1239:70 means “assign local pref of 70”  …  1239:110 means “assign local pref of 110” See RFC 1998

18. Example: ISP Setting Local-Pref Peer Customer Customers –110: Primary path –100: Secondary path –80: Primary backup path –70: Secondary backup path Peers –81-99: In between –Range for traffic engineering

19. Inbound Traffic: Selective Advertising When you don’t have enough prefixes… –Advertise one subnet to each provider –Advertise the supernet to both providers –Traffic splits due to the longest-prefix match –Supernet ensures backup connectivity after failure Provider 1 Provider 2 12.34.0.0/16 12.34.128.0/17 12.34.0.0/16 12.34.0.0/17 Causes unwanted increase in global routing tables

20. Inbound Traffic: Multiple Addresses Multiple external addresses for a service –One IP address for each entry point Use DNS to adjust mapping of name to address –Give different answers to different clients –Adjust over time to performance, cost, traffic, etc. 20 Provider 1 Provider 2 5.6.7.0/24 12.34.1.0/24 5.6.7.812.34.1.2

21. Inbound Traffic: Tunneling Cooperation with sender –Sending network picks the ingress point –Encapsulates the packet with the interface address Advantages –Sender may know best Disadvantages –Requires cooperation –Failure recovery 21 Provider 1 Provider 2 5.6.7.812.34.1.1 12.34.1.2 Encapsulate with 12.34.1.1 or 5.6.7.8

22. Looking Forward 22

23. Greater Path Diversity Performance depends on the end-to-end path –Need greater path diversity How to get more paths? –Add more providers? (expensive) –Support multipath routing? Ways to support multipath routing –Source routing –Provider offering multipath as a service –Overlays (e.g., RON) –Deflecting packets through intermediate Ases –Etc. 23

24. Overlay Example: RON Premise: by building application overlay network, can increase performance and reliability of routing Two-hop (application-level) Berkeley-to-Princeton route application-layer router Princeton Yale Berkeley http://nms.csail.mit.edu/ron/

25. Overlay Example: RON Exchange the results of the probes –Each host shares results with every other host –Essentially running a link-state protocol! –So, every host knows the performance properties Forward through intermediate host when needed A C B B

26. Discussion: Who’s in Charge? Routers –Reduces complexity and overhead the end hosts –Avoids placing (unwarranted?) trust in end hosts –Managed based on an organization’s goals –Lower performance measurement overhead End hosts –Best equipped to judge if performance is good enough –Decisions customized to the specific application –Finer-granularity of load-balancing decisions –Monitor performance by observing their own traffic 26

27. Other Discussion At what layer or component (e.g., DNS, BGP, overlays, application, etc.)? How to get extra path diversity? Who pays for extra resources? How to prevent global inefficiency or instability? What is a good clean-slate solution? 27