TIE Breaking: Tunable Interdomain Egress Selection Renata Teixeira Laboratoire d’Informatique de Paris 6 Université Pierre et Marie Curie with Tim Griffin (Cambridge), Mauricio Resende (AT&T), and Jennifer Rexford (Princeton)

2 Internet as a Communication Infrastructure Internet Highly-sensitive to transient and persistent performance problems

3 Two-Tier Routing Architecture Internet UCSD Sprint AT&T Verio AOL User Web Server UCSD AT&T Verio AOL Interdomain routing (BGP) Selects AS-level path based on policies Intradomain routing (IGP) Most common: OSPF, IS-IS Selects shortest path from ingress to egress based on link weights

4 UCSD Sprint AT&T Verio AOL Selecting Among Multiple Egresses Today User Web Server UCSD AT&T Verio Hot-potato routing BGP selects closest egress by comparing IGP distances NY SF LA B’s IGP distance d(B,NY): 2 d(B,SF): 31 d(B,LA): 26 B

5 However, Hot-Potato Routing is…  Too disruptive  Small changes inside can lead to big disruptions A B C D G E F A B dst Consequences -Transient forwarding instability -Traffic shift (largest traffic variations) -BGP updates to other domains

6 However, Hot-Potato Routing is…  Too disruptive  Small changes inside can lead to big disruptions  Too restrictive  Egress selection mechanism dictates a policy  Too convoluted  IGP metrics determine BGP egress selection  IGP paths and egress selection are coupled

7 Maybe a Fixed Ranking?  Goal: No disruptions because of internal changes  Solution  Each router has a fixed ranking of egresses  Select the highest-ranked egress for each destination  Use tunnels from ingress to egress  Disadvantage  Sometimes changing egresses would be useful A B C D G E F A B dst

8 Egress Selection Mechanisms automatic adaptation robustness to internal changes hot-potato routing fixed ranking Explore trade-off

9 Metrics for Ranking Egresses  Egress selection mechanisms are based on a metric (m) that each ingress router (i) uses to rank each egress router (e) for a destination  Hot-potato routing m is the intradomain distance (d(i,e))  Fixed ranking m is a constant

10 Goals of New Metric  Configurable  Implement a wide-range of egress selection policies  Simple computation  Compute on-line, in real-time  Based on information already available in routers (distance)  Easy to optimize  Expressive for a management system to optimize  Fine control  Each ingress can implement its own ranking policy for each destination

11  Decouples egress selection from internal paths  Egress selection is done by tuning  and   Allow a wide variety of egress selection policies  Hot-potato:  =1,  = 0  Fixed ranking:  =0,  = constant rank  Requirements  Small change in router decision logic  Use of tunnels (as with fixed ranking) m (e) =  (e). d(i,e) +  (e) TIE: Tunable Interdomain Egress Selection m i (dst,e) =  i (dst,e). d(i,e) +  i (dst,e) weighted intradomain distance constant

12 Routers Management System Using TIE Run optimization ,  Configure routers Path computation using m i (dst,e) Forwarding table Administrator defines policy Upon  and  change or routing change

13 Configuring TIE to Minimize Sensitivity Simulation Phase Optimization Phase Network topology Set of egress routers per prefix Set of failures system of inequalities Management System configure routers with values  i (dst,e) and  i (dst,e) that minimize sensitivity

14 At design time: m C (dst,A) < m C (dst,B) 9.  C (dst,A) +  C (dst,A) < 10.  C (dst,B) +  C (dst,B) Simulation Phase A B C dst  C (dst,A) +  C (dst,A) > 10.  C (dst,B) +  C (dst,B) 11.  C (dst,A) +  C (dst,A) < 10.  C (dst,B) +  C (dst,B) Output of simulation phase  C (dst,A)=1,  C (dst,A)=1,  C (dst,B) =2,  C (dst,B) =0

15 Optimization Phase  One system of inequalities per (node, prefix) pair  (num egresses – 1) x (num failures +1)  Practical requirements for setting parameters  Finite-precision parameter values  Limiting the number of unique values  Robustness to unplanned events  Running time  37 seconds (Abilene network) and 12 minutes (ISP network) 196MHz MIPS R10000 processor on an SGI Challenge Integer programming Objective function: min  (  +  )   1

16 Evaluation of TIE on Operational Networks  Topology and egress sets  Abilene network (U.S. research network)  Set link weight with geographic distance  Configuration of TIE  Considering single-link failures  Threshold of delay ratio: 2    [1,4] and 93% of  i (dst,e)=1    {0,1,3251} and 90% of  i (dst,e)=0  Evaluation against hot-potato and fixed ranking  Simulate single-node failures  Measure routing sensitivity and delay

17 Sensitivity to Node Failures fraction prefixes affected CCDF of (node,failure) pairs 15% of egress changes can be avoided without harming delay

18 Delay under Node Failures CCDF of (node, destination,failure) tuples ratio of delay after failure to design time delay Under threshold, TIE has longer delay than hot-potato It is better than fixed ranking for 60% of tuples

19 Conclusion  TIE mechanism for selecting egresses  Decouples interdomain and intradomain routing  Designed for being easy to optimize  Small change to router implementation  Operators can optimize TIE for other policies  Traffic engineering  Robust traffic engineering  Planning for maintenance

20 More details

21 UCSD Sprint AT&T Verio AOL Multiple Interdomain Egresses User Web Server UCSD AT&T Verio Multiple egresses for a destination are common! ISPs usually peer in multiple locations and customers buy multiple connections to one or more ISPs for reliability and performance NY SF LA AOL

22 Why Hot-Potato Routing?  Independent and consistent egress decision  Forward packet to neighbors that have selected same (closest) egress  Minimize resource consumption  Limits consumption of bandwidth by sending traffic to next domain as early as possible A B C D G E F A B dst

23 Summary of BGP Decision Process  BGP decision process  Ignore if exit point unreachable  Highest local preference  Lowest AS path length  Lowest origin type  Lowest MED (with same next hop AS)  Lowest IGP cost to next hop  Lowest router ID of BGP speaker

24 Other Policies  Traffic engineering  Configure TIE parameters to select egresses to obtain optimal link utilization  Solution: Path-based multi-commodity flow  Robust traffic engineering  Combine minimizing sensitivity with traffic engineering problem  Preparing for maintenance

25 Traffic Engineering with TIE  Problem definition  Balance utilization of internal links  Configure TIE parameters to select egresses to obtain optimal link utilization  No need to set intradomain link weights  Solution  Path-based multicommodity flow  No need to tweak routing protocols  Avoid routing convergence

26 Example Policy: Minimizing Sensitivity  Problem definition  Minimize sensitivity to equipment failures  No delay more than twice design time delay  Would be a simple change to routers  If distance is more than twice original distance Change to closest egress  Else Keep using old egress point  But cannot change routers for all possible goals We can do this with TIE just by setting  and 