ISP and Egress Path Selection for Multihomed Networks Amogh Dhamdhere Constantine Dovrolis (amogh,dovrolis)@cc.gatech.edu Networking and Telecommunications Group College of Computing Georgia Tech

Multihoming Multihoming: Connection of a stub network to multiple ISPs 70% of stub networks are multihomed Redundancy primary/backup relationships Load Balancing Distribute outgoing traffic among ISPs Cost Effectiveness Lower cost ISP for bulk traffic, higher cost ISP for performance-sensitive traffic Performance Intelligent Route Control 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Major Questions How to select the set of upstream ISPs ? Low monetary cost Good performance (low delay, loss rate) Path diversity to major traffic destinations – improves robustness to network failures How to allocate egress traffic to the set of selected ISPs ? Objective: Avoid congestion on the upstream paths Also maintain low cost 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Related Work Wang et al. (Infocom 2004) ISP selection to minimize cost to subscriber Did not consider performance constraints and path diversity Applicable only to percentile based charging pricing function IRC systems (RouteScience, Internap, Radware..) Path switching for better performance Work on short timescales – Can lead to oscillations Goldenberg et al. (Sigcomm 2004) IRC algorithm for optimizing latency and cost over short timescales Applicable to the percentile based charging model 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Problem Definition Two phase problem Phase I – ISP Selection: Select K upstream ISPs K depends on monetary and performance constraints “Static” operation Change only when major changes in the traffic destinations or ISP pricing Phase II – Egress Path Selection Allocate egress traffic to selected ISPs Avoid long term congestion and minimize cost “Semi-static” operation, performed every few hours or days 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Assumptions We provision only the egress traffic of S The set of M major destinations is known Average rates to the major destinations are known Number of ISPs to choose (K) and the set of possible ISPs (I) is known An ISP charges based on the volume of traffic routed through it (volume based charging) Assume increasing and concave pricing functions 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Objectives of ISP selection ISP selection should consider both monetary cost and performance Minimum monetary cost Estimate the cost that “would be” incurred if a set of ISPs was selected Minimum AS-level path lengths Longer paths can lead to larger delays and increase vulnerability to inter-domain routing failures AS-level paths can be measured offline using Looking Glass Servers Maximum Path diversity AS-level paths to destinations should be as “different” as possible 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

ISP Selection K ISPs to be selected out of |I| Associate a cost with each performance metric Monetary cost, path length cost and path diversity cost Total cost of a selection of ISPs C: ct(C): = αmcm(C) + αpcp(C) + αdcd(C) Optimization problem: Find the set C* with the minimum total cost Brute Force approach is feasible E.g. For |I|=15 and K=4, there are 1365 combinations Solution approach: Evaluate the cost of each selection and choose the set with the minimum cost 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Monetary and Path Length Cost Lower level optimization problem: Given a set of ISPs C, what is the minimum monetary and path length cost of routing egress flows ? Find the mapping G* of items to bins that minimizes the cost of the assignment (Bin Packing) Flows = items ISPs = bins To find: Least cost assignment of flows to ISPs NP hard ! Use First Fit Decreasing (FFD) heuristic Generated mapping G* very close to optimal Monetary and path length costs of C are then calculated using the mapping G* 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Path Diversity Cost A selection C gives K paths to each destination d K-shared link to d: A link which is shared by all K paths to d If a K-shared link fails, destination d is unreachable Minimize the number of K-shared links Should give best performance for single-link failures Define metric k(d,C): number of K-shared links to d in selection C Choose the selection with the minimum k(d,C) averaged over all destinations 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Evaluation - Bin Packing FFD heuristic used to find the minimum monetary and path length costs Simulations Need exhaustive search to identify optimal cost Restrict network to 3 ISPs and 15 destinations FFD heuristic finds a solution with high probability, when average load is below 60-70% In high load conditions, the probability of finding a solution decreases Cost ratio is close to 1, even at high load conditions FFD heuristic is close to the optimal in terms of cost 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Evaluation – Path Diversity AS-level paths and traffic rates are input to simulator 9 ISPs, 250 destinations Given K, find the selection C* with the minimum path diversity cost For each selection C, u(C) = total traffic lost due to the failure of each link in topology Calculate Δu(C) = u(C) – u(C*) for each selection C Replace graphs with new graphs increasing the line thickness Possibly color ? Single link failures: C* is the optimal selection 2,3 link failures: C* is close to the optimal selection 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Cannot know a priori whether a mapping will cause congestion Egress Path Selection After Phase-I, S has K upstream ISPs Problem: How to map outgoing traffic to the ISPs M flows: KM mappings of flows to ISPs Some mappings may cause congestion to flows ! Flows can be congested at access links or further upstream Objective: Find the loss-free mapping with the minimum cost Challenges: Upstream topology and capacities are unknown Cannot know a priori whether a mapping will cause congestion Iterative routing approaches required 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Egress Path Selection Step 1: Use the FFD heuristic to map flows to ISPs Assume initially that the access links are bottlenecks Access capacities are known FFD heuristic for bin packing gives a cost close to the best possible cost Some flows may be congested Bottlenecks in upstream networks Use as the starting point for the stochastic search 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Egress Path Selection Step 2: Use iterative stochastic search to find a loss-free solution Stochastic search by simulated annealing Iterative combinatorial optimization algorithm Route traffic, measure congestion, decide next action Action: Flows re-routed from one ISP to another can accept moves which increase congestion Accepting “bad” moves can help to escape local minima 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Evaluation of Stochastic Search Stochastic search involves iterative routing Traffic has to be re-routed Some traffic may be dropped due to congested links Evaluation metrics Probability of finding a solution (high) Number of iterations to find a solution (low) Amount of traffic re-routed (low) Amount of dropped traffic (low) Compare against other heuristics Only bin packing (access-link) Greedy iterative algorithm (greedy-single) Moves which increase congestion are not accepted Variants of Simulated Annealing 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Evaluation of Stochastic Search SA-slow and greedy show similar probability of finding a solution Other algorithms have a significantly lower probability of finding a solution On the average, SA-slow needs fewer iterations to find a feasible solution Accepting “worse” solutions can actually help find a loss-free solution faster SA drops less traffic on the average than greedy-single SA re-routes less traffic on the average than greedy-single 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Summary of Contributions Proposed practical algorithms for ISP selection and egress traffic allocation among selected ISPs ISP selection algorithm takes into account both monetary and performance constraints Formulated as a bin-packing problem Applicable for general pricing functions Can be extended to incorporate more performance metrics Egress path selection without knowledge of upstream topology Proposed simulated annealing for stochastic search Performs better than other simple iterative algorithms 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Thank You ! 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Evaluation of Bin Packing Simulations Need exhaustive search to identify optimal cost Restrict network to 3 ISPs and 15 destinations FFD-like heuristic finds a solution with high probability, when average load is below 60-70% In high load conditions, the probability of finding a solution decreases Cost ratio is very close to 1, even at high load conditions Heuristic algorithm is close to the optimal in terms of cost 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Stochastic Search – Probability of finding a solution bneck_loc=0.5 (bottlenecks in the middle of network) and bneck_shar=0 (no shared bottlenecks) SA-slow and greedy-single show similar probability of finding a solution Access-link, greedy-mult and SA-fast show significantly lower probability of finding a solution Henceforth, compare only greedy-single and SA-slow 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Stochastic Search – Number of Iterations How many iterations before solution is found ? Number of iterations required increases with offered load (average flow rate) SA-slow performs better than greedy-single on the average Accepting solutions with increasing congestion can actually help find a non-congested solution quicker 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Stochastic Search – Results What is the total rate of traffic that is re-routed ? What is the total rate of traffic that is dropped due to congestion ? SA-slow drops less traffic on the average than greedy-single SA-slow re-routes less traffic on the average than greedy-single 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006

Assumptions Provision the egress traffic of S The set of M major destinations is known Average rates to the major destinations are known Number of ISPs to choose (K) and the set of possible ISPs (I) is known An ISP charges based on the volume of traffic routed through it (volume based charging) Assume increasing and concave pricing functions 9/20/2018 Amogh Dhamdhere IEEE Infocom 2006