1. Improved BGP convergence via Ghost Flushing Yehuda Afek Anat Bremler-Barr Shemer Schwarzd המרכז הבינתחומי הרצליה

2. Problem: BGP Convergence [Labovitz,Ahuja,Bose,Jahanian] BGP may take up to 15 minutes to converge. Here: Reduce the worst case from minutes to seconds, in a practical way

3. Problem: BGP Convergence [Labovitz,Ahuja,Bose,Jahanian] Events Time (sec ’ s, minRouteAdver=30) E-Down 30 n n  10,000, up to 15 minutes E-Up 30d d E-Up 30 d d  30, d=diameter E-Longer 2 30 l l == path length E-Shorter 30d E-Shorter 30 d Here: E-down = l time units (unit = link delay) E-Longer = 30 d

4. Agenda BGP overview The BGP convergence problem Ghost buster rule Ghost flushing rule Simulation results

5. BGP protocol Distance (Path) vector protocol Receive AS-path from the neighbors Chooses the best one (shortest) Eliminates Routing loops using the AS-path Two kinds of messages: Announcements and Withdrawal

6. Problem: Ghost information One Ghost (old information) makes many, and in the network it continues recursively dst 2 1 3 4 dst: 0 0 t=0 withdraw

7. Problem: Ghost information dst 2 1 3 4 dst: 2 0 dst: 1 0 0 dst: {} t=1 annc:2 0 annc:1 0 One Ghost (old information) makes many, and in the network it continues recursively

8. Problem: Ghost information dst 2 1 3 4 dst: {} dst: 1 2 0 0 dst: 3 1 0 dst: {} t=2 withdraw One Ghost (old information) makes many, and in the network it continues recursively

9. MinRouteAdver Effect MinRouteAdver – wait 30 sec ’ s before sending an announcement again Applies to announcements only, not on withdrawals Motivation to reduce messages

10. Problem: Ghost information dst 2 1 3 4 dst: {} dst: 2 1 0 0 dst: 3 1 0 dst: {} t=3t=4t=5t=6 One Ghost (old information) makes many, and in the network it continues recursively minRouteAdver: Wait 30 sec ’ s before sending the next announcement (BGP) t=28t=27t=24t=22t=20t=17t=16t=14t=11t=10t=8t=7t=31 annc: 3 1 0 annc: 2 1 0

11. Problem: MinRouteAdver Effect MinRouteAdver – delays the elimination of ghost information dst 2 1 3 4 dst: {} dst: 2 1 0 0 dst: 3 1 0 dst: {} t=31 annc: 3 1 0 annc: 2 1 0

12. E_Down convergence MessageTime nE30nBGP In the clique (size 4) example the scenario ends after 62 sec (= 30(n-2) )

13. Without MinRouteAdver Avalanche of Messages O(n!) Explore all possible paths of length 1, 2 … dst 2 1 3 4 dst: 0 0 dst: {} t=0 1 : 1 0 3 : 3 0 4 : 4 0 1 : 1 0 2 : 2 0 3 : 3 0 1 : 1 0 2 : 2 0 4 : 4 0 2 : 2 0 3 : 3 0 4 : 4 0 withdrawal

14. Without MinRouteAdver dst 2 1 3 4 dst: 2 0 dst: 1 0 0 dst: {} t=0.1 1 : 1 0 3 : 3 0 4 : 4 0 1 : 1 0 3 : 3 0 4 : 4 0 1 : 1 0 3 : 3 0 4 : 4 0 2 : 2 0 3 : 3 0 4 : 4 0 annc: 2 0 annc: 1 0 Avalanche of Messages O(n!) Explore all possible paths of length 1, 2 …

15. Without MinRouteAdver dst 2 1 3 4 dst: 20 0 dst: 3 0 dst: {} t=0.2 1 : 1 2 0 3 : 3 0 4 : 4 0 1 : 1 2 0 2 : 2 0 3 : 3 0 1 : 1 2 0 2 : 2 0 4 : 4 0 2 : 2 0 3 : 3 0 4 : 4 0 annc: 2 0 annc:3 0 annc:2 0 Avalanche of Messages O(n!) Explore all possible paths of length 1, 2 …

16. Without MinRouteAdver dst 2 1 3 4 dst: 3 0 dst: 4 0 dst: 3 0 0 dst: {} t=0.3 1 : 1 2 0 3 : 3 0 4 : 4 0 1 : 1 2 0 2 : 2 1 0 3 : 3 0 1 : 1 2 0 2 : 2 1 0 4 : 4 0 2 : 2 3 0 3 : 3 0 4 : 4 0 annc:3 0 annc:4 0 Avalanche of Messages O(n!) Explore all possible paths of length 2, 3 … annc:3 0

17. Without MinRouteAdver dst 2 1 3 4 dst: 4 0 dst: 1 2 0 0 dst: 4 0 dst: {} t=0.4 1 : 1 2 0 3 : 3 1 0 4 : 4 0 1 : 1 2 0 2 : 2 1 0 3 : 3 1 0 1 : 1 2 0 2 : 2 1 0 4 : 4 0 2 : 2 3 0 3 : 3 1 0 4 : 4 0 annc:4 0 annc: 1 2 0 annc:4 0 Avalanche of Messages O(n!) Explore all possible paths of length 2, 3 … annc:4 0

18. E_Down convergence MessageTime nE30nBGP with MinRouteAdver n!Ehn h=one link delay BGP without MinRouteAdver

19. Related Work Introducing the problem [Labovitz,Ahuja,Bose,Jahanian], [Labovitz,Wattenhofer,Venkatachary,Ahuja] real life evidence theoretical analysis Experimental analysis [Griffin,Premore] Solution Works in Counting to Infinity: Adding states [Garcia-Luna-Aceves] – EIGRP like … Route Poisoning with Hold-down [Cisco:Rutgers] – IGRP like... Routes consistency [Pei,Zhao,Wang,Massey,Mankin,Wu,Zhang]

20. Ghost flushing rule If ASpath to dst is longer and cannot send annoucement (due to minRouteAdver rule ) then send withdrawal Motivation: Flush the ghost information ASAP

21. Ghost Flushing example dst 2 1 3 4 dst: 0 0 t=0 withdraw

22. Ghost Flushing example dst 2 1 3 4 dst: 2 0 dst: 1 0 0 dst: {} t=1 annc:2 0 annc:1 0

23. Ghost Flushing example dst 2 1 3 4 dst: {} dst: 1 2 0 0 dst: 3 1 0 dst: {} t=2 withdraw 2,3,4 send “ flushing ” withdrawal: since their ASpath is changed and minRouteAdver timer did not elapsed. withdraw

24. Ghost Flushing example dst 2 1 3 4 dst: {} dst: 1 2 0 0 dst: 3 1 0 dst: {} t=2 withdraw Longer ASpath & minRouteAdver timer  Send “ flushing ” withdrawal withdraw

25. Ghost Flushing example dst 2 1 3 4 dst: {} 0 t=3 withdraw

26. Analysis: Time convergence of ghost flushing rule, E_down In each time unit (=h, maximum link delay), ghost information is erased to a distance greater by one  After k time units, ghost information ASpath with length < k has disappeared.  Longest Ghost ASpath = n (in theory).  Hence (worst case) time convergence: nh

27. E_Down convergence MessageTime nE30nBGP with MinRouteAdver n!Ehn h=one link delay BGP without MinRouteAdver 2Ehn/30hnGhost flushing

28. Ghost Buster Rule The convergence time is better than expected !!!! Explanation: The minRouteAdver blocks the propagation of ghost information, while the flushing withdrawal “ eats ” the ghost information. Bad (wrong) news propagate slowly

29. Analysis: Ghost buster rule Add to the ghost flushing rule: Router sends announcement, only after delta time MinRouteAdver similar to delta: Common implementation: MinRouteAdver per peer And, timer almost always on (lots of BGP announcements !)

30. Analysis: Time convergence of ghost buster rule  The ghost information disappears at time t: d+t/(delta+h) = t/h  Every delta+h time the length of the maximum ghost ASpath is increased by one.  Every h time, the length of the minimum ghost ASpath is increased by one.  After the failure the length of the maximum ghost ASpath is d (diameter).  Hence: t = kdh/(k-1)  d, where k = (delta+h)/h is the rate of the algorithm

31. E_Down convergence MessageTime nE30nBGP with MinRouteAdver n!Ehn h=one link delay BGP without MinRouteAdver 2Ehn/30hnGhost flushing 2Ehkd/30(k-1) kdh/k-1  dh d=diameter k=(delta+h)/h Ghost flushing With Ghost buster

32. The effect on E_longer dst 2 1 3 4 0 5 6 7 BGP: Convergence time dominated: 1. Time until ghost information vanishes 2. Time until backup path propagates in Ghost flushing: helps the first factor

33. The effect on E_longer Original BGP may err: MinRouteAdver  peer stores wrong ASPath BGP may err and send the packet in the wrong direction Ghost flushing: send withdrawal to a peer. Perhaps by a chance there may be an alternative path there.

34. Simulation: BGP code Shortest path metric Delay on link between 0.2 to 2 sec MinRouteAdver randomly in 0 to 30 sec

35. Simulation: Clique E-down

36. Simulation: ISP topology 1 3 7 5 8 4 9 dst

37. Example: Core Internet (ASes) 5.1498110 62981111119 5411112188 2.52617 3392024206 36103411615 501017271124 361031433 5189817522 2296310451 Ghost Flushing BGPIn-degreeOut-degree

38. E_longer: Convergence Time dst 2 1 3 4 0 5 6 7

39. E_longer: ISP Topology

40. Conclusion Reduced convergence time from minutes to sec ’ s. Does not hurt in other cases Ghost flushing - no change at BGP messages Ghost buster solution – a new counting to infinity solution BGP very sensitive to minor modifications.