5 BGP Overview (contd.) Path Vector protocol Similar to Distance Vector routingLoop detection done using AS_PATH fieldR1R2Peering session (TCP)Exchange full routing table at startUpdates sent incrementally
6 The volume of BGP messages exchanged is Key PointThe volume of BGP messages exchanged isabnormally highMost messages are redundant / unnecessary and do notcorrespond to and topology or policy changes
7 Consequence: Instability Normal data packets handled by dedicated hardwareBGP packet processing consumes CPU timeSevere CPU processing overhead takes the router offlineRoute Flap Storm:BRouter A temporarily failsWhen A becomes alive B & Csend full routing tablesAB & C fail…cascading effectCHow do we avoid /lessen the impact of these problems?
8 Route Dampening Router does not accept frequent route updates to a destinationMight signal that network has erratic connectivityIncrement counter for destination when route changesCounter exceeds threshold stop accepting updatesDecrement counter with timeImpediment-legitimate updates delayedProblem:Future legitimate announcements are accepted onlyafter a delay
9 Prefix Aggregation/Super-netting Core router advertises a less specific network prefixReduces size of routing tables exchangedProblems:Prefix aggregation is not effective because:- Internet addresses largely non-hierarchically assignedImpediment: multihoming- Domain renumbering not done when changing ISP’s- 25% of prefixes multi-homed- Multi-homed prefixes should be exposed at the core
10 Route Servers O(N) peering sessions per Router 1 peering session per routerRouteServerIn spite of this data exposes instability-due to non-hierarchical nature of networkIn-spite of all these measures the BGP message overheadis unexpectedly high
11 Evaluation Methodology Data from Route Server at M.A.E west (D.C) peering pointPeering point for more than 60 major ISP’sNine month logTime series analysis of message exchange events
12 Observation: Lot’s of redundant updates Duplicate route with-drawlsISPNumber of With-drawlsUniqueRatioA2327643445F86417124357I14112175One Reason:- Stateless BGP- No state of previous with-drawls maintained
13 Observation: Instability Proportional to Activity After removing duplicate messages:Lessermessages10:00 AMISP infrastructure up-gradeInstability density with time6:0012:0018:0024:00Time of dayLogarithmic Z-axis
14 Evidence from Fine Grained Structure Number of instability events7 days24 hoursPower spectral densityFrequency (1/hour)Conjecture:BGP packets are competing with data packets duringhigh bandwidth activity.
15 Observation: Instability & size uncorrelated WADiffProportion of routing tableProportion of announcementsAADiffProportion of routing tableProportion of announcementsWADupProportion of routing tableProportion of announcementsISP’s serving more network prefixesmay not contribute more to instability
16 Observation: Instability distributed over routes 75% medianCumulative proportion10# of announcements per prefix+AS20% to 90% of routes change 10 times or lessNo single route contributes significantly to instability
17 Observation: Synchronized updates AADiffInter Arrival Time distribution forAADiff’sProportion30s1minInter-arrival times ofupdates shows periodicity30 s and 1 minute patternsSome routers collect and sendUpdates once every 30 sPossible reasons:Routers get synchronizedBorder router- Internal router: interaction misconfigured??
18 End-to-end Perspective Chinoy: “Dynamics of Internet routing information” (SIGCOMM 93)Measurements on NSFNET showed:- Processing and forwarding latency of BDP updateis 3 orders of magnitude more than the latency incurred inforwarding data packets- Will lead to packet drops during the intervening periodPaxson: “End-to-End routing behavior in the internet” (SIGCOMM 96)Routing loops introduce loops into other router’s routing tablesAn end-to-end route changes every 1.5 hours on an average
20 Summary and Conclusions Redundant routing information flows in coreInstability distributed across autonomous systemsPossible reasons for instability:Stateless BGP updatesMisconfigured routersSynchronizationClocks driving the links not synchronized (link “flaps”)
21 Follow-up work & impact “Origins of Internet Routing Instability”-1999Migration from stateless to stateful BGP decreased duplicate withdrawalsby an order of magnitudeBut Duplicate Announcements (AADup) doubledReason: Non-transitive attribute filtering not implemented- BGP specification: “never propagate non-transitive attributes”..- ASPATH is transitive attribute- MED (Multi Exit Discriminator) is NOT transitive