1AT&T Labs - Research SNMP Simple Network Measurements Please! Matthew Roughan

2AT&T Labs - Research Outline zPart I: SNMP traffic data ySimple Network Management Protocol yWhy? How? What? zPart II: Modeling yPutting time series and traffic modeling together xTraffic modeling deals with stationary processes (typically) xTime series gives us a way of getting a stationary process xBut the analysis requires an understanding of the traffic model yApplications, and algorithms

3AT&T Labs - Research Part I: SNMP Traffic Data

4AT&T Labs - Research Data Availability – Traffic Data

5AT&T Labs - Research Data Availability – packet traces Packet traces limited availability – like a high zoom snap shot special equipment needed (O&M expensive even if box is cheap) lower speed interfaces (only recently OC48 available, no OC192) huge amount of data generated

6AT&T Labs - Research Data Availability – flow level data Flow level data not available everywhere – like a home movie of the network historically poor vendor support (from some vendors) large volume of data (1:100 compared to traffic) feature interaction/performance impact

7AT&T Labs - Research Data Availability – SNMP SNMP traffic data – like a time lapse panorama MIB II (including IfInOctets/IfOutOctets) is available almost everywhere manageable volume of data no significant impact on router performance

8AT&T Labs - Research SNMP zAdvantages yComparatively simple yRelatively low volume yIt is used already (lots of historical data) zDisadvantages yData quality – an issue with any data source xAmbiguous xMissing data xIrregular sampling yOctets counters only tell you link utilizations xHard to get a traffic matrix xCan’t tell what type of traffic xCan’t easily detect DoS, or other unusual events yCoarse time scale (>1 minute typically) xLack of well tested relationship between coarse time-scale averages and performance (hence active perf. measurement)

9AT&T Labs - Research Datasets zSNMP yOC48 tool xHistorical view of high speed and peering links from CBB yAT&T Broadband xA sample of broadband access traffic yCompass xThe whole CBB, not just traffic, also router view zRelated datasets yNetflow (see Carsten Lund, Fred True) xDrill down to see what sort of traffic yNetdb (see Joel Gottlieb) xNetwork topology and configuration data

10AT&T Labs - Research SNMP traffic data SNMP Polls SNMP Octets Counter poller router poll data Like an Odometer Management system agent

11AT&T Labs - Research Irregularly sampled data zWhy? yMissing data (transport over UDP, often in-band) yDelays in polling (jitter) yPoller sync xMultiple pollers xStaggered polls zWhy care? yTime series analysis yComparisons between links xDid traffic shed from link A go to link B xCalculation of traffic matrices yTotals (e.g. total traffic to Peer X) yCorrelation to other data sources xDid event BGP route change at time T effects links A,B,C,…

12AT&T Labs - Research Applications zCapacity planning yNetwork at the moment is “hand-crafted” yWant to automate processes yProvisioning for failure scenarios requires adding loads zTraffic engineering yEven if done by hand, you need to see results yBGP zEvent detection yDetect network problems zBusiness cases yHelp sales and marketing make cases zBilling y95 th percentile billing

13AT&T Labs - Research Part II: Modeling zPutting together theory from yTime series analysis yTraffic theory zTo SNMP data yIn particular for backbone traffic

14AT&T Labs - Research Total traffic into a city for 2 weeks

15AT&T Labs - Research Model zTraffic data has several components yTrend, T t xLong term changes in traffic ySeasonal (periodic) component, S t xDaily and weekly cycles yStationary stochastic component, W t xNormal variation yTransient anomalies, I t xDoS, Flash crowds, Rerouting (BGP, link failures) zmany ways you could combine these components ystandard time series analysis xSum X t = T t + S t + W t + I t xProduct X t = T t S t W t I t xBox-Cox transform

16AT&T Labs - Research A Simple Model (for backbone traffic) zBased on Norros model zNon-stationary mean zStochastic component unspecified (for the moment)

17AT&T Labs - Research Why this model? zBehaves as expected under multiplexing zGood model for backbone traffic yLots of multiplexing zSimple, estimable parameters, flexible, can make predictions, data supports it

18AT&T Labs - Research What does a model get you? zDecomposition yMA for trend (window > period of seasonal component) ySMA for seasonal component (average at same time of day/week) ySeveral methods for segmenting I t zAnomaly detection zInterpolation yLinear, or wavelet based for short gaps (<3 hours) yModel based for long gaps (>3 hours) zHow smooth is backbone traffic (is it LRD) zCapacity planning yEstimation of traffic matrices

19AT&T Labs - Research Example: decomposition Data => Decomposition

20AT&T Labs - Research Example: interpolation zModel based vs linear

21AT&T Labs - Research Example: anomaly detection zWavelet based

22AT&T Labs - Research Conclusion zSNMP is a good data source yAvailable everywhere yYou need to do some work to extract useful data zTraffic model gives you more yA framework for other algorithms yA way to decide what information is important yA way of seeing how smooth traffic really is xEffect of multiplexing zThere is still more info. to get ypacket traces, flow data, … zAlgorithms are applicable to other traffic data zFor more details have a look at