1. 1AT&T Labs - Research SNMP Simple Network Measurements Please! Matthew Roughan roughan@research.att.com http://swallow.research.att.com/roughan/SNAP roughan@research.att.com

2. 2AT&T Labs - Research Outline zPart I: SNMP traffic data ySimple Network Management Protocol yWhy? How? What? zPart II: Modeling yPutting time series and traffic modeling together xTraffic modeling deals with stationary processes (typically) xTime series gives us a way of getting a stationary process xBut the analysis requires an understanding of the traffic model yApplications, and algorithms

3. 3AT&T Labs - Research Part I: SNMP Traffic Data

4. 4AT&T Labs - Research Data Availability – Traffic Data

5. 5AT&T Labs - Research Data Availability – packet traces Packet traces limited availability – like a high zoom snap shot special equipment needed (O&M expensive even if box is cheap) lower speed interfaces (only recently OC48 available, no OC192) huge amount of data generated

6. 6AT&T Labs - Research Data Availability – flow level data Flow level data not available everywhere – like a home movie of the network historically poor vendor support (from some vendors) large volume of data (1:100 compared to traffic) feature interaction/performance impact

7. 7AT&T Labs - Research Data Availability – SNMP SNMP traffic data – like a time lapse panorama MIB II (including IfInOctets/IfOutOctets) is available almost everywhere manageable volume of data no significant impact on router performance

8. 8AT&T Labs - Research SNMP zAdvantages yComparatively simple yRelatively low volume yIt is used already (lots of historical data) zDisadvantages yData quality – an issue with any data source xAmbiguous xMissing data xIrregular sampling yOctets counters only tell you link utilizations xHard to get a traffic matrix xCan’t tell what type of traffic xCan’t easily detect DoS, or other unusual events yCoarse time scale (>1 minute typically) xLack of well tested relationship between coarse time-scale averages and performance (hence active perf. measurement)

9. 9AT&T Labs - Research Datasets zSNMP yOC48 tool xHistorical view of high speed and peering links from CBB yAT&T Broadband xA sample of broadband access traffic yCompass xThe whole CBB, not just traffic, also router view zRelated datasets yNetflow (see Carsten Lund, Fred True) xDrill down to see what sort of traffic yNetdb (see Joel Gottlieb) xNetwork topology and configuration data

10. 10AT&T Labs - Research SNMP traffic data SNMP Polls SNMP Octets Counter poller router poll data Like an Odometer 999408 Management system agent

11. 11AT&T Labs - Research Irregularly sampled data zWhy? yMissing data (transport over UDP, often in-band) yDelays in polling (jitter) yPoller sync xMultiple pollers xStaggered polls zWhy care? yTime series analysis yComparisons between links xDid traffic shed from link A go to link B xCalculation of traffic matrices yTotals (e.g. total traffic to Peer X) yCorrelation to other data sources xDid event BGP route change at time T effects links A,B,C,…

12. 12AT&T Labs - Research Applications zCapacity planning yNetwork at the moment is “hand-crafted” yWant to automate processes yProvisioning for failure scenarios requires adding loads zTraffic engineering yEven if done by hand, you need to see results yBGP zEvent detection yDetect network problems zBusiness cases yHelp sales and marketing make cases zBilling y95 th percentile billing

13. 13AT&T Labs - Research Part II: Modeling zPutting together theory from yTime series analysis yTraffic theory zTo SNMP data yIn particular for backbone traffic

14. 14AT&T Labs - Research Total traffic into a city for 2 weeks

15. 15AT&T Labs - Research Model zTraffic data has several components yTrend, T t xLong term changes in traffic ySeasonal (periodic) component, S t xDaily and weekly cycles yStationary stochastic component, W t xNormal variation yTransient anomalies, I t xDoS, Flash crowds, Rerouting (BGP, link failures) zmany ways you could combine these components ystandard time series analysis xSum X t = T t + S t + W t + I t xProduct X t = T t S t W t I t xBox-Cox transform

16. 16AT&T Labs - Research A Simple Model (for backbone traffic) zBased on Norros model zNon-stationary mean zStochastic component unspecified (for the moment)

17. 17AT&T Labs - Research Why this model? zBehaves as expected under multiplexing zGood model for backbone traffic yLots of multiplexing zSimple, estimable parameters, flexible, can make predictions, data supports it

18. 18AT&T Labs - Research What does a model get you? zDecomposition yMA for trend (window > period of seasonal component) ySMA for seasonal component (average at same time of day/week) ySeveral methods for segmenting I t zAnomaly detection zInterpolation yLinear, or wavelet based for short gaps (<3 hours) yModel based for long gaps (>3 hours) zHow smooth is backbone traffic (is it LRD) zCapacity planning yEstimation of traffic matrices

19. 19AT&T Labs - Research Example: decomposition Data => Decomposition

20. 20AT&T Labs - Research Example: interpolation zModel based vs linear

21. 21AT&T Labs - Research Example: anomaly detection zWavelet based

22. 22AT&T Labs - Research Conclusion zSNMP is a good data source yAvailable everywhere yYou need to do some work to extract useful data zTraffic model gives you more yA framework for other algorithms yA way to decide what information is important yA way of seeing how smooth traffic really is xEffect of multiplexing zThere is still more info. to get ypacket traces, flow data, … zAlgorithms are applicable to other traffic data zFor more details have a look at http://swallow.research.att.com/roughan/SNAP