Estimation and identification of long-range dependence in Internet traffic Thomas Karagiannis University of California, Riverside / CAIDA

samsi 2003 Long-range dependence (LRD) LRD captures the memory of the behavior Past values affect the presentPast values affect the present LRD describes scaling properties of a series Statistical properties are independent of scale of observationStatistical properties are independent of scale of observation It is quantified by a single scalar number Hurst power-law exponentHurst power-law exponent LRD appears in many aspects of networks Traffic load, arrival times, delaysTraffic load, arrival times, delays Shift from the traditional Poisson modeling and independence assumption to LRD and heavy tail modelingShift from the traditional Poisson modeling and independence assumption to LRD and heavy tail modeling

samsi 2003 Questions regarding LRD How can we identify LRD? How can we estimate the intensity of LRD? Where does LRD exist? What causes LRD? What are the effects of LRD? How can we use LRD?

samsi 2003 LRD estimation Long-range dependence: Demonstrate the failure of the estimatorsDemonstrate the failure of the estimators Inaccuracy Inaccuracy Sensitivity Sensitivity Provide practical guidelines in LRD estimationProvide practical guidelines in LRD estimation SELFIS (Self – Similarity Analsysis) Software tool for LRD estimationSoftware tool for LRD estimation

samsi 2003 LRD estimation 1.Evaluating the accuracy of the estimators Synthetic fractional Gaussian noise (FGN) and fractional ARIMA time-seriesSynthetic fractional Gaussian noise (FGN) and fractional ARIMA time-series Large difference in estimator results for synthesized LRD series with known Hurst exponentLarge difference in estimator results for synthesized LRD series with known Hurst exponent 2.Evaluating the robustness of the estimators Periodicity, noise, trend, short-term correlations Periodicity, noise, trend, short-term correlations Estimation methodologies significantly affected Estimation methodologies significantly affected 3.Towards robust estimation: Provide practical guidelines and algorithms to achieve robust estimation (under submission) Provide practical guidelines and algorithms to achieve robust estimation (under submission)

samsi 2003 LRD Estimation: The SELFIS tool

samsi 2003 LRD Estimation: The SELFIS tool Design: Java-based, platform-independentJava-based, platform-independent Free, modularFree, modular Classes of function Self-Similarity and long-range dependence analysisSelf-Similarity and long-range dependence analysis Fractional Gaussian noise generatorsFractional Gaussian noise generators Data processing algorithms (smoothing, stationarity tests)Data processing algorithms (smoothing, stationarity tests)Benefits: Repeatability and consistencyRepeatability and consistency Leverage of expertise among different disciplinesLeverage of expertise among different disciplines Ease of useEase of use 200 Downloads from researchers spanning various disciplines and organizations

samsi 2003 LRD in backbone traffic Examine Internet traffic in the backbone (to appear in INFOCOM 2004) : OC48 link (2.4Gbps) traces taken by CAIDA monitors at a Tier 1 Internet Service Provider (ISP)OC48 link (2.4Gbps) traces taken by CAIDA monitors at a Tier 1 Internet Service Provider (ISP) Traces from the WIDE backbone (WIDE project)Traces from the WIDE backbone (WIDE project) Large number of independent sources Huge traffic multiplexingHuge traffic multiplexing

samsi 2003 LRD in backbone traffic Examine Poisson assumptions Study LRD in byte/packet counts at different time-scales Compare findings to historical Bellcore traces Identify the relevant time scale for LRD

samsi 2003 LRD in backbone traffic: Findings Time dependent Poisson characterization of network traffic that when viewed across very long time scales, exhibits the observed long- range dependence Backbone traffic Sub-second time-scales : PoissonSub-second time-scales : Poisson Multi-second time-scales : Piecewise-linear nonstationarityMulti-second time-scales : Piecewise-linear nonstationarity Large time-scales : Long-range dependenceLarge time-scales : Long-range dependence

samsi 2003 Conclusions LRD estimation: Identify caveatsIdentify caveats Propose solutionsPropose solutions LRD identification: LRD in backbone trafficLRD in backbone traffic Identify characteristics in different time-scalesIdentify characteristics in different time-scales