Self-Similar through High-Variability: Statistical Analysis of Ethernet LAN Traffic at the Source Level Walter Willinger, Murad S. Taqqu, Robert Sherman, Daniel V. Wilson Bellcore, Boston University SIGCOMM’95

Outline Introduction Self-similarity through high-variability Ethernet LAN traffic measurements at the source level Implications of the Noah Effect in practice Conclusion

Introduction Actual traffic exhibits correlations over a wide range of time scales (i.e. has long- range dependence). Traditional traffic models focus on a very limited range of time scales and are thus short-range dependent in nature.

Introduction Two problems that cause the resistance toward self-similar traffic modeling What is a physical “explanation” for the observed self-similar nature of measured traffic from today’s packet networks? What is the impact of self-similarity on network and protocol design and performance analysis?

Introduction The superposition of many ON/OFF sources whose ON-periods and OFF-periods exhibit the Noah Effect produces aggregate network traffic that features the Joseph Effect. Noah Effect: high variability or infinite variance Joseph Effect: self-similar or long-range dependent

Self-Similarity through High-Variability Idealized ON/OFF model An ON-period can be followed by an ON- period and an OFF-period can succeed another OFF-period. The distributions of the ON and OFF times may vary.

Idealized ON/OFF Model Reward sequence {W( l ), l = 0,1,2,…} {W( l )} is a 0/1-valued discrete time stochastic process. W( l ) = 1 or 0 depends on whether or not there is a packet at time l. {W( l )} consists of a sequence of 1’s (“ON- periods”) and 0’s (“OFF-periods”)

Idealized ON/OFF Model The lengths of the ON- and OFF-periods are i.i.d. positive random variables, denoted U k, k = 1,2,… Let S k = S 0 + U 2 + … + U k, k  0 be the corresponding renewal times.

Idealized ON/OFF Model Suppose there are M i.i.d. sources The m th source has its own reward sequence {W( l ), l  0} Superposition reward (“packet load”) b: non-overlapping time blocks j: the aggregation block number

Idealized ON/OFF Model Suppose that U has a hyperbolic tail distribution, as M  and b  , adequately normalized is fractional Gaussian noise, which is self-similar with Hurst parameter ½  H <1

Idealized ON/OFF Model Property (1) is the infinite variance syndrome or the Noah Effect.   2 implies E(U 2 ) =   > 1 ensures that E(U) < , and that S 0 is not infinite

Idealized ON/OFF Model Theorem 1. For large enough source Number M and Block aggregation size b, the cumulative load behaves statistically as where and. More precisely, where L lim means convergence in the sense of the finite- dimensional distributions (convergence in law)

Ethernet LAN Traffic Measurements at the Source Level Location Bellcore Morristown Research and Engineering Center The first set The busy hour of the August 1989 Ethernet LAN measurements About 105 sources, 748 active source-destination pairs 95% of the traffic was internal The second set 9 day-long measurement period in December 1994 About 3,500 sources, 10,000 active pairs Measurements are made up entirely of remote traffic

Textured Plots of Packet Arrival Times

Checking for the Noah Effect Complementary distribution plots Hill’s estimate Let U 1, U 2,…, U n denote the observed ON-(or OFF-)periods and write U (1)  U (2)  …  U (n) for the corresponding order statistics

A Robustness Property of the Noah Effect As far as the Noah Effect is concerned, it does not matter how the OFF-periods have been defined. The similar investigation of sensitivity of the ON- period distributions to the choice of threshold value reveals the same appealing robustness feature of the Noah Effect.

Self-Similarity and the Noah Effect: 1989 Traffic Traces 181(out of 748) source-destination pairs generated more than 93% of all the packets are considered. The data at the source-destination level are consistent with ON/OFF modeling assumption Noah Effect for the distribution of ON/OFF-periods  -values for the ON- and OFF-periods may be different.

Self-Similarity and the Noah Effect: 1994 Traffic Traces Non-Mbone traffic 300 (out of 10,000) pairs responsible for 83% of the traffic are considered. Self-similarity property of the aggregate packet stream is mainly due to the relative strong presence of the Noah Effect in the OFF-periods.

Self-Similarity and the Noah Effect: 1994 Traffic Traces Mbone traffic Only an analysis of the aggregate packet stream is performed. The strong intensity of the Joseph Effect become obvious only after aggregation levels beyond 100ms. There is no Noah Effect for ON-periods. Reason: The use of unsophisticated compression algorithms resulted in packets bursts separated by comparatively large idle periods.

Traffic Modeling and Generation Although network traffic is intrinsically complex, parsimonious modeling is still possible. Estimating a single parameter  (intensity of the Noah Effect) is enough.

Performance and Protocol Analysis The queue length distribution Traditional (Markovian) traffic: decreases exponentially fast Self-similar traffic: decreases much more slowly Protocol design should be expected to take into account knowledge about network traffic such as the presence or absence of the Noah Effect.

Conclusion The presence of the Noah Effect in measured Ethernet LAN traffic is confirmed. The superposition of many ON/OFF models with Noah Effect results in aggregate packet streams that are consistent with measured network traffic, and exhibits the self-similar or fractal properties.