Download presentation

Presentation is loading. Please wait.

1
CMPT 855Module 7.0 1 Network Traffic Self-Similarity Carey Williamson Department of Computer Science University of Saskatchewan

2
CMPT 855Module 7.0 2 Introduction n A recent measurement study has shown that aggregate Ethernet LAN traffic is self-similar [Leland et al 1993] n A statistical property that is very different from the traditional Poisson-based models n This presentation: definition of network traffic self-similarity, Bellcore Ethernet LAN data, implications of self-similarity

3
CMPT 855Module 7.0 3 Measurement Methodology n Collected lengthy traces of Ethernet LAN traffic on Ethernet LAN(s) at Bellcore n High resolution time stamps n Analyzed statistical properties of the resulting time series data n Each observation represents the number of packets (or bytes) observed per time interval (e.g., 10 4 8 12 7 2 0 5 17 9 8 8 2...)

4
CMPT 855Module 7.0 4 Self-Similarity: The Intuition n If you plot the number of packets observed per time interval as a function of time, then the plot looks ‘‘the same’’ regardless of what interval size you choose n E.g., 10 msec, 100 msec, 1 sec, 10 sec,... n Same applies if you plot number of bytes observed per interval of time

5
CMPT 855Module 7.0 5 Self-Similarity: The Intuition n In other words, self-similarity implies a ‘‘fractal-like’’ behaviour: no matter what time scale you use to examine the data, you see similar patterns n Implications: u Burstiness exists across many time scales u No natural length of a burst u Traffic does not necessarilty get ‘‘smoother” when you aggregate it (unlike Poisson traffic)

6
CMPT 855Module 7.0 6 Self-Similarity: The Mathematics n Self-similarity is a rigourous statistical property (i.e., a lot more to it than just the pretty ‘‘fractal-like’’ pictures) n Assumes you have time series data with finite mean and variance (i.e., covariance stationary stochastic process) n Must be a very long time series (infinite is best!) n Can test for presence of self-similarity

7
CMPT 855Module 7.0 7 Self-Similarity: The Mathematics n Self-similarity manifests itself in several equivalent fashions: n Slowly decaying variance n Long range dependence n Non-degenerate autocorrelations n Hurst effect

8
CMPT 855Module 7.0 8 Slowly Decaying Variance n The variance of the sample decreases more slowly than the reciprocal of the sample size n For most processes, the variance of a sample diminishes quite rapidly as the sample size is increased, and stabilizes soon n For self-similar processes, the variance decreases very slowly, even when the sample size grows quite large

9
CMPT 855Module 7.0 9 Variance-Time Plot n The ‘‘variance-time plot” is one means to test for the slowly decaying variance property n Plots the variance of the sample versus the sample size, on a log-log plot n For most processes, the result is a straight line with slope -1 n For self-similar, the line is much flatter

10
Variance-Time Plot Variance m

11
Variance-Time Plot Variance m Variance of sample on a logarithmic scale 0.0001 0.001 10.0 0.01 100.0

12
Variance-Time Plot Variance m Sample size m on a logarithmic scale 11010010 4567

13
Variance-Time Plot Variance m

14
Variance-Time Plot Variance m

15
Variance-Time Plot Variance m Slope = -1 for most processes

16
Variance-Time Plot Variance m

17
Variance-Time Plot Variance m Slope flatter than -1 for self-similar process

18
CMPT 855Module 7.0 18 Long Range Dependence n Correlation is a statistical measure of the relationship, if any, between two random variables n Positive correlation: both behave similarly n Negative correlation: behave as opposites n No correlation: behaviour of one is unrelated to behaviour of other

19
CMPT 855Module 7.0 19 Long Range Dependence (Cont’d) n Autocorrelation is a statistical measure of the relationship, if any, between a random variable and itself, at different time lags n Positive correlation: big observation usually followed by another big, or small by small n Negative correlation: big observation usually followed by small, or small by big n No correlation: observations unrelated

20
CMPT 855Module 7.0 20 Long Range Dependence (Cont’d) n Autocorrelation coefficient can range between +1 (very high positive correlation) and -1 (very high negative correlation) n Zero means no correlation n Autocorrelation function shows the value of the autocorrelation coefficient for different time lags k

21
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient

22
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient Maximum possible positive correlation

23
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient Maximum possible negative correlation

24
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient No observed correlation at all

25
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient

26
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient Significant positive correlation at short lags

27
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient No statistically significant correlation beyond this lag

28
CMPT 855Module 7.0 28 Long Range Dependence (Cont’d) n For most processes (e.g., Poisson, or compound Poisson), the autocorrelation function drops to zero very quickly (usually immediately, or exponentially fast) n For self-similar processes, the autocorrelation function drops very slowly (i.e., hyperbolically) toward zero, but may never reach zero n Non-summable autocorrelation function

29
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient

30
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient Typical short-range dependent process

31
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient

32
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient Typical long-range dependent process

33
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient Typical long-range dependent process Typical short-range dependent process

34
CMPT 855Module 7.0 34 Non-Degenerate Autocorrelations n For self-similar processes, the autocorrelation function for the aggregated process is indistinguishable from that of the original process n If autocorrelation coefficients match for all lags k, then called exactly self-similar n If autocorrelation coefficients match only for large lags k, then called asymptotically self-similar

35
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient

36
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient Original self-similar process

37
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient Original self-similar process

38
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient Original self-similar process Aggregated self-similar process

39
CMPT 855Module 7.0 39 Aggregation n Aggregation of a time series X(t) means smoothing the time series by averaging the observations over non-overlapping blocks of size m to get a new time series X (t) m

40
CMPT 855Module 7.0 40 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... n Then the aggregated series for m = 2 is:

41
CMPT 855Module 7.0 41 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... n Then the aggregated series for m = 2 is:

42
CMPT 855Module 7.0 42 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... n Then the aggregated series for m = 2 is: 4.5 4.5

43
CMPT 855Module 7.0 43 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... n Then the aggregated series for m = 2 is: 4.5 8.0 4.5 8.0

44
CMPT 855Module 7.0 44 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... n Then the aggregated series for m = 2 is: 4.5 8.0 2.5 4.5 8.0 2.5

45
CMPT 855Module 7.0 45 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... n Then the aggregated series for m = 2 is: 4.5 8.0 2.5 5.0 4.5 8.0 2.5 5.0

46
CMPT 855Module 7.0 46 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... n Then the aggregated series for m = 2 is: 4.5 8.0 2.5 5.0 6.0 7.5 7.0 4.0 4.5 5.0... 4.5 8.0 2.5 5.0 6.0 7.5 7.0 4.0 4.5 5.0...

47
CMPT 855Module 7.0 47 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... Then the aggregated time series for m = 5 is:

48
CMPT 855Module 7.0 48 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... Then the aggregated time series for m = 5 is:

49
CMPT 855Module 7.0 49 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... Then the aggregated time series for m = 5 is: 6.0 6.0

50
CMPT 855Module 7.0 50 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... Then the aggregated time series for m = 5 is: 6.0 4.4 6.0 4.4

51
CMPT 855Module 7.0 51 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... Then the aggregated time series for m = 5 is: 6.0 4.4 6.4 4.8... 6.0 4.4 6.4 4.8...

52
CMPT 855Module 7.0 52 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... Then the aggregated time series for m = 10 is:

53
CMPT 855Module 7.0 53 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... Then the aggregated time series for m = 10 is: 5.2 5.2

54
CMPT 855Module 7.0 54 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... Then the aggregated time series for m = 10 is: 5.2 5.6 5.2 5.6

55
CMPT 855Module 7.0 55 Aggregation: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1... Then the aggregated time series for m = 10 is: 5.2 5.6... 5.2 5.6...

56
Autocorrelation Function +1 0 lag k 0 100 Autocorrelation Coefficient Original self-similar process Aggregated self-similar process

57
CMPT 855Module 7.0 57 The Hurst Effect n For almost all naturally occurring time series, the rescaled adjusted range statistic (also called the R/S statistic) for sample size n obeys the relationship E[R(n)/S(n)] = c n E[R(n)/S(n)] = c nwhere: R(n) = max(0, W,... W ) - min(0, W,... W ) S (n) is the sample variance, and W = X - k X for k = 1, 2,... n H 11nn 2 kin i =1 n

58
CMPT 855Module 7.0 58 The Hurst Effect (Cont’d) n For models with only short range dependence, H is almost always 0.5 n For self-similar processes, 0.5 < H < 1.0 n This discrepancy is called the Hurst Effect, and H is called the Hurst parameter n Single parameter to characterize self-similar processes

59
CMPT 855Module 7.0 59 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n There are 20 data points in this example

60
CMPT 855Module 7.0 60 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n There are 20 data points in this example n For R/S analysis with n = 1, you get 20 samples, each of size 1:

61
CMPT 855Module 7.0 61 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n There are 20 data points in this example n For R/S analysis with n = 1, you get 20 samples, each of size 1: Block 1: X = 2, W = 0, R(n) = 0, S(n) = 0 n1

62
CMPT 855Module 7.0 62 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n There are 20 data points in this example n For R/S analysis with n = 1, you get 20 samples, each of size 1: Block 2: X = 7, W = 0, R(n) = 0, S(n) = 0 n1

63
CMPT 855Module 7.0 63 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 2, you get 10 samples, each of size 2:

64
CMPT 855Module 7.0 64 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 2, you get 10 samples, each of size 2: Block 1: X = 4.5, W = -2.5, W = 0, R(n) = 0 - (-2.5) = 2.5, S(n) = 2.5, R(n)/S(n) = 1.0 n12

65
CMPT 855Module 7.0 65 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 2, you get 10 samples, each of size 2: Block 2: X = 8.0, W = -4.0, W = 0, R(n) = 0 - (-4.0) = 4.0, S(n) = 4.0, R(n)/S(n) = 1.0 n12

66
CMPT 855Module 7.0 66 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 3, you get 6 samples, each of size 3:

67
CMPT 855Module 7.0 67 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 3, you get 6 samples, each of size 3: Block 1: X = 4.3, W = -2.3, W = 0.3, W = 0 R(n) = 0.3 - (-2.3) = 2.6, S(n) = 2.05, R(n)/S(n) = 1.30 n12 3

68
CMPT 855Module 7.0 68 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 3, you get 6 samples, each of size 3: Block 2: X = 5.7, W = 6.3, W = 5.7, W = 0 R(n) = 6.3 - (0) = 6.3, S(n) = 4.92, R(n)/S(n) = 1.28 n12 3

69
CMPT 855Module 7.0 69 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 5, you get 4 samples, each of size 5:

70
CMPT 855Module 7.0 70 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 5, you get 4 samples, each of size 4: Block 1: X = 6.0, W = -4.0, W = -3.0, W = -5.0, W = 1.0, W = 0, S(n) = 3.41, R(n) = 1.0 - (-5.0) = 6.0, R(n)/S(n) = 1.76 n12 345

71
CMPT 855Module 7.0 71 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 5, you get 4 samples, each of size 4: Block 2: X = 4.4, W = -4.4, W = -0.8, W = -3.2, W = 0.4, W = 0, S(n) = 3.2, R(n) = 0.4 - (-4.4) = 4.8, R(n)/S(n) = 1.5 n12 345

72
CMPT 855Module 7.0 72 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 10, you get 2 samples, each of size 10:

73
CMPT 855Module 7.0 73 R/S Statistic: An Example n Suppose the original time series X(t) contains the following (made up) values: 2 7 4 12 5 0 8 2 8 4 6 9 11 3 3 5 7 2 9 1 n For R/S analysis with n = 20, you get 1 sample of size 20:

74
CMPT 855Module 7.0 74 R/S Plot n Another way of testing for self-similarity, and estimating the Hurst parameter n Plot the R/S statistic for different values of n, with a log scale on each axis n If time series is self-similar, the resulting plot will have a straight line shape with a slope H that is greater than 0.5 n Called an R/S plot, or R/S pox diagram

75
R/S Pox Diagram R/S Statistic Block Size n

76
R/S Pox Diagram R/S Statistic Block Size n R/S statistic R(n)/S(n) on a logarithmic scale

77
R/S Pox Diagram R/S Statistic Block Size n Sample size n on a logarithmic scale

78
R/S Pox Diagram R/S Statistic Block Size n

79
R/S Pox Diagram R/S Statistic Block Size n Slope 0.5

80
R/S Pox Diagram R/S Statistic Block Size n Slope 0.5

81
R/S Pox Diagram R/S Statistic Block Size n Slope 0.5 Slope 1.0

82
R/S Pox Diagram R/S Statistic Block Size n Slope 0.5 Slope 1.0

83
R/S Pox Diagram R/S Statistic Block Size n Slope 0.5 Slope 1.0 Self- similar process

84
R/S Pox Diagram R/S Statistic Block Size n Slope 0.5 Slope 1.0 Slope H (0.5 < H < 1.0) (Hurst parameter)

85
CMPT 855Module 7.0 85 Summary n Self-similarity is an important mathematical property that has recently been identified as present in network traffic measurements n Important property: burstiness across many time scales, traffic does not aggregate well n There exist several mathematical methods to test for the presence of self-similarity, and to estimate the Hurst parameter H n There exist models for self-similar traffic

Similar presentations

© 2020 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google