1. Advanced Technology Laboratories Traffic Matrix Estimation in Non- Stationary Environments Presented by R. L. Cruz Department of Electrical & Computer Engineering University of California, San Diego Joint work with Antonio Nucci Nina Taft Christophe Diot NISS Affiliates Technology Day on Internet Tomography March 28, 2003

2. Advanced Technology Laboratories page 2 The Traffic Matrix Estimation Problem Formulated in Y. Vardi, “Network Tomography: Estimating Source-Destination Traffic From Link Data,” JASA, March 1995, Vol. 91, No. 433, Theory & Methods

3. Advanced Technology Laboratories page 3 The Traffic Matrix Estimation Problem ingress egress XjXj XjXj YiYi PoP (Point of Presence) Y = A X Link Measurement Vector Routing Matrix “Traffic Matrix”

4. Advanced Technology Laboratories page 4 The Traffic Matrix Estimation Problem Importance of Problem: capacity planning, routing protocol configuration, load balancing policies, failover strategies, etc. Difficulties in Practice –missing data –synchronization of measurements (SNMP) –Non-Stationarity (our focus here) long convergence time needed to obtain estimates

5. Advanced Technology Laboratories page 5 What is Non-Stationary? Traffic Itself is Non-Stationary

6. Advanced Technology Laboratories page 6 What is Non-Stationary? Also, Routing is Non-Stationary –e.g. Due to Link Failures –Essence of Our Approach Purposely reconfigure routing in order to help estimate traffic matrix –More information leads to more accurate estimates Effectively increases rank of A We have developed algorithms to reconfigure the routing for this purpose (beyond the scope of this talk)

7. Advanced Technology Laboratories page 7 Outline of Remainder of Talk Describe the “Stationary” Method –Stationary traffic, non-stationary routing –Stationary traffic assumption is reasonable if we always measure traffic at the same time of day (e.g. “peak period” of a work day) Briefly Describe the “Non-Stationary” Method –Both non-stationary traffic and non-stationary routing –More complex but allows estimates to be obtained much faster

8. Advanced Technology Laboratories page 8 Network and Measurement Model Network with L links, N nodes, P=N(N-1) OD pair flows –K measurement intervals, 1 ≤ k ≤ K –Y(k) is the link count vector at time k: (L x 1) –A(k) is the routing matrix at time k: (L x P) –X(k) is the O-D pair traffic vector at time k: (P x 1) X(k) = (x 1 (k), x 2 (k), … x P (k)) T Y(k) = A(k) X(k) Y(k) and A(k) can be truncated to reflect missing and redundant data

9. Advanced Technology Laboratories page 9 Traffic Model: Stationary Case X(k) is the O-D pair traffic vector at time k: (P x 1) X(k) = (x 1 (k), x 2 (k), … x P (k)) T X(k) = X + W(k) W(k) : “Traffic Fluctuation Vector Zero mean, covariance matrix B B = diag(X)

10. Advanced Technology Laboratories page 10 Matrix Notation where: Linear system of equations: [LK][LK][P][LK][KP][KP][P] Choose Routing Configurations such that Rank(A) = P

11. Advanced Technology Laboratories page 11 Traffic matrix Estimation-Stationary Case Initial Estimate: Use Psuedo-Inverse of A: - does not require statistics of W (covariance B) Gauss-Markov Theorem: Assume B is known - Unbiased, minimum variance estimate - Coincides with Maximum Likelihood Estimate if W is Gaussian Y = AX + CW

12. Advanced Technology Laboratories page 12 Traffic matrix Estimation-Stationary Case Y = AX + CW Minimum Estimation Error: (assumes B is known)

13. Advanced Technology Laboratories page 13 Traffic matrix Estimation-Stationary Case Recall we assume B = cov(W) satisfies B = diag(X) Set Recursion for Estimates:

14. Advanced Technology Laboratories page 14 Traffic matrix Estimation-Stationary Case Our estimate is a solution to the equation: Open questions for fixed point equation: - Existence of Solution? - Uniqueness? - Is solution an un-biased estimate?

15. Advanced Technology Laboratories page 15 Numerical Example-Stationary case N=10 nodes, L=24 links and P=90 connections. Three set of OD pairs with mean x equal to: –500 Mbps, 2 Gbps and 4 Gbps. Gaussian Traffic Fluctuations:

16. Advanced Technology Laboratories page 16 Stationary case: b=1 Samples/Snapshot=1

17. Advanced Technology Laboratories page 17 Stationary case: b=1 Samples/Snapshot=1

18. Advanced Technology Laboratories page 18 Stationary case: b=1 Samples/Snapshot=15

19. Advanced Technology Laboratories page 19 Stationary case: b=1 Samples/Snapshot=15

20. Advanced Technology Laboratories page 20 Stationary case: b=1.4 Samples/Snapshot=1

21. Advanced Technology Laboratories page 21 Stationary case: b=1.4 Samples/Snapshot=1

22. Advanced Technology Laboratories page 22 Stationary case: b=1.4 Samples/Snapshot=15

23. Advanced Technology Laboratories page 23 Stationary case: b=1.4 Samples/Snapshot=15

24. Advanced Technology Laboratories page 24 Stationary and Non-Stationary traffic 20 snapshots / 4 samples per snapshot / 5 min per sample Stationary Approach: 20 min per day (same time) / 20 days Non-Stationary Approach: aggregate all the samples in one window time large 400 min (7 hours)

25. Advanced Technology Laboratories page 25 Traffic Model: Non-Stationary Case Each OD pair is cyclo-stationary: Each OD pair is modeled as: Fourier series expansion:

26. Advanced Technology Laboratories page 26 Mean estimation Results-Non Stationary case Three set of OD pairs where are linear independent Gaussian variables with:

27. Advanced Technology Laboratories page 27 Non Stationary case: b=1 Link Count

28. Advanced Technology Laboratories page 28 Non Stationary case: b=1 Mean estimation