1. Multiscale Traffic Processing Techniques for Network Inference and Control
Richard Baraniuk Edward Knightly
Robert Nowak Rolf Riedi
Rice University INCITE Project
April 2001

2. Rice University | INCITE.rice.edu | April 2001
InterNet Control and Inference Tools at the Edge
Overall Objective: Scalable, edge-based tools for on-line network analysis, modeling, and measurement
Theme for DARPA NMS Research: Multiscale traffic analysis, modeling, and processing via multifractals
Expertise: Statistical signal processing, mathematics, network QoS
Rice University | INCITE.rice.edu | April 2001

3. Rice University | INCITE.rice.edu | April 2001
Technical Challenges
Poor understanding of origins of complex network dynamics
Lack of adequate modeling techniques for network dynamics
Internal network inaccessible
Need: Manageable, reduced-complexity models with characterizable accuracy
Rice University | INCITE.rice.edu | April 2001

4. Rice University | INCITE.rice.edu | April 2001
Multiscale modeling
Rice University | INCITE.rice.edu | April 2001

5. Rice University | INCITE.rice.edu | April 2001
Multiscale Analysis
Time
Multiscale statistics
Var1
Scale
Var2
Var3
Analysis: flow up the tree by adding
Varj
Start at bottom with trace itself
Rice University | INCITE.rice.edu | April 2001

6. Rice University | INCITE.rice.edu | April 2001
Multiscale Synthesis
Time
Start at top with total arrival
Multiscale parameters
Var1
Scale
Var2
Var3
Synthesis: flow down via innovations
Varj
Signal: bottom nodes
Rice University | INCITE.rice.edu | April 2001

7. Multifractal Wavelet Model (MWM)
Random multiplicative innovations Aj,k on [0,1] eg: beta
Parsimonious modeling (one parameter per scale)
Strong ties with rich theory of multifractals
Rice University | INCITE.rice.edu | April 2001

8. Multiscale Traffic Trace Matching
Auckland 2000
MWM match
4ms
16ms
64ms
Rice University | INCITE.rice.edu | April 2001

9. Rice University | INCITE.rice.edu | April 2001
Multiscale Queuing
Rice University | INCITE.rice.edu | April 2001

10. Rice University | INCITE.rice.edu | April 2001
Probing the Network
Rice University | INCITE.rice.edu | April 2001

11. Probing Ideally: delay spread of packet pair spaced by T sec
correlates with
cross-traffic volume at time-scale T
Rice University | INCITE.rice.edu | April 2001

12. Probing Uncertainty Principle
Should not allow queue to empty between probe packets
Small T for accurate measurements
but probe traffic would disturb cross-traffic (and overflow bottleneck buffer!)
Larger T leads to measurement uncertainties
queue could empty between probes
To the rescue: model-based inference
Rice University | INCITE.rice.edu | April 2001

13. Multifractal Cross-Traffic Inference
Model bursty cross-traffic using MWM
Rice University | INCITE.rice.edu | April 2001

14. Efficient Probing: Packet Chirps
MWM tree inspires geometric chirp probe
MLE estimates of cross-traffic at multiple scales
Rice University | INCITE.rice.edu | April 2001

15. Chirp Probe Cross-Traffic Inference
Rice University | INCITE.rice.edu | April 2001

16. Rice University | INCITE.rice.edu | April 2001
ns-2 Simulation
Inference improves with increased utilization
Low utilization (39%)
High utilization (65%)
Rice University | INCITE.rice.edu | April 2001

17. ns-2 Simulation (Adaptivity)
Inference improves as MWM parameters adapt
MWM parameters
Inferred x-traffic
Rice University | INCITE.rice.edu | April 2001

18. Adaptivity (MWM Cross-Traffic)
Eg: Route changes
Rice University | INCITE.rice.edu | April 2001

19. Comparing Probing schemes
Rice University | INCITE.rice.edu | April 2001

20. Comparing probing schemes
`Classical’:
Bandwidth estimation by packet pairs and
trains
Novel: Traffic estimation, probing best by
Uniform?
Poisson?
Chirp?
Rice University | INCITE.rice.edu | April 2001

21. Rice University | INCITE.rice.edu | April 2001
Model based Probing
Chirp:
model based, superior
Uniform:
Uncertainty increases error
Rice University | INCITE.rice.edu | April 2001

22. Impact of Probing on Performance
Heavy
Heavy probing
- reduces bandwidth
- increases loss
- inflicts time-outs
NS-simulation:
Same `web-traffic’ with
variable probing rates
Light
Rice University | INCITE.rice.edu | April 2001

23. Influence of probing rate on error
Chirp probing performing uniformly good
Uniform requires higher rates to perform
Rice University | INCITE.rice.edu | April 2001

24. Rice University | INCITE.rice.edu | April 2001
Synergies
SAIC (Warren):
MWM code for real time simulator
SLAC (Cottrell, Feng):
Modify PingER for chirp-probing
High performance networks
Demo:
C-code for real world chirp-probing
using NetDyn (TCP) + simple Daemon at receiver
(INRIA France, UFMG Brazil, Michigan State)
Rice University | INCITE.rice.edu | April 2001

25. INCITE: Near-term / Ongoing
Verification with real Internet experiments
Rice testbed (practical issues)
SAIC (real time algorithms)
SLAC / ESNet (real world verification)
Enhancements:
rigorous statistical error analysis
deal with random losses
multiple bottleneck queues (see demo)
passive monitoring (novel models)
closed loop paths/feedback (ns-simulation)
Rice University | INCITE.rice.edu | April 2001

26. INCITE: Longer-Term Goals
New traffic models, inference algorithms
theory, simulation, real implementation
Applications to Control, QoS, Network Meltdown early warning
Leverage from our other projects
ATR program (DARPA, ONR, ARO)
RENE (Rice Everywhere Network:NSF)
NSF ITR
DoE
Rice University | INCITE.rice.edu | April 2001

27. Stationary multifractals
40 Minutes
35 Minutes: instead of 3 slides on multifractal bursts just one summary (see end)
Rice University | INCITE.rice.edu | April 2001

28. Stationary multiplicative models
j(s): stationary, indep., E[j(s)]=1
A(t) = lim 0t 1(s) 2(s)… n(s) ds
May degenerate (compare: MWM is conservative)
stationary increments
Assume j(2j s) are i.i.d.; Renewal reward
Compare MWM: j(2j s) constant over [k,k+1]
If Var()<1: Convergence in L2 ; E[A(t)]=t
Multifractal function: T(q)=q-log2E[q]
Compare: MWM has Lambda that is constant in integer intervals: deterministic geometry
Rice University | INCITE.rice.edu | April 2001

29. Rice University | INCITE.rice.edu | April 2001
Simulation
L2 criterion for convergence translates to
T(2)>0
Conjecture: For q>1 converge in Lq if T(q)>0
Thus non-degenerate
iff T’(1)>0, ie E[ L log (L /2) ] >0
Rice University | INCITE.rice.edu | April 2001

30. Rice University | INCITE.rice.edu | April 2001
Parameter estimation
No conservation: can’t isolate multipliers
Possible correlation within multipliers
IID values:
Z(s) = log [ 1(s) 2(s)… n(s) ]
Cov(Z(t)Z(t+s))= Si=1..n exp(-lis)Var i(s)
`LRD-scaling’ at medium scales, but SRD. Multifractal subordination -> true LRD.
One can thus fit the auto-correlation and induce the parameters li and Var i(s)
For MWM in log domain:
Cov(Z(t)Z(t+s))= S Var log[i(s)]
Where the sum goes now over all I such that t and t+s are in the same
Dyadic interval of order I.
Recursive computation of var possible.
Advantage of working in real time (not log time): can compute the actual
i(s) for all i and s (step functions) because of conservation of mass, thus
Obtain distr, not only var.
Rice University | INCITE.rice.edu | April 2001

31. Rice University | INCITE.rice.edu | April 2001