1. Rice University, SPiN Group spin.rice.edu Multiscale Traffic Processing Techniques for Network Inference and Control R. Baraniuk R. Nowak E. Knightly R. Riedi V. Ribeiro S. Sarvotham A. Keshavarz Rice University NMS PI meeting Chicago November 2002 Signal Processing in Networking

2. Rice University, SPiN Group spin.rice.edu Objective : Reduced complexity, multiscale link models with known accuracy Innovative Ideas Multifractal analysis Multiplicative modeling Multiscale queuing Chirps for probing Impact Congestion control performance improvement Dynamical streaming monitoring/anomaly detection

3. Rice University, SPiN Group spin.rice.edu Chirp Probing always-on, non-intrusive on-line decision anomaly detection Effort 1

4. Rice University, SPiN Group spin.rice.edu Efficient probing: PathChirp Traditional probing paradigm: –Produce (light) congestion –PacketPair: Sample the traffic –Pathload: flood at variable rate intolerable level of congestion –TOPP: PacketPairs at variable spacing New: –PathChirp: Variable rate within a train of probes More efficient, light

5. Rice University, SPiN Group spin.rice.edu PathChirp Performance Real world tool available performs comparably to –PathLoad –PacketPair –TOPP …at smaller probing rate …more robust to bursty traffic conditions Ongoing work: –Exploit dispersion information to become robust against multiple hops

6. Rice University, SPiN Group spin.rice.edu Non-intrusive: Queue Samplers Traffic arrival process and Queue process equivalent Advantages of sampling the Queue –Slim probe packets smaller probing rate or higher accuracy –Robust to Multi-Hop Delay simply accumulates while …probe dispersion is vulnerable (pair-compression at fast links) Delphi: –further advantage through Model-based inference (MSQ) Overcomes uncertainty principle Ongoing work: –optimal probing pattern Uniform / Back2Back / variable rate –optimal probing paradigm Traffic samplers / Queue samplers / Delphi –Integration into network simulators 30% util 80% util 50% util Error against scale Queue error Traffic error Delphi error Traffic error Queue error

7. Rice University, SPiN Group spin.rice.edu Multiple Hops: Collaboration Monitor packets at intermediate point of path –IP-tunneling –Coordinated Measuremens (IPEX: Krishnan) Integrating probing tools into simulators –JavaSim (UIUC: Hou) –NS-2 (ISI: Heidemann) Probing buffer at core router –Passive inference (Sprint Labs: Moon)

8. Rice University, SPiN Group spin.rice.edu Connection-level Analysis and Modeling of Network Traffic understanding the cause of bursts control and improve performance detect changes of network state Effort 2

9. Rice University, SPiN Group spin.rice.edu Non-Gaussianity and Dominance Systematic study: time series separation For each bin of 500 ms: remove packets of the ONE strongest connection Leaves “Gaussian” residual traffic Overall traffic Residual traffic1 Strongest connection = + Mean 99%

10. Rice University, SPiN Group spin.rice.edu Origins of Alpha Traffic Mostly TCP –flow & congestion controlled HTTP, SMTP, NNTP, etc. Several gateways, Abilene Core Alpha connections tend to have the same source- destination IP addresses. Any large volume connection with the same e2e IP addresses as an alpha connection is also alpha.  Systematic cause of alpha: Large file transfers over high bandwidth e2e paths.

11. Rice University, SPiN Group spin.rice.edu Simple Connection Taxonomy This is the only systematic reason Bursts arise from large transfers over fast links.

12. Rice University, SPiN Group spin.rice.edu Cwnd or RTT? Correlation coefficient=0.68 RTT has strong influence on bandwidth and dominance. peak-rate (Bps) 1/RTT (1/s) Correlation coefficient=0.01 10 3 4 5 6 2 3 4 5 peak-rate (Bps) cwnd (B) Colorado State University trace, 300,000 packets BetaAlphaBetaAlpha

13. Rice University, SPiN Group spin.rice.edu Examples of Alpha/Beta Connections Alpha connections burst because of short round trip time, not large TCP window Appear to be flow (not congestion) controlled

14. Rice University, SPiN Group spin.rice.edu Alpha traffic: Flow or Congestion Control? Alpha connection show –unusually small advertised window –drastic drop in advertised window (to zero) –…which correlates with burst arrival Alpha connections with same END host –Share same high peak rate –End-host becomes bottleneck, network does not control  TCP is known to be unfair to long RTT connections  TCP appears to foster also the Alpha bursts  What determines the alpha-peak rates?

15. Modeling Alpha Traffic ON/OFF model revisited: High variability in connection rates (RTTs) Low rate = betaHigh rate = alpha fractional Gaussian noise stable Levy noise + = + + =

16. Rice University, SPiN Group spin.rice.edu Impact: Network Simulation Simulation: ns topology to include alpha links Simple: equal bandwidth Realistic: heterogeneous end-to-end bandwidth Congestion control Design and management

17. Rice University, SPiN Group spin.rice.edu Impact: Traffic simulation -Classical ON/OFF model captures only -Gaussian Beta-component -Large scale fluctuations -Variable rate ON/OFF captures also -burstiness Realistic traffic contains ON/OFF sources with heterogeneous rates

18. Rice University, SPiN Group spin.rice.edu Ongoing work: simulation Relate physical network parameters to traffic dynamics –File size distributions –Network topology / RTT distribution –Network devices (links, servers) –Congestion control vs flow control Predict performance from simple set of parameters Integration into network simulators to abstract network clouds

19. Rice University, SPiN Group spin.rice.edu Impact: Performance Beta Traffic rules the small Queues Alpha Traffic causes the large Queue-sizes (despite small Window Size) All Alpha PacketsQueue-size with Alpha Peaks

20. Rice University, SPiN Group spin.rice.edu Ongoing Work Total true Queue: –Q = Q  +Q  (contributions from alpha and beta component) Model assumption: –Alpha=stable, Beta=fractional Brownian motion Optimal setup of two individual Queues driven by alpha and beta components to come closest to Q  and Q  quantify impact of components on queuing

21. Rice University, SPiN Group spin.rice.edu Separation on Connection Level Alpha connections: dominant. Properties: –Few, light load –Responsible for violent bursts –Responsible for large queuing delays –Peak rate > mean arrival rate + 1 std dev –Typically short RTT –Typically FLOW-CONTROLLED (limited at receiver) Beta connections: Residual traffic –Main load –Gaussian, LRD –Typically bandlimited at bottleneck link C+ analyzer (order of sec)

22. Rice University, SPiN Group spin.rice.edu Summary: Goals Network/user-driven traffic model –Obtain performance parameters directly from network and user specifications Non-intrusive, always-on, global traffic estimation –Anomaly detection in network (change in network state) Collaborations & Tech Transfer Monitoring tool using chirps (Caida, SLAC, Sprint [students] ) Verification/”internal” measurements using –IP-tunneling, coordinated measurements (IPEX) –Integration of PathChirp into network simulators (UIUC, ISI) Online traffic probing and analysis tools for –Integrated demo (GaTech) –Demystify self-similarity ( UC Riverside)

23. Rice University, SPiN Group spin.rice.edu spin.rice.edu