Presentation is loading. Please wait.

Presentation is loading. Please wait.

Bandwidth estimation in computer networks: measurement techniques & applications Constantine Dovrolis College of Computing Georgia Institute of Technology.

Similar presentations

Presentation on theme: "Bandwidth estimation in computer networks: measurement techniques & applications Constantine Dovrolis College of Computing Georgia Institute of Technology."— Presentation transcript:

1 Bandwidth estimation in computer networks: measurement techniques & applications Constantine Dovrolis College of Computing Georgia Institute of Technology

2 The Internet as a “black box” Several network properties are important for applications and transport protocols: Delay, loss rate, capacity, congestion, load, etc But routers and switches do not provide direct feedback to end-hosts (except ICMP, also of limited use) Mostly due to scalability, policy, and simplicity reasons

3 Can we guess what is in the black box? End-systems can infer network state through end-to-end (e2e) measurements Without any feedback from routers Objectives: accuracy, speed, non-intrusiveness probe packets

4 Bandwidth estimation in packet networks Bandwidth estimation (bwest) area: Inference of various throughput-related metrics with end-to-end network measurements Early works: Keshav’s packet pair method for congestion control (‘89) Bolot’s capacity measurements (’93) Carter-Crovella’s bprobe and cprobe tools (’96) Jacobson’s pathchar - per-hop capacity estimation (‘97) Melander’s TOPP method for avail-bw estimation (’00) In last 3-5 years: Several fundamentally new estimation techniques Many research papers at prestigious conferences More than a dozen of new measurement tools Several applications of bwest methods Significant commercial interest in bwest technology

5 Overview This talk is an overview of the most important developments in bwest area over the last 10 years A personal bias could not be avoided.. Overview Bandwidth-related metrics Capacity estimation Packet pairs and CapProbe technique Available bandwidth estimation Iterative probing: Pathload, Pathvar and PathChirp Comparison of tools Applications SOcket Buffer Auto-Sizing (SOBAS)

6 Network bandwidth metrics Ravi S.Prasad, Marg Murray, K.C. Claffy, Constantine Dovrolis, “Bandwidth Estimation: Metrics, Measurement Techniques, and Tools,” IEEE Network, November/December 2003.

7 Capacity definition Maximum possible end-to-end throughput at IP layer In the absence of any cross traffic Achievable with maximum-sized packets If C i is capacity of link i, end-to-end capacity C defined as: Capacity determined by narrow link

8 Available bandwidth definition Per-hop average avail-bw: A i = C i (1-u i ) u i : average utilization A.k.a. residual capacity End-to-end avg avail-bw A: Determined by tight link ISPs measure per-hop avail-bw passively (router counters, MRTG graphs)

9 The avail-bw as a random process Instantaneous utilization u i (t): either 0 or 1 Link utilization in (t, t+  ) Averaging timescale:  Available bandwidth in (t, t+  ) End-to-end available bandwidth in (t, t+  )

10 Available bandwidth distribution Avail-bw has significant variability Need to estimate second-order moments, or even better, the avail-bw distribution Variability depends on averaging timescale  Larger timescale, lower variance Distribution is Gaussian-like, if  > msec and with sufficient flow multiplexing

11 E2E Capacity estimation (a’): Packet pair technique Constantine Dovrolis, Parmesh Ramanathan, David Moore, “Packet Dispersion Techniques and Capacity Estimation,” In IEEE/ACM Transactions on Networking, Dec 2004.

12 Packet pair dispersion Packet Pair (P-P) technique Originally, due to Jacobson & Keshav Send two equal-sized packets back-to-back Packet size: L Packet trx time at link i: L/C i P-P dispersion: time interval between last bit of two packets Without any cross traffic, the dispersion at receiver is determined by narrow link:

13 Cross traffic interference Cross traffic packets can affect P-P dispersion P-P expansion: capacity underestimation P-P compression: capacity overestimation Noise in P-P distribution depends on cross traffic load Example: Internet path with 1Mbps capacity

14 Multimodal packet pair distribution Typically, P-P distribution includes several local modes One of these modes (not always the strongest) is located at L/C Sub-Capacity Dispersion Range (SCDR) modes: P-P expansion due to common cross traffic packet sizes (e.g., 40B, 1500B) Post-Narrow Capacity Modes (PNCMs): P-P compression at links that follow narrow link

15 E2E Capacity estimation (b’): CapProbe Rohit Kapoor, Ling-Jyh Chen, Li Lao, Mario Gerla, M. Y. Sanadidi, "CapProbe: A Simple and Accurate Capacity Estimation Technique," ACM SIGCOMM 2004

16 Compression of p-p dispersion First packet queueing => compressed dispersion Capacity overestimation

17 Expansion of p-p dispersion Second packet queueing => expansion of dispersion Capacity underestimation

18 CapProbe Both expansion and compression are result queuing delays Key insight: packet pair that sees zero queueing delay would yield exact estimate measure RTT for each probing packet of packet pair estimate capacity from pair with minimum RTT-sum Capacity Capacity

19 Measurements - Internet, Internet2 To UCLA-2UCLA-3UANTNU TimeCapacityTimeCapacityTimeCapacityTimeCapacity Cap Probe 0’035.50’01960’02980’0797 0’035.60’01970’04790’0797 0’035.50’02970’17830’2297 Pathrate 6’105.60’16985’19860’2997 6’145.40’16985’20880’2597 6’55.70’16985’181330’2597 Pathchar 21’ ’49183 hr343 hr34 20’ ’41183 hr343 hr ’47183 hr303 hr35 CapProbe implemented using PING packets, sent in pairs

20 Avail-bw estimation (a’): Pathload Manish Jain, Constantine Dovrolis, “End-to-End Available Bandwidth: Measurement Methodology, Dynamics, and Relation with TCP Throughput,” IEEE/ACM Transactions on Networking, Aug 2003.

21 Probing methodology Sender transmits periodic packet stream of rate R K packets, packet size L, interarrival T = L/R Receiver measures One-Way Delay (OWD) for each packet D(k) = t arv (k) - t snd (k) OWD variations: Δ(k) = D(k+1) – D(k) Independent of clock offset between sender/receiver With stationary & fluid-modeled cross traffic: If R > A, then Δ(k) > 0 for all k Else, Δ(k) = 0 for all k

22 Self-loading periodic streams Increasing OWDs means R>A Almost constant OWDs means R

23 Example of OWD variations 12-hop path from U-Delaware to U-Oregon K=100 packets, A=74Mbps, T=100μsec R left = 97Mbps, R right =34Mbps Ri

24 Iterative probing and Pathload Iterative probing in Pathload: Send multiple probing streams (duration τ ) at various rates Each stream samples path at different time interval Outcome of each stream is either R i > A or R i < A Estimate upper and lower bound for avail-bw variation range Avail-bw time series from NLANR trace

25 Avail-bw estimation (b’): percentile estimation Manish Jain, Constantine Dovrolis, “End-to-End Estimation of the Available Bandwidth Variation Range,” ACM SIGMETRICS 2005

26 Avail-bw distribution and percentiles Avail-bw random process, in timescale  A  (t) Assume stationarity Marginal distribution of A  : F  (R) = Prob [A  ≤ R] A p :p th percentile of A , such that p = F  (A p ) Objective: Estimate variation range [A L, A H ] for given averaging timescale  A L  and A H are p L and p H percentiles of A  Typically, p L =0.10 and p H =0.90

27 Percentile sampling Question : Which percentile of A  corresponds to rate R? Given R and  estimate F  (R) Assume that F  (R) is inversible Sender transmits periodic packet stream of rate R Length of stream = measurement timescale  Receiver classifies stream as Type-G if A  ≤ R: I(R)= 1 with probability F  (R) Type-L otherwise: I(R)= 0 with probability 1-F  (R) Collect N samples of I(R) by sending N streams of rate R Number of type-G streams: I(R,N)= Σ i I i (R) E[ I(R,N) ] = F  (R) * N Percentile rank of probing rate R: F  (R) = E[I(R,N)] / N Use I(R,N)/N as estimator of F  (R)

28 Non-parametric estimation Does not assume specific avail-bw distribution Iterative algorithm Stationarity requirement across iterations N-th iteration: probing rate R n Use percentile sampling to estimate percentile rank of R n To estimate the upper percentile A H with p H = F  (A H ): f n = I(R n,N)/N If f n is between p H ±  report A H = R n Otherwise, If f n > p H + , set R n+1 < R n If f n R n Similarly, estimate the lower percentile A L

29 Validation example Verification using real Internet traffic traces b=0.05 b=0.15 Non-parametric estimator tracks variation range within 10-20% Selection of b depends on traffic Traffic spikes/dips may not be detected if b is too small But larger b causes larger MSRE

30 Parametric estimation Assume Gaussian avail-bw distribution Justified assumption for large degree of traffic multiplexing And/or for long averaging timescale (>200msec) Gaussian distribution completely specified by Mean  and standard deviation   p th percentile of Gaussian distribution A p =  +    -1 (p) Sender transmits N probing streams of rates R 1 and R 2 Receiver determines percentiles ranks corresponding to R 1 and R 2  and   can be then estimated by solving R 1 =  +    -1 (p 1 ) R 2 =  +    -1 (p 2 ) Variation range is then calculated from: A H =  +    -1 (p H ) A L =  +    -1 (p L )

31 Validation example Gaussian trafficnon-Gaussian traffic Verification using real Internet traffic traces Evaluated with both Gaussian and non-Gaussian traffic Parametric algorithm is more accurate than non-parametric algorithm, when traffic is good match to Gaussian model in non-stationary conditions

32 Avail-bw estimation (c’): PathChirp Vinay Ribeiro, Rolf Riedi, Jiri Navratil, Rich Baraniuk, Les Cottrell, “PathChirp: Efficient Available Bandwidth Estimation,” PAM 2003

33 Chirp Packet Trains Exponentially decrease packet spacing within packet train Wide range of probing rates Efficient: few packets

34 Chirps vs. CBR Trains Multiple rates in each chirping train Allows one estimate per-chirp Potentially more efficient estimation

35 CBR Cross-Traffic Scenario Point of onset of increase in queuing delay gives available bandwidth

36 Comparison with Pathload 100Mbps links pathChirp uses 10 times fewer bytes for comparable accuracy Available bandwidth EfficiencyAccuracy pathchirppathloadpathChirp 10-90% pathload Avg.min-max 30Mbps0.35MB3.9MB19-29Mbps16-31Mbps 50Mbps0.75MB5.6MB39-48Mbps39-52Mbps 70Mbps0.6MB8.6MB54-63Mbps63-74Mbps

37 Experimental comparison of avail-bw estimation tools Alok Shriram, Marg Murray, Young Hyun, Nevil Brownlee, Andre Broido, k claffy, ”Comparison of Public End-to-End Bandwidth Estimation Tools on High-Speed Links,” PAM 2005

38 Testbed topology

39 Accuracy comparison - SmartBits Direction 1, Measured AB Direction 2, Measured AB Actual AB

40 Accuracy comparison - TCPreplay Actual Available Bandwidth Measured Available Bandwidth

41 What’s wrong with Spruce? Spruce was proposed as fast & accurate estimation method Strauss et al., IMC’03: example of direct probing With a single-link model: Sender sends periodic stream with input rate R i Receiver measures output rate R o Avail-bw A given by: Assumptions: Tight link capacity C t is known Input rate R i can be controlled by source But what would happen in a multi-hop path? R i can be less than source rate (due to previous link with lower capacity) and, even worse, it can be unknown!

42 Measurement latency comparison Abing: 1.3 to 1.4 s Spruce: 10.9 to 11.2 s Pathload: 7.2 to 22.3 s Patchchirp: 5.4 s Iperf: 10.0 to 10.2 s

43 Probing traffic comparison

44 Applications of bandwidth estimation

45 Large TCP transfers and congestion control Bandwidth-delay product estimation Socket buffer sizing Streaming multimedia Adjust encoding rate based on avail-bw Intelligent routing systems Overlay networks and multihoming Select best path based on capacity or avail-bw Content Distribution Networks (CDNs) Choose server based on least-loaded path SLA and QoS verification Monitor path load and allocated capacity End-to-end admission control Network “spectroscopy” Several more..

46 Application-1: Improved TCP Throughput with BwEst Ravi S. Prasad, Manish Jain, Constantine Dovrolis, “Socket Buffer Auto-Sizing for High-Performance Data Transfers,” Journal of Grid Computing, 2003

47 TCP performs poorly in fat-long paths Basic idea in TCP congestion control: Increase congestion window until packet loss occurs Then reduce window by a factor of two (multiplicative decrease) Consequences Increased loss-rate Avg. throughput less than available bandwidth Large delay-variation Example: 1Gbps path, 100msec RTT, 1500B pkts Single loss ≈ 4000 pkts window reduction Recovery from single loss ≈ 400 sec Need very small loss probability (ρ<2*10 -8 ) to saturate path

48 TCP background Socket-layer buffers Send buffer: B s Receive buffer: B r Socket Buffer SenderReceiver Socket Layer TCP Layer Wr Ws TCP windows Send: W s <= B s Congestion: W c Receiver (advertised): W r <= B r W s = min {B s, W c, W r } Ideally, TCP send window W s should be set to connection’s bandwidth-delay product “Bandwidth”: connection’s fair share “Delay”: connection’s RTT Achieve efficiency, fairness, no congestion

49 Can we avoid congestive losses w/o changing TCP? W s = min {W c, W r } But W r <= S S: rcv socket buffer size We can limit S so that connection is limited by receive window: W s = S Non-congested path: Throughput R(S) = S/T(S) R(S) increases with S until it reaches MFT MFT: Maximum Feasible Throughput (onset of congestion) Objective: set S close to MFT, to avoid any losses and stabilize TCP send window to a safe value Congested path: Path with losses before start of target transfer Limiting S can only reduce throughput

50 Socket buffer auto-sizing (SOBAS) Apply SOBAS only in non-congested paths Receiving application measures rcv-throughput R rcv When receive throughput becomes constant: Connection has reached its maximum throughput R max If the send window becomes any larger, losses will occur Receiving application limits rcv socket buffer size S: S = RTT * R max With congestion unresponsive cross traffic: R max = A With congestion responsive cross traffic: R max = MFT SOBAS does not require any changes in TCP But estimation of RTT and congestion-status requires “ping-like” probing

51 Measurements at WAN path Path: GaTech to LBL,CA SOBAS flow get higher average throughput than non-SOBAS flow because former avoids all congestion losses SOBAS does not significantly increase path’s RTT Non-SOBAS SOBAS

52 To conclude..

53 Things I have not talked about Other estimation tools & techniques Abing, netest, pipechar, STAB, pathneck, IGI/PTR, … Per-hop capacity estimation (pathchar, clink, pchar) Other applications Bwest in new TCPs (e.g., TCP Westwood, TCP FAST) Bwest in wireless nets Bwest in overlay routing Bwest in multihoming Bwest in bottleneck detection Scalable bandwidth measurements in networks with many monitoring nodes Practical issues Interrupt coalescence Traffic shapers Non-FIFO queues

54 Conclusions We know how to do the following: Estimate e2e capacity & avail-bw Apply bwest in several applications We know that we cannot do the following: Estimate per-hop capacity with VPS techniques Avoid avail-bw estimation bias when traffic is bursty and/or path has multiple bottlenecks We do not yet know how to do the following: Scalable bandwidth measurements Integrate probing traffic in application data Many research questions are still open Help wanted!

Download ppt "Bandwidth estimation in computer networks: measurement techniques & applications Constantine Dovrolis College of Computing Georgia Institute of Technology."

Similar presentations

Ads by Google