Amogh Dhamdhere, Hao Jiang and Constantinos Dovrolis Buffer Sizing for Congested Internet Links Amogh Dhamdhere, Hao Jiang and Constantinos Dovrolis (amogh,hjiang,dovrolis)@cc.gatech.edu Networking and Telecommunications Group, College of Computing, Georgia Tech.

Outline Motivation and related work Objectives and traffic model The utilization constraint alone Utilization and loss rate constraints Parameter estimation and simulation results 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Motivation Router buffers are important in packet networks Absorb rate variations of incoming traffic Prevent packet losses during traffic bursts Increasing buffer space increases the utilization of the link and decreases the loss rate Increasing buffer also increases queuing delays ! So smaller buffers are desirable Fundamental Question: What is the minimum buffer requirement to satisfy constraints on the utilization, loss rate and queuing delay ? 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Rules of Thumb Some router vendors suggest 500ms of buffering. Why 500ms ? Bandwidth Delay Product rule: Capacity of link times the “typical” RTT (B = CT) Which RTT should we use ? Many TCP flows with different RTTs ? How do different types of flows (large vs small) affect the buffer requirement ? Several variants of this rule e.g. Capacity times link delay 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Related Work Approaches based on queuing models e.g. M/M/1/k TCP is not open-loop. TCP flows are reactive Modeling Internet traffic is difficult “Stanford” model (Appenzeller et al. Sigcomm 2004) Buffer requirement for full utilization decreases with square root of N Did not consider the loss rate at the link Assumed that flows are completely desynchronized Applicable when the number of flows is large Morris (1997 and 2000) Buffer proportional to the number of flows (B = 6*N) Considered all flows active at the link 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Outline Objectives and traffic model Motivation and related work The utilization constraint alone Utilization and loss rate constraints Parameter estimation and simulation results 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Our Objectives Full utilization: Maximum loss rate: The average utilization of the link should be at least % when the offered load is sufficiently high Maximum loss rate: The loss rate p should not exceed , typically 1-2% for a saturated link Minimum queuing delays: High queuing delay causes higher transfer latencies and jitter Also increases cost and power consumption Should satisfy utilization and loss rate constraints with minimum amount of buffering possible All of these objectives may not be feasible ! 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Traffic Classes Locally Bottlenecked Persistent (LBP) TCP flows Large TCP flows limited by losses at the target link Loss rate p is equal to the loss rate at the target link Remotely Bottlenecked Persistent (RBP) TCP flows Large TCP flows limited by losses at target link and other links Loss rate is greater than loss rate at target link Window Limited Persistent TCP flows Large TCP flows, throughput limited by the advertised window Short TCP flows and non-TCP traffic 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Assumption Key Assumption: LBP flows account for most of the traffic at the target link (80-90 %) In this case, we can ignore the buffering requirement of non-LBP flows non-LBP flows also contribute to the utilization and loss rate at the target link Contribution is small if fraction of non-LBP traffic is small Our model is applicable in links where this assumption holds Edge links and links in access networks are candidates 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Outline The utilization constraint alone Motivation and related work Objectives and traffic model The utilization constraint alone Utilization and loss rate constraints Parameter estimation and simulation results 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

TCP Window Dynamics Saw-tooth behavior of TCP Padhye (1998) TCP throughput can be approximated by Average window size is independent of RTT Valid when loss rate is small 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Util. Constraint - Multiple TCP Flows heterogeneous LBP flows with RTTs Consider initially the worst-case scenario: Global Loss Synchronization. All flows decrease windows simultaneously in response to losses. We derive that As a bandwidth-delay product Where is the harmonic mean of the RTTs 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Util. Constraint - Multiple TCP Flows is called the effective RTT of the flows Influenced more by smaller values Intuition: Flows with smaller RTTs have larger portion of their window in the bottleneck buffer Hence have larger influence on the required buffer Flows with large RTTs have larger portion of their window “on the wire” Practical Implication: A few connections with very large RTTs cannot significantly influence the buffer requirement, as long as most flows have small RTTs 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Partial Synchronization Model In practice, flows are not completely synchronized Loss Burst Length: Number of packets lost by flows during a congestion event Empirical observation: Loss burst length increases almost linearly with i.e. A simple probabilistic argument gives us, Partial loss synchronization reduces the buffer requirement. 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Validation ns2 simulations. Heterogeneous flows, % Partial synchronization model accurately predicts the buffer requirement. Deterministic model overestimates the buffer requirement ! 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Outline Utilization and loss rate constraint Motivation and related work Objectives and traffic model The utilization constraint alone Utilization and loss rate constraint Parameter estimation and simulation results 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Utilization and Loss Rate End-user perceived service is poor when the loss rate is more than 5-10% Particularly for short and interactive flows Results by Morris (1997) High variability in the completion times of short transfers Some “unlucky” flows suffer repeated losses and timeouts The buffer size controls the loss rate Upper bound the loss rate to . Assume is 1% 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Relation between loss rate and N homogeneous LBP flows at the target link. Link capacity C, flow RTTs T Assume that the flows saturate the link and their throughput is given by p is proportional to the square of Hence to maintain loss rate at less than But this requires admission control Such schemes not deployed yet 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Flow Proportional Queueing First proposed by Morris (2000) Don’t limit Increase RTTs to decrease loss rate Increase RTT by increasing buffer, which increases queuing delay Solving for B gives Where Practically, packets for , and packets for 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Flow Proportional Queueing (contd.) Intuition: packets per flow, either in buffer (B term) or “on the wire” ( term) Differences with Morris’ FPQ scheme Morris did not take into account the term Set arbitrarily to 6 packets Applied the rule for all flows active at the link Increasing RTTs may violate delay constraint In that case, choose the minimum buffer that can satisfy utilization and loss constraints 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Integrated Model Separate results for utilization and loss rate constraints Satisfy the most stringent of the two requirements B for utilization decreases with , while B for loss rate increases with : Crossover point Called the BSCL formula 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Integrated Model - Validation Simulations using ns2. Heterogeneous flows, varied from 1 to 200. Utilization % and loss constraint % Utilization constraint Loss rate constraint 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Outline Parameter estimation and simulation results Motivation and related work Objectives and traffic model The utilization constraint alone Utilization and loss rate constraints Parameter estimation and simulation results 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Parameter Estimation Flow Classification: Number of LBP flows: Zhang et al. (2002): Classify TCP flows based on rate limiting factors Number of LBP flows: LBP flows: all rate reductions due to packet losses at target link RBP flows: Some rate reductions due to losses elsewhere Effective RTT: Jiang et al. (2002): Passive algorithm to measure TCP Round Trip Times from packet traces Loss Synchronization: Measure loss burst length from trace or use approximation 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Evaluation - Setup ns2 simulations. Multi-level tree topology with wide range of RTTs (20ms to 550ms). Target link capacity 50Mbps. varied from 1 to 400. 20 RBP flows, 10 window limited flows. Mice flows with average size 14 packets, exponential inter-arrivals. Non-LBP traffic (R) is varied between 5% and 20% of C. 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Results – Loss Rate 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Results – Loss Rate 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Results – Loss Rate 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Results – Loss Rate BSCL can bound loss rate close to the target, if R is less than 10%. Accuracy decreases as fraction of non-LBP traffic increases. Stanford model and the rule of thumb cannot bound loss rate. 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Results - Utilization For a large number of flows, all three schemes achieve full utilization. For smaller number of flows, BSCL sometimes leads to underutilization. Due to the probabilistic nature of loss synchronization. 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Summary Derived a buffer sizing formula (BSCL) for congested links, taking into account both utilization and loss rate of the target link. Applicable for links in which 80-90% of the traffic comes from large locally bottlenecked TCP flows. Account for the effects of heterogeneous RTTs and partial loss synchronization. Validated the results through simulations. 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Thank You ! 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Parameter estimation - Distinguishing between LBP and RBP flows: Intuition: For a LBP flow, rate reduction should be preceded by a loss at the target link. For RBP flows, rate reduction will not always be accompanied by a loss at the target link (due to losses in other links). 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Why is Buffer Size Important ? Router buffer size affects: Utilization of the link. Loss rate of the link. Fairness among TCP connections. Results by Morris (1997): A very small buffer can lead to underutilization. Loss rate increases as the square of N. 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Partial Synchronization Model (contd.) Consider a congestion event with the average loss-burst length . A simple probabilistic argument gives us, Remarks: For global loss synchronization, and the buffer requirement becomes B = CT. Partial loss synchronization reduces the buffer requirement. For heterogeneous connections, replace T with the effective RTT. 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Outline Motivation and related work Objectives and traffic model The utilization constraint alone Utilization and loss rate constraints Parameter estimation and simulation results 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005

Results - Loss Rate BSCL can bound loss rate close to the target, if R is less than 10%. Accuracy decreases as fraction of non-LBP traffic increases. Stanford model and the rule of thumb cannot bound loss rate. 11/19/2018 Amogh Dhamdhere IEEE Infocom 2005