Modeling TCP in Small-Buffer Networks Mark Shifrin and Isaac Keslassy Technion (Israel)

How much buffering do routers need? Problem How much buffering do routers need?

Why Does Buffer Size Matter? Buffers are costly. Today’s buffers: 1/2 board space 1/3 power consumption Small buffers: On chip buffers Higher density Lower cost Scalability [N. McKeown, Stanford]

How much Buffer does a Router need? Universally applied rule-of-thumb: A router needs a buffer size: 2T is the round-trip propagation time (or just 250ms) C is the capacity of the outgoing link Background Mandated in backbone and edge routers. Appears in RFPs and IETF architectural guidelines. Has major consequences for router design. Comes from dynamics of TCP congestion control. Villamizar and Song: “High Performance TCP in ANSNET”, CCR, 1994. Based on 2 to 16 TCP flows at speeds of up to 40 Mb/s.

Synchronized Flows Aggregate window has same dynamics Therefore buffer occupancy has same dynamics Rule-of-thumb still holds. t

Many TCP Flows Probability Distribution B Buffer Size

Stanford Model [Appenzeller et al., ’04 | McKeown and Wischik, ’05 | McKeown et al., ‘06] Assumption 1: TCP Flows modeled as i.i.d.  total window W has Gaussian distribution Assumption 2: Queue is the only variable part  queue has Gaussian distribution: Q=W-CONST  use smaller buffer than in rule of thumb

Impact on Router Design 40Gb/s linecard with 1,000,000 flows Rule of thumb: Buffer = 10Gbits Requires external, slow DRAM Stanford model: Buffer = 10Mbits Can use on-chip, fast SRAM Delays halved for short-flows

Motivation In a small-buffer world… Assumption 1: TCP Flows modeled as i.i.d.  total window W has Gaussian distribution Assumption 2: Queue is the only variable part  queue has Gaussian distribution: Q=W-CONST OK Queue is small  negligible? Gaussian part is… on the lines!

Contributions Distribution models for: Lines Arrival rates to queues Queue sizes Packet loss rates General closed-loop model for small-buffer networks Result: queues are not the only variable part in the network

Model Development Flow li1 pdf cwnd total arrival rate l1i arrival rates Q pdf packet loss

Model of li1 li1 pdf cwnd packet loss

Bursty Model of Window Distribution dest. source li5 li1 li3 li2 li4 li6 rtti=tp1i+tp2i+tp3i+tp4i+tp5i+tp6i Common Approach 1: Uniform packet distribution. l1(t)i = wi(t)* tpi1/rtti. Approach 2: Bursty Packet Distribution

Bursty Model of Window Distribution Assumption: All packets in a flow move in a single burst Conclusion 1: All packets belonging to an arbitrary flow i are present almost always on the same link. Conclusion 2: The probability of burst of flow i being present on the certain link is equal to the ratio of its propagation latency to the total rtti.

Simulation Results

Model Development Flow li1 pdf cwnd total arrival rate l1i arrival rates

Rate Transmission Derivation The objective is to find the pdf of the number of packets sent on some link i, ri, in a time unit δt. Assumptions: The rate on each one of the links in L1 is statistically independent We assume that the transmissions are bursty. We assume that the rate is proportional to the distribution of l1i and to the ratio δt/tp1i.

Arrival rate of a single flow B source li2 li1 li1 *δt/tp1i δt tp1i We find the arrival distribution for every flow in δt msec.

Total Rate Result: Proof based on the Lindeberg condition. Generalizes Central Limit Theorem for non-identically distributed components Holds if the share of each flow comparatively to the sum is negligible as the number of the flows grows. Argument for the proof: cwnd is limited by maximum value  same for l1i and ri.

Instantaneous Rate Model - Results Probability Total arrival rate – number of packets per δt

Model Development Flow li1 pdf cwnd total arrival rate l1i arrival rates Q pdf packet loss

PDF for Q To find the queue size distribution: Run Markov Chain simulation (compared with [Tran-Gia and Ahmadi, ‘88]) Use samples of R for the transitions Packet loss p is derived from the queue size distribution.

PDF of Q Probability Q state – 0 to 585 packets

Fixed Point Solution: p=f(p) li1 pdf cwnd total arrival rate l1i arrival rates Q pdf packet loss

Packet loss - results Model gives about 10%-25% of discrepancy. Case 1: Measured: p=2.7%, Model: p=3% Case 2: Measured: p=0.8%, Model: p=0.98% Case 3: Measured: p=1.4%, Model: p=1.82% Case 4: Measured: p=0.452%, Model: p=0.56%

The Gaussian distributions L4 L2 L5 L6 L1 W NS2 simulation of 500 flows with different propagation times

Are these lines really Gaussian?

Summary Introduced general closed-loop model for small-buffer networks Proved wrong the usual assumption that queues are the only variable part in the network

Thank you.