Presentation on theme: "1 EE384Y: Packet Switch Architectures Part II Sizing Router Buffers (Recent work by Guido Appenzeller) Nick McKeown Professor of Electrical Engineering."— Presentation transcript:
1 EE384Y: Packet Switch Architectures Part II Sizing Router Buffers (Recent work by Guido Appenzeller) Nick McKeown Professor of Electrical Engineering and Computer Science, Stanford University
2 How much Buffer does a Router need? Universally applied rule-of-thumb: A router needs a buffer size: 2T is the round-trip propagation time (or just 250ms) C is the capacity of the outgoing link Background Mandated in backbone and edge routers. Appears in RFPs and IETF architectural guidelines. Has major consequences for router design. Comes from dynamics of TCP congestion control. Villamizar and Song: High Performance TCP in ANSNET, CCR, Based on 2 to 16 TCP flows at speeds of up to 40 Mb/s.
3 Example 10Gb/s linecard or router Requires 300Mbytes of buffering. Read and write new packet every 32ns. Memory technologies SRAM: require 80 devices, 1kW, $2000. DRAM: require 4 devices, but too slow. Problem gets harder at 40Gb/s Hence RLDRAM, FCRAM, etc.
4 TCP TCP adapts to congestion Sender sends packets, receiver sends ACKs Sending rate is controlled by Window W At any time, only W unacknowledged packets may be outstanding W is adjusted for each packet (in CA mode): If ACK received: W = W+1/W (W=W+1 for each W packets) If packet is lost: W = W/2 (W halved in case of loss) The sending rate of TCP is:
5 Single TCP Flow Router with large enough buffers for full link utilization B Dest CC > C Source t Window size Buffer size and RTT For every W ACKs received, send W+1 packets
6 Over-buffered Link
7 Under-buffered Link
8 Buffer = Rule-of-thumb Interval magnified on next slide
9 Microscopic TCP Behavior When sender pauses, buffer drains one RTT Drop
10 Origin of rule-of-thumb Before and after reducing window size, the sending rate of the TCP sender is the same Inserting the rate equation we get The RTT is part transmission delay T and part queuing delay B/C. We know that after reducing the window, the queueing delay is zero.
11 Rule-of-thumb Rule-of-thumb makes sense for one flow Typical backbone link has > 20,000 flows Does the rule-of-thumb still hold? Answer: If flows are perfectly synchronized, then Yes. If flows are desynchronized then No.
12 Buffer size is height of sawtooth t B 0
13 If flows are synchronized Aggregate window has same dynamics Therefore buffer occupancy has same dynamics Rule-of-thumb still holds. t
14 Two TCP Flows Two TCP flows can synchronize
15 If flows are not synchronized Aggregate window has less variation Therefore buffer occupancy has less variation The more flows, the smaller the variation Rule-of-thumb does not hold. t
16 If flows are not synchronized Probability Distribution Buffer Size B 0
17 Quantitative Model Model congestion window of a flow as random variable model aswhere For many de-synchronized flows We assume congestions windows are independent All congestion windows have the same probability distribution Now central limit theorem gives us queue length distribution
18 Required buffer size Simulation
19 Required buffer size 99.9% 98.0% 99.5% 2×2×
20 Small buffers help short flows Average flow completion times of 14 packet flows that share a congested bottleneck link with long-lived flows.
21 Experiments with backbone router GSR 12000, OC3 Line Card TCP Flows Router BufferLink Utilization PktsRAMModelSimExp x 1 x 2 x 3 x Mb 2Mb 4Mb 8Mb 96.9% 99.9% 100% 94.7% 99.3% 99.9% 99.8% 94.9% 98.1% 99.8% 99.7% x 1 x 2 x 3 x kb 1Mb 2Mb 4Mb 99.7% 100% 99.2% 99.8% 100% 99.5% 100% 99.9% Thanks: Experiments conducted by Paul Barford and Joel Sommers, U of Wisconsin
22 What about Short Flows? So far we assumed long flows in congestion avoidance mode. What if traffic is mainly short flows in slow-start? Answer: Behavior is different, but In mixes of flows, long flows drive buffer requirements Required buffer for short flows is independent of line speed and RTT (same for 1Mbit/s or 40 Gbit/s)
23 A single, short-lived TCP flow Flow length 62 packets, RTT ~140 ms RTT syn fin ack received Flow Completion Time (FCT)
24 Modelling TCP Flows vs. independent bursts Inter-Burst Arrival Time is greater than buffer size Therefore, we assume bursts are independent. Poisson arrivals of flows Arrivals of length L flow (the flow length in packets) Poisson arrivals of bursts Four different poisson arrival processes of lengths 2,4,...
25 The M/G/1 Model TCP traffic is modelled as an M/G/1 arrival process: poisson arrivals of jobs with an arrival rate of Average queue length in jobs is: This gives us an average queue length in packets of Let's see if this works in practice...
26 Average Queue length
27 Queue Distribution To determine the required buffer, we need the queue distribution. Or at least the tail end of the queue distribution P(Q = x) Q Buffer B Packet Loss For M/G/1 queues there is no general solution for the queue distribution. We did two things (details are in the paper): – Use M/G/1 processor sharing model (bad) – Use Frank Kelly's effective bandwidth (good)
28 In Summary Buffer size is dictated by long TCP flows. 10Gb/s linecard with 200,000 x 56kb/s flows Rule-of-thumb: Buffer = 2.5Gbits Requires external, slow DRAM Becomes: Buffer = 6Mbits Can use on-chip, fast SRAM Completion time halved for short-flows 40Gb/s linecard with 40,000 x 1Mb/s flows Rule-of-thumb: Buffer = 10Gbits Becomes: Buffer = 50Mbits