Router Buffer Sizing and Reliability Challenges in Multicast Aditya Akella 02/28

TCP Performance Can TCP saturate a link? Congestion control –Increase utilization until… link becomes congested –React by decreasing window by 50% –Window is proportional to rate * RTT Doesn’t this mean that the network oscillates between 50 and 100% utilization? –Average utilization = 75%??Average utilization = 75%?? –No…this is *not* right!

Single TCP Flow Router without buffers

Summary Unbuffered Link t W Minimum window for full utilization The router can’t fully utilize the link –If the window is too small, link is not full –If the link is full, next window increase causes drop –With no buffer it still achieves 75% utilization

TCP Performance In the real world, router queues play important role –Window is proportional to rate * RTT But, RTT changes as well the window –Window to fill links = propagation RTT * bottleneck bandwidth If window is larger, packets sit in queue on bottleneck link

TCP Performance If we have a large router queue  can get 100% utilization100% utilization –But, router queues can cause large delays How big does the queue need to be? –Windows vary from W  W/2 Must make sure that link is always full W/2 > RTT * BW W = RTT * BW + Qsize Therefore, Qsize > RTT * BW –Ensures 100% utilizationEnsures 100% utilization –Delay? Varies between RTT and 2 * RTT

Single TCP Flow Router with large enough buffers for full link utilization

Example 10Gb/s linecard –Requires 300Mbytes of buffering. –Read and write 40 byte packet every 32ns. Memory technologies –DRAM: require 4 devices, but too slow. –SRAM: require 80 devices, 1kW, $2000. Problem gets harder at 40Gb/s –Hence RLDRAM, FCRAM, etc. Rule-of-thumb makes sense for one flow –Typical backbone link has > 20,000 flows –Does the rule-of-thumb still hold?

If flows are synchronized Aggregate window has same dynamics Therefore buffer occupancy has same dynamics Rule-of-thumb still holds. t

If flows are not synchronized Probability Distribution B 0 Buffer Size

Central Limit Theorem CLT tells us that the more variables (Congestion Windows of Flows) we have, the narrower the Gaussian (Fluctuation of sum of windows) –Width of Gaussian decreases with –Buffer size should also decreases with

Loss Recovery in Multicast Sender-reliable –Wait for ACKs from all receivers. Re-send on timeout or selective ACK –Per receiver state in sender not scalable –ACK implosion Receiver-reliable –Receiver NACKs (resend request) lost packet –Does not provide 100% reliability –NACK implosion

R1 Implosion S R3R4 R2 21 R1 S R3R4 R2 Packet 1 is lostAll 4 receivers request a resend Resend request

Retransmission Re-transmitter –Options: sender, other receivers How to retransmit –Unicast, multicast, scoped multicast, retransmission group, … Problem: Exposure

R1 Exposure S R3R4 R2 21 R1 S R3R4 R2 Packet 1 does not reach R1; Receiver 1 requests a resend Packet 1 resent to all 4 receivers 1 1 Resend request Resent packet

Ideal Recovery Model S R3R4 R2 2 1 S R3R4 R2 Packet 1 reaches R1 but is lost before reaching other Receivers Only one receiver sends NACK to the nearest S or R with packet Resend request 11 Resent packet Repair sent only to those that need packet R1

Scalable Reliable Multicast Originally designed for wb (whiteboard) Receiver-reliable –NACK-based Every member may multicast NACK or retransmission

R1 SRM Request Suppression S R3 R2 21 R1 S R3 R2 Packet 1 is lost; R1 requests resend to Source and Receivers Packet 1 is resent; R2 and R3 no longer have to request a resend 1 X X Delay varies by distance X Resend request Resent packet

Deterministic Suppression d d d d 3d Time data nack repair d 4d d 2d 3d = Sender = Repairer = Requestor Delay = C 1  d S,R

SRM Star Topology S R2 21 R3 Packet 1 is lost; All Receivers request resends Packet 1 is resent to all Receivers X R4 Delay is same length S R2 1 R3R4 Resend request Resent packet

SRM: Stochastic Suppression data d d d d Time NACK repair 2d session msg Delay = U[0,D 2 ]  d S,R = Sender = Repairer = Requestor

SRM (Summary) NACK/Retransmission suppression –Delay before sending –Delay based on RTT estimation –Deterministic + Stochastic components Periodic session messages –Full reliability –Estimation of distance matrix among members