TCP and Congestion Control(2) Dan LI CS Department, Tsinghua University Today we’ll talk about TCP and congestion control. 2018/11/7

Today’s Lecture Transport Introduction Error Recovery Window Flow Control Analysis of TCP Congestion Control TCP Variations Beyond TCP Congestion Control Reading list 2018/11/7

Congestion 10 Mbps 100 Mbps 1.5 Mbps Different sources compete for resources inside network: Link bandwidth and queue space Why is it a problem? Sources are unaware of current state of resource Sources are unaware of each other In many situations will result in < 1.5 Mbps of throughput (congestion collapse) 2018/11/7

Causes & Costs of Congestion Four senders – multihop paths Timeout/retransmit Q: What happens as rate increases? 2018/11/7

Causes & Costs of Congestion When packet dropped, any “upstream transmission capacity used for that packet was wasted! 2018/11/7

Congestion Collapse Definition: Increase in network load results in decrease of useful work done Many possible causes Spurious retransmissions of packets still in flight Classical congestion collapse Solution: better timers and TCP congestion control Undelivered packets Packets consume resources and are dropped elsewhere in network Solution: congestion control for ALL traffic 2018/11/7

Other Congestion Collapse Causes Control traffic Large percentage of traffic is for control Headers, routing messages, DNS, etc. Stale or unwanted packets Packets that are delayed on long queues “Push” data that is never used 2018/11/7

Congestion Control and Avoidance Desirable properties: Scalability: # flows, range of capacities, range of delays Do well in entire range! Efficiency: High network utilization Fairness Works-ness: avoids collapse! Congestion collapse is not just a theory Has been frequently observed in many networks 2018/11/7

Fairness Jain’s fairness index Goal: Something that works well enough f = (Sxi)2/n(Sxi2) All x equal: 1 k/n get service: k/n Goal: Something that works well enough 2018/11/7

Objectives Simple router behavior Distributed algorithm Efficiency: Xknee = Sxi(t) Fairness: (Sxi)2/n(Sxi2) Power: (throughputa/delay) Convergence: control system must be stable 2018/11/7

Design Questions Congestion Control questions How is congestion signaled? Either mark or drop packets When is a router congested? Drop tail queues – when queue is full Average queue length – at some threshold Control questions How do senders react to congestion? How do senders determine capacity for flow? 2018/11/7

Congestion Control Upon congestion, flows must reduce rate Decrease algorithm If no congestion, flows might try sending more Increase algorithm Let’s assume window-based flow control Sender maintains “cwnd”: # of unacknowledged packets in the network at any time Transmission rate: cwnd / rtt 2018/11/7

Linear Control Many different possibilities for reaction to congestion and probing Examine simple linear controls Window(t + 1) = a + b Window(t) Different ai/bi for increase and ad/bd for decrease Supports various reaction to signals Increase/decrease additively Increased/decrease multiplicatively Which of the four combinations is optimal? 2018/11/7

Phase Plots Simple way to visualize behavior of competing connections over time User 2’s Allocation x2 User 1’s Allocation x1 2018/11/7

Phase Plots What are desirable properties? What if flows are not equal? Fairness Line Overload User 2’s Allocation x2 Optimal point Underutilization Efficiency Line User 1’s Allocation x1 2018/11/7

Additive Increase/Decrease Both X1 and X2 increase/decrease by the same amount over time Additive increase improves fairness and additive decrease reduces fairness Fairness Line T1 User 2’s Allocation x2 T0 Efficiency Line 2018/11/7 User 1’s Allocation x1

Multiplicative Increase/Decrease Both X1 and X2 increase by the same factor over time Extension from origin – constant fairness Fairness Line T1 User 2’s Allocation x2 T0 Efficiency Line User 1’s Allocation x1 2018/11/7

What is the Right Choice? Constraints limit us to AIMD AIMD moves towards optimal point x0 x1 x2 Efficiency Line Fairness Line User 1’s Allocation x1 User 2’s Allocation x2 2018/11/7

TCP and Linear Control Upon congestion: While probing w(t+1) = a*w(t) 0 < a < 1 While probing w(t+1) = w(t) + b 0 < b << wmax TCP sets a = 1/2, b = 1 (packet) 2018/11/7

TCP Congestion Control Motivated by ARPANET congestion collapse Underlying design principle: packet conservation At equilibrium, inject packet into network only when one is removed Basis for stability of physical systems Why was this not working? Connection doesn’t reach equilibrium Spurious retransmissions Resource limitations prevent equilibrium 2018/11/7

TCP Congestion Control - Solutions Reaching equilibrium Slow start Eliminates spurious retransmissions Accurate RTO estimation Fast retransmit Adapting to resource availability Congestion avoidance 2018/11/7

TCP Congestion Control Basic principles AIMD Packet conservation Reaching steady state quickly ACK clocking 2018/11/7

AIMD: Now You Grok the Sawtooth Distributed, fair and efficient Packet loss is seen as sign of congestion and results in a multiplicative rate decrease Factor of 2 TCP periodically probes for available bandwidth by increasing its rate Rate Time 2018/11/7

Congestion Avoidance If loss occurs when cwnd = W Upon receiving ACK Network can handle 0.5W ~ W segments Set cwnd to 0.5W (multiplicative decrease) Upon receiving ACK Increase cwnd by (1 packet)/cwnd What is 1 packet?  1 MSS worth of bytes After cwnd packets have passed by  approximately increase of 1 MSS Implements AIMD 2018/11/7

Congestion Avoidance Sequence Plot Sequence No The picture show the seq-time, the sequence no increases with time, but after some period, congestion happens, and some packets are lost, so the seq stop increasing. And then as the rate is decreased, the seq no increase again. Packets Acks Time 2018/11/7

Packet Conservation At equilibrium, inject packet into network only when one is removed Sliding window and not rate controlled But still need to avoid sending burst of packets  would overflow links Need to carefully pace out packets (ack clocking)! Helps provide stability Need to eliminate spurious retransmissions Accurate RTO estimation Better loss recovery techniques (e.g. fast retransmit) 2018/11/7

TCP Packet Pacing Congestion window helps to “pace” the transmission of data packets In steady state, a packet is sent when an ack is received Data transmission remains smooth, once it is smooth Self-clocking behavior Pb Pr Sender Receiver As Ar Ab 2018/11/7

Reaching Steady State Doing AIMD is fine in steady state but slow… How does TCP know what is a good initial rate to start with? Should work different network speed Quick initial phase to help get up to speed (slow start) 2018/11/7

Slow Start Packet Pacing How do we get this clocking behavior to start? Initialize cwnd = 1 Upon receipt of every ack, cwnd = cwnd + 1 Implications Window actually increases to W in RTT * log2(W) Can overshoot window and cause packet loss 2018/11/7

Slow Start Example 1 One RTT 0R 2 1R 3 4 2R 5 6 7 8 3R 9 10 11 12 13 One pkt time 0R 2 1R 3 4 2R 5 6 7 8 3R 9 10 11 12 13 14 15 2018/11/7

Slow Start Sequence Plot . Sequence No Packets Acks Time 2018/11/7

Return to Slow Start If packet is lost we lose our self clocking as well Need to implement slow-start and congestion avoidance together When timeout occurs set ssthresh to 0.5w If cwnd < ssthresh, use slow start Else use congestion avoidance 2018/11/7

TCP Saw Tooth Behavior Congestion Window Time Timeouts may still occur Slowstart to pace packets Fast Retransmit and Recovery Initial Slowstart 2018/11/7

TCP Modeling Given the congestion behavior of TCP can we predict what type of performance we should get? What are the important factors Loss rate Affects how often window is reduced RTT Affects increase rate and relates BW to window RTO Affects performance during loss recovery MSS Affects increase rate 2018/11/7

Simple TCP Model Some additional assumptions Fixed RTT No delayed ACKs In steady state, TCP loses packet each time window reaches W packets Window drops to W/2 packets Each RTT window increases by 1 packetW/2 * RTT before next loss BW = MSS * avg window/RTT = MSS * (W + W/2)/(2 * RTT) = .75 * MSS * W / RTT 2018/11/7

Simple Loss Model What was the loss rate? BW = .75 * MSS * W / RTT Packets transferred = (.75 W/RTT) * (W/2 * RTT) = 3W2/8 1 packet lost  loss rate = p = 8/3W2 W = sqrt( 8 / (3 * loss rate)) BW = .75 * MSS * W / RTT BW = MSS / (RTT * sqrt (2/3p)) 2018/11/7

TCP Friendliness What does it mean to be TCP friendly? TCP is not going away Any new congestion control must compete with TCP flows Should not clobber TCP flows and grab bulk of link Should also be able to hold its own, i.e. grab its fair share, or it will never become popular How is this quantified/shown? Has evolved into evaluating loss/throughput behavior If it shows 1/sqrt(p) behavior it is ok 2018/11/7

TCP Performance Can TCP saturate a link? Congestion control Increase utilization until… link becomes congested React by decreasing window by 50% Window is proportional to rate * RTT Doesn’t this mean that the network oscillates between 50 and 100% utilization? Average utilization = 75%?? No…this is *not* right! 2018/11/7

Summary Unbuffered Link W Minimum window for full utilization t The router can’t fully utilize the link If the window is too small, link is not full If the link is full, next window increase causes drop With no buffer it still achieves 75% utilization 2018/11/7

TCP Performance In the real world, router queues play important role Window is proportional to rate * RTT But, RTT changes as well the window Window to fill links = propagation RTT * bottleneck bandwidth If window is larger, packets sit in queue on bottleneck link 2018/11/7

TCP Performance (Cont.) If we have a large router queue  can get 100% utilization But, router queues can cause large delays How big does the queue need to be? Windows vary from W  W/2 Must make sure that link is always full W/2 > RTT * BW W = RTT * BW + Qsize Therefore, Qsize > RTT * BW Ensures 100% utilization Delay? Varies between RTT and 2 * RTT 2018/11/7

Today’s Lecture Transport Introduction Error Recovery Window Flow Control Analysis of TCP Congestion Control TCP Variations Beyond TCP Congestion Control Reading list 2018/11/7

TCP Congestion Control Tahoe Slow Start Congestion Avoidance Fast Retransmit Reno Fast Recovery Vegas New Congestion Avoidance RED Probabilistic marking REM Clear buffer, match rate Too many others 2018/11/7

Variants Tahoe & Reno AQM(Active Queue Management) NewReno SACK Rate-halving Mod.s for high performance AQM(Active Queue Management) RED, ARED, FRED, SRED BLUE, SFB REM, PI 2018/11/7

TCP Tahoe (Jacobson 1988) How to adjust window in each phase? time SS CA SS: Slow Start CA: Congestion Avoidance How to adjust window in each phase? When to switch phase? 2018/11/7

Tahoe (Cont.) Basic ideas Gently probe network for spare capacity Drastically reduce rate on congestion Windowing: self-clocking Other functions: round trip time estimation, error recovery for every ACK { if (W < ssthresh) then W++ (SS) else W += 1/W (CA) } for every loss { ssthresh = W/2 W = 1 2018/11/7

TCP Reno (Jacobson 1990) window Fast retransmission/fast recovery time The picture show TCP tahoe revised by jacobson in 1990. It adds Fast retransmission/fast recovery SS CA SS: Slow Start CA: Congestion Avoidance 2018/11/7

Fast Retransmit Wait for a timeout is quite long Immediately retransmits after 3 dupACKs without waiting for timeout Adjusts ssthresh flightsize = min(awnd, cwnd) ssthresh  max(flightsize/2, 2) Enter Slow Start (cwnd = 1) 2018/11/7

Fast Recovery Motivation: prevent “pipe” from emptying after fast retransmit Idea: each dupACK represents a packet having left the pipe (successfully received) Enter FR/FR after 3 dupACKs Set ssthresh  max(flightsize/2, 2) Retransmit lost packet Set cwnd  ssthresh + ndup (window inflation) Wait till W=min(awnd, cwnd) is large enough; transmit new packet(s) On non-dup ACK (1 RTT later), set cwnd  ssthresh (window deflation) Enter CA 2018/11/7

Example: FR/FR Fast retransmit Fast recovery 7 4 4 4 11 4 cwnd 8 2 3 4 5 6 8 7 1 7 4 9 4 4 11 10 time Exit FR/FR 4 time R 8 cwnd 8 ssthresh Fast retransmit Retransmit on 3 dupACKs Fast recovery Inflate window while repairing loss to fill pipe 2018/11/7

NewReno FR/FR S 1 2 3 4 5 6 7 8 9 10 1 3 timeout time R 2 time 8 9 10 5 8 unack’d pkts On 3 dupACKs, receiver has packets 2, 4, 6, cwnd=8, retransmits pkt 1, enter FR/FR Next dupACK increment cwnd to 9 After a RTT, ACK arrives for pkts 1 & 2, exit FR/FR, cwnd=5, 8 unack’ed pkts No more ACK, sender must wait for timeout TCP New Reno improves retransmission during the fast recovery phase of TCP Reno. During fast recovery, for every duplicate ACK that is returned to TCP New Reno, a new unsent packet from the end of the congestion window is sent, to keep the transmit window full. For every ACK that makes partial progress in the sequence space, the sender assumes that the ACK points to a new hole, and the next packet beyond the ACKed sequence number is sent. Because the timeout timer is reset whenever there is progress in the transmit buffer, this allows New Reno to fill large holes, or multiple holes, in the sequence space - much like TCP SACK. Because New Reno can send new packets at the end of the congestion window during fast recovery, high throughput is maintained during the hole-filling process, even when there are multiple holes, of multiple packets each. When TCP enters fast recovery it records the highest outstanding unacknowledged packet sequence number. When this sequence number is acknowledged, TCP returns to the congestion avoidance state. For example as in this picture: On 3 dupACKs, receiver has packets 2, 4, 6, cwnd=8, retransmits pkt 1, enter FR/FR Next dupACK increment cwnd to 9 After a RTT, ACK arrives for pkts 1 & 2, exit FR/FR, cwnd=5, 8 unack’ed pkts No more ACK, sender must wait for timeout 2018/11/7

NewReno Fall & Floyd ‘96 Motivation: multiple losses within a window Partial ACK acknowledges some but not all packets outstanding at start of FR Partial ACK takes Reno out of FR, deflates window Sender may have to wait for timeout before proceeding Idea: partial ACK indicates lost packets Stays in FR/FR and retransmits immediately Retransmits 1 lost packet per RTT until all lost packets from that window are retransmitted Eliminates timeout 2018/11/7

SACK Mathis, Mahdavi, Floyd, Romanow ’96 Motivation: Reno & NewReno retransmit at most 1 lost packet per RTT Pipe can be emptied during FR/FR with multiple losses Idea: SACK provides better estimate of packets in pipe SACK TCP option describes received packets On 3 dupACKs: retransmits, halves window, enters FR Updates pipe = packets in pipe Increment when lost or new packets sent Decrement when dupACK received Transmits a (lost or new) packet when pipe < cwnd Exit FR when all packets outstanding when FR was entered are acknowledged 2018/11/7

TCP Vegas (Brakmo & Peterson 1994) window time SS CA Converges, no retransmission … provided buffer is large enough 2018/11/7

Vegas CA Algorithm for every RTT { if W/RTTmin – W/RTT < a then W ++ if W/RTTmin – W/RTT > b then W -- } for every loss W := W/2 queue size 2018/11/7

Implications Congestion measure = end-to-end queueing delay At equilibrium Zero loss Stable window at full utilization Approximately weighted proportional fairness Nonzero queue, larger for more sources Convergence to equilibrium Converges if sufficient network buffer Oscillates like Reno otherwise 2018/11/7

Today’s Lecture Transport Introduction Error Recovery Window Flow Control Analysis of TCP Congestion Control TCP Variations Beyond TCP Congestion Control Reading list 2018/11/7

Congestion Router must buffer packets because input > output End-to-end delay increases as buffer fills When buffer is full, “tail drop” occurs 2018/11/7

TCP without ECN Congestion Detection Congestion Avoidance Retransmit Timeout 3 duplicate ACKs Congestion Avoidance Happens after congestion has already occurred (Multiplicative decrease of cwnd AFTER loss) Current TCP does something like congestion ‘recovery’ Network is treated as a black box, no way to know of impending doom 2018/11/7

Active Queue Management Detect “incipient” (early) congestion Try to keep average queue size in “good” range Randomly choose IP-PDUs to notify about congestion (how?) 2018/11/7

Explicit Congestion Notification ECN is an AQM mechanism Routers notify TCP about incipient congestion Use TCP/IP headers to send ECN signals TCP treats ECN signals exactly the same as when a single dropped packet is detected BUT – Packets are NOT actually dropped 2018/11/7

Advantages of ECN Prevents unnecessary packet drops at routers  less retransmissions  improvement in the “GOODPUT” Avoids timeouts by getting faster notification to end hosts Less retransmissions also means less traffic on the network 2018/11/7

ECN Performance Improvements ECN+ - allow SYN ACKs to be marked Internet draft currently RED* - mark packets using ECN, don’t drop 2018/11/7