1. TCP and Congestion Control(2)
Dan LI
CS Department, Tsinghua University
Today we’ll talk about TCP and congestion control.
2018/11/7

2. Today’s Lecture Transport Introduction Error Recovery
Window Flow Control
Analysis of TCP Congestion Control
TCP Variations
Beyond TCP Congestion Control
Reading list
2018/11/7

3. Congestion
10 Mbps
100 Mbps
1.5 Mbps
Different sources compete for resources inside network: Link bandwidth and queue space
Why is it a problem?
Sources are unaware of current state of resource
Sources are unaware of each other
In many situations will result in < 1.5 Mbps of throughput (congestion collapse)
2018/11/7

4. Causes & Costs of Congestion
Four senders – multihop paths Timeout/retransmit
Q: What happens as rate increases?
2018/11/7

5. Causes & Costs of Congestion
When packet dropped, any “upstream transmission capacity used for that packet was wasted!
2018/11/7

6. Congestion Collapse
Definition: Increase in network load results in decrease of useful work done
Many possible causes
Spurious retransmissions of packets still in flight
Classical congestion collapse
Solution: better timers and TCP congestion control
Undelivered packets
Packets consume resources and are dropped elsewhere in network
Solution: congestion control for ALL traffic
2018/11/7

7. Other Congestion Collapse Causes
Control traffic
Large percentage of traffic is for control
Headers, routing messages, DNS, etc.
Stale or unwanted packets
Packets that are delayed on long queues
“Push” data that is never used
2018/11/7

8. Congestion Control and Avoidance
Desirable properties:
Scalability:
# flows, range of capacities, range of delays
Do well in entire range!
Efficiency: High network utilization
Fairness
Works-ness: avoids collapse!
Congestion collapse is not just a theory
Has been frequently observed in many networks
2018/11/7

9. Fairness Jain’s fairness index Goal: Something that works well enough
f = (Sxi)2/n(Sxi2)
All x equal: 1
k/n get service: k/n
Goal: Something that works well enough
2018/11/7

10. Objectives Simple router behavior Distributed algorithm
Efficiency: Xknee = Sxi(t)
Fairness: (Sxi)2/n(Sxi2)
Power: (throughputa/delay)
Convergence: control system must be stable
2018/11/7

11. Design Questions Congestion Control questions
How is congestion signaled?
Either mark or drop packets
When is a router congested?
Drop tail queues – when queue is full
Average queue length – at some threshold
Control questions
How do senders react to congestion?
How do senders determine capacity for flow?
2018/11/7

12. Congestion Control Upon congestion, flows must reduce rate
Decrease algorithm
If no congestion, flows might try sending more
Increase algorithm
Let’s assume window-based flow control
Sender maintains “cwnd”: # of unacknowledged packets in the network at any time
Transmission rate: cwnd / rtt
2018/11/7

13. Linear Control
Many different possibilities for reaction to congestion and probing
Examine simple linear controls
Window(t + 1) = a + b Window(t)
Different ai/bi for increase and ad/bd for decrease
Supports various reaction to signals
Increase/decrease additively
Increased/decrease multiplicatively
Which of the four combinations is optimal?
2018/11/7

14. Phase Plots
Simple way to visualize behavior of competing connections over time
User 2’s Allocation x2
User 1’s Allocation x1
2018/11/7

15. Phase Plots What are desirable properties?
What if flows are not equal?
Fairness Line
Overload
User 2’s Allocation x2
Optimal point
Underutilization
Efficiency Line
User 1’s Allocation x1
2018/11/7

16. Additive Increase/Decrease
Both X1 and X2 increase/decrease by the same amount over time
Additive increase improves fairness and additive decrease reduces fairness
Fairness Line T1
User 2’s Allocation x2 T0
Efficiency Line
2018/11/7
User 1’s Allocation x1

17. Multiplicative Increase/Decrease
Both X1 and X2 increase by the same factor over time
Extension from origin – constant fairness
Fairness Line T1
User 2’s Allocation x2 T0
Efficiency Line
User 1’s Allocation x1
2018/11/7

18. What is the Right Choice?
Constraints limit us to AIMD
AIMD moves towards optimal point x0 x1 x2
Efficiency Line
Fairness Line
User 1’s Allocation x1
User 2’s Allocation x2
2018/11/7

19. TCP and Linear Control Upon congestion: While probing
w(t+1) = a*w(t) < a < 1
While probing
w(t+1) = w(t) + b 0 < b << wmax
TCP sets a = 1/2, b = 1 (packet)
2018/11/7

20. TCP Congestion Control
Motivated by ARPANET congestion collapse
Underlying design principle: packet conservation
At equilibrium, inject packet into network only when one is removed
Basis for stability of physical systems
Why was this not working?
Connection doesn’t reach equilibrium
Spurious retransmissions
Resource limitations prevent equilibrium
2018/11/7

21. TCP Congestion Control - Solutions
Reaching equilibrium
Slow start
Eliminates spurious retransmissions
Accurate RTO estimation
Fast retransmit
Adapting to resource availability
Congestion avoidance
2018/11/7

22. TCP Congestion Control
Basic principles
AIMD
Packet conservation
Reaching steady state quickly
ACK clocking
2018/11/7

23. AIMD: Now You Grok the Sawtooth
Distributed, fair and efficient
Packet loss is seen as sign of congestion and results in a multiplicative rate decrease
Factor of 2
TCP periodically probes for available bandwidth by increasing its rate
Rate
Time
2018/11/7

24. Congestion Avoidance If loss occurs when cwnd = W Upon receiving ACK
Network can handle 0.5W ~ W segments
Set cwnd to 0.5W (multiplicative decrease)
Upon receiving ACK
Increase cwnd by (1 packet)/cwnd
What is 1 packet?  1 MSS worth of bytes
After cwnd packets have passed by  approximately increase of 1 MSS
Implements AIMD
2018/11/7

25. Congestion Avoidance Sequence Plot
Sequence No
The picture show the seq-time, the sequence no increases with time, but after some period, congestion happens, and some packets are lost, so the seq stop increasing. And then as the rate is decreased, the seq no increase again.
Packets
Acks
Time
2018/11/7

26. Packet Conservation
At equilibrium, inject packet into network only when one is removed
Sliding window and not rate controlled
But still need to avoid sending burst of packets  would overflow links
Need to carefully pace out packets (ack clocking)!
Helps provide stability
Need to eliminate spurious retransmissions
Accurate RTO estimation
Better loss recovery techniques (e.g. fast retransmit)
2018/11/7

27. TCP Packet Pacing
Congestion window helps to “pace” the transmission of data packets
In steady state, a packet is sent when an ack is received
Data transmission remains smooth, once it is smooth
Self-clocking behavior Pb Pr
Sender
Receiver As Ar Ab
2018/11/7

28. Reaching Steady State Doing AIMD is fine in steady state but slow…
How does TCP know what is a good initial rate to start with?
Should work different network speed
Quick initial phase to help get up to speed (slow start)
2018/11/7

29. Slow Start Packet Pacing
How do we get this clocking behavior to start?
Initialize cwnd = 1
Upon receipt of every ack, cwnd = cwnd + 1
Implications
Window actually increases to W in RTT * log2(W)
Can overshoot window and cause packet loss
2018/11/7

30. Slow Start Example 1 One RTT 0R 2 1R 3 4 2R 5 6 7 8 3R 9 10 11 12 13
One pkt time 0R 2 1R 3 4 2R 5 6 7 8 3R 9 10 11 12 13 14 15
2018/11/7

31. Slow Start Sequence Plot.
Sequence No
Packets
Acks
Time
2018/11/7

32. Return to Slow Start
If packet is lost we lose our self clocking as well
Need to implement slow-start and congestion avoidance together
When timeout occurs set ssthresh to 0.5w
If cwnd < ssthresh, use slow start
Else use congestion avoidance
2018/11/7

33. TCP Saw Tooth Behavior Congestion Window Time Timeouts may still occur
Slowstart
to pace
packets
Fast
Retransmit
and Recovery
Initial
Slowstart
2018/11/7

34. TCP Modeling
Given the congestion behavior of TCP can we predict what type of performance we should get?
What are the important factors
Loss rate
Affects how often window is reduced
RTT
Affects increase rate and relates BW to window
RTO
Affects performance during loss recovery
MSS
Affects increase rate
2018/11/7

35. Simple TCP Model Some additional assumptions
Fixed RTT
No delayed ACKs
In steady state, TCP loses packet each time window reaches W packets
Window drops to W/2 packets
Each RTT window increases by 1 packetW/2 * RTT before next loss
BW = MSS * avg window/RTT = MSS * (W + W/2)/(2 * RTT) = .75 * MSS * W / RTT
2018/11/7

36. Simple Loss Model What was the loss rate? BW = .75 * MSS * W / RTT
Packets transferred = (.75 W/RTT) * (W/2 * RTT) = 3W2/8
1 packet lost  loss rate = p = 8/3W2
W = sqrt( 8 / (3 * loss rate))
BW = .75 * MSS * W / RTT
BW = MSS / (RTT * sqrt (2/3p))
2018/11/7

37. TCP Friendliness What does it mean to be TCP friendly?
TCP is not going away
Any new congestion control must compete with TCP flows
Should not clobber TCP flows and grab bulk of link
Should also be able to hold its own, i.e. grab its fair share, or it will never become popular
How is this quantified/shown?
Has evolved into evaluating loss/throughput behavior
If it shows 1/sqrt(p) behavior it is ok
2018/11/7

38. TCP Performance Can TCP saturate a link? Congestion control
Increase utilization until… link becomes congested
React by decreasing window by 50%
Window is proportional to rate * RTT
Doesn’t this mean that the network oscillates between 50 and 100% utilization?
Average utilization = 75%??
No…this is *not* right!
2018/11/7

39. Summary Unbuffered LinkW
Minimum window for full utilization t
The router can’t fully utilize the link
If the window is too small, link is not full
If the link is full, next window increase causes drop
With no buffer it still achieves 75% utilization
2018/11/7

40. TCP Performance In the real world, router queues play important role
Window is proportional to rate * RTT
But, RTT changes as well the window
Window to fill links = propagation RTT * bottleneck bandwidth
If window is larger, packets sit in queue on bottleneck link
2018/11/7

41. TCP Performance (Cont.)
If we have a large router queue  can get 100% utilization
But, router queues can cause large delays
How big does the queue need to be?
Windows vary from W  W/2
Must make sure that link is always full
W/2 > RTT * BW
W = RTT * BW + Qsize
Therefore, Qsize > RTT * BW
Ensures 100% utilization
Delay?
Varies between RTT and 2 * RTT
2018/11/7

42. Today’s Lecture Transport Introduction Error Recovery
Window Flow Control
Analysis of TCP Congestion Control
TCP Variations
Beyond TCP Congestion Control
Reading list
2018/11/7

43. TCP Congestion Control
Tahoe
Slow Start
Congestion Avoidance
Fast Retransmit
Reno
Fast Recovery
Vegas
New Congestion Avoidance
RED
Probabilistic marking
REM
Clear buffer, match rate
Too many others
2018/11/7

44. Variants Tahoe & Reno AQM(Active Queue Management) NewReno SACK
Rate-halving
Mod.s for high performance
AQM(Active Queue Management)
RED, ARED, FRED, SRED
BLUE, SFB
REM, PI
2018/11/7

45. TCP Tahoe (Jacobson 1988) How to adjust window in each phase?
time SS CA
SS: Slow Start
CA: Congestion Avoidance
How to adjust window in each phase?
When to switch phase?
2018/11/7

46. Tahoe (Cont.) Basic ideas Gently probe network for spare capacity
Drastically reduce rate on congestion
Windowing: self-clocking
Other functions: round trip time estimation, error recovery
for every ACK {
if (W < ssthresh) then W++ (SS)
else W += 1/W (CA) }
for every loss {
ssthresh = W/2
W = 1
2018/11/7

47. TCP Reno (Jacobson 1990) window Fast retransmission/fast recovery time
The picture show TCP tahoe revised by jacobson in It adds Fast retransmission/fast recovery SS CA
SS: Slow Start
CA: Congestion Avoidance
2018/11/7

48. Fast Retransmit Wait for a timeout is quite long
Immediately retransmits after 3 dupACKs without waiting for timeout
Adjusts ssthresh
flightsize = min(awnd, cwnd)
ssthresh  max(flightsize/2, 2)
Enter Slow Start (cwnd = 1)
2018/11/7

49. Fast Recovery
Motivation: prevent “pipe” from emptying after fast retransmit
Idea: each dupACK represents a packet having left the pipe (successfully received)
Enter FR/FR after 3 dupACKs
Set ssthresh  max(flightsize/2, 2)
Retransmit lost packet
Set cwnd  ssthresh + ndup (window inflation)
Wait till W=min(awnd, cwnd) is large enough; transmit new packet(s)
On non-dup ACK (1 RTT later), set cwnd  ssthresh (window deflation)
Enter CA
2018/11/7

50. Example: FR/FR Fast retransmit Fast recovery 7 4 4 4 11 4 cwnd 82 3 4 5 6 8 7 1 7 4 9 4 4 11 10
time
Exit FR/FR 4
time R 8
cwnd 8
ssthresh
Fast retransmit
Retransmit on 3 dupACKs
Fast recovery
Inflate window while repairing loss to fill pipe
2018/11/7

51. NewReno
FR/FR S 1 2 3 4 5 6 7 8 9 10 1 3
timeout
time R 2
time 8 9 10 5
8 unack’d pkts
On 3 dupACKs, receiver has packets 2, 4, 6, cwnd=8, retransmits pkt 1, enter FR/FR
Next dupACK increment cwnd to 9
After a RTT, ACK arrives for pkts 1 & 2, exit FR/FR, cwnd=5, 8 unack’ed pkts
No more ACK, sender must wait for timeout
TCP New Reno improves retransmission during the fast recovery phase of TCP Reno. During fast recovery, for every duplicate ACK that is returned to TCP New Reno, a new unsent packet from the end of the congestion window is sent, to keep the transmit window full. For every ACK that makes partial progress in the sequence space, the sender assumes that the ACK points to a new hole, and the next packet beyond the ACKed sequence number is sent.
Because the timeout timer is reset whenever there is progress in the transmit buffer, this allows New Reno to fill large holes, or multiple holes, in the sequence space - much like TCP SACK. Because New Reno can send new packets at the end of the congestion window during fast recovery, high throughput is maintained during the hole-filling process, even when there are multiple holes, of multiple packets each. When TCP enters fast recovery it records the highest outstanding unacknowledged packet sequence number. When this sequence number is acknowledged, TCP returns to the congestion avoidance state.
For example as in this picture:
On 3 dupACKs, receiver has packets 2, 4, 6, cwnd=8, retransmits pkt 1, enter FR/FR
Next dupACK increment cwnd to 9
After a RTT, ACK arrives for pkts 1 & 2, exit FR/FR, cwnd=5, 8 unack’ed pkts
No more ACK, sender must wait for timeout
2018/11/7

52. NewReno Fall & Floyd ‘96 Motivation: multiple losses within a window
Partial ACK acknowledges some but not all packets outstanding at start of FR
Partial ACK takes Reno out of FR, deflates window
Sender may have to wait for timeout before proceeding
Idea: partial ACK indicates lost packets
Stays in FR/FR and retransmits immediately
Retransmits 1 lost packet per RTT until all lost packets from that window are retransmitted
Eliminates timeout
2018/11/7

53. SACK Mathis, Mahdavi, Floyd, Romanow ’96
Motivation: Reno & NewReno retransmit at most 1 lost packet per RTT
Pipe can be emptied during FR/FR with multiple losses
Idea: SACK provides better estimate of packets in pipe
SACK TCP option describes received packets
On 3 dupACKs: retransmits, halves window, enters FR
Updates pipe = packets in pipe
Increment when lost or new packets sent
Decrement when dupACK received
Transmits a (lost or new) packet when pipe < cwnd
Exit FR when all packets outstanding when FR was entered are acknowledged
2018/11/7

54. TCP Vegas (Brakmo & Peterson 1994)
window
time SS CA
Converges, no retransmission
… provided buffer is large enough
2018/11/7

55. Vegas CA Algorithm for every RTT
{ if W/RTTmin – W/RTT < a then W ++
if W/RTTmin – W/RTT > b then W -- }
for every loss
W := W/2
queue size
2018/11/7

56. Implications Congestion measure = end-to-end queueing delay
At equilibrium
Zero loss
Stable window at full utilization
Approximately weighted proportional fairness
Nonzero queue, larger for more sources
Convergence to equilibrium
Converges if sufficient network buffer
Oscillates like Reno otherwise
2018/11/7

57. Today’s Lecture Transport Introduction Error Recovery
Window Flow Control
Analysis of TCP Congestion Control
TCP Variations
Beyond TCP Congestion Control
Reading list
2018/11/7

58. Congestion Router must buffer packets because input > output
End-to-end delay increases as buffer fills
When buffer is full, “tail drop” occurs
2018/11/7

59. TCP without ECN Congestion Detection Congestion Avoidance
Retransmit Timeout
3 duplicate ACKs
Congestion Avoidance
Happens after congestion has already occurred (Multiplicative decrease of cwnd AFTER loss)
Current TCP does something like congestion ‘recovery’
Network is treated as a black box, no way to know of impending doom
2018/11/7

60. Active Queue Management
Detect “incipient” (early) congestion
Try to keep average queue size in “good” range
Randomly choose IP-PDUs to notify about congestion (how?)
2018/11/7

61. Explicit Congestion Notification
ECN is an AQM mechanism
Routers notify TCP about incipient congestion
Use TCP/IP headers to send ECN signals
TCP treats ECN signals exactly the same as when a single dropped packet is detected
BUT – Packets are NOT actually dropped
2018/11/7

62. Advantages of ECN
Prevents unnecessary packet drops at routers  less retransmissions  improvement in the “GOODPUT”
Avoids timeouts by getting faster notification to end hosts
Less retransmissions also means less traffic on the network
2018/11/7

63. ECN Performance Improvements
ECN+ - allow SYN ACKs to be marked
Internet draft currently
RED* - mark packets using ECN, don’t drop
2018/11/7