1. Computer Networking
TCP

2. Overview TCP timeouts Congestion sources and collapse
Congestion control basics
TCP congestion control
TCP fast recovery
Lecture 15:

3. Reliability Challenges
Like reliability on links
Similar techniques (timeouts, acknowledgements, etc.)
New challenges
Congestion related losses
Variable packet delays
What should the timeout be?
Reordering of packets
Ensure sequences numbers are not reused
How long do packets live?
MSL = 120 seconds based on IP behavior
Lecture 15:

4. TCP = Go-Back-N Variant
Receiver can only return a single “ack” sequence number to the sender.
Acknowledges all bytes with a lower sequence number
Starting point for retransmission
But: sender only retransmits a single packet.
Reason???
Error control is based on byte sequences, not packets.
Retransmitted packet can be different from the original lost packet – why?
Packets can overlap – why?
Sliding window with cumulative acks
Ack field contains last in-order packet received
Duplicate acks sent when out-of-order packet received
Lecture 15:

5. Round-trip Time Estimation
Wait at least one RTT before retransmitting
Importance of accurate RTT estimators:
Low RTT  unneeded retransmissions
High RTT  poor throughput
RTT estimator must adapt to change in RTT
But not too fast, or too slow!
Spurious timeouts
“Conservation of packets” principle – more than a window worth of packets in flight
Lecture 15:

6. Initial Round-trip Estimator
Round trip times exponentially averaged:
New RTT = a (old RTT) + (1 - a) (new sample)
Recommended value for a:
0.875 for most TCP’s
Retransmit timer set to b RTT, where b = 2
Every time timer expires, RTO exponentially backed-off
Like Ethernet
Not good at preventing spurious timeouts
Lecture 15:

7. Jacobson’s Retransmission Timeout
Key observation:
At high loads round trip variance is high
Solution:
Base RTO on RTT and standard deviation or RRTT
rttvar =  * dev + (1- )rttvar
Dev = linear deviation
Inappropriately named – actually smoothed linear deviation
Lecture 15:

8. Retransmission Ambiguity
Original transmission
retransmission
Sample
RTT
ACK
RTO
Original transmission X
RTO
Sample
RTT
retransmission
ACK
Lecture 15:

9. Karn’s RTT Estimator Accounts for retransmission ambiguity
If a segment has been retransmitted:
Don’t count RTT sample on ACKs for this segment
Keep backed off time-out for next packet
Reuse RTT estimate only after one successful transmission
Lecture 15:

10. Timestamp Extension
Used to improve timeout mechanism by more accurate measurement of RTT
When sending a packet, insert current timestamp into option
4 bytes for seconds, 4 bytes for microseconds
Receiver echoes timestamp in ACK
Actually will echo whatever is in timestamp
Removes retransmission ambiguity
Can get RTT sample on any packet
Lecture 15:

11. Timer Granularity
Many TCP implementations set RTO in multiples of 200,500,1000ms
Why?
Avoid spurious timeouts – RTTs can vary quickly due to cross traffic
Make timers interrupts efficient
What happens for the first couple of packets?
Pick a very conservative value (seconds)
Lecture 15:

12. Delayed ACKS Problem: Solution:
In request/response programs, you send separate ACK and Data packets for each transaction
Solution:
Don’t ACK data immediately
Wait 200ms (must be less than 500ms – why?)
Must ACK every other packet
Must not delay duplicate ACKs
Lecture 15:

13. TCP ACK Generation [RFC 1122, RFC 2581]
Event
In-order segment arrival,
No gaps,
Everything else already ACKed
One delayed ACK pending
Out-of-order segment arrival
Higher-than-expect seq. #
Gap detected
Arrival of segment that
Partially or completely fills gap
TCP Receiver action
Delayed ACK. Wait up to 500ms
for next segment. If no next segment,
send ACK
Immediately send single
cumulative ACK
Send duplicate ACK, indicating seq. #
of next expected byte
Immediate ACK if segment starts
at lower end of gap
Lecture 15:

14. Overview TCP timeouts Congestion sources and collapse
Congestion control basics
TCP congestion control
TCP fast recovery
Lecture 15:

15. Congestion Different sources compete for resources inside network
10 Mbps
100 Mbps
1.5 Mbps
Different sources compete for resources inside network
Why is it a problem?
Sources are unaware of current state of resource
Sources are unaware of each other
Manifestations:
Lost packets (buffer overflow at routers)
Long delays (queuing in router buffers)
In many situations will result in < 1.5 Mbps of throughput for the above topology (congestion collapse)
Lecture 15:

16. Causes & Costs of Congestion: Scenario 1
Two senders, two receivers
One router, infinite buffers
No retransmission
Large delays when congested
Maximum achievable throughput
Lecture 15:

17. Causes & Costs of Congestion: Scenario 2
One router, finite buffers
Sender retransmission of lost packet
Lecture 15:

18. Causes & Costs of Congestion: Scenario 2l in
out =
Always: (goodput)
“Perfect” retransmission only when loss:
Retransmission of delayed (not lost) packet makes larger (than perfect case) for same l in
out
> l in l
out
“Costs” of congestion:
More work (retrans) for given “goodput”
Unneeded retransmissions: link carries multiple copies of pkt
Lecture 15:

19. Causes & Costs of Congestion: Scenario 3
Four senders
Multihop paths
Timeout/retransmit
Q: what happens as and increase ? l in l in
Lecture 15:

20. Causes & Costs of Congestion: Scenario 3
Another “cost” of congestion:
When packet dropped, any “upstream transmission capacity used for that packet was wasted!
Lecture 15:

21. Congestion Collapse
Definition: Increase in network load results in decrease of useful work done
Many possible causes
Spurious retransmissions of packets still in flight
Classical congestion collapse
How can this happen with packet conservation
Solution: better timers and TCP congestion control
Undelivered packets
Packets consume resources and are dropped elsewhere in network
Solution: congestion control for ALL traffic
Lecture 15:

22. Other Congestion Collapse Causes
Fragments
Mismatch of transmission and retransmission units
Solutions
Make network drop all fragments of a packet (early packet discard in ATM)
Do path MTU discovery
Control traffic
Large percentage of traffic is for control
Headers, routing messages, DNS, etc.
Stale or unwanted packets
Packets that are delayed on long queues
“Push” data that is never used
Lecture 15:

23. Congestion Control and Avoidance
A mechanism which:
Uses network resources efficiently
Preserves fair network resource allocation
Prevents or avoids collapse
Congestion collapse is not just a theory
Has been frequently observed in many networks
Lecture 15:

24. Approaches Towards Congestion Control
Two broad approaches towards congestion control:
End-end congestion control:
No explicit feedback from network
Congestion inferred from end-system observed loss, delay
Approach taken by TCP
Network-assisted congestion control:
Routers provide feedback to end systems
Single bit indicating congestion (SNA, DECbit, TCP/IP ECN, ATM)
Explicit rate sender should send at
Lecture 15:

25. Example: ATM Rate-based Flow Control
Provide explicit “per flow” feedback.
Host tells the network how fast it is sending
Switches calculate how fast the host should be sending and include this information in explicit flow control packets that are sent periodically
Feedback can be binary or explicit.
binary: source slows down or speeds up
explicit: switch specifies the rate 16 16 16 16 16 7 7 10 16 10
Lecture 15:

26. Example: TCP Very simple mechanisms in network
FIFO scheduling with shared buffer pool
Feedback through packet drops
TCP interprets packet drops as signs of congestion and slows down
This is an assumption: packet drops are not a sign of congestion in all networks
E.g. wireless networks
Periodically probes the network to check whether more bandwidth has become available.
Lecture 15:

27. Tradeoffs in Congestion Control
Explicit schemes that isolate traffic flows seem preferable, but require more support inside the networks
per-flow buffering, weighted fair queuing, policing
scalability???
determining nature of the explicit feedback in heterogeneous environment
Diversity in networks makes TCP approach a good solution
Dropping packets is universally a natural response to congestion
But many open issues: how to isolate poorly behaved sources, diversity in TCP implementations, ...
Lecture 15:

28. Overview TCP timeouts Congestion sources and collapse
Congestion control basics
TCP congestion control
TCP fast recovery
Lecture 15:

29. Objectives Simple router behavior Distributedness
Efficiency: X = Sxi(t)
Fairness: (Sxi)2/n(Sxi2)
Convergence: control system must be stable
Lecture 15:

30. Basic Control Model Reduce speed when congestion is perceived
How is congestion signaled?
Either mark or drop packets
How much to reduce?
Increase speed otherwise
Probe for available bandwidth – how?
Lecture 15:

31. Linear Control
Many different possibilities for reaction to congestion and probing
Examine simple linear controls
Window(t + 1) = a + b Window(t)
Different ai/bi for increase and ad/bd for decrease
Supports various reaction to signals
Increase/decrease additively
Increased/decrease multiplicatively
Which of the four combinations is optimal?
Lecture 15:

32. Phase Plots
Simple way to visualize behavior of competing connections over time
User 2’s Allocation x2
User 1’s Allocation x1
Lecture 15:

33. Phase Plots What are desirable properties?
What if flows are not equal?
Fairness Line
Overload
User 2’s Allocation x2
Optimal point
Underutilization
Efficiency Line
User 1’s Allocation x1
Lecture 15:

34. Additive Increase/Decrease
Both X1 and X2 increase/ decrease by the same amount over time
Additive increase improves fairness and additive decrease reduces fairness
Fairness Line T1
User 2’s Allocation x2 T0
Efficiency Line
User 1’s Allocation x1
Lecture 15:

35. Muliplicative Increase/Decrease
Both X1 and X2 increase by the same factor over time
Extension from origin – constant fairness
Fairness Line T1
User 2’s Allocation x2 T0
Efficiency Line
User 1’s Allocation x1
Lecture 15:

36. Convergence to Efficiency
Fairness Line xH
User 2’s Allocation x2
Efficiency Line
User 1’s Allocation x1
Lecture 15:

37. Distributed Convergence to EfficiencyxH
Efficiency Line
Fairness Line
User 1’s Allocation x1
User 2’s Allocation x2
a=0
b=1
Lecture 15:

38. Convergence to Fairness
Fairness Line xH
User 2’s Allocation x2
xH’
Efficiency Line
User 1’s Allocation x1
Lecture 15:

39. Convergence to Efficiency & Fairness
Fairness Line xH
User 2’s Allocation x2
xH’
Efficiency Line
User 1’s Allocation x1
Lecture 15:

40. What is the Right Choice?
Constraints limit us to AIMD
Can have multiplicative term in increase
AIMD moves towards optimal point x0 x1 x2
Efficiency Line
Fairness Line
User 1’s Allocation x1
User 2’s Allocation x2
Lecture 15:

41. Overview TCP timeouts Congestion sources and collapse
Congestion control basics
TCP congestion control
TCP fast recovery
Lecture 15:

42. TCP Congestion Control
Motivated by ARPANET congestion collapse
Underlying design principle: packet conservation
At equilibrium, inject packet into network only when one is removed
Basis for stability of physical systems
Why was this not working?
Connection doesn’t reach equilibrium
Spurious retransmissions
Resource limitations prevent equilibrium
Lecture 15:

43. TCP Congestion Control - Solutions
Reaching equilibrium
Slow start
Eliminates spurious retransmissions
Accurate RTO estimation
Fast retransmit
Adapting to resource availability
Congestion avoidance
Lecture 15:

44. TCP Congestion Control
Packet loss is seen as sign of congestion and results in a multiplicative rate decrease
Factor of 2
TCP periodically probes for available bandwidth by increasing its rate
Rate
Time
Lecture 15:

45. TCP Congestion Control Open Questions
How can this be implemented?
Operating system timers are very coarse – how do you accurately calculate the transmission rate?
How does TCP know what is a good initial rate to start with?
Should work both for a CDPD (10s of Kbs or less) and for supercomputer links (2.4 Gbs and growing)
Lecture 15:

46. TCP Congestion Control Implementation
Implemented using a congestion window that limits how much data can be in the network.
TCP also keeps track of how much data is in transit
Data can only be sent when the amount of outstanding data is less than the congestion window.
The amount of outstanding data is increased on a “send” and decreased on “ack”
(last sent – last acked) < congestion window
Window limited by both congestion and buffering
Sender’s maximum window = Min (advertised window, cwnd)
Lecture 15:

47. TCP Packet Pacing
Congestion window helps to “pace” the transmission of data packets
In steady state, a packet is sent when an ack is received
Data transmission remains smooth, once it is smooth
Self-clocking behavior Pb Pr
Sender
Receiver As Ar Ab
Lecture 15:

48. Congestion Avoidance If loss occurs when cwnd = W Upon receiving ACK
Network can handle 0.5W ~ W segments
Set cwnd to 0.5W (multiplicative decrease)
Upon receiving ACK
Increase cwnd by 1/cwnd
Implements AIMD
Lecture 15:

49. Congestion Avoidance Sequence Plot
Sequence No
Time
Lecture 15:

50. Congestion Avoidance Behavior
Window
Time
Cut
Congestion
Window
and Rate
Packet loss
+ Timeout
Grabbing
back
Bandwidth
Lecture 15:

51. Slow Start Packet Pacing
How do we get this clocking behavior to start?
Initialize cwnd = 1
Upon receipt of every ack, cwnd = cwnd + 1
Implications
Window actually increases to W in RTT * log2(W)
Can overshoot window and cause packet loss
Lecture 15:

52. Slow Start Example 1 One RTT 0R 2 1R 3 4 2R 5 6 7 8 3R 9 10 11 12 13
One pkt time 0R 2 1R 3 4 2R 5 6 7 8 3R 9 10 11 12 13 14 15
Lecture 15:

53. Slow Start Sequence Plot.
Sequence No
Time
Lecture 15:

54. Return to Slow Start
If packet is lost we lose our self clocking as well
Need to implement slow-start and congestion avoidance together
When timeout occurs set ssthresh to 0.5w
If cwnd < ssthresh, use slow start
Else use congestion avoidance
Lecture 15:

55. TCP Saw Tooth Behavior Congestion Window Time Timeouts may still occur
Slowstart
to pace
packets
Fast
Retransmit
and Recovery
Initial
Slowstart
Lecture 15:

56. Overview TCP timeouts Congestion sources and collapse
Congestion control basics
TCP congestion control
TCP fast recovery
Lecture 15:

57. TCP Flavors Tahoe, Reno, Vegas
TCP Tahoe (distributed with 4.3BSD Unix)
Original implementation of Van Jacobson’s mechanisms (VJ paper)
Includes:
Slow start
Congestion avoidance
Fast retransmit
Lecture 15:

58. Fast Retransmit What are duplicate acks (dupacks)?
Repeated acks for the same sequence
When can duplicate acks occur?
Loss
Packet re-ordering
Window update – advertisement of new flow control window
Assume re-ordering is infrequent and not of large magnitude
Use receipt of 3 or more duplicate acks as indication of loss
Don’t wait for timeout to retransmit packet
Lecture 15:

59. Fast Retransmit X Retransmission Duplicate Acks Sequence No Time
Lecture 15:

60. Multiple Losses X X X X Now what? Retransmission Duplicate Acks
Sequence No
Time
Lecture 15: