1. Jennifer Rexford Fall 2014 (TTh 3:00-4:20 in CS 105) COS 561: Advanced Computer Networks http://www.cs.princeton.edu/courses/archive/fall14/cos561/ TCP Congestion Control at the Network Edge

2. Original TCP Design 2 Internet

3. Wireless and Data-Center Networks 3 Internet wireless network data-center network

4. Data-Center Networks 4

5. Modular Network Topology Containers Racks –Multiple servers –Top-of-rack switches 5

6. Tree-Like Topologies Many equal-cost paths Small round-trip times (e.g., < 250 microseconds) 6 CR AR S S S S S S S S A A A A A A … S S S S A A A A A A …... S S S S S S S S A A A A A A … S S S S A A A A A A …

7. Commodity Switches Low-cost switches –Especially for top-of-rack switches Simple memory architecture –Small packet-buffer space –Shared buffer over all input ports –Simple drop-tail queues 7

8. Multi-Tier Applications 8 8 Front end Server Aggregator … Aggregator Worker … …

9. Application Mix Partition-aggregate workflow –Multiple workers working in parallel –Straggler slows down the entire system –Many workers send response at the same time Diverse mix of traffic –Low latency for short flows –High throughput for long flows Multi-tenancy –Many tenants sharing the same platform –Running network to high levels of utilization Small number of large flows on links 9

10. TCP Incast Problem Multiple workers transmitting to one aggregator –Many flows traversing the same link –Burst of packets sent at (nearly) the same time –… into a relatively small switch memory Leading to high packet loss –Some results are slow to arrive –May be excluded from the final results Developer software changes –Limit the size of worker responses –Randomize the sending time for responses 10

11. Queue Buildup Mix of long and short flows –Long flows fill up the buffers in switches –… causing queuing delay (and loss) for the short flows –E.g., queuing delay of 1-14 milliseconds Large relative to propagation delay –E.g., 100 microseconds intra-rack –E.g., 250 microseconds inter-rack –Leading to RTT variance and big throughput drop Shared switch buffers –Short flows on one port –… affected by long flows on other ports 11

12. TCP Outcast Problem Mix of flows at two different input ports –Many inter-rack flows –Few intra-rack flows –Destined for same output Burst of packet arrivals –Arriving on one input port –Causing bursty loss for the other Harmful to the intra-rack flows –Lose multiple packets –Loss detected by timeout –Irony: worse throughput despite lower RTT! 12 AR S S S S S S S S A A A A A A … S S S S A A A A A A …

13. Delayed Acknowledgments Sending ACKs can be expensive –E.g., send 40-byte ACK packet for each data packet Delay ACKs reduce the overhead –Receiver waits before sending the ACK –… in the hope of piggybacking the ACK on a response Delayed-ACK mechanism –Set a timer when the data arrives (e.g., 200 msec) –Piggyback the ACK or send ACK for every other packet –… or send an ACK after the timer expires Timeout for delayed ACK is an eternity!! –Disable delayed ACKs, or shorten the timer 13

14. Data-Center TCP (DCTCP) Key observation –TCP reacts to the presence of congestion –… not to the extent of congestion Measuring extent of congestion –Mark packets when buffer exceeds a threshold Reacting to congestion –Reduce cwnd in proportion to fraction of marked packets Benefits –React early, as queue starts to build –Prevent harm to packets on other ports –Get workers to reduce sending rate early 14

15. Poor Multi-Path Load Balancing Multiple shortest paths between pairs of hosts –Spread the load over multiple paths Equal-cost multipath –Round robin –Hash-based Uneven load –Elephant flows congest some paths –… while other paths are lightly loaded Reducing congestion –Careful routing of elephant flows 15

16. Wireless Networks 16

17. TCP Design Setting Relatively low packet loss –E.g., hopefully less than 1% –Okay to retransmit lost packets from the sender Loss is caused primarily by congestion –Use loss as an implicit signal of congestion –… and reduce the sending rate Relatively stable round-trip times –Use RTT estimate in retransmission timer End-points are always on Stable end-point IP addresses –Use IP addresses as end-point identifiers 17

18. Problems in Wireless Networks Limited bandwidth High latencies High bit-error rates Temporary disconnections Slow handoffs Mobile device disconnects to save energy, bearers, etc. 18 Internet

19. Link-Level Retransmission Retransmit over the wireless link –Hide packet losses from end-to-end –… by retransmitting lost packets on wireless link –Works for any transport protocol 19

20. Split Connection 20 Two TCP connections –Between fixed host and the base station –Between base and the mobile device Other optimizations –Compression, just-in-time delivery, etc.

21. Burst Optimization Radio wakeup is expensive –Wake up –Establish a bearer –Use battery and signaling resources Burst optimization –Send bigger chunks less often –… to allow the mobile device to go to idle state 21

22. Lossless Handover Mobile moves from one base station to another –Packets in flight still arrive at the old base station –… and could lead to bursty loss (and TCP timeout) Old base station can buffer packets –Send buffered packets to the new base station 22 Internet

23. Freezing the Connection Mobile device can predict temporary disconnection –E.g., fading, handoff Mobile can ask the fixed host to stop sending –Advertise a receive window of 0 Benefits –Avoids wasted transmission of data –Avoid loss that triggers timeouts, decrease in cwnd, etc. 23

24. Discussion CUBIC paper 24