1. TCP Westwood: Experiments over Large Pipes Cesar Marcondes Anders Persson Prof. M.Y. Sanadidi Prof. Mario Gerla NRL – Network Research Lab UCLA

2. PATHNETS 2004 - San Jose CA Background TCP NewReno is challenged on large pipes: –Slow convergence to full utilization –Not intended to handle non-congestion packet loss Large Pipes performance criteria: –Utilization –Stability –Fast Ramp Up to “Cruising Speed” from Slow start –Fairness under differing RTTs –Friendliness to NewReno Alternatives include: HS TCP, FAST, TCPW Goal of this study: Measurements of TCPW, FAST and HS TCP over large pipes

3. PATHNETS 2004 - San Jose CA TCPW Goal: high utilization, fairness, and friendliness over large leaky dynamic pipes Sender side only estimation of Eligible Rate Estimate (ERE) Estimation takes into account congestion level, capacity of the bottleneck, achieved rate Exponential filtering to time average estimates and avoid network conditions instability ERE is used to: –(1) set congestion window after packet loss –(2) repeatedly reset ssthresh to reach “cruising speed” fast from slow start

4. PATHNETS 2004 - San Jose CA RE Sampling : Packet train, fair estimate under congestion, underestimates under random loss TCPW ABSE BE Sampling : Packet pair, effective under random loss, overestimates under congestion Under Congestion Under No Congestion TkTk TkTk To obtain ERE: adapt the sample interval T k according to congestion level Congestion level is similar to that in Vegas: Expected Rate-Achieved Rate

5. PATHNETS 2004 - San Jose CA Experiments Environment (Powerful Machines) CPU: Xeon 3.06GHz Cache: 512 L2/ 1MB L3 Intel 1000PRO PCI-X BUS 133MHz NewRenoSender AdvancedTCPSender Gigabit link UCLAGigabitSwitch Gigabit link NewRenoReceiver(Alabama) Internet2 NewRenoReceiver(Caltech)

6. PATHNETS 2004 - San Jose CA UCLA Internet2 Link Traffic Our Experiments Traffic Other UCLA Users in Background

7. PATHNETS 2004 - San Jose CA Test Methodology Automated Scripts –Scheduled by Unix crontab –Automatically reinitiate the O.S. with each protocol and conduct new measurements Linux: FAST, HS-TCP and NewReno FreeBSD: TCPW Sender/Receiver buffer is set to 2 MB to enable high utilization of Gbps links Iperf traffic generation, TCPdump, Nistnet emulator

8. PATHNETS 2004 - San Jose CA Benchmark Tests Case Study I: –UCLA-Alabama (155 Mbps, 64 msec) Case Study II: –UCLA-CalTech (1 Gbps, 4msec) Group of 10 successive night time runs for each test Throughput, fairness, friendliness Artificial non-congestion loss (PER 0.1 to 0.5%)

9. PATHNETS 2004 - San Jose CA Case Study I: UCLA–Alabama NewRenoSender AdvancedTCPSender Internet2(Gigabit) ATM Atlanta – Alabama NewRenoReceiver(Alabama) 155Mbps ATM Link Bottleneck Link as measured by PathRate And confirmed later by the network admin

10. PATHNETS 2004 - San Jose CA Throughput Convergence to cruising speed varies among protocols High deviation among multiple runs in HSTCP and NewReno HSTCP deviations decrease over time (as the AIMD behavior changes) UCLA-Alabama

11. PATHNETS 2004 - San Jose CA UCLA-Alabama

12. PATHNETS 2004 - San Jose CA Transfer Completion Times On average: TCPW and FAST: 0 to 100 MB in 5.8 Sec! HSTCP: 0 to 100 MB in 7.5 Sec! NewReno: 0 to 100 MB in 11 Sec! UCLA-Alabama

13. PATHNETS 2004 - San Jose CA Friendliness UCLA-Alabama

14. PATHNETS 2004 - San Jose CA TCP FAST – Preliminary Analysis RTT Variation over Time as Observed by TCPdump Outstanding Window as Observed by TCPdump UCLA-Alabama

15. PATHNETS 2004 - San Jose CA Random Loss Emulation Induced non-congestion packet loss in emulator (PER 0.1% up to 0.5%) TCPW throughput much higher than all other schemes AdvancedTCPSender NewRenoReceiver(Alabama) UCLA – Alabama UCLA-Alabama NistnetNetworkEmulator

16. PATHNETS 2004 - San Jose CA Random Loss Emulation (Results) UCLA-Alabama

17. PATHNETS 2004 - San Jose CA Case Study II: UCLA–CalTech NewRenoSender(UCLA) AdvancedTCPSender(UCLA) Internet2(Gigabit) TCPReceiver(CalTech) 1 Gbps 4 ms

18. PATHNETS 2004 - San Jose CA Throughput TCP NewReno starts-up really high since it relies in the cached threshold and the feedback is really fast Cached Slow Start Threshold versus Adaptive Start-Up (Pros and Cons) Westwood is delayed by its own Stability Filter –Stability-based Filter dampens estimates in proportion to the variance of observation UCLA-CalTech

19. PATHNETS 2004 - San Jose CA UCLA-CalTech

20. PATHNETS 2004 - San Jose CA TCP Westwood Stability Filter versus Fixed Gain Filter Sample Estimations vary a lot due to NIC coalescing and OS issues at Gigabit/s. As variability increases, stability filter relies on a more *stable* moving average filter Solution: Use a fixed gain instead of an adaptive when we know we are dealing with Gbps range speeds TCPW ramp up as HS-TCP and FAST UCLA-CalTech

21. PATHNETS 2004 - San Jose CA TCPW Start-Up using Fixed Exponential Average UCLA-CalTech

22. PATHNETS 2004 - San Jose CA Friendliness UCLA-CalTech

23. PATHNETS 2004 - San Jose CA Conclusions TCPW and FAST performed equally well in terms of average throughput All Advanced TCP protocols have an excellent intra- protocol fairness Friendliness –FAST appears to suffer a synchronization problem Under non-congestion error scenario, TCPW shows greater robustness At Gigabit speed, measurements could be messed up by Interrupt Coalescing and other HW/Kernel bottlenecks, affecting moving average filters

24. PATHNETS 2004 - San Jose CA Future Work New algorithm that is Interrupt Coalescence-Aware for Gbps environment New Agile and Stable Filter Improve the Automated TCP Test Tool (Benchmark and New Tests)

25. PATHNETS 2004 - San Jose CA Thanks Netlab CalTech Xiaoyan Hong – CS / Alabama Univ.