1. Optimizing UDP-based Protocol Implementations Yunhong Gu and Robert L. Grossman Presenter: Michal Sabala National Center for Data Mining

2. Outline UDP Performance Characteristics and Optimizations Composable UDT: A Framework for UDP- based Protocol Implementations

3. Part I. UDP Performance Characteristics and Optimization Techniques

4. Introduction UDP-based Protocol is needed –As short-term solution to the lack of effective kernel space transport protocols for high bandwidth-delay product networks –As application specific data transfer library, e.g., Multimedia data transfer It is not an easy task to impalement a new UDP-based protocol from scratch –And may be not necessary!

5. UDP Performance Sending and receiving buffer size Packet size IO mode –Scattering/gathering (writev/readv) –Memory copy avoidance (e.g., overlapped IO of Windows Socket2) To reach same data transfer rate, UDP needs slightly less CPU time than TCP, and cause slightly less end system delay

6. UDP Performance: Impact of Buffer Size

7. UDP Performance: Impact of Packet Size Throughput CPU Util.

8. UDP-based Protocol Performance Additional overhead –Additional memory copy –Additional packet processing –Additional context switches

9. Optimization Guidelines Avoid additional memory copy Reduce the number of packets –Control packets, esp. acknowledgements Reduce overall processing time –Simpler mechanism is better Avoid burst in processing time –CPU may be too busy to process incoming packets

10. Optimization Guidelines Memory copy avoidance –UDP IO –API semantics Acknowledgements –Timer-based Acknowledging –Light ACK –Loss processing Timing, rate control, and self-clocking

11. Optimization Guidelines Disk IO –sendfile/recvfile Threading –Synchronization cost Code Optimization –sending/receiving loop Profiling

12. Part II. Composable UDT: A Framework for UDP-based Protocol Implementations

13. Composable UDT Based on the UDT (UDP-based Data Transfer library) implementation Integrated those optimization techniques described in this paper

14. Objectives Rapid development of UDP-based transport protocols and application specific data transfer libraries Easy evaluation of new congestion control algorithms Non-objectives –Replace kernel space protocol implementations –User-level TCP implementation

15. Current Status UDT/CCC: Configurable congestion control In future –Data reliability configuration –Message boundary support

16. Configurable Congestion Control Packet sending control –Rate-based, window-based, hybrid Redefinition of control event handlers –Loss, ACK, Time Out, etc. Access to internal protocol parameters –RTT, RTO, Loss Rate, etc. User customized packet formats

17. Implementation C++ class inheritance –CCC: base class for control event handing Callbacks Performance monitoring –Internal protocol parameters –Performance statistics

18. Implementation

19. Example: Simplified TCP class CTCP: public CCC { public: virtual void init() { m_dPktSndPeriod = 0.0; m_dCWndSize = 2.0; setACKInterval(2); } virtual void onACK(const int&) { m_dCWndSize += 1.0/m_dCWndSize; } virtual void onLoss(const int*, const int&) { m_dCWndSize *= 0.5; } };

20. Configurable Congestion Control

21. Future Work Continue to improve the UDT/CCC library More experimental evaluation work of the UDT/CCC library –Compare k-TCP and u-TCP in more network environments –Implement more TCP variants More pre-implemented congestion control algorithms

22. Conclusion UDP-based protocol is one of the solutions to bulk data transfer in high BDP networks Some optimization principles and techniques are discussed in this paper We further propose a composable framework in order to make it much easier to implement UDP-based protocols

23. Thank you! For more information, please visit UDT Project: http://udt.sf.net NCDM: http://www.ncdm.uic.edu

24. Backup Slides

25. UDP Performance: Experiment Setup NameCPUMemoryNICOS onno Dual Itanium2 1.5GHz 8 GB10 GbELinux 2.6.0 sara77 Dual Xeon 2.4GHz 2 GB1 GbELinux 2.4.18 ncdm171 Dual PowerPC G4 1GHz 2 GB1 GbEMac OS X win91 Dual Xeon 2.4GHz 2 GB1 GbE Windows XP Professional ncdm87 Dual Opteron 2.4GHz 4 GB1 GbELinux 2.6.8

26. UDP Performance: CPU Utilization Name UDPTCP SendingReceivingSendingReceiving onno0.220.350.230.50 sara77 0.400.450.51 ncdm171 1.221.452.222.73 win91 1.031.091.141.28 ncdm87 0.260.400.250.56

27. UDP Performance: End System Delay Name UDPTCP Delay (ms) onno0.0620.068 sara77 0.0700.086 ncdm171 0.2020.245 win91 0.2030.302 ncdm87 0.0650.087

28. UDT Profiling: Modules

29. UDT Profiling: Functionalities

30. CPU Utilization: K-TCP vs U-TCP Machines SenderReceiver K-TCPU-TCPK-TCPU-TCP onno0.230.600.500.44 sara770.510.650.510.78 ncdm1712.223.462.733.26 win911.142.071.281.20 ncdm870.250.520.560.60