1. FISP1 ECE 7750 Distributed Control Systems FISP1 Topic: Network Delay Mitigation Prepared by: Nathan Erni Dept. of Electrical and Computer Engineering Utah State University E: nate.erni@aggiemail.usu.edu; T: (435)232-7526; 2/18/2010

2. Overview Problem Statement UDP vs. TCP Things to Consider UDP – What causes Delay TCP – How are packets lost – What causes Delay Measurement Resources Summary Further Work References Questions

3. Problem Statement Network Delay

4. UDP vs. TCP User Datagram Protocol (UDP) – Connectionless Header information is reduced, thus packet size is lighter Reduction in processing time – Assumes packet arrival. Does not rebroadcast – Mainly used in short communication links (LAN) – Latency can be as low as packet Round Trip Time (RTT) Best type of protocol to reduce latency. Most common are Real Time Protocol (RTP) and Real Time Control Protocol (RTCP) run under the UDP protocol. – Can have Missing packets, duplicate packets, or packet in wrong order Transmission Control Protocol (TCP) – Connection-Oriented 3-way communication before data transfer – Rebroadcasts packets if lost Insures that packets arrive Processing time increased – Keeps track of order – Mainly used in long distance communication links (WAN)

5. Things to Consider Understand your system that you want to control and pick what protocol works best for you. – Where is your controller located compared to the plant? – How will your system respond to packet loss and delay? – How much bandwidth does your controller have on the network? Is it shared?

6. User Datagram Protocol What Causes Delay? – Hub All network data it receives on one port will be immediately transmitted out all the other ports, so each computer must take it's turn before sending data. – Switch Acts like a smart hub. All network data it receives will be directed to only the destination port. This frees up bandwidth for all other lines.

7. Two LANS Joined by a Subnet Application Presentation Session Transport Network Datalink PhysicalNetwork Datalink Physical Network Datalink Physical Network Datalink Physical Network Datalink Physical Application Presentation Session Transport Network Datalink Physical LAN 1LAN 2Transmission Line Router 1Router 2 ~http://www.neng.usu.edu/classes/ece/6600/slides.html

8. Transmission Control Protocol How are Packets Lost? – Data Collision Two packets are sent on the same line – Buffer Overflow Packets are transmitted too fast for the receiving end – Noise Noise corrupts data causing packet to be disregarded

9. Transmission Control Protocol How to Avoid Packets Lost? – Data Collision No avoidance – Buffer Overflow Receivers buffer size can be increased or senders buffer size can be decreased – Noise No avoidance (assuming no access to physical layer)

10. Transmission Control Protocol Sending and Resending Packets – Window Size Allocation of how many packets can be sent during a time duration – Sending Protocol Stop and Wait Go Back N Selective Repeat

11. Transmission Control Protocol Window Size Sender Receiver 01230123 12341234 23452345 34563456 45674567 56785678 01230123 12341234 23452345 34563456 45674567 56785678 Senders Window Receivers Window ~http://www.neng.usu.edu/classes/ece/6600/slides.html

12. Transmission Control Protocol Stop and Wait Go Back N Selective Repeat Time Station 1 Station 2 ~http://www.neng.usu.edu/classes/ece/6600/slides.html

13. Transmission Control Protocol What Causes Delay? – Address Resolution Protocol (ARP) Cache Timeout No association with MAC address and IP address of receiver – Unused Bandwidth Free bandwidth on the line – Buffer Overflow Packets are transmitted too fast for the receiving end

14. Transmission Control Protocol How to Mitigate Delay? – Address Resolution Protocol (ARP) Cache Timeout ARP associates the receivers MAC address with the current IP address. If this address is not know a broadcast packet is sent out requesting the receivers information. The association only stays in ARP cache for 4 seconds. Each time a packet is sent the ARP cache is updated, thus transmission delay can me mitigated if packets are sent less than every 4 seconds. This elevates the need for sending ARP packets. To view ARP cache type arp –a in command window

15. Transmission Control Protocol How to Mitigate Delay? – Unused Bandwidth Find the Narrow Link and push bandwidth to that limit. – Example: Consider a narrow link of 10 Mbps and a Round Trip Time (RTT) of 50 ms. The in flight data should be (10 Mbps) * (0.025 ms) = 250 Kbits. ~http://fasterdata.es.net/TCP-tuning/TCP-Tuning-Tutorial.pdf

16. Transmission Control Protocol How to Mitigate Delay? – Buffer Overflow Maximize Buffer Size – Default TCP send/receive buffers are 64 KB, and default maximum size is 256 KB. This is 10 to 150 times too small – Good buffer size » Buffer size = Bandwidth*RTT – Example: Consider a narrow link of 500 Mbps (62 Mbytes/sec) and a Round Trip Time (RTT) of 50 ms. The buffer size should be (62 Mbytes/sec) * (0.05 seconds) = 3.1 Mbytes Caution: Do not set the buffer size too big or too many packets will be sent at once and data will be lost

17. Transmission Control Protocol Comparison ~http://fasterdata.es.net/TCP-tuning/TCP-Tuning-Tutorial.pdf

18. Transmission Control Protocol One more thought. Parallel Streaming ~http://fasterdata.es.net/TCP-tuning/TCP-Tuning-Tutorial.pdf

19. Measurement Resources Tracert – measures delay to each node – Command Window tracert google.com Iperf – measures Interval, Transfer, and Bandwidth – http://dast.nlanr.net/Projects/Iperf/ Pathrate – measures the capacity of the narrow link – http://www.pathrate.org Pathload – measures the available bandwidth – http://www.pathrate.org Pipechar – measures hop-by-hop available bandwidth – http://dsd.lbl.gov/NCS/ Tcpdump – TCP header information for source/destination address – ftp://ftp.ee.lbl.gov/ Tcptrace – plots output analysis – http://www.ncne.nlanr.net?TCP/testrig/

20. Summary UDP – Latency as low as RTT – Risk packet loss, duplication, or wrong order TCP – Latency increase from UDP due to processing time – Insures packet transfer is correct Latency Mitigation – Avoid Hubs and Routers in LAN – Reduce unnecessary Nodes – Keep MAC/IP Correlation in ARP Cache – Utilize the Narrow Link Bandwidth – Maximize Buffer Settings – Use Parallel Streams

21. Further Work If time permitted, I would like to complete the below tasks. – Implementation of the tuning process in a real control system and measure the actual speed increase in both a LAN and WAN system for the UDP and TCP protocols. – Research other protocols to see if their RTT is faster or characteristics are better then UDP/TCP. – Research topics on measurement of how much delay a system can handle before becoming unstable. – Create a quick reference tuning guide/software that will allow you to measure desired information and change parameters within a network.

22. References TCP Tuning Techniques for High-Speed wide-Area Networks – http://fasterdata.es.net/TCP-tuning/TCP-Tuning-Tutorial.pdf http://fasterdata.es.net/TCP-tuning/TCP-Tuning-Tutorial.pdf Computer Networking Course Website ECE6600 – http://www.neng.usu.edu/classes/ece/6600/slides.html http://www.neng.usu.edu/classes/ece/6600/slides.html Minimizing Average Network Delay for Ultra-Wideband Wireless Networks – http://www4.ncsu.edu/~mlsichit/Research/Publications/wcncUWB.pd f http://www4.ncsu.edu/~mlsichit/Research/Publications/wcncUWB.pd f Stability Analysis of Networked Control Systems with Data Dropout and Transmission Delays – http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4594178 &isnumber=4592780 http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4594178 &isnumber=4592780 Topics in High-Performance Messaging – http://www.29west.com/docs/THPM/packet-loss-myths.html http://www.29west.com/docs/THPM/packet-loss-myths.html

23. References Stability of Networked Control Systems in the Presence of Packet Loss – http://www.ecse.rpi.edu/~babak/cdc2003_camera_ready.pdf http://www.ecse.rpi.edu/~babak/cdc2003_camera_ready.pdf Packet Loss Myths – http://www.29west.com/docs/THPM/packet-loss-myths.html http://www.29west.com/docs/THPM/packet-loss-myths.html Network Latency – http://smutz.us/techtips/NetworkLatency.html http://smutz.us/techtips/NetworkLatency.html Using UDP socket connections for low-latency and loss-tolerant scenarios in AIR 2 (Part 1) – http://www.insideria.com/2009/11/udp-socket-connections-for-los-1.html http://www.insideria.com/2009/11/udp-socket-connections-for-los-1.html Comparative analysis – TCP – UDP – http://www.laynetworks.com/Comparative%20analysis_TCP%20Vs%20UDP.ht m http://www.laynetworks.com/Comparative%20analysis_TCP%20Vs%20UDP.ht m Switches vs. Hubs – http://www.networkclue.com/hardware/network/switches-vs-hubs.aspx http://www.networkclue.com/hardware/network/switches-vs-hubs.aspx

24. Questions?

25. Extra Material