1. Geoff Salmon, Monia Ghobadi, Yashar Ganjali, Martin Labrecque, J. Gregory Steffan University of Toronto

2. What is the Precise Traffic Generator (PTG)? Uses NetThreads on the NetFPGA to generate packets. Goals 1. Send packets precisely at the requested time. 2. Allow packets to be constantly streamed from host PC. No separate “load phase”.

3. Why Are These Goals Interesting? 1. Precise transmission times: Can emulate diverse network conditions and topologies. 2. Stream packets from host PC: Allows generated traffic to be reactive ie. TCP flows with congestion control

4. Motivation: Network Testbed Experiments Testing systems which are sensitive to packet arrival times (ie. packet buffering and scheduling in routers) requires realistic test traffic. Need to explore a wide range of traffic With a small testbed, this is difficult or impossible. With a large enough testbed, this is difficult and expensive.

5. Simplified Problem A few computers can create required traffic volume. However, resulting traffic may not be realistic. How is the traffic different? The PTG focuses on controlling the packet transmission times.

6. Components of the PTG 3. pktgen Linux Kernel NetFPGA Packet Travel Direction 2. nf2 Linux Driver NetLink GigE 1. NetThreads Application PCI Host Computer

7. Userspace Program: pktgen Simple command-line utility used for testing PTG. Sends descriptions of packets and transmission times to the driver. Uses inter-transmission times and payload sizes that are either: fixed, or read from files. Communicates with driver using a NetLink socket. Easily replaced by other userspace or kernel code.

8. nf2 Linux Driver Modified version of nf2 driver. Main jobs: Receive packet descriptions from NetLink socket. Build command packets containing multiple descriptions. Send command packets to the NetThreads app over the PCI bus.

9. NetThreads Application Eight threads of execution in NetThreads: 7 threads receive command packets and prepare packets to transmit in output memory. 1 thread sends packets from the output memory at correct times. Input MemoryOutput Memory

10. PCI Bus is a Bottleneck Cannot copy all packets across PCI bus. Main idea: do not copy packet payloads Payloads are zeroed when sent from NetFPGA. Experiment often only look at packet headers anyway. PCI transaction overheads require driver to gather multiple packet descriptions into single command packet.

11. Transmission Time Precision Compare against Stanford’s Packet Generator (SPG) PTG/SPG Sending fixed rate packets NetFPGA router measures inter- arrival times Remember it takes 8ns to send 1 byte on GigE.

12. Throughput: MTU-Size Packets PreambleEthernet Hdr DataCRC 22 bytes46-1500 bytes4 bytes MTU-sized Packet = 1526 bytes Transmits in 1526 * 8 = 12208 ns Add Inter Frame Gap of 96ns. 12304ns is back to back!

13. Throughput: Smaller Packets

14. Throughput: Smaller Packets cont. Errors appear between 6000-7000ns. See the same problem for different packet sizes. The mean inter-transmission times are correct, not a PCI bottleneck problem. Most likely: the sending thread is doing too much work between packet transmissions. In 6400ns each thread has 6400 ns / 8 ns per cycle / 4 threads = 200 clock cycles

15. Future Work Implement receiving packets Currently PTG cannot receive packets and send them to the host PC, so generating reactive traffic requires another NIC. Augment existing software network emulator (ie. NISTNet) with PTG to transmit packets at the correct times.

16. Conclusion The PTG precisely controls the transmission times of packets streamed from the host PC. It has the potential to enhance the realism of software network emulators, and allow researchers to perform experiments which are currently difficult or impossible.

17. Modified Hardware Designs NetThreads design: Removed Output Queues increase accuracy of transmission times. NetFPGA reference router design used in measurements: Removed Input Arbiter and Output Port Lookup.

18. Inaccuracy of NISTNet Software network emulators schedule packet transmissions with fixed-rate timers/interrupts. Ran experiment to measure inaccuracy using NISTNet. PTG NISTNet host Delays packets by 100ms NISTNet host Delays packets by 100ms NetFPGA router measuring arrival times Measure inter-arrivals here

19. Packets Sent Every 70 μs

20. Packets Sent Every 640 μs

21. Packets Sent Every 700 μs