1. Our ‘nic.c’ module We create a ‘character-mode’ device-driver for the 82573L NIC to use in future experiments

2. No way to communicate Linux operating system kernel (no knowledge of the NIC) Network interface controller (no knowledge of the OS) HARDWARE SOFTWARE These components lack a way to interact

3. Role for a ‘device driver’ Linux operating system kernel (no knowledge of the NIC) Network interface controller (no knowledge of the OS) HARDWARE SOFTWARE device driver module (knows about both the OS and the NIC)

4. Character devices The concept of a ‘character mode’ device is that it provides a ‘stream’ of data bytes that an application-programs can access by using standard Linux system-calls: –‘open()’ –‘write()’ –‘read()’ –‘close()’ –Maybe other ‘methods’ (depending on the device)

5. Requirement The character-mode devices are known to a Linux system by their ‘names’ and by an associated pair of ID-numbers (major and minor) normally set up by a ‘superuser’ root# mknod /dev/nic c 97 0 root# chmod a+rw /dev/nic

6. Driver module module_init module_exit fops read() write() isr() my_proc_rx() nic.c required pair of module administration functions character-mode device-driver’s required ‘file-operations’ data-structure this module’s collection of ‘payload’ functions my_proc_tx()

7. Overview hardware char driver module Linux operating system Kernel networking subsystem application program standard runtime libraries user space kernel space file subsystem We bypass use of the TCP/IP protocol stack for our demo

8. Transmit operation application program user data-buffer runtime library write() Linux OS kernel nic device-driver my_write() file subsystem hardware packet buffer copy_from_user() DMA user space kernel space

9. Receive operation application program user data-buffer runtime library read() Linux OS kernel nic device-driver my_read() file subsystem hardware packet buffer copy_to_user() user space kernel space DMA

10. We ‘tweak’ our packet-format Our ‘nictx.c’ and ‘nicrx.c’ modules elected to use the Ethernet-header’s Type/Length field for storing the number of data-bytes But this allows our driver to be misled by packet’s which don’t follow that scheme So we added a ‘signature’ and relocated our ‘count’ field to the ‘payload’ region actual bytes of user-data 0 6 12 14 destination MAC-addresssource MAC-address Type/Len count

11. Using ‘echo’ and ‘cat’ Our device-driver module ‘nic.c’ is intended to allow two programs that are running on a pair of our classroom PCs to communicate via their ‘secondary’ Ethernet interfaces $ echo Hello > /dev/nic $ _ $ cat /dev/nic Hello _ Receiving…Transmitting…

12. Exploring RX Descriptor FIFO Timeout for an in-class demonstration

13. ‘push’ versus ‘pull’ Host memory transmit packet buffer transmit-FIFO push Ethernet controller receive-FIFO receive packet buffer pull to/from LAN The ‘write()’ routine in our ‘nic.c’ driver can transfer some data anytime there is at least one Tx-descriptor that is not “owned” by the network hardware, but the ‘read()’ routine in our ‘nic.c’ driver will have to wait for some data to arrive. So to avoid doing any wasteful busy-waiting, our ‘nic.c’ driver can utilize the Linux kernel’s sleep/wakeup mechanism – if it enables the NIC’s interrupts!

14. Sleep/wakeup We will need to employ a wait-queue, we will need to enable device-interrupts, and we will need to write and install the code for an interrupt service routine (ISR) So our ‘nic.c’ device-driver will have a few additional code and data components that were absent from our ‘nictx.c’ and ‘nicrx.c’ kernel-module demos

15. How NIC’s interrupts work There are four interrupt-related registers which are essential for us to understand ICR 0x00C0 0x00C8 0x00D0 0x00D8 ICS IMS IMC Interrupt Cause Read Interrupt Cause Set Interrupt Mask Set/Read Interrupt Mask Clear

16. Interrupt-event types reserved 82573L 31: INT_ASSERTED (1=yes,0=no) 31 30 18 17 16 15 14 10 9 8 7 6 5 4 2 1 0 17: ACK (Rx-ACK Frame detected) 16: SRPD (Small Rx-Packet detected) 15: TXD_LOW (Tx-Descr Low Thresh hit) 9: MDAC (MDI/O Access Completed) 7: RXT0 ( Receiver Timer expired) 6: RXO (Receiver Overrun) 4: RXDMT0 (Rx-Desc Min Thresh hit) 2: LSC (Link Status Change) 1: TXQE( Transmit Queue Empty) 0: TXDW (Transmit Descriptor Written Back)

17. Interrupt Mask Set/Read This register is used to enable a selection of the device’s interrupts which the driver will be prepared to recognize and handle A particular interrupt becomes ‘enabled’ if software writes a ‘1’ to the corresponding bit of this Interrupt Mask Set register Writing ‘0’ to any register-bit has no effect, so interrupts can be enabled one-at-a-time

18. Interrupt Mask Clear Your driver can discover which interrupts have been enabled by reading IMS – but your driver cannot ‘disable’ any interrupts by writing to that register Instead a specific interrupt can be disabled by writing a ‘1’ to the corresponding bit in the Interrupt Mask Clear register Writing ‘0’ to a register-bit has no effect on the interrupt controller’s Interrupt Mask

19. Interrupt Cause Read When interrupts occur, your driver’s ‘interrupt service routine’ (isr) discovers which specific conditions triggered the interruption if it reads the NIC’s Interrupt Cause Read register In this case your driver can clear any selection of these bits (except bit #31) by writing ‘1’s to them (writing ‘0’s to this register will have no effect) If case no interrupt has occurred, reading this register may have the side-effect of clearing it

20. Interrupt Cause Set For testing your driver’s interrupt-handler, you can artificially trigger any particular combination of interrupts by writing ‘1’s into the corresponding register-bits of this Interrupt Cause Set register (assuming your combination of bits corresponds to interrupts that are ‘enabled’ by ‘1’s being present for them in the Interrupt Mask)

21. Our interrupt-handling We decided to enable only a subset of the possible interrupt causes (those related to transmits, receives, and link-changes): The ones we take action on are TXDW, RXT0, and RXDMT0 #define INTR_MASK(1<<0)|(1<<1)|(1<<15)|(1<<2)|(1<<4)|(1<<6)|(1<<7) transmit-relatedreceive-related link-status changed

22. NIC “owns” some Rx-descriptors descriptor 0 0123456701234567 descriptor 1 descriptor 2 descriptor 3 descriptor 4 descriptor 5 descriptor 6 descriptor 7 RDT RDH RDLEN =0x80 RDBAH/RDBAL This register gets advanced by 4 when the NIC’s supply of descriptors drops below a programmed threshold This register gets initialized to 0, then gets changed by the controller as new packets are received rxhead Our ‘static’ variables pickup Keeps track of where packet-data resumes

23. In-class exercise Login to a pair of classroom machines Install our ‘nic.ko’ module on both hosts Try transferring a textfile from one of the machines to the other, by using ‘cat’: hrn23501$ cat textfile > /dev/nic hrn23502$ cat /dev/nic > recv1000.out How large a textfile can you successfully transfer using our Linux device-driver? Any problems encountered doing this?