1. Lecture 101 Lecture 10: Kernel Modules and Device Drivers ECE 412: Microcomputer Laboratory

2. Lecture 102 Objectives Review Linux environment Device classification Review Kernel modules PCMCIA example Skeleton example of implementing a device driver for a BlockRAM based device

3. Lecture 103 Review Questions What are some of the services/features that an IPIF- generated interface to the PLB/OPB bus can provide? Byte Steering for devices with narrow data widths Address range checking to detect transactions your device should handle User-defined registers Interface to the interrupt hardware Fixed-length burst transfers DMA engine Read/write FIFOs

4. Lecture 104 Linux Execution Environment Program Libraries Kernel subsystems

5. Lecture 105 Device Classification Most device drivers can be classified into one of three categories. Character devices. –Console and parallel ports are examples. –Implement a stream abstraction with operations such as open, close, read and write system calls. –File system nodes such as /dev/tty1 and /dev/lp1 are used to access character devices. –Differ from regular files in that you usually cannot step backward in a stream.

6. Lecture 106 Device Classification (cont) Block devices –A block device is something that can host a filesystem, e.g. disk, and can be accessed only as multiples of a block. –Linux allows users to treat block devices as character devices (/dev/hda1) with transfers of any number of bytes. –Block and character devices differ primarily in the way data is managed internally by the kernel at the kernel/driver interface. –The difference between block and char is transparent to the user. Network interfaces –In charge of sending and receiving data packets. –Network interfaces are not stream-oriented and therefore, are not easily mapped to a node in the filesystem, such as /dev/tty1. –Communication between the kernel and network driver is not through read/write, but rather through packet transfer functions.

7. Lecture 107 Linux Execution Environment (review) Execution paths

8. Lecture 108 Process and System Calls Process: program in execution. Unique pid. Hierarchy. User address space vs. kernel address space Application requests OS services through TRAP mechanism –x86: syscall number in eax register, exception (int $0x80) –result = read (file descriptor, user buffer, amount in bytes) –Read returns real amount of bytes transferred or error code (<0) Kernel has access to kernel address space (code, data, and device ports and memory), and to user address space, but only to the process that is currently running Current process descriptor. current pid points to current pid Two stacks per process: user stack and kernel stack Special instructions to copy parameters / results between user and kernel space

9. Lecture 109 Kernel Modules Kernel modules are inserted and unloaded dynamically –Kernel code extensibility at run time –insmod / rmmod / lsmod commands. Look at /proc/modules –Kernel and servers can detect and install them automatically, for example, cardmgr (pc card services manager) Example of the content of /proc/modules –nfs 170109 0 - Live 0x129b0000 –The first column contains the name of the module. –The second column refers to the memory size of the module, in bytes. –The third column lists how many instances of the module are currently loaded. A value of zero represents an unloaded module. –The fourth column states if the module depends upon another module to be present in order to function, and lists those other modules. –The fifth column lists what load state the module is in: Live, Loading, or Unloading are the only possible values. –The sixth column lists the current kernel memory offset for the loaded module. This information can be useful for debugging purposes, or for profiling tools such as oprofile.

10. Lecture 1010 Module Execution Modules execute in kernel space –Access to kernel resources (memory, I/O ports) and global variables ( look at /proc/ksyms) –Export their own visible variables, register_symtab (); –Can implement new kernel services (new system calls, policies) or low level drivers (new devices, mechanisms) –Use internal kernel basic interface and can interact with other modules –Need to implement init_module and cleanup_module entry points, and specific subsystem functions (open, read, write, close, ioctl …)

11. Lecture 1011 Hello World hello_world_module.c: #define MODULE #include static int __init init_module(void) { printk(" Hello, world\n"); /* is message priority. */ return 0; } static int __exit cleanup_module(void) { printk(" Goodbye cruel world\n"); } printk (basic kernel service) outputs messages to console and/or to /var/log/messages To compile and run this code: –root# gcc -c hello_world_module.c –root# insmod hello_world_module.o –root# rmmod hello_world_module

12. Lecture 1012 Linking a module to the kernel (from Rubinis book)

13. Lecture 1013 Register Capability You can register a new device driver with the kernel: –int register_chrdev(unsigned int major, const char *name, struct file_operations *fops); –A negative return value indicates an error, 0 or positive indicates success. –major: the major number being requested (a number < 128 or 256). –name: the name of the device (which appears in /proc/devices). –fops: a pointer to a global jump table used to invoke driver functions. Then give to the programs a name by which they can request the driver through a device node in /dev –To create a char device node with major 254 and minor 0, use: mknod /dev/memory_common c 254 0 –Minor numbers should be in the range of 0 to 255. (Generally, the major number identifies the device driver and the minor number identifies a particular device (possibly out of many) that the driver controls.)

14. Lecture 1014 PCMCIA Read/Write Common/Attribute Memory application data = mem_read (address, type) mem_write (address, data, type) data = mem_read (address, type) mem_write (address, data, type) /dev/… PCMCIA registered memory fops /dev/… PCMCIA registered memory fops memory_read(), memory_write() - map kernel memory to I/O window - copy from PCMCIA to user ( &buf) - copy from user to PCMCIA (&data) - map kernel memory to I/O window - copy from PCMCIA to user ( &buf) - copy from user to PCMCIA (&data) USER SPACE KERNEL SPACE Libc: file I/O PCMCIA attribute common - open(/dev/memory_[common|attribute]) - lseek(fd, address) - read(fd, buf,1); return buf; - write(fd, data, 1) - open(/dev/memory_[common|attribute]) - lseek(fd, address) - read(fd, buf,1); return buf; - write(fd, data, 1) int buf Card insertion card_memory_config: - read CIS - config I/O window - config IRQ - register R/W fops card_memory_config: - read CIS - config I/O window - config IRQ - register R/W fops Kernel memory

15. Lecture 1015 PCMCIA Button Read Interrupt handling application data = mem_read (address, type) mem_write (address, data, type) data = mem_read (address, type) mem_write (address, data, type) /dev/… PCMCIA registered memory fops /dev/… PCMCIA registered memory fops memory_button_read() - interruptible_sleep_on (PC->queue) - memory_read() - map kernel memory to I/O window - copy PC to user ( &buf) - interruptible_sleep_on (PC->queue) - memory_read() - map kernel memory to I/O window - copy PC to user ( &buf) USER SPACE KERNEL SPACE Libc: file I/O PCMCIA attribute common - open(/dev/memory_common) - lseek(fd, address) - read(fd, buf,1); return buf; - write(fd, data, 1) - open(/dev/memory_common) - lseek(fd, address) - read(fd, buf,1); return buf; - write(fd, data, 1) int buf Card insertion card_memory_config: … - config IRQ handler card_memory_config: … - config IRQ handler Kernel memory Button int. int_handler: - wake_up( PC->queue) int_handler: - wake_up( PC->queue)

16. Lecture 1016 Skeleton Example: OCM-Based BlockRAM PowerPC has an OCM (on-chip memory) bus that lets you attach fast memory to the cache Xilinx provides a core (dso_if_ocm) that handles the interface to the OCM and outputs BRAM control signals –Found under Project->Add/Edit cores –Creates an interface that detects accesses to a specified physical address range and outputs control signals for a BlockRAM

17. Lecture 1017 Software-Side Issues Xilinx core handles the BlockRAM interface from the hardware side, but need to make BlockRAM visible/accessible to software Two issues: –Programs operate on virtual addresses, even when running as root –Ideally, want to be able to make BlockRAM visible to user- mode programs User-mode programs cant set virtual->physical address mappings

18. Lecture 1018 Direct Approach -- Use mmap() Only works for code running as root fd = open(/dev/mem, O_RDWR); bram = mmap(0x40000000, 2048, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0x40000000); assert(bram == 0x40000000); Creates pointer to the /dev entry that describes the physical memory Maps 2048 bytes from /dev/mem onto the programs address space, starting at offset 0x40000000 from the start of the pointer Requests that those bytes be mapped onto addresses starting at 0x40000000 Checks (via assert) that mmap() returned the requested address, as mmap() isnt required to follow that request

19. Lecture 1019 Better Approach -- Device Driver Create device driver module and install into Linux Device driver module will map BRAM onto address space of currently-running program

20. Lecture 1020 Device Driver Device drivers provide mechanisms, not policy. –Mechanism: Defines what capabilities are provided? –Policy: Defines how those capabilities can be used? This strategy allows flexibility. The driver controls the hardware and provides an abstract interface to its capabilities. The driver ideally imposes no restrictions (or policy) on how the hardware should be used by applications. For example, X manages the graphics hardware and provides an interface to user programs. Window managers implement a particular policy and know nothing about the hardware. Kernel apps build policies on top of the driver, e.g. floppy disk, such as who has access, the type of access (direct or as a filesystem), etc. -- it makes the disk look like an array of blocks. Courtesy of UMBC

21. Lecture 1021 Device Driver Outline 1.Obtain memory map semaphore for currently running program (to prevent overlapping changes) 2.Insert new virtual memory area (VMA) for BRAM 3.Call get_unmapped_area with physical address range of BRAM 4.Allocate and initialize VMA for the BRAM 5.Call remap_page_range() to build page tables 6.Use insert_vma_struct() and make_pages_present() to enable access to new pages See Running Linux on a Xilinx XUP Board for more information (on the web, written by John Kelm).

22. Lecture 1022 Next Time Quiz 1