1. kernel module programming Nezer J. Zaidenberg

2. reference This guide is built on top of The Linux Kernel Module Programming Guide The guide is available from http://tldp.org/LDP/lkmpg/2.6/html/index.html http://tldp.org/LDP/lkmpg/2.6/html/index.html I have also put references to other sources

3. First class in kernel We will cover kernel programming I assume you now master the user space portion of this course.

4. Agenda What is kernel module How to code simple kernel modules char devices adding system calls ioctls netlink sockets

5. Next week Using procfs and sysfs Blocking Using timers to implement periodical tasks Interrupt handling

6. Kernel module Pluggable module to the operating system that adds functionality Device driver New features Since we don’t have hardware we focus on features

7. Kernel modules Can be added on the fly Allow us to change functionality on the fly Allow us to read and write

8. SIMPLEST MODULE #include /* Needed by all modules */ #include /* Needed for KERN_INFO */ int init_module(void) { printk(KERN_INFO "Hello world 1.\n"); return 0; } void cleanup_module(void) { printk(KERN_INFO "Goodbye world 1.\n"); }

9. EXPLAINING module init is called when module is loaded module cleanup is called when module is unloaded

10. printk The server can’t use stdlib due to userspace/kernel space issues Most of C library is implemented in the kernel printk is printf for kernel programs.

11. Makefile obj-m += hello-1.o all: make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules clean: make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

12. Inserting and removing module To insert insmod modprobe To remove rmmod modprobe -r

13. hello-2.c static int __init hello_2_init(void) { printk(KERN_INFO "Hello, world 2\n"); return 0; } static void __exit hello_2_exit(void) { printk(KERN_INFO "Goodbye, world 2\n"); } module_init(hello_2_init); module_exit(hello_2_exit);

14. What does it do __init macro and __exit macro tells the kernel when we use this function This allows to free kernel memory that is used only at init

15. hello-3.c static int hello3_data __initdata = 3; static int __init hello_3_init(void) { printk(KERN_INFO "Hello, world %d\n", hello3_data); return 0; } static void __exit hello_3_exit(void) { printk(KERN_INFO "Goodbye, world 3\n"); } //init … exit…

16. Hello-4.c #define DRIVER_AUTHOR "Peter Jay Salzman "p@dirac.org #define DRIVER_DESC "A sample driver" static int __init init_hello_4(void) { printk(KERN_INFO "Hello, world 4\n"); return 0; } static void __exit cleanup_hello_4(void) { printk(KERN_INFO "Goodbye, world 4\n"); }

17. Hello-4.c (cont) module_init(init_hello_4); module_exit(cleanup_hello_4); MODULE_LICENSE("GPL"); MODULE_SUPPORTED_DEVICE("testdevice"); MODULE_AUTHOR(DRIVER_AUTHOR);/* Who wrote this module? */ MODULE_DESCRIPTION(DRIVER_DESC);/* What does this module do */

18. Additions in hello-4.c We added licensing and information that removes the “kernel is tainted message” We also added some more info

19. Hello-5.c (simplified) static int myint = 420; module_param(myint, int, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH); MODULE_PARM_DESC(myint, "An integer"); static int __init hello_5_init(void) { int i; printk(KERN_INFO "Hello, world 5\n=============\n"); printk(KERN_INFO "myint is an integer: %d\n", myint); return 0; }

20. passing args We demonstrate the effect of passing args The tutorial demonstrate how more args (from different types can be given)

21. Getting info modinfo gives info on running modules we will get info for our modules 4 and 5.

22. Multiple files We build a module using two files start.c stop.c

23. start.c #include /* We're doing kernel work */ #include /* Specifically, a module */ int init_module(void) { printk(KERN_INFO "Hello, world - this is the kernel speaking\n"); return 0; }

24. stop.c #include /* We're doing kernel work */ #include /* Specifically, a module */ void cleanup_module() { printk(KERN_INFO "Short is the life of a kernel module\n"); }

25. Makefile for multiple files obj-m += startstop.o startstop-objs := start.o stop.o all: make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules clean: make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

26. What do we do with KM Everything the OS can do Create drivers Virtual devices New system calls New OS abilities (support new file systems? implement something differently/more efficiently? etc)

27. Devices All devices happen to exist under the /dev file system and represented as a file that we can work with. This include I/O devices, virtual devices etc.

28. I/O Devices I/O devices in Linux are specified as block and char device Block devices support lseek(2) : Example - disk Char devices don’t support lseek(2) : Example - keyboard.

29. Struct file operations Char device/Block device is an ABC We inherit and implement the functions we need using struct file operations.

30. Example struct file_operations fops = {.read = device_read,.write = device_write,.open = device_open,.release = device_release };

31. Registering int register_chrdev(unsigned int major, const char *name, struct file_operations *fops); regiser = puts the driver handler in the factory (same as ffmpeg codecs) Major = which driver handles this device. Minor = if we have several devices of the same type this distinguish them (like two HD on a box)

32. Unregistering Is done using unregister_chardev function If we try to rmmod and somebody is using the module - it will fail. Otherwise it will be successful. Usage count for our module is managed by the kernel. To help the kernel we have try_module_get and module_put functions.

33. Examples - blank char device #include #include /* for put_user */ int init_module(void); void cleanup_module(void); static int device_open(struct inode *, struct file *); static int device_release(struct inode *, struct file *); static ssize_t device_read(struct file *, char *, size_t, loff_t *); static ssize_t device_write(struct file *, const char *, size_t, loff_t *);

34. Example 2 #define SUCCESS 0 #define DEVICE_NAME "chardev"/* Dev name as it appears in /proc/devices */ #define BUF_LEN 80/* Max length of the message from the device */ static int Major;/* Major number assigned to our device driver */ static int Device_Open = 0; static char msg[BUF_LEN];/* The msg the device will give when asked */ static char *msg_Ptr; static struct file_operations fops = {.read = device_read,.write = device_write,.open = device_open,.release = device_release };

35. Example 3 int init_module(void) { Major = register_chrdev(0, DEVICE_NAME, &fops); if (Major < 0) { printk(KERN_ALERT "Registering char device failed with %d\n", Major); return Major; } printk(KERN_INFO "the driver, create a dev file with\n"); printk(KERN_INFO "'mknod /dev/%s c %d 0'.\n", DEVICE_NAME, Major); printk(KERN_INFO "Remove the device file and module when done.\n"); return SUCCESS; }

36. Example 4 void cleanup_module(void) { /* * Unregister the device */ int ret = unregister_chrdev(Major, DEVICE_NAME); if (ret < 0) printk(KERN_ALERT "Error in unregister_chrdev: %d\n", ret); }

37. Example 5 static int device_open(struct inode *inode, struct file *file) { static int counter = 0; if (Device_Open) return -EBUSY; Device_Open++; sprintf(msg, "I already told you %d times Hello world!\n", counter++); msg_Ptr = msg; try_module_get(THIS_MODULE); return SUCCESS; }

38. Example 6 static int device_release(struct inode *inode, struct file *file) { Device_Open--;/* We're now ready for our next caller */ /* * Decrement the usage count, or else once you opened the file, you'll * never get get rid of the module. */ module_put(THIS_MODULE); return 0; }

39. Example 7 static ssize_t device_read(struct file *filp,/* see include/linux/fs.h */ char *buffer,/* buffer to fill with data */ size_t length,/* length of the buffer */ loff_t * offset) { int bytes_read = 0; if (*msg_Ptr == 0) return 0; while (length && *msg_Ptr) { put_user(*(msg_Ptr++), buffer++); length--; bytes_read++; } return bytes_read; }

40. Explaining Normally kernel memory is kept after the MMU When we go through the MMU to put stuff in a user data we use put_user

41. Example 8 static ssize_t device_write(struct file *filp, const char *buff, size_t len, loff_t * off) { printk(KERN_ALERT "Sorry, this operation isn't supported.\n"); return -EINVAL; }

42. Communicating with user space Kernel modules communicate with user space programs in several methods new system calls dev file interface and functions (and ioctls) kernel netlink socket /proc fs

43. New system calls we can add new system calls to the linux kernel system calls have their os function table and we can register new ones we must supply header files off course (and man page if we are nice)

44. Chapter 8 Replacing System calls This chapter is very dangerous!

45. switching open - chapter 8 This is an example to replacing open(2) in the system call table. It’s dangerous(!) We can add new system calls if we want in similar way We don’t do it in our homework

46. taken from chapter 8 int init_module() { original_call = sys_call_table[__NR_open]; sys_call_table[__NR_open] = our_sys_open; return 0; }

47. Explaining We save original open function pointer from the system call table (NR_Open = open index) We put our own function We can do similar ways to add function (just find unused number)

48. Again taken from chapter 8 void cleanup_module() { if (sys_call_table[__NR_open] != our_sys_open) { printk(KERN_ALERT "Somebody else also played with the "); printk(KERN_ALERT "open system call\n"); printk(KERN_ALERT "The system may be left in "); printk(KERN_ALERT "an unstable state.\n"); } sys_call_table[__NR_open] = original_call; }

49. Completely new system calls Tutorials http://www.linuxjournal.com/article/3326 http://tldp.org/HOWTO/html_single/Implem ent-Sys-Call-Linux-2.6-i386/http://tldp.org/HOWTO/html_single/Implem ent-Sys-Call-Linux-2.6-i386/ IMHO - Linux journal tutorial is better written but its old. the 2nd tutorial is more accurate... Read BOTH

50. I will not cover new system calls but.. Take note - that you register the system call in the table and add name You register the name and handler You must recompile the entire kernel Using ioctl(2) is usually better if you can

51. Using IOCTL(2)

52. ioctl(2) IOCTL is like “a joker” system call. it gets two or more parameters. The file descriptor to work on (like device file) The operation number (like function number) Argument lists for the operation

53. IOCTL(2) - usages Add kernel operations - without adding system calls Add kernel operations - without recompiling Add kernel operations - that can be removed with the module - on the fly.

54. Parital example from chapter 7 struct file_operations Fops = {.read = device_read,.write = device_write,.ioctl = device_ioctl,.open = device_open,.release = device_release,/* a.k.a. close */ };

55. ioctl(2) implementation int device_ioctl(struct inode *inode,/* see include/linux/fs.h */ struct file *file,/* ditto */ unsigned int ioctl_num,/* number and param for ioctl */ unsigned long ioctl_param) { int i; char *temp; char ch; switch (ioctl_num) { case IOCTL_SET_MSG: temp = (char *)ioctl_param; get_user(ch, temp); for (i = 0; ch && i < BUF_LEN; i++, temp++) get_user(ch, temp); device_write(file, (char *)ioctl_param, i, 0); break; case IOCTL_GET_MSG: //... }

56. CalLing IOCTL from userspace #include "chardev.h" #include #include /* open */ #include /* exit */ #include /* ioctl */ ioctl_set_msg(int file_desc, char *message) { int ret_val; ret_val = ioctl(file_desc, IOCTL_SET_MSG, message); if (ret_val < 0) { printf("ioctl_set_msg failed:%d\n", ret_val); exit(-1); }

58. Camera driver For ex. 3!

59. What is camera driver Camera driver is just another block device or char device that produces frames. Linux has a spec that all camera drivers must follow called V4L2 (Video 4(for) Linux 2) This spec is nothing but how to use read(2),write(2), ioctl(2), mmap.

60. About V4L spec Not very interesting. Available from bytesex and thedirks Just API documentation

61. About EX3 You have to code frame grabber, a kernel quasi-webcam driver(getting the frames from flv file) and a user process that communicates with the kernel driver. We will discuss next class how to communicate between user and kernel

62. Kernel netlink sockets

63. Tutorial http://www.linuxjournal.com/article/7356

64. what is netlink socket new domain of sockets (like AF_INET, AF_UNIX) one end in the kernel space one end in userland DGRAM or RAW (raw messages - like IP) only. no STREAM

65. USER SPACE SIDE Protocol for sockets is used (protocol is used to define communication type) We may want to add new communication type In user mode we probably will under stand API easily

66. USER EXAMPLE 1 From linux journal #include #define MAX_PAYLOAD 1024 /* maximum payload size*/ struct sockaddr_nl src_addr, dest_addr; struct nlmsghdr *nlh = NULL; struct iovec iov; int sock_fd; void main() { sock_fd = socket(PF_NETLINK, SOCK_RAW,NETLINK_TEST);

67. USER EXAMPLE 2 From linux journal memset(&src_addr, 0, sizeof(src_addr)); src__addr.nl_family = AF_NETLINK; src_addr.nl_pid = getpid(); /* self pid */ src_addr.nl_groups = 0; /* not in mcast groups */ bind(sock_fd, (struct sockaddr*)&src_addr, sizeof(src_addr)); memset(&dest_addr, 0, sizeof(dest_addr)); dest_addr.nl_family = AF_NETLINK; dest_addr.nl_pid = 0; /* For Linux Kernel */ dest_addr.nl_groups = 0; /* unicast */

68. USER EXAMPLE 3 From linux journal nlh=(struct nlmsghdr *)malloc( NLMSG_SPACE(MAX_PAYLOAD)); /* Fill the netlink message header */ nlh->nlmsg_len = NLMSG_SPACE(MAX_PAYLOAD); nlh->nlmsg_pid = getpid(); /* self pid */ nlh->nlmsg_flags = 0; /* Fill in the netlink message payload */ strcpy(NLMSG_DATA(nlh), "Hello you!"); iov.iov_base = (void *)nlh; iov.iov_len = nlh->nlmsg_len; msg.msg_name = (void *)&dest_addr; msg.msg_namelen = sizeof(dest_addr); msg.msg_iov = &iov; msg.msg_iovlen = 1; sendmsg(fd, &msg, 0);

69. USER EXAMPLE 4 From linux journal /* Read message from kernel */ memset(nlh, 0, NLMSG_SPACE(MAX_PAYLOAD)); recvmsg(fd, &msg, 0); printf(" Received message payload: %s\n", NLMSG_DATA(nlh)); /* Close Netlink Socket */ close(sock_fd); }

70. now the kernel side... We have to register ourselves. (to support socket from the right type) Everything else is straight forward

71. Kernel netlink example 1 (LJ) struct sock *nl_sk = NULL; void nl_data_ready (struct sock *sk, int len) { wake_up_interruptible(sk->sleep); } void netlink_test() { struct sk_buff *skb = NULL; struct nlmsghdr *nlh = NULL; int err; u32 pid;

72. Kernel netlink example 2 (LJ) nl_sk = netlink_kernel_create(NETLINK_TEST, nl_data_ready); /* wait for message coming down from user-space */ skb = skb_recv_datagram(nl_sk, 0, 0, &err); nlh = (struct nlmsghdr *)skb->data; printk("%s: received netlink message payload:%s\n", __FUNCTION__, NLMSG_DATA(nlh)); pid = nlh->nlmsg_pid; /*pid of sending process */ NETLINK_CB(skb).groups = 0; /* not in mcast group */ NETLINK_CB(skb).pid = 0; /* from kernel */ NETLINK_CB(skb).dst_pid = pid; NETLINK_CB(skb).dst_groups = 0; /* unicast */ netlink_unicast(nl_sk, skb, pid, MSG_DONTWAIT); sock_release(nl_sk->socket); }

73. Homework 3 Tape library

74. what is vtl In many case we work with disks to implement tape libraries faster and cheaper then robuts but backup application still want tape API so we create VIRTUAL TAPE LIBRARY a char device - that writes to real disk that acts as tape library

75. In the homework You have been requested to write a user space program that manages a tape library (bunch of files each representing a tape) A kernel application that acts as the tape The two programmes communicate via netlink socket The kernel application gets commands via ioctl(2)

76. Goal - VTL We implement a virtual tape library We create a char device - the virtual tape We communicate with the char device via ioctls, read, write

77. how to communicate read, write = read from tape, write to tape ioctl = rewind, start new tape, destroy tape, replace tape

78. what to implement 3 programs Userland programme that handles the virtual library - compress and decompress tapes and files (talk to kernel via netlink socket) Kernel module that communicate with Userland programmes Test utility - talk to kernel with read/write and ioctls

79. Details I will implement over the weekend and put exact implementation details for automatic checking I will also publish my test programmes

80. VTL companies IBM VTS on mainframe Falconstor Sepaton (no-tapes) - now sungate Diligent (now IBM) It’s a major industry in storage world!

81. Compression Libbz2

82. Is easy http://www.bzip.org/http://www.bzip.org bzip has a buffer compression function and assortment of file functions