1. Linux Modules and Device Drivers
Computer Science & Engineering Department Arizona State University Tempe, AZ 85287
Dr. Yann-Hang Lee (480)

2. What is “kernel” The basic component of an operating system
to provide the lowest-level abstraction layer for the resources (especially processors and I/O devices).
available to application processes through inter-process communication mechanisms and system calls
Kernel space and user space
What are system calls
Which are systems calls –
cc, make, ls, cat, grep, read, open, printf, malloc, etc.
How can we set the baud rate of a serial port?
Configuration of a hyperterminal
stty -F /dev/ttyS2 ospeed 38400
Ioctl to get and set terminal attributes (defined in struct termios)
The mechanism of making system calls and passing parameters

3. Kernel Components Process management Memory management File system
Process life cycle (create and terminate), inter-process communication, scheduling (long-term, short-term)
Memory management
Virtual memory and physical management, allocation, protection
File system
Virtual file system, tree structure, buffer cache, block devices
Types -- regular file, directory, symbolic link, block-oriented device file, character-oriented device file, pipe and named pipe and socket
Character device control
Almost every system operation maps to a physical device
The code used to do those operations is called Device Driver
Networking
Collect incoming packets and de-multiplexing them
Deliver outgoing packets

4. Invocation of Kernel Functions
A process invokes a system call.
The CPU executing the process signals an exception.
The kernel handles the exception on behalf of the process that caused it.
A peripheral device issues an interrupt signal to the CPU to notify it of an event
each interrupt signal is dealt by a kernel program called an interrupt handler.
interrupts occur at unpredictable times.
A kernel thread is executed.
Because it runs in Kernel Mode, the corresponding programmust be considered part of the kernel.

5. Protocols in the Linux Kernel)
(The Linux® Networking Architecture: Design and Implementation of Network
Protocols in the Linux Kernel)

6. Linux SysCall
System call = exception (to force a switch from user mode to supervisor mode)
“int $0x80” or “sysenter” to make a call, and “iret” or “sysexit” to return to user mode
in Linux/arch/x86/include/asm/irq_vectors.h –
# define SYSCALL_VECTOR x80
in Linux/arch/x86/kernel/traps.c
set_system_trap_gate(SYSCALL_VECTOR, &system_call);

7. Example Linux Syscall Linux/arch/x86/syscalls/syscall_32.tbl # Name
Registers
Definition
eax
ebx
ecx
edx
esi
edi
sys_restart_syscall
0x00 -
kernel/signal.c:2058 1
sys_exit
0x01
int error_code
kernel/exit.c:1046 2
sys_fork
0x02
struct pt_regs *
arch/alpha/kernel/entry.S:716 3
sys_read
0x03
unsigned int fd
char __user *buf
size_t count
fs/read_write.c:391 4
sys_write
0x04
const char __user *buf
fs/read_write.c:408 5
sys_open
0x05
const char __user *filename
int flags
int mode
fs/open.c:900 6
sys_close
0x06
fs/open.c:969 7
sys_waitpid
0x07
pid_t pid
int __user *stat_addr
int options
kernel/exit.c:1771 8
sys_creat
0x08
const char __user *pathname
fs/open.c:933 9
sys_link
0x09
const char __user *oldname
const char __user *newname
fs/namei.c:2520 10
sys_unlink
0x0a
fs/namei.c:2352
Linux/arch/x86/syscalls/syscall_32.tbl

8. Generating Syscall Table
Script (Linux/arch/x86/syscalls/syscalltbl.sh) reads in table definition (Linux/arch/x86/syscalls/syscall_32.tbl)
Produce Linux/arch/x86/include/generated/asm/syscalls_32.h
in …/x86/kernel/syscall_32.c,
syscall table is initiated
Sys calls are defined in various
source programs
In Linux/fs/read_write.c
SYSCALL_DEFINE3(read, unsigned int, fd, char __user *, buf, size_t, count)
__SYSCALL_I386(0, sys_restart_syscall, )
__SYSCALL_I386(1, sys_exit, )
#ifdef CONFIG_X86_32
__SYSCALL_I386(2, sys_fork, )
#else
#endif
__SYSCALL_I386(3, sys_read, )
__SYSCALL_I386(4, sys_write, )
……..

9. Kernel Operation of Linux Syscall
Syscall entry
Architecture dependent
Linux/arch/x86/kernel/entry_32.S
Syscall table : Linux/arch/x86/kernel/syscall_32.c
ENTRY(system_call)
………
cmpl $(NR_syscalls), %eax
jae syscall_badsys
syscall_call:
call *sys_call_table(,%eax,4)
movl %eax,PT_EAX(%esp) # store the return value
syscall_exit: ……
INTERRUPT_RETURN # i.e. iret
const sys_call_ptr_t sys_call_table[__NR_syscall_max+1] = {
[0 ... __NR_syscall_max] = &sys_ni_syscall,
#include <asm/syscalls_32.h> };

10. System Calls – Passing Parameters
Ordinary C functions use stack to pass parameters
Neither the user mode or the kernel mode stacks can be used in syscalls.
system call parameters are written in the CPU registers before issuing the system call.
the kernel then copies the CPU registers onto the kernel mode stack before invoking the system call service routine.
Push ebp, edi, esi, edx, ecx, ebx into stack then make the call to syscall function
Which stack?
Each process (and LWP) has a user mode
stack and a kernel mode stack (e.g. 2 pages (8K))
Two additional interrupt stacks in kernel space
for interrupts and SoftIRQs
Switch stacks by software inLinux
EBP 
ESP 
Local variable
Old frame pointer (EBP)
EIP saved by call inst.
Parameter 3
Parameter 2
Parameter 1

11. Syscall in Glibc /gnu/glibc/sysdeps/unix/sysv/linux/i386/syscall.S
Example getpid in /gnu/glibc/sysdeps/unix/sysv/linux/
.text
24 ENTRY (syscall) 25
26 PUSHARGS_6 /* Save register contents. */
27 _DOARGS_6(44) /* Load arguments. */
28 movl 20(%esp), %eax /* Load syscall number into %eax. */
29 ENTER_KERNEL /* Do the system call. */
30 POPARGS_6 /* Restore register contents. */
31 cmpl $-4095, %eax /* Check %eax for error. */
32 jae SYSCALL_ERROR_LABEL /* Jump to error handler if error. */
33 ret /* Return to caller. */ 34
35 PSEUDO_END (syscall)
# define ENTER_KERNEL int $0x80

12. Memory Spaces (1)
The logical address space of a process is divided into two parts
– 0x to PAGE_OFFSET-1 can be addressed in either user or kernel mode
– PAGE_OFFSET to 0xffffffff can be addressed only in kernel mode
– PAGE_OFFSET is usually 0xc
Peripheral devices are controlled by writing and reading their registers. Where are these registers?
I/O Port: request an IO port region and inb (outb) from <asm/io.h>
I/O Memory:
Request a memory region
Devices at physical addresses which have to be mapped to virtual addresses for software to access.

13. (X86 virtual memory layout)
Memory Spaces (2)
Physical memory
Virtual memory (user and kernel space)
(X86 virtual memory layout)

14. Linux Kernel Modules (1)
Modules can be compiled and dynamically linked into kernel address space.
Useful for device drivers and file systems that need not always be resident until needed. (why?)
Extend the functionality of the kernel without rebuilding and rebooting the system.
Advantages:
modularized approach
Platform independence
Frugal main memory usage
No performance penalty
Cannot use modules if
to replace statically linked code or to change an already defined data structure

15. Linux Kernel Modules (2)
To enable the module mechanism, modules
must be accessible by the rest of the kernel
must know the addresses of symbols in the kernel and in other modules.
references are resolved once and for all when a module is linked.
keeping track of the use of modules, so that no module is unloaded while another module or another part of the kernel is using it.
Kernel keeps a symbol table
Symbols accessible to kernel-loadable modules appear in /proc/kallsyms.
EXPORT_SYMBOL(), which exports to any loadable module, or
EXPORT_SYMBOL_GPL(), which exports only to GPL-licensed modules.
Can call any functions exported by the kernel
no library is linked to modules

16. Kernel Modules which process runs init and exit code?
An example module
#include <linux/module.h> // Needed by all modules
#include <linux/kernel.h> // Needed for KERN_ALERT
#include <linux/init.h> // Needed for the macros
static int hello_2_init(void) {
printk(KERN_ALERT "Hello, world 2\n");
return 0; }
static void hello_2_exit(void) {
printk(KERN_ALERT "Goodbye, world 2\n"); }
module_init(hello_2_init);
module_exit(hello_2_exit);
which process runs init and exit code?
Is a part of kernel – printf or printk
Syscalls:
i init_module sys_init_module
i delete_module sys_delete_module

17. Linking and Unlinking Modules

18. Programs for Linking and Unlinking Modules
lsmod
reads /proc/modules
rmmod
From reads the name of the module to be unlinked.
Invokes the query_module( )
Invokes the delete_module( ) system call, with the QM_REFS subcommand several times, to retrieve dependency information on the linked modules.
modprobe
loads a module into the kernel.
check any module dependency and load any other modules that are required – stacking of modules

19. Module Implementation
The kernel considers only modules that have been loaded into RAM by the insmod program and for each of them allocates memory area containing:
a module object
null terminated string that represents module's name
the code that implements the functions of the module
Building
Linux kbuild
make -C ~/kernel-2.6 M=`pwd` modules to build modules.ko
“M=“ is recognized and kbuild is used
~/kernel-2.6 is the kernel source directory
pwd ??

20. Make File to Build Modules
CC = i586-poky-linux-gcc
ARCH = x86
CROSS_COMPILE = i586-poky-linux-
SROOT=/opt/clanton-full/1.4.2/sysroots/i586-poky-linux/
EXTRA_CFLAGS += -g
obj-m = HelloModule.o
# the kbuild system know that it has to build “HelloModule.ko" and will look for source file
all: make ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE) -C
$(SROOT)/usr/src/kernel M=$(PWD) modules
clean: make ARCH=x86 CROSS_COMPILE=i586-poky-linux- -C $(SROOT)/usr/src/kernel
M=$(PWD) clean
The source file is compiled to a “HelloModule.o“
"HelloModule.mod.c" is created to contain the information about the module and is compiled to "HelloModule.mod.o".
HelloModule.o and HelloModulemod.o are linked together to create the "modulename.ko".
.ko is an elf file

21. Linking and Loading Modules
Insmod
Prepare the module in user space
Reads from the name of the module to be linked and Locates the file containing the module's object code
Allocate memory area needed to store the module code, its name, and the module object in user space and bring in the module to the buffer
Invoke sys_init_module which will
allocate kernel memory to hold module.
copy the relocatable object code of the module to the kernel space.
get the kernel symbol table in order to link the module.
use the kernel symbol table and the module symbol tables to relocates the object code included in the module's file.
registers the kobject included in the mkobj field of the module object
releases the user mode memory area
invoke the init method of the module
Linking is done in kernel since 2.6

22. Building Executable Code
Compile source programs to object code files
Linker to take one or more objects and combines them into a single executable program
Unix ld, GNU linker
dynamic linker, the part of an OS
that loads and links the shared
libraries for an executable
symbol resolution
relocation
Linker script – to describe how the sections in the input files should be mapped into the output file, and to control the memory layout.
main.o
delay.o
libc.a
linker
executable
library
object

23. Linker Script
SECTIONS {
.text.start (_KERNEL_BASE_) : {
startup.o( .text ) }
.text : ALIGN(0x1000) {
_TEXT_START_ = .;
*(.text)
_TEXT_END_ = .;
.data : ALIGN(0x1000) {
_DATA_START_ = .;
*(.data)
_DATA_END_ = .;
.bss : ALIGN(0x1000) {
_BSS_START_ = .;
*(.bss)
_BSS_END_ = .;
A text file that describe how the sections in the input files should be mapped into the output file, and to control the memory layout of the output file.
ENTRY linker script command to set the entry point.
SECTIONS command to describe memory layout
`.' indicates the location counter
MEMORY command describes the location and size of blocks of memory in the target
MEMORY {
ROM (rx) : ORIGIN = 0, LENGTH = 256k
RAM (wx) : org = 0x , len = 1M }

24. ELF -- Executable and Linking Format (1)
The old one – a.out for UNIX community
ELF –
standard for Linux
better support for cross-compilation, dynamic linking, initializer/finalizer
An ELF file can be one of the 4 types
Relocatable
Executable
Shared object
Core file
Two views
Compilers, assemblers, and
linkers – a set of logical sections.
System loader – a set of segments

25. ELF -- Executable and Linking Format (2)
Partial list of the ELF sections
.init - Startup
.text - String
.fini - Shutdown
.rodata - Read Only
.data - Initialized Data
.tdata - Initialized Thread Data
.tbss - Uninitialized Thread Data
.ctors - Constructors
.dtors - Destructors
.got - Global Offset Table
.bss - Uninitialized Data

26. GCC –g Option
Produce debugging information in the operating system's native format, including DWARF
objdump –h to dump
the section headers
.debug_line -- full
mapping between
lines in the C source
code and machine
code addresses
26 .debug_aranges
CONTENTS, READONLY, DEBUGGING
27 .debug_pubnames
28 .debug_info cc f
29 .debug_abbrev a b
30 .debug_line b d5
Decoded dump of debug contents of section .debug_line:
CU: /home/eliben/eli/eliben-code/debugger/tracedprog2.c:
File name Line number Starting address
tracedprog2.c x
tracedprog2.c x804860a
tracedprog2.c x
tracedprog2.c x804861c

27. Device Driver Basics Purpose: A software interface to hardware devices
a well defined and consistent interface to handle requests for device operations
isolate device-specific code in the drivers
A software interface to hardware devices
resides in kernel or user spaces
Classification
character device (terminal)
block (disk) -- with buffer cache
network
pseudodevice
When to call a device driver
configuration, I/O operations, and interrupt
OS specific
code
I/O class
specific code
device
driver
Hardware
specific code
I/O adapters

28. Char and Block Devices Character Devices Block Devices
Accessed as a stream of bytes (like a file)
Typically just data channels or data areas, which allow only sequential access
Char devices are accessed by means of filesystem nodes
Example: /dev/tty1 and /dev/lp0
Driver needs to implement at least the open, close, read, and write system calls
Block Devices
provides access to devices that transfer randomly accessible data in fixed-size blocks
Examples?
Design Challenges
Concurrency, performance, and portability

29. Char Device
cdev -- the kernel internal structure to represents a character device file
Defined in <linux/cdev.h>
struct cdev {
struct kobject kobj;
struct module *owner;
struct file_operations *ops;
struct list_head list;
dev_t dev;
unsigned int count; };
struct file_operations ldd_fops = {
.owner = THIS_MODULE,
.llseek = ldd_llseek,
.read = ldd_read,
.write = ldd_write,
.ioctl = ldd_ioctl,
.open = ldd_open,
.release = ldd_release };

30. Driver Interface for Character Devices
struct file_operations { // defined in inlcude/linux/fs.h struct module *owner; loff_t (*llseek) (struct file *, loff_t, int); ssize_t (*read) (struct file *, char __user *, size_t, loff_t *); ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *); ssize_t (*aio_read) (struct kiocb *, const struct iovec *, unsigned long, loff_t); ssize_t (*aio_write) (struct kiocb *, const struct iovec *, unsigned long, loff_t); long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long); long (*compat_ioctl) (struct file *, unsigned int, unsigned long); int (*mmap) (struct file *, struct vm_area_struct *); int (*open) (struct inode *, struct file *); int (*flush) (struct file *, fl_owner_t id); int (*release) (struct inode *, struct file *); int (*fsync) (struct file *, int datasync); int (*fasync) (int, struct file *, int); int (*lock) (struct file *, int, struct file_lock *); ………………… };

31. User Program & Kernel Interface

32. Basic Device Driver Structure
“/tty0/” 1
“/xx1/” 3
“/pty0/”
Device-
dependent
data
Device list
(of device descriptors)
Driver table
(function pointers)
File descriptor
table 2
Major number to identify the driver associated with the device
Minor number provides a way for the driver to differentiate multiple devices.
dev_t type in <linux/types.h>: 8bis for major and 8 bits for minor in kernel 2.4, 12bis for major and 20 bits for minor in kernel 2.6.

33. Register Char Device (1)
static dev_t myDev = 0;
static char myName[] = "TestDevice";
static int result = 0;
static int hello_init(void) {
if((result = alloc_chrdev_region(&myDev,0,1,myName))!= 0){
printk("Unable to allocate device number\n");
return -1; }
printk(KERN_ALERT "My major is %d, minor is %d\n“,
MAJOR(myDev),MINOR(myDev));
return 0;
static void hello_exit(void)
if(result == 0){
unregister_chrdev_region(myDev, 1);

34. Register Char Device (2)
Device numbers registration and release
Manually-assigned registration
Dynamically-allocated registration
Release
int register_chrdev_region(
dev_t first, # The beginning device number
unsigned int count, # Number of continuous device number
char *name # Device name (shown in /proc/devices) );
int alloc_chrdev_region(
dev_t *dev, # output-only that holds the first device
unsigned int firstminor, # The beginning of first minor number
unsigned int count,
char *name );
void unregister_chrdev_region(
dev_first, # The first device number going to release
unsigned int count );

35. Device Registration Allocate and initialize a cdev struct
cdev_alloc or cdev_init
cdev gets initilized with a file_opreations structure
cdev_add to register operatons of your device driver
int cdev_add(struct cdev *dev, dev_t num, unsigned int count)
cdev_del — remove a cdev from the system

36. Open System Call
The open system maps onto an open operation registered by a driver module
The kernel first does some processing (e.g., finds the inode struct and file struct for the driver
Invokes the open operation
int open(const char *pathname, int flags);
int open(const char *pathname, int flags, mode_t mode);
// return the new file descriptor,
int (*open) (struct inode *, struct file *);

37. File Struct The file structure represents an open file.
struct file {
struct list_head f_list;
struct dentry *f_dentry;
struct vfsmount *f_vfsmnt;
struct file_operations *f_op;
atomic_t f_count;
unsigned int f_flags;
mode_t f_mode;
loff_t f_pos;
unsigned long f_reada, f_ramax, f_raend, f_ralen, f_rawin;
struct fown_struct f_owner;
unsigned int f_uid, f_gid;
int f_error;
unsigned long f_version;
void *private_data; };
The file structure represents an open file.
a kernel structure
created by the kernel on open and is passed to any function that operates on the file
defined in <linux/fs.h>
mode_t f_mode : readable or writable
loff_t f_pos: current reading or writing position.
struct file_operations *f_op
void *private_data

38. inode Each object in the filesystem is represented by an inode
File type (executable, block special etc), permissions (read, write etc), Owner, Group, File Size,
File access, change and modification time, deletion time Number of links (soft/hard), extended attribute such as append only
Access Control List (ACLs)
Includes
dev_t i_rdev;
struct block_device *i_bdev;
struct char_device *i_cdev;

39. Memory Access – User and Kernel Spaces
You might receive some pointers that point to user space in your driver function
Do not dereference it directly
Access it via kernel functions (<asm/uaccess.h>): copy_to_user() and copy_from_user()
They would check whether the user space pointer is valid
unsigned long copy_to_user(
void __user *to,
const void *from,
unsigned long count);
unsigned long copy_from_user(
void *to,
const void __user *from,

40. The Read/Write Method

41. Container_of
If strcut x contains y, can we find the pointer to x given the pointer to y
defined in <linux/kernel.h>:
container_of(pointer, container_type, container_field);
takes a pointer to a field of type container_field, within a structure of type container_type,
returns a pointer to the containing structure.