1. National Taiwan University OS Project 0 & 1 Advisor: Dr. Chih-Wen Hsueh Student: Tang-Hsun Tu 台灣大學 網媒所 / 資工所 Wireless Networking and Embedded Systems Laboratory Real-Time System Software Group September 7, 2015

2. National Taiwan University Tang-Hsun Tu /602  Linux Installation  Linux Kernel Compilation  System Call  Project Hints Outline

3. National Taiwan University Linux Installation

4. National Taiwan University Tang-Hsun Tu /604  Including shells, libraries, tools, compiler, servers, applications.  Ubuntu, Redhat, Fedora, Mandrake, SuSE, Debian, Ubuntu, Gentoo …  This slides are based on Ubuntu 10.04 LTS. 11.04 11.10 Beta  Linux 3.0  http://www.ubuntu-tw.org/modules/tinyd0/ http://www.ubuntu-tw.org/modules/tinyd0/ Linux Distribution

5. National Taiwan University Tang-Hsun Tu /605  Windows can be also installed together.  Modify the boot sequence to boot your computer from CD-ROM.  Make sure your hardware and device.  A clear head and relaxed mind.  Some drinks and food. Before Installation

6. National Taiwan University Tang-Hsun Tu /606

7. National Taiwan University Tang-Hsun Tu /607

8. National Taiwan University Tang-Hsun Tu /608  /dev/hda, /dev/hdb, /dev/hdc, … /dev/hda1, /dev/hda2, …  /dev/sda, /dev/scd0, …  Mount Points / /swap … Disks and Partitions

9. National Taiwan University Tang-Hsun Tu /609

10. National Taiwan University Tang-Hsun Tu /6010  Swap partition is usually twice as RAM when it is less than 1GB.  No more than 4 primary partitions including / partition and swap partition.  If the primary partitions are not enough, you can use logical partitions.  Make sure all your mount points are correct. Partition Division

11. National Taiwan University Tang-Hsun Tu /6011

12. National Taiwan University Tang-Hsun Tu /6012

13. National Taiwan University Tang-Hsun Tu /6013  After dividing partitions, you only need to click your mouse.  After installation, reboot and enjoy your Linux!  Installation by text mode is similar to graphic mode. End of Installation

14. National Taiwan University Tang-Hsun Tu /6014  Install packages in Ubuntu. apt-cache search xxxx apt-get install xxxx  GNU Compiler Collection. gcc –o test test.c./test  Makefile. make  Command-line editor. vim  SSH server and client. ssh –oPort=port accout@ip Some Useful Commands

15. National Taiwan University Tang-Hsun Tu /6015  Install ssh server. apt-get install ssh  A ssh client on Windows. http://ntu.csie.org/~piaip/pietty/ Some Useful Tools

16. National Taiwan University Tang-Hsun Tu /6016  鳥哥的私房菜 http://linux.vbird.org/  Ubuntu 正體中文站 http://www.ubuntu-tw.org/  Ubuntu Homepage http://www.ubuntu.com/ Reference

17. National Taiwan University Linux Kernel Compilation

18. National Taiwan University Tang-Hsun Tu /6018  Kernel is the core of an operating system. Process management Memory management Scheduler Filesystem Virtualization … What is Linux Kernel?

19. National Taiwan University Tang-Hsun Tu /6019  You can configure your Linux by compiling a new kernel. Add new features, eg. patch kernel. Support new hardware. Disable functions you don’t need. Develop your own kernel.... When Should We Compile Kernel?

20. National Taiwan University Tang-Hsun Tu /6020  Kernel Website, http://www.kernel.orghttp://www.kernel.org  You also can download the source codes patched by Ubuntu. Where to D/L Linux Kernel?

21. National Taiwan University Tang-Hsun Tu /6021  You need to be root to compile kernel. /usr/src/… sudo -i  Download the necessary tools. apt-get update apt-get install kernel-package gcc libncurses5-dev automake gcc libc6-dev build-essential  Go to System/Administration/synaptic package Manager or http://www.kernel.org/.http://www.kernel.org/ Get the kernel source codes e.g. 3.0.1 Prepare Your Kernel Source Code

22. National Taiwan University Tang-Hsun Tu /6022  Unzip kernel source code. cd /usr/src tar jxvf linux-source-3.X.tar.bz2  You may have many versions of Linux source codes. Prepare Your Kernel Source Code (Cont)

23. National Taiwan University Tang-Hsun Tu /6023  There are many ways to configure. make config make menuconfig make xconfig...  If you do not know how to configure, you can copy the old config file from /boot. make mrproper cp /boot/config-`uname -r`.config make oldconfig make menuconfig Configure your Kernel

24. National Taiwan University Tang-Hsun Tu /6024 Configure your Kernel (Cont)

25. National Taiwan University Tang-Hsun Tu /6025  If you meet some problems (SATA). Device Drivers --->  SCSI device support --->  SCSI device support  SCSI disk support Device Drivers --->  SCSI device support --->  SCSI low-level drivers --->  [*] Serial ATA (SATA) support Configure your Kernel (Cont)

26. National Taiwan University Tang-Hsun Tu /6026  The first compilation (about 30min ~ 1.5hr): #make clean #make bzImage #make modules  (long time) #make install #make modules_install #mkinitramfs –o /boot/initrd.img-3.x 3.x  do this only if you reinstall modules then reboot!  The second compilation (about 5~10min in our project): #make bzImage #make install then reboot! Kernel Compilation

27. National Taiwan University Tang-Hsun Tu /6027  See how many cores or hyper-threading on your machine. cat /proc/cpuinfo | grep processor | wc –l e.g. 4  Compile with the number of jobs (threads). make –j4 bzImage make –j4 modules... Speed up Kernel Compilation

28. National Taiwan University Tang-Hsun Tu /6028  Setup your boot manager. vim /boot/grub/menu.lst  Add the following section (Grub1): title Ubuntu 10.04 LTS, kernel 3.0.1 uuid xxxx kernel /boot/vmlinuz-3.0.1 root=UUID=xxxx ro splash initrd /boot/initrd.img-3.0.1 Configure your Boot Menu

29. National Taiwan University Tang-Hsun Tu /6029  Setup your boot manager. vim /boot/grub/grub.cfg  Add the following section (Grub2): Configure your Boot Menu (Cont)

30. National Taiwan University Tang-Hsun Tu /6030  鳥哥的私房菜 http://linux.vbird.org/  Google http://www.google.com/ Reference

31. National Taiwan University System Call

32. National Taiwan University Tang-Hsun Tu /6032  System call is the mechanism used by an application program to request service from the OS.  Users can use it to communicate with kernel.  Here are two approaches developing our own system calls. Using kernel module Modify the source codes of Linux directly Introduction

33. National Taiwan University Tang-Hsun Tu /6033  Building system calls in kernel module is more flexible than modifying kernel.  When we want to use our system call, just install our kernel modules. Also, if we don’t need it right away, just remove modules. Modifying kernel is not necessary. (But you still need to modify your kernel for OS project 1.) Using Kernel Module

34. National Taiwan University Tang-Hsun Tu /6034  For sys_call_table, your should extern it in a file such as /arch/x86/kernel/i386_ksyms_32.c. Export sys_call_table extern void* sys_call_table[]; /*variable should be exported. */ EXPORT_SYMBOL(sys_call_table); 01 02 03

35. National Taiwan University Tang-Hsun Tu /6035  sys_call_table is read-only after kernel version 2.6.23.  If you want to try this method using kernel version which is higher than 2.6.23, you have to modify your current kernel source codes and recompile it. Export sys_call_table (Cont)

36. National Taiwan University Tang-Hsun Tu /6036  Firstly, check your compiled kernel version. uname –a  In x86 32bit vim /usr/src/linux-3.x/arch/x86/kernel/entry_32.S.section.rodata, “a” .section.data, “aw” Export sys_call_table (Cont)

37. National Taiwan University Tang-Hsun Tu /6037  In x86 64bit vim /usr/src/linux-3.x/arch/x86/kernel/syscall_64.c line 22: delete the “const” Export sys_call_table (Cont)

38. National Taiwan University Tang-Hsun Tu /6038  Add to export symbol. vim /usr/src/linux-3.x/kernel/kallsyms.c extern void *sys_call_table; EXPORT_SYMBOL(sys_call_table); Export sys_call_table (Cont)

39. National Taiwan University Tang-Hsun Tu /6039  vim makefile  vim myservice.c Write Your Makefile

40. National Taiwan University Tang-Hsun Tu /6040  Include and Define.  Extern the “sys_call_table”.  Write your own system call. Write Kernel Module #include /* We're doing kernel work */ #include #define __NR_mysyscall 200 /* define the number of our system call */ 01 02 03 typedef void (*sys_call_ptr_t)(void); extern sys_call_ptr_t sys_call_table[]; sys_call_ptr_t orig_sys_call; 01 02 03 /* Our system call */ asmlinkage int mysyscall(int arg1) { printk("enter mysyscall()\n"); return arg1 * 10; } 01 02 03 04 05

41. National Taiwan University Tang-Hsun Tu /6041  Initialize the kernel module.  Extern the “sys_call_table”. Write Kernel Module (Cont) /* Initialize the module - replace the system call */ int init_module() { printk("Insert mysyscall module\n"); orig_sys_call = sys_call_table[__NR_mysyscall]; sys_call_table[__NR_mysyscall] = mysyscall; return 0; } 01 02 03 04 05 06 07 /* Cleanup - unregister the appropriate file from /proc */ void cleanup_module() { printk("Remove mysyscall module\n"); sys_call_table[__NR_mysyscall] = orig_sys_call; } 01 02 03 04 05

42. National Taiwan University Tang-Hsun Tu /6042  Compile. make  Insert the module to kernel. insmod./myservice.ko  Remove the module from kernel. rmmod myservice  List the modules in kernel. lsmod Use Kernel Module

43. National Taiwan University Tang-Hsun Tu /6043  Write an application to use your system call. vim ap.c  Compile and execute. gcc ap.c –o ap./ap 10 User Application #include #define _GNU_SOURCE #include #define __NR_mysyscall 200 int main(int argc,char *argv[]) { printf("%d\n", syscall(__NR_mysyscall, atoi(argv[1]))); return 0; } 01 02 03 04 05 06 07 08 09 10 11 12 13

44. National Taiwan University Tang-Hsun Tu /6044  Here are two approaches developing our own system calls. Using kernel module Modify the source codes of Linux directly Build Your Own System Calls

45. National Taiwan University Tang-Hsun Tu /6045  Create a new file in /usr/src/linux- 3.x/kernel/. vim myservice.c  Add your system call. Write Your System Call #include asmlinkage int sys_myservice(int arg1) { printk("my service is invoked!\n"); return arg1 * 10; } 01 02 03 04 05 06 07 08

46. National Taiwan University Tang-Hsun Tu /6046  In x86 32bit /usr/src/linux-3.x/arch/x86/include/asm/ unistd_32.h  The index must be the last in the list. e.g. #define __NR_myservice 347 Write Your System Call (Cont) - x86_32

47. National Taiwan University Tang-Hsun Tu /6047  Create an entry (function name) in system call table. /usr/src/linux-3.x/arch/x86/kernel/ syscall_table_32.S.long sys_myservice Write Your System Call (Cont) - x86_32

48. National Taiwan University Tang-Hsun Tu /6048  In x86 64bit /usr/src/linux-3.x/arch/x86/include/asm/ unistd_64.h  The index must be the last in the list. #define __NR_myservice 309 __SYSCALL(__NR_myservice, sys_myservice) Write Your System Call (Cont) - x86_64

49. National Taiwan University Tang-Hsun Tu /6049  Define the prototype. /usr/src/linux-3.x/include/linux/syscalls.h asmlinkage int sys_myservice(int arg1); Write Your System Call (Cont)

50. National Taiwan University Tang-Hsun Tu /6050  Add to makefile to compile. vim /usr/linux-3.x/kernel/Makefile obj-y += myservice.o  Now, you can recompile your kernel. Write Your System Call (Cont)

51. National Taiwan University Tang-Hsun Tu /6051  Write an application to use your system call. vim ap.c  Compile and execute. gcc ap.c –o ap./ap 10 User Application #include #define _GNU_SOURCE #include #define __NR_myservice 309 int main(int argc,char *argv[]) { printf("%d\n", syscall(__NR_myservice, atoi(argv[1]))); return 0; } 01 02 03 04 05 06 07 08 09 10 11 12 13

52. National Taiwan University Tang-Hsun Tu /6052  Socket programming server: socket/bind/listen/accept client: socket/connect  New system calls sendmsg() recvmsg() sendmmsg(), 3.0 ↑ recvmmsg(), 2.6.33 ↑ Hints (1/4)

53. National Taiwan University Tang-Hsun Tu /6053  Since we want measure the number of invoking time of sendmsg()/recvmsg() in sendmmsg()/recvmmsg(), we might need to add counters in /usr/src/Linux- 3.x/net/socket.c. e.g. int nr_called_sendmsg;  If you want to use this variable in your system call or kernel module, you have to export it. EXPORT_SYMBOL(nr_called_sendmsg);  printk can print messages in kernel, use dmesg to check. Hints (2/4)

54. National Taiwan University Tang-Hsun Tu /6054 Hints (3/4)  A generic interface: syscall(int no,...) __NR_sendmmsg and __NR_recvmmsg syscall() sendmmsg()/recvmmsg()

55. National Taiwan University Tang-Hsun Tu /6055  Results Hints (4/4) server client 1. syscall number of “recvmmsg()” 2. received messages 3. the invoking times of “recvmsg()” in the kernel 4. time of invoking recvmmsg() 1. typed message 2. syscall number of “sendmmsg()” 3. the invoking times of “sendmsg()” in the kernel 4. time of invoking sendmmsg()

56. National Taiwan University Tang-Hsun Tu /6056  LXR, http://rswiki.csie.org/lxr/http/sourcehttp://rswiki.csie.org/lxr/http/source Some Useful Tool

57. National Taiwan University Tang-Hsun Tu /6057  Manpages apt-get install manpages-dev Some Useful Tool (Cont)

58. National Taiwan University Tang-Hsun Tu /6058  Kernel Website http://www.kernel.org  Socket Programming http://www.tenouk.com/Module41b.html  LXR http://rswiki.csie.org/lxr/http/source http://lxr.linux.no/linux/  Google http://www.google.com/  Recvmmsg() http://lwn.net/Articles/334532/  Sendmmsg() http://lwn.net/Articles/441169/ Reference

59. National Taiwan University Tang-Hsun Tu /6059  We set up a desktop PC with virtualization in Lab202. cd /home/lab202/os_cfg Start Xen:./runxen.sh Start Ubuntu VM:./runubu.sh or./runubu2.sh Start WinXP VM:./runxp.sh Start VGA pass-through WinXP VM:  sudo./runvgaxp.sh &  You can compile your kernels on the VMs!  Create your own VM disk. dd or qemu-img-xen create vmdisk.img 8G  Backups are in /home/lab202/backup.  Account and password. Announced in the courses or send a mail to TA. Resource in Lab202

60. National Taiwan University Tang-Hsun Tu /6060 Q & A