1. Sogang University Advanced Operating Systems (Inter-Process Communication - Linux) Advanced Operating Systems (Inter-Process Communication - Linux) Sang Gue Oh, Ph.D. Email : sgoh@macroimpact.com Email : sgoh@macroimpact.com

2. Sogang University Inter-Process Communication - Linux Page 2 Inter-Process Communication (IPC) in Linux n Inter-Process Communication : Mechanism for various processes to communicate among them. t Different processes run on different address space. t Kernel needs to provide mechanisms to communicate -> IPC. n Types of IPCs in Linux t Traditional UNIX IPCs : Signal, Pipe, Socket t System V IPCs : Message Queue, Semaphore, Shared Memory n Will cover both user library routines (APIs) and kernel implementations.

3. Sogang University Inter-Process Communication - Linux Page 3 SignalsSignals n OS mechanism for notifying an application process that some event (asynchronous event) has occurred. n Types t Occurrence of a hardware event : Pressing Ctrl+C key, Divide by 0. t Notification of a software condition : User’s interval timer expired. A signal is represented by a name of the form SIGXXX (e.g., SIGINT). (include/asm/signal.h) n The number of supported signals t Limited to the word size of the processor (32 bits can have 32 signals). t Defined by the constant NSIG in include/asm/signal.h. t Users are not allowed to create new signals but can use SIGUSR1 or SIGUSR2.

4. Sogang University Inter-Process Communication - Linux Page 4 Types of Signals Listing signals : kill command (kill -l), 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP 6) SIGIOT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1 11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP 21) SIGTTIN 22) SIGTTOU 23) SIGURG 24) SIGXCPU 25) SIGXFSZ 26) SIGVTALRM 27) SIGPROF 28) SIGWINCH 29) SIGIO 30) SIGPWR

5. Sogang University Inter-Process Communication - Linux Page 5 Actions for the Signals Default actions. t Ignore the signal. t Terminate the program. t Terminate the program and generate a file core. n Users can install corresponding handlers for the signals. Users can block/unblock/suspend signals. n Neither the SIGSTOP (causes a process to halt) nor the SIGKILL (causes a process to exit) signals can be ignored. Their default actions may not be modified.

6. Sogang University Inter-Process Communication - Linux Page 6 Signal Information in the Kernel (task_struct) Signals information in the task_struct t Field signal : Currently pending signals. t Field blocked : A mask of blocked signals. t Field sig : A table containing the addresses of routines (NSIG routines) that will handle the signals (e.g., sigaction[_NSIG]). unsigned long signal; unsigned long blocked; /* bitmap of masked signals */ struct signal_struct *sig; /* include the sigaction field */

7. Sogang University Inter-Process Communication - Linux Page 7 Signal Generation t Normal processes can only send signals to processes in the same process group (same uid and gid). t Generated by library calls (e.g., kill, raise, alarm) or command (e.g., kill). t By setting the appropriate bit in the task_struct's signal field. t The signal field is checked before returning from the system call (i.e., before the scheduler is invoked) and change the process’s state to TASK_RUNNING. t Signals are not presented to the process immediately they are generated, they must wait until the process is running again.

8. Sogang University Inter-Process Communication - Linux Page 8 System Calls Related to Signal (1) n Signal Generation t int kill(pid_t pid, int signo); t int raise(int signo); t unsigned int alarm(unsigned int seconds); n Signal Mask and Signal Set t int sigemptyset(sigset_t *set); -> initialize a signal set to contain no signals. t int sigfillset(sigset_t *set); -> initialize a signal set to contain all of the signals. t int sigaddset(sigset_t *set, int signo); -> add a specified signal into the set. t int sigdelset(sigset_t *set, int signo); -> delete a specified signal from the set. t int sigismember(const sigset_t *set, int signo); -> check if the signal is in the set. t int sigprocmask(int how, const sigset_t *set, sigset_t *oset); -> int how : the manner in which the signal mask is to be modified (e.g., SIG_BLOCK, SIG_UNBLOCK, SIG_SETMASK). sigset_t *set : set of signals to be used for the modification. sigset_t *oset : set of signals that were blocked before the call to sigprocmask.

9. Sogang University Inter-Process Communication - Linux Page 9 System Calls Related to Signal (2) n Catching and Ignoring Signals t int sigaction(int signo, const struct sigaction *act, struct sigaction *oact); struct sigaction { void (*sa_handler) ();/* SIG_DFL, SIG_IGN, or pointer to function */ sigset_t sa_mask;/* additional signals to be blocked */ int sa_flags;/* special options */  } n Waiting for Signals t int pause(void); t int sigsuspend(const sigset_t *sigmask);

10. Sogang University Inter-Process Communication - Linux Page 10 Use of Signal - Example 1 (1) #include char handmsg[] = "I found ^C !\n"; struct sigaction act; int flag = 0; void catch_ctrl_c(int signo) { write(STDERR_FILENO, handmsg, strlen(handmsg)); flag++; }

11. Sogang University Inter-Process Communication - Linux Page 11 Use of Signal - Example 1 (2) main() { act.sa_handler = catch_ctrl_c; sigemptyset(&act.sa_mask); act.sa_flags = 0; if ( sigaction(SIGINT, &act, NULL) < 0 ) fprintf(stderr, "Error !\n"); while ( flag < 5 ) ; }

12. Sogang University Inter-Process Communication - Linux Page 12 PipePipe Pipe t Unidirectional byte streams which connect one process into the other process. t The data written at one end of the channel is read at the other end. t From a logical point of view, a pipe can be compared to a FIFO queue of characters. t No structured communication : it is not possible to know the size, sender/receiver of data contained in the pipe. t The management of pipes is integrated in the file system. Access to pipes is achieved by reading/writing from/to file descriptors. Communication Pipe Task A Task B writeread

13. Sogang University Inter-Process Communication - Linux Page 13 Pipe Used by Commands One current usage of the pipe mechanism is performed by means of the command line interpreter when commands are linked: (e.g., > ps -aux | grep root | tail ) [ /home/parksy ] ps -aux | grep root | tail root 9232 0.0 0.1 1392 664 ? S Aug26 0:00 in.identd -e -o root 9233 0.0 0.1 1392 664 ? S Aug26 0:00 in.identd -e -o ….. root 11679 0.0 0.1 2100 680 ? S Aug28 0:14 sendmail: accepti root 23172 0.0 0.0 1176 488 ? S Sep05 0:00 lpd root 16763 0.2 0.1 1704 876 ? S 10:32 0:00 in.telnetd root 16764 0.1 0.2 2320 1232 pts/0 S 10:32 0:00 login -- parksy parksy 16800 0.0 0.1 2236 528 pts/0 S 10:33 0:00 grep root ps -auxgrep roottail outinoutin

14. Sogang University Inter-Process Communication - Linux Page 14 Anonymous Pipe n Created by a process and the transmission for associated descriptors is achieved only by inheritance by its descendants. (i.e., by creating a child process using fork() system call) n Restrictive in that it only allows communication between processes with a common ancestor which is a creator of a pipe. Creation of an anonymous pipe : (fs/pipe.c) t int pipe(int filesdes[2]); -> filesdes[0] : read descriptor, filesdes[1] : write descriptor. Anonymous Pipe Writing Reading write(filesdes[1])read(filesdes[0])

15. Sogang University Inter-Process Communication - Linux Page 15 Example of Anonymous Pipes n Case : ps -aux | grep root | tail P1 P2 P3 fork Creation pipe1 Creation pipe2 exec … ps -aux Pipe 1 exec … grep root Pipe 2 dup2 exec … tail dup2 exit 3 dup2(int oldfd, int newfd) : duplicate file descriptor

16. Sogang University Inter-Process Communication - Linux Page 16 Named Pipe (FIFO) n Remove the constraint of anonymous pipe (i.e., no name, only used between processes that have a parent process in common) because they are entries in the file system. n Has a name and handled exactly like files with respect to file operations (e.g., open, close, read, write). n Created by mkfifo or mknod commands. Can be created by C functions : mkfifo(). ( fs/fifo.c ) t int mkfifo(const char *path, mode_t mode); n Reading from / writing to a named pipe can be achieved by using standard read() and write() system calls.

17. Sogang University Inter-Process Communication - Linux Page 17 Pipe in the Kernel n R/W synchronization : use locks, wait queues and signals.

18. Sogang University Inter-Process Communication - Linux Page 18 Use of Pipe - Example #include int main(void) { inr n, fd[2], pid; char line[100]; if (pipe(fd) < 0) exit(-1); if ( (pid = fork()) < 0 ) exit(-1); else if (pid > 0) { close(fd[0]); write(fd[1], “Hello World\n”, 12); } else { close(fd[1]); n = read(fd[0], line, MAXLINE); write(stdout, line, n); } Parent Child fd[0] fd[1] PIPE fork Kernel

19. Sogang University Inter-Process Communication - Linux Page 19 System V IPC System V Inter-Process Communication Method t Message Queue, Semaphore, Shared Memory. t All share common authentication methods (page 21 - ipc_perm structure). t Processes may access these resources only by passing a unique reference identifier (IPC id) to the kernel via system calls. t This IPC id (similar to a file descriptor) is returned when users invoke system calls for creating/opening the each IPC structure. t Unlike the file descriptor, the IPC ids are system-wide and are not process-specific. t Increase the slot usage sequence number in ipc_perm structure whenever a given IPC structure is removed.

20. Sogang University Inter-Process Communication - Linux Page 20 Key Management n To create IPCs, or simply to access them, it is necessary to possess a key. n This is a number that identifies the IPC uniquely at the system level, and it is therefore necessary to possess it in order to access the resource. n To generate a key, the library function ftok is called. n The key value is calculated with the following formula: key = (st_ino & 0xFFFF) | ((st_dev & 0xFF) >> 16) | (proj << 24), where st_ino is inode # and st_dev is device #. #include key_t ftok(char *pathname, char proj);

21. Sogang University Inter-Process Communication - Linux Page 21 Access Permission Structure ipc_perm (include/linux/ipc.h) - defined in for system call. struct ipc_perm { __kernel_key_t key;/* IPC key */ __kernel_uid_t uid;/* Owner’s user id */ __kernel_gid_t gid;/* Owner’s group id */ __kernel_uid_t cuid;/* Creator’s user id */ __kernel_gid_t cgid;/* Creator’s group id */ __kernel_mode_t mode;/* Read-write permission */ unsigned short seq;/* Slot usage sequence number */ };

22. Sogang University Inter-Process Communication - Linux Page 22 Creating and Opening IPC Channels n Call ftok, passing it a pathname and id, or n Specify a key of IPC_PRIVATE, which guarantees that a new, unique IPC object is created. Open or create IPC channel Access IPC channel ftok() msgget() semget() shmget() Key of IPC_PRIVATE key_t key char *pathname int id int identifier msgctl(), msgsnd(), msgrcv() semctl(), semop() shmctl(), shmat(), shmdt()

23. Sogang University Inter-Process Communication - Linux Page 23 Programs - ipcs and ipcrm (1) n ipcs command : prints various information about the System V IPC such as current allocation status, limits, etc. [ /usr/src/linux/include/linux ] ipcs ------ Shared Memory Segments -------- key shmid owner perms bytes nattch status 0x00280267 1 root 644 1048576 1 ------ Semaphore Arrays -------- key semid owner perms nsems status 0x00280269 0 root 666 14 ------ Message Queues -------- key msqid owner perms used-bytes messages

24. Sogang University Inter-Process Communication - Linux Page 24 Programs - ipcs and ipcrm (2) n ipcs command : ipcs -l -> display current limits. [ /usr/src/linux/include/linux ] ipcs -l ------ Shared Memory Limits -------- max number of segments = 128 max seg size (kbytes) = 32768 max total shared memory (kbytes) = 16777216 min seg size (bytes) = 1 ------ Semaphore Limits -------- max number of arrays = 128 max semaphores per array = 250 max semaphores system wide = 32000 max ops per semop call = 32 semaphore max value = 32767 ------ Messages: Limits -------- max queues system wide = 128 max size of message (bytes) = 4056 default max size of queue (bytes) = 16384

25. Sogang University Inter-Process Communication - Linux Page 25 Programs - ipcs and ipcrm (3) n ipcrm command : removes a System V message queue, semaphore set, or shared memory segment. n ipcrm [ shm | msg | sem ] id

26. Sogang University Inter-Process Communication - Linux Page 26 Message Queue (1) msgque vector : maintains a list of message queues (ipc/msg.c). t Each element of which points to a msqid_ds. t static struct msqid_ds *msgque[MSGMNI]; -> default MSGMNI is 128. msqid_ds : maintains a list of message (include/linux/msg.h). struct msqid_ds { struct ipc_perm msg_perm; struct msg *msg_first; /* first message on queue */ struct msg *msg_last; /* last message in queue */ __kernel_time_t msg_stime; /* last msgsnd time */ __kernel_time_t msg_rtime; /* last msgrcv time */ __kernel_time_t msg_ctime; /* last change time */ struct wait_queue *wwait; struct wait_queue *rwait; unsigned short msg_cbytes; /* current number of bytes on queue */ unsigned short msg_qnum; /* number of messages in queue */ unsigned short msg_qbytes; /* max number of bytes on queue */ __kernel_ipc_pid_t msg_lspid; /* pid of last msgsnd */ __kernel_ipc_pid_t msg_lrpid; /* last receive pid */ };

27. Sogang University Inter-Process Communication - Linux Page 27 Message Queue (2) struct msg { struct msg *msg_next;/* next message on queue */ long msg_type; char *msg_spot;/* message text address */ time_t msg_stime;/* msgsnd time */ short msg_ts;/* message text size */ }; t msg.c contains the implementation of system calls such as sys_msgsnd, sys_msgrcv, sys_msgget, and sys_msgctl. t Corresponding library calls are provided to the users (#include ). l int msgget(key_t key, int oflag); l int msgsnd(int msqid, const void *ptr, size_t length, int flag); l ssize_t msgrcv(int msqid, void *ptr, size_t length, long type, int flag); l int msgctl(int msqid, int cmd, struct msqid_ds *buff);

28. Sogang University Inter-Process Communication - Linux Page 28 Message Queue Structure in Kernel

29. Sogang University Inter-Process Communication - Linux Page 29 Semaphore (1) semary vector : maintains a list of semid_ ds (ipc/sem.c). t Each element of which points to a semid_ds.  static struct semid_ds *semary[SEMMNI]; -> default SEMMNI is 128. semid_ds : a semaphore array (include/linux/sem.h). struct semid_ds { struct ipc_perm sem_perm; /* permissions.. see ipc.h */ __kernel_time_t sem_otime; /* last semop time */ __kernel_time_t sem_ctime; /* last change time */ struct sem *sem_base; /* ptr to first semaphore in array */ struct sem_queue *sem_pending; /* pending operations to be processed */ struct sem_queue **sem_pending_last; /* last pending operation */ struct sem_undo *undo; /* undo requests on this array */ unsigned short sem_nsems; /* no. of semaphores in array */ };.

30. Sogang University Inter-Process Communication - Linux Page 30 Semaphore (2) /* One semaphore structure for each semaphore in the system. */ struct sem { int semval;/* current value */ int sempid;/* pid of last operation */ }; /* One queue for each semaphore set in the system. */ struct sem_queue { struct sem_queue *next;/* next entry in the queue */ struct sem_queue **prev;/* previous entry in the queue, *(q->prev) == q */ struct wait_queue *sleeper;/* sleeping process */ struct sem_undo *undo;/* undo structure */ int pid;/* process id of requesting process */ int status;/* completion status of operation */ struct semid_ds *sma;/* semaphore array for operations */ struct sembuf *sops;/* array of pending operations */ int nsops;/* number of operations */ int alter;/* operation will alter semaphore */ };

31. Sogang University Inter-Process Communication - Linux Page 31 Semaphore (3) t sem.c contains the implementation of system calls such as sys_semget, sys_semctl, and sys_semop. t Corresponding library calls are provided to the users (#include ). l int semget(key_t key, int nsems, int oflag); l int semop(int semid, struct sembuf *opsptr, size_t nops); l int semctl(int semid, int semnum, int cmd); struct sembuf { short sem_num;/* semaphore # : 0, 1,.. nsems-1 */ short sem_op;/* semaphore operation */ short sem_flg;/* operation flag */ }

32. Sogang University Inter-Process Communication - Linux Page 32 Semaphore Structure in Kernel

33. Sogang University Inter-Process Communication - Linux Page 33 Shared Memory (1) shm_segs : maintains a list of shmid_ds (ipc/shm.c) t Each element of which points to a shmid_ds.  static struct shmid_ds *shm_segs[SHMMNI]; -> default SHMMNI is 128. shmid_ ds : each shared memory area (include/linux/shm.h). struct shmid_ds { struct ipc_perm shm_perm;/* operation perms */ int shm_segsz;/* size of segment (bytes) */ __kernel_time_t shm_atime;/* last attach time */ __kernel_time_t shm_dtime;/* last detach time */ __kernel_time_t shm_ctime;/* last change time */ __kernel_ipc_pid_t shm_cpid;/* pid of creator */ __kernel_ipc_pid_t shm_lpid;/* pid of last operator */ unsigned short shm_nattch;/* no. of current attaches */ unsigned short shm_unused;/* compatibility */ void *shm_unused2;/* ditto - used by DIPC */ void *shm_unused3;/* unused */ unsigned long shm_npages; /* size of segment (pages) */ unsigned long *shm_pages; /* array of ptrs to frames -> SHMMAX */ struct vm_area_struct *attaches; /* descriptors for attaches */ };

34. Sogang University Inter-Process Communication - Linux Page 34 Shared Memory (2) n shm.c contains the implementation of system calls such as sys_smget, sys_shmctl, sys_shmat, and sys_shmdt. t Corresponding library calls are provided to the users (#include ). l int shmget(key_t key, size_t size, int oflag); l void *shmat(int shmid, const void *shmaddr, int flag); l int shmdt(const void *shmaddr); l int shmctl(int shmid, int cmd, struct shmid_ds *buff);

35. Sogang University Inter-Process Communication - Linux Page 35 Shared Memory Structure in Kernel