1. Instructor: Junfeng Yang
W4118 Operating Systems
Instructor: Junfeng Yang

2. Logistics Homework 2 out: system call fault injector
System call can fail for a variety of reasons
Many programs must correctly these failures
Our fault injector can test this
How? You add a system call fail(), to fail one of the future system calls a process issues
We’ll use VM for kernel programming assignments
Three ways to get the class VM
Download from the course website
Go to office hours and copy from me or TAs
We’ll make a few DVDs
Who’re looking for teammates?

3. Last lecture Process: a good way to manage concurrent activities
Address space
Mechanism: Process dispatching
Policy: process scheduling
Dispatcher gains control via periodic timer interrupt
Dispatcher saves process state to PCB on context switch
Dispatcher maintains scheduling queues of processes
Common process operations
Process creation
What are processes
- program in execution
- code + stuff affected
- what can be affected?
- process control block
Process v.s. program
- proc > prog: more than just static code
- prog > proc: one prog can invoke multiple processes
- proc > prog: one prog can have many instantiations
Address space
- workspace
- 1 to 1 mapping of proc <-> addr space
- linear memory, range [0 – N-1], N = 2^32, 2^64
- why good?
Why need them?
- multiprogramming v.s. uniprogramming
Multiprogramming
- policy & mechanism
policy: how to schedule (later)
mechanism: how are processed dispatched (today)
Dispatch diagram
- gaining control. Cooperative v.s. preemptive
- cooperative. trust process to give up CPU. Sys call (sleep())
- preemptive. trust no one (interrupt, timer, device)
- what states must be saved?
- context switch
- how to implement. Code needs registers
- how expensive
process creation
- from scratch
- clone

4. Today Processes (cont.) Processes in Linux Process termination
Interprocess communication
Processes in Linux
Where is the relevant code
task_struct
Context Switch in Linux. switch stack.
all pointer paramters are untrusted and from user.

5. Process Termination
Process executes last statement and asks the operating system to delete it (exit(int status)). In exit():
OS notifies parent process child exit status
Parent gets this status via wait(int* stat_loc)
0: success, non-zero: failure
OS deallocates child’s resources
Processes can be terminated by other processes
E.g. Parent may terminate execution of children processes
Child has exceeded allocated resources
Task assigned to child is no longer required
If parent is exiting
Some operating system do not allow child to continue if its parent terminates: All children terminated - cascading termination

6. Notes on UNIX Process Termination
What if child exits before parent?
Child process becomes a zombie process
Parent must call wait() to “reap” child. OS will notify parent about child’s termination
What if parent exits before child?
Orphaned processes
Re-parented to process 1, the init process
while (1) {
write (1, "$ “, 2);
parse_cmd (command, args); // parse user input
switch(pid = fork ()) {
case -1: perror (“fork”); break;
case 0: // child
execv (command, args, 0); break;
default: // parent
wait (0); break; // wait for child to terminate }

7. Today Processes (cont.) Processes in Linux Process termination
Interprocess communication
Processes in Linux
Relevant files
Data structures
Context switch implementation
all pointer paramters are untrusted and from user.

8. Cooperating Processes
Independent process cannot affect or be affected by the execution of another process.
Cooperating process can affect or be affected by the execution of another process
Advantages of process cooperation
Information sharing
Computation speed-up
Modularity/Convenience

9. Interprocess Communication Models
Message Passing Shared Memory

10. Message Passing v.s. Shared Memory
Why good? Simpler. All sharing is explicit
Why bad? Overhead. Data copying, cross protection domains
Shared Memory
Why good? Performance. Set up shared memory once, then access w/o crossing protection domains
Why bad? Synchronization

11. IPC Example: Unix signals
A very short message: just a small integer
A fixed set of available signals. Examples:
2: SIGINT, sent (usuallly) when you press ctrl+C
9: SIGKILL, to kill a process
11: SIGSEGV, sent when there is a memory error
Send a signal to a process
kill(pid_t pid, int sig)
Signal can be sent by users, kernel, or other processes
What to do when receiving a signal? Installing a handler for a signal
sighandler_t signal(int signum, sighandler_t handler);

12. IPC Example: Unix pipe int pipe(int fd[2]); Operations on pipes:
Returns two file descriptors in fd[0] and fd[1];
Writes to fd[1] will be read on fd[0]
When last copy of fd[1] closed, fd[0] will return EOF
Return 0 on success, -1 on error
Operations on pipes:
read/write/close --- as with files
When fd[1] closed, read(fd[0]) returns 0 bytes
When fd[0] closed, write(fd[1]):
Kills process with SIGPIPE, or if blocked
Failes with EPIPE

13. IPC Example: Unix pipe (cont.)
int pipefd[2];
pipe(pipefd);
switch(pid=fork()) {
case -1: perror("fork"); exit(1);
case 0: close(pipefd[0]);
// write to fd[1]
break;
default: close(pipefd[1]);
// read from fd[0] }
pipe: sets up pipefd
fork: copies file descriptors
In child: closes useless read end pipefd[0]
writes to write end pipefd[1]
In parent: closes useless write end pipefd[0]
reads from read end pipefd[1]

14. IPC Example: Unix Shared Memory
int shmget(key_t key, size_t size, int shmflg);
Create a shared memory segment, and return its id
key: unique identifier of a shared memory segment, or IPC_PRIVATE (means create a new shared mem seg)
void* shmat(int shmid, const void *addr, int flg)
Attach shared memory segment to address space of the calling process. Return a pointer to shared memory
shmid: id returned by shmget()
int shmdt(const void *shmaddr);
Detach from shared memory

15. IPC Example: Unix Shared Memory (cont.)
int id = shmget(IPC_PRIVATE, sizeof(int),
IPC_CREAT | 0666);
int *x = (int*)shmat(id, NULL, 0);
*x = 0;
switch(pid=fork()) {
case -1: perror("fork"); exit(1);
case 0: while(1) { ++*x; sleep(1); }
default: while(1) { printf(“x = %d\n”, *x); sleep(1); } }
Problem: synchronization! (later)
How to share between two processes that are not parent-child?
Key must be agreed upon offline

16. Today Processes (cont.) Processes in Linux Process termination
Interprocess communication
Processes in Linux
Process data structures
Process operations: fork() and exit()
Context switch implementation
Linux source tree
PCB: task_struct
fields
kernel stack
each thread has a kernel stack
how to find kernel stack?
tr -> entry in gdt -> stack
how to find task_struct?
thread_info -> task_struct
assembly code

17. Find process info: /proc/<pid>
ps to get process id
For each process, there is a corresponding directory /proc/<pid> to store this process information in the /proc pseudo file system

18. Process-related files
Header files
include/linux/sched.h – declarations for most task data structures
include/linux/wait.h – declarations for wait queues
include/asm-i386/system.h – architecture-dependent declarations
Source files
kernel/sched.c – task scheduling routines
kernel/signal.c – signal handling routines
kernel/fork.c – process/thread creation routines
kernel/exit.c – process exit routines
fs/exec.c – executing program
arch/i386/kernel/entry.S – kernel entry points
arch/i386/kernel/process.c – architecture-dependent process routines
We’ll use not the latest kernel, but matches ULK book
include/ hearders
include/linux arch-independent
Include/asm-* arch-dependent
kernel/ core os code. e.g. scheduler, processes
arch/* arch-dependent code

19. Linux: Processes or Threads?
Linux uses a neutral term: tasks
Tasks represent both processes and threads
Threads = tasks that share AS data structures
When processes trap into the kernel, they share the Linux kernel’s address space  kernel threads
Task data structure
task_struct: process control block
kernel stack: work space for systems calls (the kernel executes on the user process’s behalf) or interrupt handlers

20. Process Control Block in Linux
task_struct (process descriptor in ULK)
include/linux/sched.h
Each task has a unique task_struct

21. Task States: state
TASK_RUNNING – the thread is running on the CPU or is waiting to run
TASK_INTERRUPTIBLE – the thread is sleeping and can be awoken by a signal (EINTR)
TASK_UNINTERRUPTIBLE – the thread is sleeping and cannot be awakened by a signal
TASK_STOPPED – the process has been stopped by a signal or by a debugger
TASK_TRACED – the process is being traced via the ptrace system call
include/linux/sched.h

22. Exit States
EXIT_ZOMBIE – the process is exiting but has not yet been waited for by its parent
EXIT_DEAD – the process has exited and has been waited for

23. Process IDs process ID: pid thread group ID: tgid
pid of first thread in process
getpid() returns this ID, so all threads in a process share the same process ID
many system calls identify a process by its PID
Linux kernel uses pidhash to efficiently find processes by pids
(see include/linux/pid.h, kernel/pid.c)

24. Other PCB data structures
user: user_struct – per-user information (for example, number of current processes)
mm, active_mm: mm_struct – memory areas for the process (address space)
fs: fs_struct – current and root directories associated with the process
files: files_struct – file descriptors for the process
signal: signal_struct – signal structures associated with the process

25. Process Relationships
Processes are related: children, sibling
Parent/child (fork()), siblings
Possible to "re-parent"
Parent vs. original parent
Parent can "wait" for child to terminate
Process groups: signal_struct->pgrp
Possible to send signals to all members
Sessions: signal_struct->session
Processes related to login

26. How Linux manages processes
In order for Linux to efficiently manage the scheduling of its various ‘tasks’, separate queues are maintained for ‘running’ tasks and for tasks that temporarily are ‘blocked’ while waiting for a particular event to occur (such as the arrival of new data from the keyboard, or the exhaustion of prior data sent to the printer)
These queues are implemented using doubly-linked list (struct list_head in include/linux/list.h)

27. Some tasks are ‘ready-to-run’
init_task list
run_queue
Those tasks that are ready-to-run comprise a sub-list of all the tasks,
and they are arranged on a queue known as the ‘run-queue’
(struct runqueue in kernel/sched.c)
Those tasks that are blocked while awaiting a specific event to occur
are put on alternative sub-lists, called ‘wait queues’, associated with
the particular event(s) that will allow a blocked task to be unblocked
(wait_queue_t in include/linux/wait.h and kernel/wait.c)

28. Kernel Wait Queues
wait_queue_head_t can have 0 or more wait_queue_t chained onto them
However, usually just one element
Each wait_queue_t contains a list_head of tasks
All processes waiting for specific "event“
Used for timing, synch, device i/o, etc.
wait_queue_t
wait_queue_head_t
waitqueue
waitqueue
waitqueue
waitqueue

29. Kernel stack
Each process in Linux has two stacks, a user stack and a kernel stack (8KB by default)
Kernel stack can only be accessed in kernel mode
Interrupt and trap handlers run on kernel stack
User stack cannot be trusted
Q1: switching address spaces is costly. Can we avoid this overhead when entering kernel mode from user mode?
Q2: how does the hardware find the current task’s kernel stack ?

30. Q1: A task’s virtual-memory layout4G
User space
Kernel space
process descriptor
and
kernel-mode stack
kernel mode 3G
User-mode stack-area
Kernel space is also mapped into user space  from user mode to kernel mode, no need to switch address spaces
Protection?
Kernel space is only accessible when mode bit = 0
user mode
Shared runtime-libraries
Task’s code and data
0xC

31. Q2: Finding current task’s kernel stack (on x86)
Global Descriptor Table
initialized in startup_32 in arch/i386/boot/compressed/head.S
CPU0 tr
kern stack top
esp
CPU0
Changes on each context switch (__switch_to in arch/i386/kernel/process.c)
Task’s kernel-stack
Hardware retrieves kernel stack top and load it into %esp, also saves previous %esp, for return to user mode
8-KB
Still need to find task_struct !
push %eax  movl %eax, [esp], esp = esp - 4
pop %eax  movl [esp], %eax, esp = esp + 4
Per-cpu task segment descriptor in global descriptor table (setup at boot time)
Task segment descriptor has a pointer to currernt task’s stack

32. Connections between task_struct and kernel stack
Linux uses part of a task’s kernel-stack to store a structure thread_info
thread_info contains low-level data that low-level code (e.g. entry.S) can immediate access, and a pointer to the task’s task_struct
0xe
esp
Task’s kernel-stack
struct task_struct
8-KB
Task’s
process-descriptor
Task’s thread-info
0xe800e000
8KB aligned

33. How to find thread_info?
movl $0xFFFFE000, %eax
andl %esp, %eax (mask out last 13 bits)
0xe
esp
Task’s kernel-stack
struct task_struct
8-KB
Task’s
process-descriptor
Task’s thread-info
0xe800e000
8KB aligned

34. How to find thread_info? (cont)
Macro current_thread_info implements this computation
current macro yields the task_struct of current task
include/asm-i386/current.h
Why good?
Fast ! 2 instructions to find current from %esp
current is not a static variable, useful for SMP

35. Today Processes (cont.) Processes in Linux Process termination
Interprocess communication
Processes in Linux
Process data structures
Process operations: fork() and exit()
Context switch implementation
Linux source tree
PCB: task_struct
fields
kernel stack
each thread has a kernel stack
how to find kernel stack?
tr -> entry in gdt -> stack
how to find task_struct?
thread_info -> task_struct
assembly code

36. fork() call chain libc fork() system_call (arch/i386/kernel/entry.S)
sys_clone() (arch/i386/kernel/process.c)
do_fork() (kernel/fork.c)
copy_process() (kernel/fork.c)
p = dup_task_struct(current) // shallow copy
copy_* // copy point-to structures
copy_thread () // copy stack, regs, and eip,
// and set child return value
// to 0 via
// childregs->eax = 0;
wake_up_new_task() // set child runnable

37. exit() call chain libc exit(code)
system_call (arch/i386/kernel/entry.S)
sys_exit() (kernel/exit.c)
do_exit() (kernel/exit.c)
exit_*() // free data structures
exit_notify() // tell other processes we exit
// reparent children to init
// EXIT_ZOMBIE
// EXIT_DEAD

38. Today Processes (cont.) Processes in Linux Process termination
Interprocess communication
Processes in Linux
Process data structures
Process operations: fork() and exit()
Context switch implementation
Linux source tree
PCB: task_struct
fields
kernel stack
each thread has a kernel stack
how to find kernel stack?
tr -> entry in gdt -> stack
how to find task_struct?
thread_info -> task_struct
assembly code

39. Context switch call chain
schedule() (kernel/sched.c) (talk about scheduling later)
context_switch()
swtich_mm (include/asm-i386/mmu_context.h)
switch address space
switch_to (include/asm-i386/system.h)
switch stack, regs, and %eip
__swtich_to (arch/i386/kernel/process.c)

40. Context switch by stack swtich: the idea
Kernel stack captures process states
Registers
Task_struct through thread_info
Changing the stack pointer changes the process
Task’s kernel-stack
Task’s
process-descriptor
Task’s thread-info

41. Context switch by stack switch: the implementation (simplified)
swtich_to(p0,p1)
save registers on stack
p0->esp = %esp
p0->eip = ret_addr;
%esp = p1->esp;
push p1->eip;
ret
ret_addr:
pop registers from stack
P0 stack
P1 stack
eax …
eax …
ret_addr
thread_info
thread_info
esp
eip
esp
eip
p0->eip = ret_addr
esp
eip
eax …
CPU