1. Process Management

2. Processes and threads
A process is a program (object code stored on some media) in execution.
A program itself is not a process; a process is an active program and related resources.
Threads are the objects of activity within the process.
To Linux, a thread is just a special kind of process.
The clone system call – creating a thread

3. System calls (Linux) fork() & vfork() & clone() exec() exit() wait4()
to create a process/thread
exec()
to create a new address space and load a new program into it.
exit()
wait4()
to enable a process to wait for the termination of a specific process

4. Process control block (task_struct, thread_info)
The task_struct structure is allocated via the slab allocator.
To support “4k task_struct“, thread_info was created that lives at the bottom of the stack.

5. thread_info

6. task_struct
The process descriptor (task_struct) contains all the information about a specific process.

7. task_struct State and execution information
such as pending signals, binary format, pid, pointers to parents and other related processes, priorities, and CPU time
Information on allocated virtual memory.
Process credentials such as user and group ID, capabilities.
Files used: filesystem information on all files handled by the process must be saved.
Thread information, which records the CPU-specific
Information on interprocess communication (IPC)
Signal handlers used by the process.

8. task_struct and thread_info

9. Identifying the current process
current is calculated by masking out the 13 (depends on the stack size) least significant bits of the stack pointer to obtain the thread_info structure.
8KB

10. linux/include/asm-x86_64/pda.h
In AMD64 and IA-32 architectures, each CPU in the system has a per-processor private data area which holds various interesting items of information. For AMD64, it is defined as follows:

11. linux/include/asm-x86_64/pda.h
The segment selector register gs always points to the data structure, and elements of it can therefore be simply addressed as offsets to the segment. This structure includes the pcurrent pointer that points to the task_struct instance of the current process.

12. Process State The process becomes runnable if it receives a signal.
the task does not respond to signals in this state

13. Process Context Normal program execution occurs in user-space.
When a program executes a system call or triggers an exception (e.g., segmentation fault), it enters kernel-space.
the kernel is said to be "executing on behalf of the process" and is in process context.
Interrupt context, on the other hand, is not associated with a process.

14. The Process Family Tree
All processes are descendants of the init process, whose PID is 1.

15. linked list

16. The task list
for_each_process(task) { /* this pointlessly prints the name and PID of each task */ printk("%s[%d]\n", task->comm, task->pid); }

17. fork, vfork, clone
int sys_fork(struct pt_regs *regs) { return do_fork(SIGCHLD, regs->sp, regs, 0, NULL, NULL); } int sys_vfork(struct pt_regs *regs) return do_fork(CLONE_VFORK | CLONE_VM | SIGCHLD, regs->sp, regs, 0, NULL, NULL); int sys_clone(unsigned long clone_flags, unsigned long newsp, void __user *parent_tid, void __user *child_tid, struct pt_regs *regs) if (!newsp) newsp = regs->sp; return do_fork(clone_flags, newsp, regs, 0, parent_tid, child_tid);

18. Note: __user
The kernel uses __user to identify pointers to areas in user address space
The kernel needs to ensure that the page frame in RAM that backs the destination is actually present
Reference:
/include/linux/compiler.h

19. clone & libc
CLONE(2) Linux Programmer's Manual CLONE(2) NAME clone, __clone2 - create a child process SYNOPSIS #define _GNU_SOURCE /* See feature_test_macros(7) */ #include <sched.h> int clone(int (*fn)(void *), void *child_stack, int flags, void *arg, ... /* pid_t *ptid, struct user_desc *tls, pid_t *ctid */ ); DESCRIPTION clone() creates a new process, in a manner similar to fork(2). It is actually a library function layered on top of the underlying clone() system call, hereinafter referred to as sys_clone. A description of sys_clone is given towards the end of this page.

20. do_fork clone_flags: A flag set to specify duplication properties.
long do_fork(unsigned long clone_flags, unsigned long stack_start,
struct pt_regs *regs, unsigned long stack_size,
int __user *parent_tidptr, int __user *child_tidptr)
clone_flags: A flag set to specify duplication properties.
stack_start: The start address of the user mode stack to be used.
regs: holding all registers in the order in which they are saved on the parent’s kernel stack when a system call is executed
parent_tidptr and child_tidptr: two pointers to addresses in userspace that hold the TIDs of the parent and child processes.

21. Code flow diagram for do_fork

22. copy_process
/* Perform scheduler related setup. Assign this task to a CPU. */

23. flags (CLONE_XXX & copy_XXX)

24. flags (CLONE_XXX & copy_XXX)

25. Kernel Threads
kernel threads do not have an address space (in fact, their mm pointer is NULL).
Kernel threads are, however, schedulable and preemptable as normal processes.
Example: pdflush task and the ksoftirqd task.
A kernel thread can be created only by another kernel thread.
int kernel_thread(int (*fn)(void *), void * arg, unsigned long flags)

26. the kernel threads
run the command ps –ef. You should see something like this…
Kernel process names are surrounded by square brackets ([]).
UID PID PPID C STIME TTY TIME CMD
root Jan04 ? :00:00 /sbin/init
root Jan04 ? :00:00 [kthreadd]
root Jan04 ? :00:02 [ksoftirqd/0]
root Jan04 ? :00:00 [kworker/u:0]
root Jan04 ? :00:00 [migration/0]
root Jan04 ? :00:00 [migration/1]
root Jan04 ? :00:00 [ksoftirqd/1]
root Jan04 ? :00:02 [kworker/0:1]

27. ksoftirqd
ksoftirqd is used to process deferrable “driver functions.”

28. kworker
The largest part of the actual processing for the kernel is performed by “kworker”
and particularly in the case, where there are timers, input/output, interrupts, etc.
And they normally keep up a correspondence to the greater part of any allocated “system” time to run processes

29. Starting New Programs
New programs are started by replacing an existing program with new code.
Linux provides the execve system call for this purpose
execve() ->...->sys_execve() -> do_execve()
<-user mode----><-kernel mode >

30. do_execve filename: the executable file
int do_execve(char * filename,
char __user *__user *argv,
char __user *__user *envp,
struct pt_regs * regs)
filename: the executable file
argv & envp: The argument vector and environment
regs: holding all registers in the order in which they are saved on the kernel stack when the system call is executed

31. Note: __user
The kernel uses __user to identify pointers to areas in user address space
The kernel needs to ensure that the page frame in RAM that backs the destination is actually present
Reference:
/include/linux/compiler.h

32. Code flow diagram for do_execve

33. mm_alloc
/linux/mm/slab.c#L3645

34. bprm_init
call mm_alloc to generate a new instance of mm_struct to manage the process address space.
call init_new_context to initial architecture-specific functions that initializes the process/program
call bprm_mm_init to set up an initial stack.

35. Prepare_binprm int prepare_binprm(struct linux_binprm *bprm)
Various parameters of the new process are, for the sake of simplicity, combined into a structure of type linux_binprm (that is bprm).
including:
a number of parent process values
SUID and SGID bits

36. search_binary_handler
search_binary_handler is used at the end of do_execve to find a suitable binary format for the particular file.
each format can be recognized by reference to special characteristics (usually a ‘‘magic number‘‘ at the beginning of the file).

37. Magic number
Compiled Java class files (bytecode) start with hex 0xCAFEBABE.
When compressed with Pack200 the bytes are changed to 0xCAFED00D.

38. Note: Magic number
readelf -h ls ELF Header: Magic: 7f 45 4c Class: ELF64 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: EXEC (Executable file) Machine: Advanced Micro Devices X86-64 Version: 0x1 Entry point address: 0x4026e0 Start of program headers: 64 (bytes into file) Start of section headers: (bytes into file)

39. search_binary_handler
It releases all resources used by the old process.
It maps the application into virtual address space.
text, pre-initialized data, heap, stack
It sets the instruction pointer of the process and some other architecture-specific registers

40. Interpreting Binary Formats
<binfmts.h> struct linux_binfmt { struct linux_binfmt * next; struct module *module; int (*load_binary) (struct linux_binprm *, struct pt_regs * regs); int (*load_shlib) (struct file *); int (*core_dump) (long signr, struct pt_regs * regs, struct file * file); unsigned long min_coredump; /* minimal dump size */ };

41. Process termination
set the PF_EXITING flag in the flags member of the task_struct.
calls del_timer_sync() to remove any kernel timers (that are software timers).
calls __exit_mm() to release the mm_struct
calls exit_sem(). If the process is queued waiting for an IPC semaphore, it is dequeued here.

42. Process termination
calls __exit_files(), __exit_fs(), exit_namespace(), and exit_sighand() to decrement the usage count of objects
sets the task's exit code, stored in the exit_code member of the task_struct
calls exit_notify() to send signals to the task's parent
calls schedule()

43. Removal of the Process Descriptor
After the parent has obtained information on its terminated child (by invoking wait4), or signified to the kernel that it does not care, the child's task_struct is deallocated.

44. Summary
how Linux stores and represents processes (with task_struct and thread_info),
how processes are created (via clone() and fork()),
how new executable images are loaded into address spaces (via the exec()),
how parents gather information about their children (via the wait4())