1. Managing Processors Jeff Chase Duke University

2. The story so far: protected CPU mode user mode kernel mode kernel “top half” kernel “bottom half” (interrupt handlers) syscall trap u-startu-returnu-start fault u-return fault clock interrupt interrupt return Kernel handler manipulates CPU register context to return to selected user context. Any kind of machine exception transfers control to a registered (trusted) kernel handler running in a protected CPU mode.

3. The kernel syscall trap/returnfault/return interrupt/return The handler accesses the core register context to read the details of the exception (trap, fault, or interrupt). It may call other kernel routines. Every entry to the kernel is the result of a trap, fault, or interrupt. The core switches to kernel mode and transfers control to a handler routine. OS kernel code and data for system calls (files, process fork/exit/wait, pipes, binder IPC, low-level thread support, etc.) and virtual memory management (page faults, etc.) I/O completionstimer ticks

4. synchronous caused by an instruction asynchronous caused by some other event intentional happens every time unintentional contributing factors trap: system call open, close, read, write, fork, exec, exit, wait, kill, etc. fault invalid or protected address or opcode, page fault, overflow, etc. interrupt caused by an external event: I/O op completed, clock tick, power fail, etc. “software interrupt” software requests an interrupt to be delivered at a later time Exceptions: trap, fault, interrupt Every entry to the kernel is the result of a trap, fault, or interrupt. The core sets its mode to protected kernel mode and transfers control to the corresponding handler.

5. Kernel Stacks and Trap/Fault Handling data Processes execute user code on a user stack in the user virtual memory in the process virtual address space. Each process has a second kernel stack in kernel space (VM accessible only to the kernel). stack System calls and faults run in kernel mode on the process kernel stack. syscall dispatch table Kernel code running in P’s process context (i.e., on its kstack) has access to P’s virtual memory. The syscall handler makes an indirect call through the system call dispatch table to the handler registered for the specific system call.

6. The kernel syscall trapsfaults interrupts sleep queueready queue A trap or fault handler may suspend (sleep) the current thread, leaving its state (call frames) on its kernel stack and a saved context in its TCB. The TCB for a blocked thread is left on a sleep queue for some synchronization object. A later event/action may wakeup the thread.

7. Thread states and transitions running readyblocked Scheduler governs these transitions. wait, STOP, read, write, listen, receive, etc. sleep STOP wait wakeup Sleep and wakeup are internal primitives. Wakeup adds a thread to the scheduler’s ready pool: a set of threads in the ready state. yield

8. Contention on ready thread queues A large-scale system may have significant contention on the spinlock for the ready thread queue. – Each core removes a thread from the ready queue with GetNextThreadToRun() on each context switch. – Every wakeup adds a thread to the ready queue. On average, the frequency of these events is linear with the number of cores. – What is the average wait time for the spinlock? To reduce contention, an OS may have a separate run queue for each machine partition. – Each queue serves a partition of N cores.

9. Per-CPU ready queues Let’s talk about priority… lock per runqueue preempt on queue insertion recalculate priority on expiration

10. CPU dispatch and ready queues In a typical OS, each thread has a priority, which may change over time. When a core is idle, pick the (a) thread with the highest priority. If a higher-priority thread becomes ready, then preempt the thread currently running on the core and switch to the new thread. If the quantum expires (timer), then preempt, select a new thread, and switch

11. Priority Most modern OS schedulers use priority scheduling. – Each thread in the ready pool has a priority value. – The scheduler favors higher-priority threads. – Threads inherit a base priority from the associated application/process. – User-settable relative importance within application – Internal priority adjustments as an implementation technique within the scheduler. – How to set the priority of a thread? How many priority levels? 32 (Windows) to 128 (OS X)

12. Per-CPU ready queues On most architectures, a find-first-bit-set instruction is used to find the highest priority bit set in one of five 32- bit words (for the 140 priorities).