1. Chapter 5 CPU Scheduling
Basic Concepts
Scheduling Criteria
Scheduling Algorithms
Thread Scheduling
Multiple-Processor Scheduling
Real-Time CPU Scheduling
Operating-System Examples
Algorithm Evaluation
ALL COMPUTER RESOURCES ARE SCHEDULED BEFORE USE

2. 5.1 Basic Concepts
Types of scheduling

4. Maximum CPU utilization obtained with multiprogramming
5.1 Basic Concepts
Maximum CPU utilization obtained with multiprogramming
Overlap computation with I/O waiting
CPU-I/O burst cycle
Process execution consists of a cycle of CPU execution and I/O wait
CPU burst distribution
Depends on process and computer

5. 5.1 Basic Concepts: Alternating Sequence of CPU and I/O Bursts
1st CPU burst
I/O bound: many very short CPU bursts
CPU bound: a few very long CPU bursts
Last, CPU burst

6. 5.1 Basic Concepts: Histogram of CPU-Burst Times
Many short CPU bursts
A few long CPU bursts
All processes on different computers show similar curve
Curve: exponential or hyperexponential

7. 5.1 Basic Concepts: CPU Scheduler
Selects from among the processes in memory that are ready to execute, and allocates the CPU to one of them
CPU scheduling decisions may take place when a process …
Switches from running to waiting state
Select one (if any) from the Ready queue
2. Switches from running to ready state
3. Switches from waiting to ready
Terminates

8. 5.1 Basic Concepts: CPU Scheduler
Scheduling under 1 and 4 is nonpreemptive (不可插隊)
A process gives up the CPU voluntarily
This scheduling does not require special hardware support
All other scheduling is preemptive
OS forces the process to give up the CPU
Requires special hardware support (例如 timer)
Costly:
A process is preempted while updating shared data and the cooperating process gets the CPU

9. CPU Scheduling vs. Process State Diagram
New
Admitted
Terminated
Interrupt
(4)
(2)
Exit
Ready
Running
Preemptive
Non-Preemptive
(3)
(1)
I/O or event completion
Waiting
I/O or event wait

10. 5.1 Basic Concepts: CPU Scheduler (Preemptive Scheduling)
All other scheduling is preemptive (續)
Effects the design of the OS kernel
Kernel is busy with an activity on behalf of a process
Changing kernel data (e.g. I/O queue)
The process is preempted in the middle of these changes
Kernel needs to read/modify the same structure
CHAOS could ensue (接踵而來)

11. 5.1 Basic Concepts: CPU Scheduler (Preemptive Scheduling)
All other scheduling is preemptive (續)
Solutions to chaos
Wait until System Call completes or I/O block take place before a context switch
Simple kernel structure
Not good for real-time computing and multiprocessing
OS needs to accept interrupts at almost all times
Sections of code must be guarded from simultaneous use

12. 5.1 Basic Concepts: Dispatcher
Dispatcher module gives control of the CPU to the process selected by the short-term scheduler
This involves:
Switching context
Switching to user mode
Jumping to the proper location in the user program to restart that program
Dispatch latency
Time taken for the dispatcher to stop one process and start another running

13. 5.2 Scheduling Criteria
CPU utilization (利用率) – keep CPU as busy as possible
Utilization in real systems ranges from 40% to 90%
Throughput – # of completed processes per time unit
Turnaround time – time from submission to completion
Waiting time in ready queue + CPU burst time + I/O time
Waiting time – amount of time a process has been waiting in the Ready queue
Response time – amount of time it takes from when a request was submitted until the first response is produced, not output (for time-sharing environment)
The amount of time it takes to responding, but not the time it takes to output that response

14. 5.2 Scheduling Criteria: Optimization Considerations
Max CPU utilization
Max throughput
Min turnaround time
Min waiting time
Min response time
Minimize average response time vs. minimize variance in response time?

15. 5.3 Scheduling Algorithms
First-come, first-served (FCFS) scheduling
Shortest-job-first scheduling
Priority scheduling
Round-robin scheduling
Multilevel queue scheduling
Multilevel feedback queue scheduling

16. 5.3 Scheduling Algorithms: FCFS Scheduling
Process Burst Time
P1 24
P2 3
P3 3
Suppose that processes arrive in the order: P1 , P2 , P3 The Gantt Chart for the schedule:
Waiting time for P1 = 0; P2 = 24; P3 = 27
Average waiting time: ( )/3 = 17
FCFS scheduling is non-preemptive (不可插隊)
Henry Gantt, P1 P2 P3 24 27 30

17. 5.3 Scheduling Algorithms: FCFS Scheduling
Suppose that processes arrive in the order P2 , P3 , P1
The Gantt chart for the schedule:
Waiting time for P1 = 6; P2 = 0; P3 = 3
Average waiting time: ( )/3 = 3
Much better than the previous case
Convoy effect (short process behind long process)
All the I/O-bound processes wait for a CPU-bound process to get off the CPU
Lower CPU and device utilization P1 P3 P2 6 3 30

18. FCFS Scheduling: The Convoy Effect
Ready queue
CPU
IO-bound
IO-bound
CPU-
bound
IO-bound
Ready queue
IO-bound
CPU-
bound
CPU
IO-bound
IO-bound
CPU
IO-bound
IO-bound
I/O
IO-bound
CPU-
bound
I/O
I/O
queue

19. 5.3 Scheduling Algorithms: Shortest-Job-First Scheduling
Associate with each process the length of its next CPU burst. Use these lengths to schedule the process with the shortest time
Two schemes
Nonpreemptive – once a process is given CPU, it cannot be preempted until completed
Preemptive – if a new process arrives with CPU burst length less than remaining time of current executing process, preempt  Shortest-Remaining-Time-First (SRTF)
SJF is optimal – min. average waiting time for a given set of processes
Suitable for long-term scheduler

20. Example of Non-Preemptive SJF
Process Arrival Time Burst Time P P P P
SJF (non-preemptive)
Average waiting time = ( )/4 = 4 P1 P3 P2 7 3 16 P4 8 12

21. Example of Preemptive SJF
Process Arrival Time Burst Time P P P P
SJF (preemptive)
Average waiting time = ( )/4 = 3 P1 P3 P2 4 2 11 P4 5 7 16

22. Shortest-Job-First Scheduling: Determining the Length of Next CPU Burst
Can only estimate the length
Can be done by using the length of previous CPU bursts, using exponential averaging

23. Shortest-Job-First Scheduling: Prediction of the Length of the Next CPU Burst
 = 0.5

24. Examples of Exponential Averaging
若  = 0，則
n+1 = n (recent history does not count)
若  = 1，則
n+1 = tn (only the actual last CPU burst counts)
Expanding the formula, we have n+1 =  tn+(1-)  tn-1 +…+(1-)j  tn-j +…+(1 - )n+1 0
Since both  and (1 - ) ≦ 1, each successive term has less weight than its predecessor

25. 5.3 Scheduling Algorithms: Priority Scheduling
Priority number associated with each process
CPU is allocated to the process with the highest priority (smaller integer indicates higher priority)
Preemptive, nonpreemptive
SJF is a priority scheduling where priority is the predicted next CPU burst time
Starvation (indefinite blocking) problem
Low priority processes may never execute
Solution: aging technique
Gradually increase the priority of processes waiting in the system for a long time

26. 5.3 Scheduling Algorithms: Round Robin (RR)
Each process gets a small unit of CPU time (time quantum), usually ms. After this time has elapsed, the process is preempted and added to the Ready queue
Given n processes in the Ready queue and the time quantum q, each process gets 1/n of the CPU time in chunks of at most q time units at once. No process waits more than (n-1)q time units
Performance
q large  FCFS
q small  q must be large with respect to context switch, otherwise overhead is too high
Processor sharing: as if there are n processors with 1/n times slower than original

27. Example of RR (Time Quantum = 20)
Process Burst Time
P1 53
P2 17
P3 68
P4 24
Gantt chart
Typically, higher average turnaround than SJF, but better response P1 P2 P3 P4 20 37 57 77 97
117
121
134
154
162

28. Time Quantum and Context Switch Time
Small quantum  frequent context switch
Time = 9 x ContextSwitch + 10

29. Turnaround Time Varies with Time Quantum
Rule of thumb: 80% of the CPU bursts should be shorter than the time quantum

30. 5.3 Scheduling Algorithms: Multilevel Queue
Processes are classified into different groups
By different response-time requirements
Ready queue partitioned into interactive (foreground) or batch (background) queues
Each queue has own scheduling algorithm
Foreground, RR
Background, FCFS

31. 5.3 Scheduling Algorithms: Multilevel Queue
Scheduling must be done between the queues
Fixed priority scheduling
Serve foreground processes and then background processes
In fixed-priority preemptive scheduling, no process in batch queue can be run unless all higher-priority queues are empty  possibility of starvation
Time slice
Each queue gets a certain amount of CPU time which it can schedule among its processes
例: 80% foreground in RR; 20%background in FCFS

32. 5.3 Scheduling Algorithms: Multilevel Feedback Queue
A process can move between the various queues
Using aging technique: a process waiting too long in the lower-priority queue is moved to a higher-priority queue
Basic idea
Separate processes with different CPU burst characteristics
Too much CPU time  move to lower-priority queue
I/O bound and interactive processes in the higher-priority queues

33. 5.3 Scheduling Algorithms: Multilevel Feedback Queue
Multilevel-feedback-queue scheduler is defined by the following parameters
Number of queues
Scheduling algorithms for each queue
Method to determine when to upgrade a process
Method to determine when to demote a process
Method used to determine which queue a process will enter when that process needs service

34. Example of Multilevel Feedback Queue (1/2)
Three queues
Q0 – time quantum 8 ms
Q1 – time quantum 16 ms
Q2 – FCFS
Scheduling
A new job enters Q0 which is served FCFS. When it gains CPU, job receives 8 ms. If it does not finish in 8 ms, job is moved to Q1
At Q1 job is again served FCFS and receives 16 additional ms. If it still does not complete, it is preempted and moved to Q2

35. Example of Multilevel Feedback Queue (2/2)
Highest priority to any process with a CPU burst of 8 ms
(finish CPU burst and start I/O burst immediately)
Process needing more than 8 but less than 24 ms is served quickly (goes second level)
Long processes sink the lowest queue

36. 5.4 Thread Scheduling
不列入考試範圍

37. 5.5 Multiple-Processor Scheduling
CPU scheduling becomes complex when multiple CPUs are available
This text concentrates on homogeneous processor system
Load sharing
A common Ready queue, rather than a separate queue for each processor
All processes go into one queue and are scheduled onto any available processor
Two approaches
Each processor is self-scheduling (needs to access/update common data structure)
Appoint one processor as scheduler for the other processes  master-slave structure

38. 5.5 Multiple-Processor Scheduling
Asymmetric multiprocessing
All system activities are handled by one specific processor. Others only execute user code

39. 5.5 Multiple-Processor Scheduling
Processor affinity (姻親、親和力、吸引力)
Process has affinity for processor on which it is currently running
例：Consider what happens to cache memory when a process has been running on a specific processor. The data most recently accessed by the process populate the cache for the processor.
Most symmetric multiprocessing systems try to avoid migration of processes from one processor to another and instead attempt to keep such processes running on the same processor.
Soft affinity vs. hard affinity

40. 5.5 Multiple-Processor Scheduling
Processor affinity (續)
The main-memory architecture of a system can affect processor affinity issues.
例：Non-uniform memory access (NUMA)

41. 5.5 Multiple-Processor Scheduling
Multicore processor
Recent trend has been to place multiple processor cores on same physical chip
Each core has a register set to maintain its architectural state and thus appear to the OS to be a separate physical processor
Faster and consume less power
Multiple threads per core also growing
Takes advantage of memory stall to make progress on another thread while memory retrieve happens

42. 5.5 Multiple-Processor Scheduling
Multicore processor (續)
Memory stall: When a processor accesses memory, it spends a significant amount of time waiting for the data to become available
Recent designs have two (more) hardware threads assigned to each core. If one thread stalls while waiting for memory, the core switches to another thread

43. 5.6 Real-Time Scheduling: Hard Real-Time Systems
Required to complete a critical task within a guaranteed amount of time
Need resource reservation for admission control
Provide the amount of time in which the process needs to complete or perform I/O
Scheduler admits if the deadline can be met; otherwise reject
Scheduler knows exactly how long each type of OS function takes to perform

44. Real-Time Scheduling: Soft Real-Time Computing
Requires that critical processes receive priority over less fortunate ones
May cause longer delay or starvation
Support multimedia, high-speed interactive graphics on a general purpose computer
Scheduler design
System has priority scheduling and real-time processes have the non-degrading highest priority
Dispatch latency must be small
Waiting for a system call or for an I/O block
Allow preemptive system calls – preemption points

45. Real-Time Scheduling: Preemptible Kernel
What happens if the higher-priority process needs to read/modify kernel data that are currently accessed by a lower-priority process?
Higher-priority process would be waiting for a lower-priority one to finish (priority inversion)
Priority inversion can be solved by the priority inheritance protocol
All processes accessing resources that the high-priority process needs inherit the high priority until they are done with the resource
Upon release of the resource, they are set to their original priority

46. Real-Time Scheduling: Dispatch Latency
Solaris: preemptive dispatch latency = 2 ms. non-preemptive > 100 ms
Conflict phase: (1) preemption of any process running in the kernel;
(2) release by low-priority processes resources needed by the high priority process

47. Scheduling of a Real-Time Process*

48. 5.7 Operating System Examples
不列入考試範圍

49. How to evaluate a selected algorithm?
5.8 Algorithm Evaluation
Criteria to select a CPU scheduling algorithm may include several measures, such as:
Maximize CPU utilization under the constraint that the maximum response time is 1 second
Maximize throughput such that turnaround time is (on average) linearly proportional to total execution time
How to evaluate a selected algorithm?
Deterministic modeling
Queueing models
Simulations
Implementation

50. 5.8 Algorithm Evaluation: Deterministic Modeling
Takes a particular predetermined workload and defines the performance of each algorithm for that workload
Simple and fast
Gives the exact numbers, allows the algorithms to be compared
Requires exact numbers of input, and its answers apply to only those cases
In general, deterministic modeling is too specific and requires too much exact knowledge to be useful

51. Deterministic Modeling Example
Process Burst Time P P P P P
FCFS P1 P2 P3 P4 P5
Average Waiting Time = 28 ms
SJF P3 P4 P1 P5 P2
Average Waiting Time = 13 ms RR
(q=10) P1 P2 P3 P4 P5 P2 P5 P2
Average Waiting Time = 23 ms

52. 5.8 Algorithm Evaluation: Queueing Models
Processes running on many systems vary from day to day
There is no static set of processes (and times) to use for deterministic modeling
What can be determined?
The distribution of CPU and I/O bursts
A mathematical formula describing the probability of a particular CPU burst
The distribution of processes arrival time
These distribution may be measured and then approximated or simply estimated

53. 5.8 Algorithm Evaluation: Queueing Models
Computer system is described as a network of servers
Each server has a queue of waiting processes
CPU is a server with its ready queue
Queuing-network analysis
Given
Arrival rates and service rates
To compute
Utilization, average queue length, average waiting time, etc.

54. 5.8 Algorithm Evaluation: Queueing Models
Little‘s formula (或稱Little’s rule)
n : average queue length
W: average waiting time in the queue
 : average arrival rate for new processes
We can use this formula to compute one of the three variables, if the other two are known
Useful in comparing scheduling algorithms, but has limitations
Arrival and service distributions are often defined in unrealistic but mathematically tractable ways
John Little, 1928~
n =  × W
process arrival rate
≒ process departure rate

55. 5.8 Algorithm Evaluation: Simulations
Simulations get a more accurate evaluation of scheduling algorithms
Distribution-driven simulation
Randomly generates data
The distribution may be defined mathematically or empirically
Simulation with trace tapes
A trace tape can be created by monitoring the real system, recording the sequence of actual events

56. 5.8 Algorithm Evaluation: Simulations