1 Using Hierarchical Scheduling to Support Soft Real-Time Applications in General-Purpose Operating Systems John Regehr March 20, 2001
2 Outline Motivation and Approach Guarantees HLS Design Augmented Reservations Conclusions and Demo
3 Overview of Contributions A general hierarchy of soft real-time schedulers can provide guaranteed scheduling behavior The Hierarchical Loadable Scheduler (HLS) architecture… Can be efficiently implemented in a general- purpose OS Increases application performance compared to case where scheduler and apps are mismatched Augmented CPU reservations increase predictability when the OS steals CPU time from real-time applications
4 Motivation People use general-purpose OSs (GPOSs) for many kinds of tasks e.g. Unix, Windows, MacOS variants Compatibility, commodity, convenience Applications have diverse scheduling requirements Time-sharing Soft real-time Isolation Co-scheduling between processors or machines
5 The Problem One size does not fit all Supporting evidence: companies sell scheduler extensions Ensim: ServerXchange Sun: Solaris Resource Manager Aurema: Active Resource Management TimeSys: Linux/RT
6 Motivating Data: Unfair CPU Allocation Between Users User 1 gets little CPU time when User 2 creates many threads N1N1 % 1 00 N 2 %
7 Approach Allow a hierarchy of CPU schedulers to control processor allocation Thesis Statement: It is useful and feasible to extend a GPOS with a general, heterogeneous scheduling hierarchy. A heterogeneous hierarchy employs different schedulers A general hierarchy does not impose a fixed scheduling model
8 Example Hierarchy FP Voice Recognition RES J Video Player Word Processor SFQ H L FP:Fixed Priority RES:CPU Reservation J:Join SFQ:Start-Time Fair Queuing
9 Time Scales Long: System is used in a particular way Duration: usually uptime of a system or longer Medium: Applications start, end, and change requirements Duration: seconds, minutes, or longer Short: Individual scheduling decisions made by schedulers within the hierarchy Duration: milliseconds or 10s of ms
10 What Microsoft Should Do Put HLS into consumer Windows! By default Support interactive, batch, and multimedia applications for a single user However, also include Library of useful schedulers and API for composing them API for implementing new schedulers
11 Reminder HLS is an architecture for flexibly composing schedulers
12 Outline Motivation and Approach Guarantees HLS Design Augmented Reservations Conclusions and Demo
13 Guarantee Definition: Ongoing lower (and possibly upper) bound on CPU allocation Goals: Describe the scheduling behavior required and provided by schedulers, and required by applications Permit these behaviors to be reasoned about Syntax: TYPE p1 p2 …
14 Example Guarantees 100% of a CPU: ALL Strictly best-effort scheduling: NULL Proportional share: PS s PSBE s δ CPU Reservations: RESBS x y, RESBH x y RESCS x y, RESCH x y
15 CPU Reservation Guarantees Hard / Soft: “Hard CPU reservation” ≠ hard real-time Soft reservations guarantee a lower bound Hard reservations also guarantee an upper bound Basic / Continuous:
16 Guarantee Conversion by Schedulers Schedulers require and provide guarantees SFQ: PSBE → PSBE + Rez: ALL → RESBH + Schedulers determine if specific guarantees can be provided ALL → RESBH 5 10, RESBH EDF-based reservation scheduler X Naïve rate monotonic reservation scheduler
17 Selected Conversions by Schedulers Full table contains 23 schedulers SchedulerConversions Fixed Priorityany → any, NULL + SFQPSBE → PSBE +, PS → PS + EEVDFALL → PSBE + Lottery, StridePS → PS + Rialto, Rialto/NTALL → RESCS + Rez, CBSALL → RESBH + Linux/RTALL → RESBS +, RESBH + Time SharingNULL → NULL +
18 Guarantee Conversion by Rewrite Rules Guarantees can be converted without a scheduler, using rewrite rules Trivial examples: PSBE s δ → PS s RESBH x y → RESBS x y Non-trivial examples: RESBS x y → RESCS x (2y-x+c) for any c ≥ 0 RESCS x y → PSBE ( x / y ) x / y (y-x)
19 Partial Rewrite Rule Table ALLTFTFTTTT RESBHFTTFTTTT RESBSFTTTTTTT RESCHFTTTTTTT RESCSFFTFTTTT PSBEFFTFTTTT PSFFFFFFTT NULLFFFFFFFT →ALLRESBHRESBSRESCHRESCSPSBEPSNULL T = rewrite rule exists F = rewrite rule does not exist
20 Another View of Some Rewrite Rules RESCH RESBHRESBSPSBE PSRESCS
21 Guarantees in Action FP RES J Video Player Voice Recognition SFQ ALL NULL ALL RESBH 5 33 * RESBH * RESBS → Word Processor PSBE PSBE PSBE
22 Related Work for Hierarchical Guarantees Hierarchical start-time fair queuing [Goyal et al. 96] Uniformly slower processors Open environment for real-time applications [Deng et al. 99] BSS-I and PShED [Lipari et al. 00]
23 Guarantee Contributions Created a framework for reasoning about composition of schedulers Derived rewrite rules Integrated more than 20 schedulers Guarantees provide a model of soft real- time CPU allocation Independent of particular scheduling algorithms Developers can program to Users can ensure that application requirements are met
24 Outline Motivation and Approach Guarantees HLS Design Augmented Reservations Conclusions and Demo
25 Inside a Non-Hierarchical CPU Scheduler Scheduling decisions are made in response to timers and thread state changes T1T2T3T4T5 CPU WWRW SCHEDULER
26 Inside a Hierarchical CPU Scheduler Distinguishing feature of hierarchical schedulers: revocation VP2VP3VP4VP5VP6 VP1 WWRW R R SCHEDULER child virtual processors parent virtual processor
27 Hierarchical Scheduler Infrastructure (HSI) Design Schedulers are implemented by code libraries loaded into the kernel Scheduler instances are active entities in the hierarchy Virtual Processors Represent potential for physical processor allocation Used to notify schedulers
28 Scheduler Notifications Hierarchical infrastructure notifies a scheduler whenever: A child VP requests or releases a processor A parent VP grants or revokes a processor A timer expires Threads implicitly notify schedulers when they block and unblock Notifications are cheap: virtual function calls
29 HSI and Scheduler Implementation HSI runs in Windows 2000 kernel Serialized by a spinlock Added ~3100 lines of code Loadable schedulers: Time sharing / fixed priority, CPU reservation, proportional share, join A representative set of schedulers, but not a complete one Implemented Rez in about two days, PS scheduler in a few hours
30 Performance Test machine is a 500MHz Pentium III Most mode change operations run in less than 40μs Create / destroy scheduler instance, begin / end CPU reservation, etc. Median context switch time Unmodified Windows 2000: 7.1μs HLS time-sharing scheduler: 11.7μs Cost of each extra level in the scheduling hierarchy is 0.96μs Many opportunities for optimization
31 Performance Data: Isolating Resource Principals N1N1 % 1 00 N 2 % 2 Without Isolation (1-level) With Isolation (2-level)
32 Scheduling a Real-Time, CPU-Bound Application Synthetic test application Represents a virtual environment or game Must provide 1 frame per 33ms (~30 FPS) 10ms of CPU time for each frame More frames are acceptable and desirable Machine also runs background work Goals Application should never miss a deadline Non-real-time work should make progress
33 Performance Data: CPU Bound Real-Time Application 30-second runs App. Guarantee%a%a FPSMisses%b%b NULL (high pri.) NULL (def. pri.) NULL (low pri.) RESBH RESBS
34 Related Work for HSI Scheduler activations [Anderson et al. 91] CPU inheritance scheduling [Ford and Susarla 96] Vassal [Candea and Jones 98]
35 Contributions Virtual processors are a novel extension of scheduler activations [Anderson et al. 91] Permit a wide range of schedulers to dynamically create a scheduling hierarchy in kernel of a GPOS First system to do this Simplifies scheduling programming model by hiding OS and hardware complexities
36 Outline Motivation and Approach Guarantees HLS Design Augmented Reservations Conclusions and Demo
37 Problem: Stolen Time We have assumed until now that the OS does not interfere with guarantees However, while processing asynchronous data the OS may steal CPU time from applications Time used by bottom-half mechanisms is not accounted for correctly A solution: Augmented CPU reservations
38 Time Stolen by Network Receive Processing
39 Augmented Reservations Strategy: accurately measure stolen time in order to compensate threads Rez-C: avoid “charging” threads for stolen time Rez-FB: use a feedback loop to assign a larger CPU reservation so requirements are met even when time is stolen Implemented as HLS schedulers
40 Augmented Reservation Performance
41 OS Design Rule Mechanisms that will be used often must be lightweight Interrupts – very lightweight – non- preemptible, scheduled in hardware DPCs and bottom-half handlers – high priority, simple scheduler, non-preemptive, no thread context Threads – least lightweight
42 Related Work for Augmented Reservations Scheduling bottom-half activity Mach [Rashid et al. 89] Nemesis [Leslie et al. 96] Scheduling bottom-half activity in a GPOS FreeBSD [Jeffay et al. 98] Feedback-based scheduling FC-EDF [Lu et al. 99] Moving code into scheduled contexts Soft modems [Jones and Saroiu 01]
43 Augmented Reservation Contributions Rez-C and Rez-FB 6% over-reservation to eliminate most deadline misses vs. 24% over-reservation for plain Rez Quantified severity of stolen time Windows Rez and Linux/RT Network, disk, software modem, USB
44 Outline Motivation and Approach Guarantees HLS Design Augmented Reservations Conclusions and Demo
45 Conclusions HLS is feasible: Composition of soft real-time schedulers can be reasoned about Implemented and performs well HLS is useful: Permits scheduling behavior to be tailored to specific needs New schedulers can be implemented more easily
46 HLS Demonstration 1. All threads scheduled by default time- sharing scheduler 2. Create a CPU reservation 3. Make proportional share scheduler the default and move time-sharing threads 4. End CPU reservation PSTS RES FP L H L TTTTTT 1. All threads scheduled by default time- sharing scheduler TTTTTTT 1. All threads scheduled by default time- sharing scheduler 2. Create a CPU reservation 3. Make proportional share scheduler the default and move time-sharing threads 4. End CPU reservation 1. All threads scheduled by default time- sharing scheduler 2. Create a CPU reservation 3. Make proportional share scheduler the default and move time-sharing threads 4. End CPU reservation TTTTTT 1. All threads scheduled by default time- sharing scheduler 2. Create a CPU reservation 3. Make proportional share scheduler the default and move time-sharing threads 4. End CPU reservation TTTTTTT
47 The End Papers and more info at: