Realizing Compositional Scheduling through Virtualization Jaewoo Lee, Sisu Xi, Sanjian Chen, Linh T.X. Phan Chris Gill, Insup Lee, Chenyang Lu, Oleg Sokolsky

Virtualization  The benefits of virtualization  Consolidate legacy systems  Integrate large, complex systems  Key challenges of virtualization for safety-critical systems  Temporal isolation  Real-time guarantee Hypervisor Legacy System Virtualization Platform Domains Legacy System 2

Compositional Scheduling  Compositional Scheduling Framework (CSF)  Provides temporal isolation and real-time guarantee  Computes the minimum-bandwidth resource model for the component  The gap between CSF theory and system  Realizing CSF though virtualization can bridge the gap Resource Model Parent component Child components Workload Periodic Tasks Component Scheduler Rate Monotonic Scheduler Scheduler Periodic Resource Model (period, budget) 3

Contributions  Compositional Scheduling Architecture (CSA)  Confederation of compositional scheduling and virtualization Enhancement to periodic server design in CSA Extension to CSF for quantum-based platforms  Performance evaluation of CSA  Synthetic workloads and avionic workloads  First open-source real-time virtualization with CSF  Extensible with new domain-scheduling algorithms 4

Overview of Our Work  Compositional Scheduling Architecture (CSA)  Component  domain  Periodic Resource Model (PRM)  Periodic Server (PS)  Task model: independent, CPU-intensive, periodic task  Scheduling algorithm: rate monotonic App Domain Hypervisor RT-Xen Hardware Task Component Root Component Compositional Scheduling PSPRM S. Xi, J. Wilson, C. Lu, C. Gill, RT-Xen: Real-Time Virtualization Based on Hierarchical Scheduling, EMSOFT, 2011 Domain PS 5

Theory Pessimism in CSF  Interface considers the worst case: U PRM – U W ≥ 0  Interface overhead leads to underutilization of the component  Resource model  periodic server in CSA  Periodic server does not fully utilize its budget  Slacks : tasks do not always execute for WCETs  Interface overhead  Underutilization of periodic server  long response times of real-time tasks  Using idle times, we propose enhanced periodic servers 6 Interface Overhead

Periodic Server Design  Purely Time-driven Periodic Server (PTPS)  If currently scheduled domain is idle, its budget is wasted  Not work-conserving t Δ DHDH DLDL Budget time Task Release Task Complete Execution of tasks in D H Execution of tasks in D L Current Domain 7

Periodic Server Design  Work-Conserving Periodic Server (WCPS)  If currently scheduled domain is idle, the hypervisor picks a lower-priority domain that has tasks to execute  Early execution of the lower-priority domain during idle period does not affect schedulability t Δ DHDH DLDL Budget time Task Release Task Complete Execution of tasks in D H Execution of tasks in D L Current Domain 8

Periodic Server Design  Capacity Reclaiming Periodic Server (CRPS)  If currently scheduled domain is idle, we can re-assign this idled budget to any other domain that has tasks to execute  Early execution of the other domain during idle period does not affect schedulability t Δ DHDH DLDL Budget time Task Release Task Complete Execution of tasks in D H Execution of tasks in D L Current Domain 9

CSF Extension for Quantum-based Platforms P:  To find the minimum-bandwidth resource model for workload W. 10 Real-number-based resource model 3 tasks Task periodTask execution time of resource model B/P:

CSF Extension for Quantum-based Platform infeasible bandwidth Real-number-based resource model Quantum-based resource model Necessary condition for schedulability  To find the minimum-bandwidth resource model for workload W. the upper bound of the period to find min-BW resource model? 11 Non-decreasing P: of resource model B/P: 1 2

CSF Extension for Quantum-based Platform the upper bound of the period to find min-BW resource model? infeasible bandwidth Non-decreasing Real-number-based resource model Quantum-based resource model Necessary condition for schedulability  To find the minimum-bandwidth resource model for workload W. Min-BW resource model 12 P: B/P: 1 2 of resource model

System Architecture  Implemented in Xen 4.0  only re-compile Xen, keep Kernel untouched  All source code available at RT-Xen website:  Current Limitations:  one VCPU per domain (single core)  CPU intensive workload 13 Xen Scheduling Framework Real-Time Sub Framework PTPSWCPSCRPS

Evaluation – Setup 14 VCPU Core 0 Core 1 Schedulers (PTPS, CRPS, WCPS) Dom0 App VCPU Dom1 VCPU Dom5 App Scheduling Algorithms (PTPS, CRPS, WCPS) Parameters for each Domain IDLE … Responsiveness: response time / deadline Deadline Miss Ratio Use Rate Monotonic within Domain …

Evaluation – Synthetic workloads  Randomly generate task sets, then compute interface  Sources of idle time:  interface overhead: U PRM – U W  slacks: over-estimation of tasks’ execution time  Range workload periods -> different interface overhead  U W : 0.7, 0.8, 0.9, 1.0  Periods: [550ms, 650ms], [350ms, 850ms], [100ms, 1100ms]  Range Execution Time Factor (ETF) -> different slacks  For all tasks in highest three priorities domains: 100%, 50%, 10%  Using period [550ms, 650ms], pick U w from 0.7, 0.8, 0.9, typical overloaded situation most interface overhead uniform distribute [wcet*ETF, wcet] extremely overloaded situation

Evaluation – Interface Overhead % 60% 0% CRPS ≥ WCPS ≥ PTPS deadline miss CDF Plot, Probability

Evaluation – ETF 17 Sched ETF = 100 %ETF = 50 %ETF = 10 % median95 %maxmedian95 %maxmedian95 %max PTPS WCPS CRPS ( Response Time / Deadline ) for the Lowest Priority Domain PTPS: non work conservative, can not utilize slacks WCPS: consumes budget in parallel, still miss deadlines CRPS: ‘reclaim’ budget more aggressively, utilize slacks effectively

Evaluation – ARINC-653 Workload 18  7 harmonic workloads, each represent a set of domains scheduled on a single processor, with each domain consisting of a set of periodic tasks A. Easwaran, I. Lee, O. Sokolsky, and S. Vestal, A Compositional Framework for Avionics (ARINC-653) Systems, Tech Report MS-CIS-09-04, 2009, University of Pennsylvania  U PRM = U W  if using real number parameters  Overheads comes from rounding up the budget  period is fixed  CRPS > WCPS > PTPS Interface Overhead (8%) CDF Plot, Probability Response Time / Deadline

Conclusion  Compositional Scheduling Architecture (CSA)  Enhanced version of the Pure Time-driven Periodic Server (PTPS) WCPS: work conserving, consume budget in parallel CRPS: aggressively reclaiming budget  Extension of CSF for quantum-based platforms  Extensive evaluation on synthetic and avionics workloads  Open Source:  RT-Xen Website: 19

Questions? 20 RT-Xen or just Google RT-Xen *^_^*

Backup : Interface overhead  Interface considers the worst case: U PRM – U W ≥ 0  For example, a task T= (p = 3, e = 1) in a component Resource model (3, 1) cannot schedule T Resource model (3, 2) can schedule T 21 U PRM – U W =2/3 – 1/3 = 1/3 Interface Overhead Task Release Task Deadline Deadline miss Resource supply of resource model (3,1) Resource supply of resource model (3,2) 1 st period of the resource supply 2 nd period of the resource supply Supplied resource

Backup : Simple rounding up does not work  The minimum-bandwidth resource model  CSF allow real number in budget.  But, budget should be an integer multiple of the scheduling quantum in quantum-based platforms  Ex:  Optimal algorithm : (1,0.38)  rounding up  (1,1)  Only integer : (1,1), (2,1), (3,2), (4,2),… Among feasible resource models, (5,2) is minimum bandwidth for quantum- based platforms

Backup: Upper bound of the period P B/P infeasible bandwidth Non-decreasing Real-number-based resource model Quantum-based resource model Necessary condition for schedulability  We can easily find the upper bound of the period for a given bandwidth 23 A given bandwidth The upper bound of the period for a given bandwidth

Backup : Difference from reservation-based system  CSA on RT-Xen virtualization  Support different local scheduler for each domain ( by running different guest OS)  Clean separation between local scheduler and global scheduler Local OS does not know other task or other domain inside the system Global scheduler does not know task information inside domain  Reservation-based native system  Local scheduler is a part of the operating system We cannot provide a component with a local scheduler  No clean separation between local scheduler and global scheduler Malformed local scheduler can affect global scheduler or other local schedulers 24

Backup : Related Work  Crespo et al., “XtratuM”, EDDC ’10  A bare VMM with a fixed cyclic scheduling policy  Cucinota et al., “Respecting Temporal Isolation...”, COMSAC ’09  KVM with a hard reservation behavior  CSA is different from above two works  Only CSA support compositional scheduling  CSA is implemented on Xen, different architecture from KVM KVM is integrated into the manager domain 25

CSF Extension for Quantum-based Platforms 26