Minimizing Cache Overhead via Loaded Cache Blocks and Preemption Placement John Cavicchio, Corey Tessler, and Nathan Fisher Department of Computer Science Wayne State University Detroit, MI, USA This research has been supported in part by US National Science Foundation (Grant no.: CNS ).

1. Refined CRPD Calculation; 2. Preemption Placement Algorithm; 3. Simulation of Actual Code Cache-Behavior. Preemption Placement to Minimize CRPD in Sequential Flowgraphs. Sequential (Non-Branching) Control Flowgraphs. Cache-Related Preemption Delay (CRPD) & Limited Preemption Scheduling. 1 Outline Background Model Problem Statement Solution

Write A Read A Write B2 Cache-Related Preemption Delay (CRPD) Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions CRPD Cache evictions due to preempting higher-priority jobs. Cache TAG A B Cache eviction Cache reload

Write B2 Cache-Related Preemption Delay (CRPD) Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions CRPD Cache evictions due to preempting higher-priority jobs. 1. Delay preemption to maintain cache locality (Limited/Deferred Preemptions). Write A Read A Techniques to Reduce CRPD

2 Cache-Related Preemption Delay (CRPD) Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions CRPD Cache evictions due to preempting higher-priority jobs. 1. Delay preemption to maintain spatial locality (Limited/Deferred Preemptions). 2. Decrease CRPD analysis pessimism by considering interdependencies between adjacent preemptions. Techniques to Reduce CRPD Write A Write B Read A Write B writeread. Independent Preemption (Single-Valued) Analysis: include a reload of A in the CRPD for each preemption of J 1 between the write and the read. writeread. Interdependent Preemption Analysis: CRPD of each preemption depends on location of next. Only account for one reload of A for each preemption of J 1 between the write and the read.

3 Opportunity: Interdependent Preemption Placement Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Interdependent CRPD (Best-Case) Single Valued CRPD Preemption Location in the Sequential Flowgraph CRPD Cost Opportunity for CRPD Reduction Reduce CRPD via preemption placement by exploiting preemption interdependencies while preserving system schedulability. Our goal

4 Related Work: CRPD Calculation Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Independent Preemption Calculation Interdependent Preemption Calculation Cache-Block Categorization & Counting Techniques (UCBs, ECBs, DC-UCBs,…) Lee et al. ['96, ’97, ‘98] Tomiyamay and Dutt. ['00] Negi, et al. [‘03] Tan and Mooney [‘04] Staschulat and Ernst [‘05] Altmeyer and Burguiere [‘11] Luniss et al. [’13] …. Cache Partitiong /Coloring Mueller [’95] Lietdtke et al. [‘96] Mancuso et al. [‘13] … Memory Access Patterns Ramaprasad and Mueller [‘06] Probabilistic Davis et al. [‘13] Our Work

5 Related Work: Limited Preemption Model Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Fixed preemption point models Floating preemption point models EDF scheduled systems Burns ['94] Baruah ['05] Bertogna and Baruah ['10] Fixed Priority scheduled systems Burns ['94], Bril et al. ['09], Bertogna et al. ['11], Davis et al. ['12] Yao et al. ['09] Single valued CRPD Bertogna et al.['10, '11] Peng, Fisher, Bertogna ['13] Interdependent CRPD Our work

6 Model: Sequential Control Flowgraphs Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions An arbitrary control sequence of instructions with a single entry point and a single exit point. Basic Block (Region).. Basic block numeric identifier. Block ID Task numeric identifier. Task ID Block non- preemptive worst case execution time. Block WCET

7 Model: Sequential Control Flowgraphs Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Potential Preemption Point (PPP) Each edge is a potential program location to (statically) insert a preemption point. CRPD Cost Matrix Interpretation: Entry  i (  i,  i ) represents the CRPD of preemption inserted after  i if a preemption is also inserted after  i. j j k k

8 Preemption Point Placement Problem Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Given: A set of tasks {  1,  2, …,  n }. Each task  i is characterized by: 1.Timing parameters (e.g., deadline and period). 2.Maximum non-preemptive region Q i. Note: Determined from limited-preemption schedulability analysis (e.g., Baruah, RTSS’05)

9 Preemption Point Placement Problem Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Given: A set of tasks {  1,  2, …,  n }. Each task  i is characterized by: 1.Timing parameters (e.g., deadline and period). 2.Maximum non-preemptive region Q i. 3.Sequential Control Flowgraph & CRPD Matrix: Goal: Determine the choice of preemption points to minimize the WCET + CRPD of each task such that any non-preemptive region is at most Q i.  i (  i,  i ) +  b i  Q i j j+1ℓ ℓ=j+1 k

9 Preemption Point Placement Problem Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Given: A set of tasks {  1,  2, …,  n }. Each task  i is characterized by: 1.Timing parameters (e.g., deadline and period). 2.Maximum non-preemptive region Q i. 3.Sequential Control Flowgraph: Goal: Determine the choice of preemption points to minimize the WCET + CRPD of each task such that any non-preemptive region is at most Q i. Let  be selected preemption points. Minimize   i (  i,  i ) +  b i j j+1ℓ ℓ=1 # of blocks  i  j

I.Calculation of CRPD for Interdependent Preemptions. II.Preemption Placement Algorithm. III.Estimation of CRPD via Cache Simulation. 10 Outline of Solutions Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions

11 CRPD Computation: UCBs Definition: Useful Cache Block (UCB) [Lee et al., ‘96] Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Task  1 Write X Read X X maps to Cache Block 1 {} {1,2} {1,2,4,8}{1,2,8}{1,2,7,8} {1,2} UCBs:

12 CRPD Computation: ECBs Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions A memory block m is called an evicting cache block, at basic block  i, if it may be accessed during execution of that basic block. Definition: Evicting Cache Block (ECB) [Lee et al., ‘96] j Task  1 {} {1,2} {1,2,4,8}{1,2,8}{1,2,7,8} {1,2} UCBs: {} {1,2} {3,4,8}{4,5,6,8}{1,2,7,8} {1,2} ECBs: Write X Read X X maps to Cache Block 1 ECB(τ 1 ) =  ECB(  1 ) ={1, 2, 3, 4, 5, 6, 7, 8} Task-Level View j j

13 CRPD Computation: AUCBs Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Definition: Accessed Useful Cache Block (AUCB): A memory block m is called an accessed useful cache block, if it may be accessed during execution of that basic block and is useful. Task  1 {} {1,2} {1,2,4,8}{1,2,8}{1,2,7,8} {1,2} UCBs: {} {1,2} {3,4,8}{4,5,6,8}{1,2,7,8} {1,2} ECBs: {} {1,2} {4,8}{8}{1,2,7,8} {1,2} AUCBs: AUCB(  i ) = UCB(  i )  ECB(  i ) j jj

14 CRPD Computation: LCBs A memory block m of the preempted task is called a loaded cache block, if it may be re-loaded during the non-preemptive region defined by two adjacent selected preemption points. Definition: Loaded Cache Block (LCB): Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions LCB(  i,  i ) = [UCB(  i )  [  AUCB(  i )]]  [  ECB(  k )] curr next curr v=curr+1 next v k  hp(i) Useful blocks of task  i at time of a preemption.Blocks accessed by  i between two adjacent preemptions. Contention with higher-priority tasks. Task  1 ii curr ii next

14 CRPD Computation: LCBs A memory block m of the preempted task is called a loaded cache block, if it may be re-loaded during the non-preemptive region, defined by two adjacent selected preemption points. Definition: Loaded Cache Block (LCB): Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions LCB(  i,  i ) = [UCB(  i )  [  AUCB(  i )]]  [  ECB(  k )] curr next curr v=curr+1 next v k  hp(i) {} {1,2} {1,2,4,8}{1,2,8}{1,2,7,8} {1,2} UCBs: {} {1,2} {4,8}{8}{1,2,7,8} {1,2} AUCBs: Task  1 ii curr ii next Assume [  ECB(  k )] = {1, 2, …, 8}: LCB(  1,  1 ) = {1, 2, 8} k  hp(i) curr next

15 CRPD Computation: Interdependent Cost Structure Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Interdependent CRPD cost between each pair of PPPs. CRPD Cost Consider a pair of PPPs. Interdependence  i (  i,  i ) = |LCB(  i,  i )|  BlockReloadTime curr next curr next

16 CRPD Computation: Single Valued Cost Structure Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Compared to the interdependent CRPD cost, the single valued approach must use the maximum CRPD cost at each PPP. Single Valued CRPD Cost

17 Outline of Solutions Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions I.Calculation of CRPD for Interdependent Preemptions. II.Preemption Placement Algorithm. III.Estimation of CRPD via Cache Simulation.

18 Preemption Placement Algorithm: Dynamic Programming Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Optimal Substructure Assume optimal set of preemption points selected. Subset of select preemption points must also be optimal for subproblem.

18 Preemption Placement Algorithm: Dynamic Programming Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Optimal Substructure SP1 SP2 SP3 …  Q i Result: Optimal preemption placement for sequential flowgraphs can be done in quadratic complexity in the number of basic blocks.

19 Outline of Solutions Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions I.Calculation of CRPD for Interdependent Preemptions. II.Preemption Placement Algorithm. III.Estimation of CRPD via Cache Simulation.

Select ten random tasks from the MRTC Benchmarks. Compute enhanced preemption cost matrix.  Each task compiled for GRSIM LEON3 target  Absint’s a 3 used to get CFG and WCET.  Cache snapshots used to obtain LCB estimates. Plot CRPD cost as function of preemption location:  Illustrate preemption-cost interdependence & variation. Breakdown Utilization Evaluation. 20 Simulations: Methodology Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions ADPCMBSORT100CNTCOVERCRC FFT1FIBCALLLMSNDESRECURSION

21 Simulations: Recursion Task CRPD Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Emphasizes the interdependence of preemption points Preemption cost trend

22 Simulations: LMS Task CRPD Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Large cost variations Preemption cost trend

23 Simulations: ADPCM Task CRPD Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Large cost variations Preemption cost trend

24 Simulations: Breakdown Utilization Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Preemption Placement & Interdependent Preemptions Preemption Placement & Independent Preemptions (Bertogna et al.) UCB-Only (Lee et al., Lunnis et al..)

Conlusions:  Enhance CRPD calculation to account for preemption interdependence.  Proposed quadratic dynamic programming algorithm for selection of optimal preemption points.  Simulations over code show potential for significant CRPD reduction. Future work:  Extend the sequential flow graph structure to an arbitrary flow graph structure. (E.g., conditional blocks, loops, function calls, and gotos.)  Develop automated & integrated compiler/schedulability tools.  Extend the technique to utilize other CRPD set-theoretic concepts. (E.g., UCB-U Multiset, ECB-U Multiset, DC-UCB). 25 Conclusions and Future Work Background Background Model Model Problem Statement Problem Statement Solutions Solutions Future Work Future Work

26 Questions? Thank You!

Deadline: July 24th Lille, November rd International Conference on Real-Time Networks & Systems

34 Related Work EDF scheduled systems Fixed preemption point models Burns ['94] : [11] A. Burns. Preemptive priority based scheduling: An appropriate engineering approach EDF scheduled systems Floating preemption point models Baruah ['05] : [4] S. Baruah. The limited-preemption uniprocessor scheduling of sporadic task systems. Bertogna and S. Baruah ['10] : [5] M. Bertogna and S. Baruah. Limited preemption EDF scheduling of sporadic task systems. Fixed Priority scheduled system Fixed preemption point models Burns ['94] : [11] A. Burns. Preemptive priority based scheduling: An appropriate engineering approach Bril et al.['12]: [10]Worst-case response time analysis of real-time tasks under fixed-priority scheduling with deferred preemption. Bertogna et al. ['11]: [7] M. Bertogna et al. Improving feasibility of fixed priority tasks using non-preemptive regions. Davis et al. ['12] : [13] R. Davis and M. Bertogna. Optimal fixed priority scheduling with deferred preemption. Fixed Priority scheduled system Floating preemption point models Yao et al. ['09]: [19] G. Yao, G. Buttazzo, and M. Bertogna. Bounding the maximum length of non-preemptive regions under fixed priority scheduling. Linear code structure Fixed preemption point models Bertogna et al. ['10]: [6] M. Bertogna et al. Preemption points placement for sporadic task sets. Bertogna et al. ['11]: [8] M. Bertogna et al. Optimal selection of preemption points to minimize preemption overhead. Conditional code structure Fixed preemption point models Peng et al. ['13]: [6] B. Peng et al. Preemption points placement for sporadic task sets.

35 Iterative CRPD Reduction (via Limited Preemptions) Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions WCET+CRPD analysis Real-time schedulability analysis Limited preemption scheduling Reduce CRPD

36 Preemption Point Placement: Single Valued CRPD Find a selection of EPPs that minimizes the WCET+CRPD of a sequential (linear) flow graph. The cost of any non-preemptive region must be less than Q i. Uses a single valued preemption cost. Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions A B CD E FGHI Cost=7 Cost=3 Cost = 24 Q i =8 Bertogna et al.['11] Optimal selection of EPPs in sequential flowgraphs BEPP

37 Preemption Point Placement: Single Valued CRPD Find a selection of EPPs that minimizes the WCET+CRPD of a series-parallel flow graph. The cost of any non-preemptive region must be less than Q i. Uses single valued preemption cost. Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions  Upper path: 28 Lower path: 25  Combined result: 32(U) Optimal result: 27(L) WCET+CRPD A B C D E F JKL G H I Q i =8 Peng et al.['13] Optimal selection of EPPs in series-parallel flowgraphs

Preemption Placement Algorithm: Dynamic Programming Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Q i =12 Cost matrices reflect optimal substructures Optimal Solution Preemption Costs Minimum Preemption Cost Previous Minimum Preemption Point Optimal Preemption Points

24 Simulations: Breakdown Utilization Background Background Model Model Problem Statement Problem Statement Solutions Solutions Conclusions Conclusions Start with a task system that may or may not be feasible. Assume the CRPD of the task system is initially zero. Execute the preemption point placement algorithm. Is the task system EDF- schedulable? Increase system utilization via binary search. Decrease system utilization via binary search. The breakdown utilization is given by U. Is the utilization change less than some  (threshold)? Yes No