1. University of Illinois at Urbana-Champaign Real-Time Capacity of Networked Data Fusion University of Illinois at Urbana-Champaign Forrest Iandola (University of Illinois) Fatemeh Saremi (University of Illinois) Tarek Abdelzaher (University of Illinois) Praveen Jayachandran (IBM Research) Aylin Yener (Pennsylvania State University)

2. Motivation and Goals Develop a theoretical bound for the capacity of data fusion systems Enable data fusion systems to run at full capacity without missing deadlines Forrest Iandola Illustration of a data fusion system with merging

3. Outline Introduce data fusion system model Scheduling theory background: Feasible Region Calculus Derive a capacity utilization bound for data fusion pipelines Extend this bound to capture merging pipelines Performance evaluation Forrest Iandola

4. “Data Fusion System” refers to… Distributed sensor networks Control systems that receive one or more data feeds “Real-Time Capacity” = data packets transmitted within time constraints Forrest Iandola Data Fusion System Model (1/3)

5. Data Fusion System Model (2/3) Workflow i is denoted as F i Invocation of F i is a “job” q D i = deadline of F i P i = period of F i R i = 1/P i = “Rate” C i,j = computation of F i on stage j Forrest Iandola

6. Data Fusion System Model (3/3) System constraints reflect a realistic data fusion system Non-preemptive earliest deadline first (EDF) scheduling Workflows are periodic. D i >> P i (in other words, many invocations of F i may be active simultaneously.) Forrest Iandola

7. Scheduling Theory Background: Feasible Region Calculus (FRC) A pipeline task set can be reduced to a uniprocessor equivalent: Assume q N is the lowest-priority workflow Forrest Iandola

8. For simplicity, let us refer to the “modified” equivalent of the lowest- priority task as q Forrest Iandola Scheduling Theory Background: Feasible Region Calculus (FRC)

9. Deriving Capacity Bound from FRC Testing schedulability of equivalent uniprocessor from as defined by FRC Remember: we assume non-preemptive EDF scheduling Forrest Iandola

10. Testing schedulability of equivalent uniprocessor from as defined by FRC Remember: we assume non-preemptive EDF scheduling Forrest Iandola Deriving Capacity Bound from FRC Basic utilization formula: Combining utilization formula with FRC definitions: To avoid deadline misses, utilization must be less than 1.

11. Simplifying the Capacity Bound to Reduce Computation Overhead Stage-additive component is very small when D i >> P i Can approximate the utilization even if we ignore stage-additive component Forrest Iandola Reduce computation time by dropping lowest-priority invocation: Replace ceiling function with (D i R i +1):

12. Handling Merging Flows Forrest Iandola

13. Handling Merging Flows

14. Let’s discuss the intuition behind this. Step 1: Reduce child pipelines to equivalent uniprocessor workflow sets Step 2: Obtain two-stage pipeline Ignore all but the largest equivalent pipeline per workflow Step 3: Calculate equivalent uniprocessor for two-stage pipeline Forrest Iandola Handling Merging Flows

15. Fundamental Results Forrest Iandola

16. Evaluation of Capacity Bound Comparing predicted useful work of a data fusion tree to actual useful work (just before onset of deadline misses) Note: Utilization due to jobs/flows that miss deadlines is not counted as useful work. Forrest Iandola Observations: Capacity bound accurately predicts ability to do useful work

17. Evaluation of Overload Behavior Comparing overload behavior of a data fusion tree with admission control (based on new capacity result) to one without Note: Utilization due to jobs/flows that miss deadlines is not counted as useful work. Forrest Iandola Observations: Capacity bound accurately predicts ability to do useful work At high load, significant degradation is observed in the absence of admission control due to excessive deadline misses

18. Conclusions Derived a capacity utilization bound for data fusion systems Simplified the bound into an easy-to-use approximation Extended this result for merging workflows Evaluation demonstrates accuracy of bound Forrest Iandola