1. Fjording the Stream: An Architecture for Queries over Streaming Sensor Data Samuel Madden, Michael J. Franklin University of California, Berkeley Proceedings of the 18th International Conference on Data Engineering (ICDE’02)

2. Outline Introduction Architecture Main characteristics Experiments Conclusions

3. Introduction Fjord  also fiord, long narrow inlet of the sea between high cliffs, as in Norway  “Framework in Java for Operators on Remote Data streams” Sensor infrastructure  Cooperators Berkeley Highway Lab (BHL) California Department of Transportation (CalTrans)  Location Bay Area Freeways  Objective Monitoring traffic conditions

4. Sensor limitations Push-based data  Waiting for queries wastes power Power  Sensors with battery 100mAh CPU: 3.5 hours TRM-1000 radio: 14MB  Tradeoff It is often worth spending many CPU cycles to conserve just a few bytes of radio traffic.

5. Issues in data stream systems Operators  Aware of the infinite nature of streams  Modified versions of AVERAGE, COUNT, SORT, JOIN hash-join A. Wischut, P.Apers. Dataflow query execution in a parallel main-memory environment. blocking operators (ex: average) specify a subset of the stream for them to operate over Query plan optimization  no mention Architecture

6. Architecture (1/2) Components  Operators has a set of input queues a set of output queues  Queues has one input operator one output operator  Sensor proxy

7. Architecture (2/2) Strategy  State based execution model  Rather than placing each pushing operator in its own thread Advantages  Better control over priority  Lower overhead output current state input new state

8. Main characteristics (1/2) Integrating streaming data with disk-based data  Example Relations between average speeds and traffic incidents  Means Using queues as data sources Combining multiple queries into a single plan  Reason Several queries need data from the same sensors. Duplication wastes bandwidth and power.  Means Using the sensor proxy

9. Main characteristics (2/2) Intergrating streaming data with disk- based data  Queue pull push putget transitionget input operatoroutput operator

10. Code snippet

11. Sensor proxy Functions  Adjust the sample rate of the sensors, based on user demand  Direct the sensor to aggregate samples in predefined ways  Let user queries share the same tuple data

12. Experiments

13. Traffic queries

14. Fjord

15. Performance Output queues become slower when there are more than a few thousand elements on them.

16. Scalability

17. Simulations

18. Speed, length of a vehicle Speed Length

19. Sensor parameters

20. Power consumption Scenario  The sensor 1.reads from it’s A-to-D input 2.transmits the sample 3.sleeps until the next sample period arrives

21. Power consumption Scenario  Sensors observe when a car passes over them transmit the { t 0, t 2 } or { t 1, t 3 } relay only a few samples per second

22. Power consumption Scenario  The sensors Only relay a count of the number of vehicles that passed in the previous second

23. Conclusions Addressing the low level infrastructure issues in a sensor stream query processing via  Fjord combines proxies, non-blocking operators and conventional query plans  Sensor proxies serve as intermediaries between sensors and query plan