1. Sensor Data Management with Model-based View LSIR, EPFL

2. Motivation Building scientific models is essential to environmental science applications data cleaning (statistical analysis) visualization (interpolation models) event detection (statistical analysis, prediction models... ) simulations

3. Motivation Database Sensor Readings Query Data Files ArcGIS / Matlab

4. Model Based View View Specificatio n database access Sensor Readings Create Interpolation View … ArcGIS / Matlab

5. Model Based View Building models as database views Convenient and elegant Uniform access to both raw data and model-derived data Real-time Visualization Optimize the computation procedure Parallel computation

6. Visualization Example – Snow Cover Distribution Measured value Interpolated value Snowcover – in mm > 1000mm 500 - 1000mm 200 - 500 mm < 200 mm User-defined model

7. Interpolation Raw sensor reading at sampled places and the sampled times. Build a interpolation model view Query values at any place any time Visualization

8. Interpolation Procedures Linear Interpolation Neighbor search Weight computation Value estimation

9. Interpolation Models Nearest Neighbor Average Inverse Distance Triangulation Kriging

10. Storage Management Materialized Non-materialized Partially Materialized Materializing Internal Variables weights

11. Historical Data Access t=3 ID Weight i j k Wi Wj Wk t=3 vivi vjvj vkvk

12. Real-time View t=1

13. Real-time View t=2

14. Real-time View t=3

15. Optimizations Assembles relational join operations Apply join optimization techniques

16. Data Cube for Interpolated Data Display/Visualize aggregate measurements a time interval, a particular area etc. quickly zoom in/out in both space and time

17. Data Cube for Interpolated Data Data cube Multi-dimensional and hierarchical aggregates time: 5 minute, 30 minute, hour, day, week, month, year, all sensor: sensor, region, site, all area: 10 m 2,100 m 2, 1 km 2 Efficient drill down, roll up

18. Data Cube all site sensor Location Dimension Measurement Type Dimension Time Dimension measurement type all year week day hour minute data measures

19. System Design (Alternative 1) Data cube with view materialization View Computation Data View Sensor data stream SQL Server Database Engine Analysis Service Client Application Archive

20. System Design (Alternative 1) Data cube with view materialization  Storage explosion Large region, fine granularity, high update rate  Cost inefficiency What if data of interest only constitute a small portion of the entire cube How about materialize the view only when it is explicitly requested?

21. System Design (Alternative 2)  Utilize the cube on raw sensor data  Store internal variables  Compute the cube (i.e. compute the view) only when it is explicitly requested by the users Data cube without view materialization The computation has to be simple and fast !

22. System Design (Alternative 2) Cube Construction Sensor data stream SQL Server Database Engine Analysis Service Client Application Internal Variables Data cube without view materialization (lazy evaluation)

23. System Design (Alternative 2) About internal variables  weights Cube built from measured data ID Weight i j k Wi Wj Wk Internal Variables

24. Support common modeling operations within database Interpolation Hidden Markov Model etc Interactions with external tools, s.a. Matlab / ArcGIS Research Plan