U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Re-thinking Data Management for Storage-Centric Sensor Networks Deepak Ganesan University of Massachusetts Amherst With: Yanlei Diao, Gaurav Mathur, Prashant Shenoy

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 2 Sensor Network Data Management Live Data Management: Queries on current or recent data. Applications: Real-time feeds/queries: Weather, Fire, Volcano Detection and Notification: Intruder, Vehicle Techniques: Push-down Filters/Triggers: TinyDB, Cougar, Diffusion, … Acquisitional Query Processing: BBQ, PRESTO, … Archival Data Management: Querying or Mining of past data Applications: Scientific Analysis of past events: Weather, Seismic, … Historical trends: Traffic analysis, habitat monitoring Our focus is on designing an efficient archival data management architecture for sensor networks

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 3 Archival Querying in Sensor Networks Data Gathering with centralized archival query processing Efficient for low rate, small volume sensors such as weather sensors (temp, humidity, …). Inefficient energy-wise for “rich” sensor data (acoustic, video, high- rate vibration). Lossless aggregation DBMS Internet Gateway

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 4 Archival Querying in Sensor Networks Acoustic stream Store data locally at sensors and push queries into the sensor network Flash memory energy- efficiency, cost, capacity. Limited capabilities of sensor platforms. Internet Gateway Image stream Flash Memory Push query to sensors

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 5 Technology Trends in Storage Generation of Sensor Platform CC1000 CC2420 Telos STM NOR Atmel NOR Communication Storage Micron NAND 128MB Energy Cost (uJ/byte)

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 6 Outline Case for Storage-centric Sensor Networks Challenges in a Storage-centric Sensor Database StonesDB Architecture Local Database Architecture Distributed Database Architecture Conclusion

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 7 Optimize for Flash and RAM Constraints Flash Memory Constraints Data cannot be over-written, only erased Pages can often only be erased in blocks (16-64KB) Unlike magnetic disks, cannot modify in-place Challenges: Memory: Minimize use of memory for flash database. Energy: Organize data on flash to minimize read/write/erase operations Aging: Need to efficiently delete old data items when storage is insufficient Load block 2.Into Memory 3. Save block back Erase block Memory 2. Modify in-memory ~16-64 KB ~4-10 KB

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 8 SQL-style Queries: Min, max, count, average, median, top-k, contour, track, etc Similarity Search: Was a bird matching signature S observed last week? Classification Queries: What type of vehicles (truck, car, tank, …) were observed in the field in the last month? Wireless Sensor Network Support Rich Archival Querying Capability Signal Processing: Perform an FFT to find the mode of vibration signal between time ?

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 9 StonesDB Goals Our goal is to design a distributed sensor database for archival data management that: Supports energy-efficient sensor data storage, indexing, and aging by optimizing for flash memories. Supports energy-efficient processing of SQL-type queries, as well as data mining and search queries. Is configurable to heterogeneous sensor platforms with different memory and processing constraints.

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 10 StonesDB Architecture

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 11 Example: Indexing in StonesDB Naïve Design: Consider a value-based index on entire stream Deletion/Aging of data triggers in-place updates involving energy-intensive block read/write/erase operations.

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 12 Indexed Storage StonesDB Design: Split data stream into partitions and build index on each partition. Age partitions as a whole cheaply. Flash Block

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 13 Outline Case for Storage-centric Sensor Networks Challenges in a Storage-centric Sensor Database StonesDB Architecture Local Database Architecture Distributed Database Architecture Conclusion

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 14 StonesDB: Data Mining Queries Similarity Search: Was a bird matching signature S observed last week? Proxy Cache of Image Summaries

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 15 StonesDB: System Operation Similarity Search: Was a bird matching signature S observed last week? Query Engine Partitioned Access Methods

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 16 Research Issues Local Database Layer Impact of RAM limitations on storage organization Energy-optimized indexing and aging. New cost models for self-tuning energy-efficient sensor databases. Distributed Database Layer Intelligent split of query processing between proxy and sensor tiers Adaptively tuning quality of data cached at sensor proxy based on query needs

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science The End STONES: STOrage-centric Networked Embedded Systems

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 18 Sensor Data Management Taxonomy Timeline vs Prior Knowledge Querying Mining CurrentRecent Past Acquisitional Query Processing (BBQ, …) Pushdown Filters (TinyDB, Cougar, …) Timeline of data being processed Search/Mining on Archived Sensor Data Type of data processing

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 19 Technology Trends in Sensor Platforms Cyclops Camera+ Mica2 Mote 128 x 128 resolution images 4 KB RAM, 10 MHz microcontroller OmniVision Camera + iMote2 128 x 128 resolution images 64KB - 32MB RAM, 10 MHz microcontroller Spectrum of sensing devices with different power, capability, resource constraints.