When the Sensors Hit the Road The CarTel Project Sam Madden MIT CSAIL With Hari Balakrishnan, Vladimir Bychkovsky, Jakob Eriksson, Bret Hull, Yang Zhang, Kevin Chen, Waseem Daher, Michel Goraczko, Hongyi Hu, Sejoon Lim, Allen Miu, Daniela Rus, Eugene Shih, Arvind Thiagarajan, Sivan Toledo Stanford Database Group

1 st Generation Sensor Networks Periodic monitoring –Wake up and sense –Sleep for Minutes Event-based monitoring –Transmit on external event Low data rates, duty cyles Static, small area Extremely underpowered devices “Mote” ~=

Participatory Sensing “Crowd-sourced” sensing –Individuals contribute their data –Rather than relying on fixed deployments Examples: –Personalized Environmental Impact Report –Microsoft SensorMap –Dartmouth BikeNet –MapMyRun, etc. CarTel

Cars as a Vehicle for Participatory Sensing Observation: –Static sensing infeasible over very wide areas –Some apps do not need high temporal fidelity Real-world problems: –Civil infrastructure monitoring –Road-surface conditions –Visual mapping –Commute optimization –Carpool finding –Speed trap mapping

Opportunistic Mobility Rather than deploy new mobile nodes, take advantage of existing mobility Example: cellphones w/ sensors –1.5 billion phones worldwide –High spatial coverage –High-performance processor Cars equipped with sensors –650 million cars on the road –Abundance of power and space –Have >100 embedded sensors What system architecture is best suited for mobile, wide-area sensing?

Data collection servers Clients Open Wi-Fi GPRS Collect / Process Deliver Visualize CarTel System Overview Portal Cabernet / QuickWifi (Carry & Forward Network) ICEDB (Intermittently Connected & Embedded DB) Per-node sensing & computation hardware

Roadmap Overview CarTel Components –Portal –IceDB –Cabernet Managing missing and uncertain data Case Studies –Traffic Analysis –WiFi Mapping –Potholes

Portal Web-based visualization framework Apps retrieve sensor data by issuing queries to ICEDB Visualize sensor data using map overlays Continuous queries to direct sensing –Pushed to remote nodes using Cabernet Local ad-hoc queries read streaming results Visualize / Analyze Web server Portal Applications TrafficWi-Fi ICEDB server CQ Rel. DB Cabernet Portal Data Viz. Streaming sensor data Cont. queries

Visualize / Analyze Data Collection Demo

ICEDB: Intermittently Connected Embedded DB Relational model is a convenient abstraction for data collection ICEDB: –Queries written in extended SQL –Continuous query processor –Distributed Bandwidth is variable 1.Buffer query and DDL commands 2.Buffer query results 3.Support “drill down” queries 4.Prioritize results Collect / Process SELECT img FROM camera, gps WHERE gps.pos in [x,y] AND camera.time= gps.time SAMPLE 1s

ICEDB Query Processing DB CQ Ad-hoc Query Processor Output Buffers Cabernet ADAPTERSADAPTERS sensor Collect / Process Remote Node 2 data paths to cope with limited BW ICEDB Server ICEDB Remote results queries

Inter-query Prioritization Remote node BISECT Portal SELECT lat, lon FROM gps WHERE insert_time > cqtime – 5 EVERY 5 seconds BUFFER IN gpsbuf DELIVERY ORDER bisect(gpsbuf) Portal Remote node FIFO Portal t=1t=2t=3 SELECT lat, lon FROM gps WHERE insert_time > cqtime – 5 EVERY 5 seconds BUFFER IN gpsbuf DELIVERY ORDER fifo(gpsbuf) Collect / Process

Global Prioritization Collect / Process to ICEDB Server to ICEDB Server SELECT lat, lon, image FROM camera WHERE insert_time > cqtime – 5 EVERY 5 seconds BUFFER IN cambuf SUMMARIZE AS SELECT floor(lat/100), floor(lon/100) FROM cambuf GROUP BY floor(lat/100), floor(lon/100)

Global Prioritization Collect / Process ICEDB Server

Specifying Priorities Three SQL language extensions: 1.Global prioritization: SUMMARIZE AS 2.Inter-query prioritization: PRIORITY 3.Intra-query prioritization: DELIVERY ORDER BY For more details: ICDE ’06 Collect / Process

Cabernet Disconnection-tolerant transport layer Buffer data until connectivity becomes available –Not connection oriented QuickWifi: Fast connection establishment CTP: Unlike TCP, wireless losses ≠ congestion losses Deliver QuickWifi Jakob Eriksson et al – Mobicom 08

Challenge: Connection Establishment is Slow Process –Scan 11 channels (600 ms average), receive beacon –Authenticate (1 round trip, 3s timeout) –Associate (1 round trip, 3s timeout) –DHCP Discovery (1 round trip, 3s timeout) –DHCP Request (1 round trip, 3s timeout) –ARP Request (1 round trip, 2s timeout) Many messages, loss rates high, retries often needed Default Linux stack took 13s to associate, on average (whole connection lasts only 19s avg!) Deliver

Connection Optimizations Scan more popular channels first (1,6,11) Since authentication in open networks always succeeds, do it in parallel with association Set timeouts to 100 ms Mean conn. time: 370 ms DHCP via broadcast addr

Roadmap Overview CarTel Components –Portal –Cabernet Managing missing and uncertain data Case Studies –Traffic Analysis –WiFi Mapping –Potholes

Noisy Data Challenge Challenge: how to store and query all of this data? Discrete points don’t work well Most apps don’t actually want raw data! –Prefer trajectories, fields, fit functions –Idea: support these as first class objects inside the DBMS

Model-Based Views Proposed in MauveDB [SIGMOD 06] –Models can be queried like database views –Basic idea: compute model grid modeled area use model to compute values at each cell of grid answer queries using grid FunctionDB: efficient implementation for an important class of models –Continuous functions of one or more variables

Benefits Declarative Queries –No need to write procedures to manipulate models, or re-implement or re-optimize each new query View provides intuitive SQL-like interface If data is already in database, do not need to move data to/from math package like MATLAB

FunctionDB: Key Idea Database system that fits continuous functions to data MauveDB-like query Interface (SQL + Grids) Algebraic query processor Raw data (temp readings) Query: Report when temp crosses threshold SELECT time WHERE temp = thresh Regression Function temp(t) Solve equation temp(t) = thresh temp time

FunctionDB: System Architecture Raw Data “Function Table” (Storing Piecewise Model) XY …… StartEndSlop e Intercep t User fits or imports functions Algebraic Query Processor (Operates on algebraic representation) User QueryQuery Result

Query Results Grid semantics: all queries yield discrete points sampled at user-specified interval (“grid size”) SELECT x,y WHERE temp < 20 GRID x 8, y temp < 20

Algebraic Execution: Overview FunctionDB supports efficient algebraic execution for piecewise polynomial functions Restriction helps achieve grid semantics even when no closed form solutions are available E.g., Non-linear polynomials, multi-variate polynomials, complex constraints X^2 + Y^3 < 25

SELECT * WHERE Temp < 20 GRID X 8, Y 8 Efficient Algebraic Implementation User X:[0,20), Y:[0,20), Temp = X+Y X:[0,20), Y:[0,20), Temp = X+Y, Temp < 20 Substitute Approx X:[0,20), Y:[0,20), Temp = X+Y, X+Y-20 < 0 Grid Symbolic Filter (Temp < 20) Function Table Temp = F(X,Y) Gridded Result Hypercubes+Boundary Points X+Y < 20

Hypercube Approximation Use technique from graphics rasterization (Taubin’s test) Tests if polynomial F(X,Y, …) can have zeros within a given hypercube H (i.e., F lies on boundary of H) Hypercube H Z = F(X,Y) Z = 0 Y X Zero of F Allows efficient pruning by testing corners of hypercube

Subdivision Algorithm Predicate Bounding Box Check centers of all grid cells smaller than grid size

More Algebraic Operators Solver for single-variable equations Function Inference E.g., X+Y = 20  X = 20-Y (Eliminates independent variables) Continuous aggregates E.g., Average  Integration

Evaluation: Temperature Data Data: 54 sensors, 10 days of data, 1 million raw data points Fit regression model temp = F(X,Y,time) Degree-2 piecewise polynomial with 22 pieces Evaluation queries: Find regions where temp < threshold (Filter) Area of region where temp < threshold (Aggregate) Compared algebraic approach to two baselines: Evaluating all grid points Pruning search using a precomputed B-Tree

Result 1: Benefits Of Algebraic Execution Depend On Selectivity

Result 2: Algebraic Execution Wins Significantly For Aggregate Queries Wider grid size is faster, but too wide a grid size results in low accuracy owing to discretization error

Roadmap Overview CarTel Components –Portal –Cabernet Managing missing and uncertain data Case Studies –Traffic Analysis –WiFi Mapping –Potholes

Route Planning Match traces to map Compute Gaussian delay for each segment –Assume independence Minimize 3 Objectives –Distance Google Maps –Expected delay –Pr(missing time goal)

Max. Probability Planning Travel time of each edge is a Gaussian –If indepdendent, travel time of a path is also Gaussian Goal: find path with max. probability of reaching destination by deadline Unlike standard shortest paths, no optimal substructure –If AxCyB is best path from A to B, AxC is not necessarily the best path from A to C Implies cannot use A* or Dijkstra 2 ABC 1 3 Lim et al. “Stochastic Motion Planning and Applications to Traffic.” To Appear, Workshop on Algorithmic Foundations of Robotics, 2008

Finding Potholes

Classification-based Approach Classifier differentiates between several types of anomalies Window data, compute features per window Variety of features: –Range of X,Y,Z accel –Energy in certain frequency bands –Car speed –… See Erikkson et al, MobiSys 2008

WiFi As A Sensor Is WiFi feasible as an uplink from cars? –Study with taxis; 30,000 distinct APs Answer: –Significant connectivity “in the wild” ~ 200 kB / minute Even at normal driving speeds –Connections last about 20 seconds –See an access point about every 20 seconds! See MOBICOM 2006

Measuring Urban Wi-Fi SFSeattle Chicago What are the performance properties of Wi-Fi networks from moving cars? Images from WiGLE.net

Experimental Setup 9 cars with CarTel software –Driving normally in Boston area –~32K access points (APs) –290 km of roads –~300 drive hours Fast scanning of WiFi access points, caching of AP parameters to speed up connection establishment Careful TCP data transfers to measure throughput, latency, loss rates Track performance statistics: connection durations, throughput distributions, etc. Area shown: ~20x15 km

Experimental Method scan associate get ip No access points found open AP found success e2e ping success local AP ping small tcp test upload ping success success 3 seconds of lost pings 9 cars with CarTel software Driving normally in Boston area ~32K access points (APs) 290 km of roads ~300 drive hours 9 cars with CarTel software Driving normally in Boston area ~32K access points (APs) 290 km of roads ~300 drive hours

Association Duration scanassociateget ipAP pingloss association duration time Last AP ping received 1 st AP ping received

Association Duration Distribution Several second associations - Across a range of speeds - Including fast scanning optimizations

Impact of Car’s Speed

Connections establish at a range of speeds Very little data at higher speeds

Estimating AP Coverage Procedure: 1.Note locations 2.Find bounding box 3.Report diagonal 200 ft 100 m location at the time of connection

Coverage Region Median coverage about 80 meters, tail surprisingly long

Disconnection Duration time scan associate get ip scan … … loss scan associate get ip tcp test upload delay between join attempts delay between end-to-end connections … …

Disconnection Duration Distribution Mean end-to-end disconnection time 260 s Mean time between assoc. attempts 23 s Will improve with higher openness Intriguing possibility for vehicular access

Bytes Transferred Per Connection Fraction of connections Data received (Kbytes) Non-trivial number of bytes per connection (mean=600 kB) Throughput consistent with home broadband access

WiFi as Sensor Part 2: vTrack Goal: Eliminate GPS –Adds cost (+$50 on $40 device) –Complicates deployment Requires line of site to sky Difficult in unattended phone use Approach –estimate position using WiFi + GPS logs –use road map and speed constraints to constrain estimates –match to roads Wifi Access Point

Raw WiFi

Demo, Result Summary Many tricky issues: –Moving APs –Looping routes –Crazy roads 2 Approaches –Iterative shortest paths –Particle-filters Good road and speed accuracy –~80% road segment accuracy

Roadmap Overview CarTel Components –Portal –ICEDB –Cabernet Deployments and Case Studies –User and cab deployment –Traffic Analysis –Wi-Fi Measurements Managing missing and uncertain data

Related Work Mobility in sensornets [Princeton/Zebranet,UCLA/NIMS,MIT/Underwater] DTN protocols: primarily focused on routing, but cf. [Rutgers/infostations,Intel/DTN-stack,Toronto/DTN-stack,CMU/DOT] Dist. query & stream processing: much work, but little when connectivity is intermittent [TinyDB,Cougar,Olston,Borealis] Back-end DBMS (geo-spatial, moving object databases) Traffic monitoring [JamBayes,traffic.com,smartraveler.com,inrix, Rutgers/TrafficLab,Umass/Diesel] –Focused on sensors on roads and/or real-time –CarTel instruments cars, mostly historic modeling Wireless / Wi-Fi monitoring –Very few previous studies of vehicular Wi-Fi

Conclusion Mobile and participatory sensornets can sense at much higher scale over larger areas than static networks Applications: traffic, fleet management, automotive diagnostics, wireless network monitoring, civil/environmental monitoring, traffic planning,… CarTel technologies: Portal, IceDB, Cabernet/QuickWifi, FunctionDB Platform enabled research results: Pothole Patrol, Traffic, WiFi as Uplink For more info: