1. When the Sensors Hit the Road The CarTel Project Sam Madden MIT CSAIL http://cartel.csail.mit.edu With Hari Balakrishnan, Vladimir Bychkovsky, Jakob Eriksson, Bret Hull, Yang Zhang, Kevin Chen, Waseem Daher, Michel Goraczko, Hongyi Hu, Sejoon Lim, Allen Miu, Daniela Rus, Eugene Shih, Arvind Thiagarajan, Sivan Toledo Stanford Database Group

2. 1 st Generation Sensor Networks Periodic monitoring –Wake up and sense –Sleep for Minutes Event-based monitoring –Transmit on external event Low data rates, duty cyles Static, small area Extremely underpowered devices “Mote” ~=

3. Participatory Sensing “Crowd-sourced” sensing –Individuals contribute their data –Rather than relying on fixed deployments Examples: –Personalized Environmental Impact Report –Microsoft SensorMap –Dartmouth BikeNet –MapMyRun, etc. CarTel

4. Cars as a Vehicle for Participatory Sensing Observation: –Static sensing infeasible over very wide areas –Some apps do not need high temporal fidelity Real-world problems: –Civil infrastructure monitoring –Road-surface conditions –Visual mapping –Commute optimization –Carpool finding –Speed trap mapping

5. Opportunistic Mobility Rather than deploy new mobile nodes, take advantage of existing mobility Example: cellphones w/ sensors –1.5 billion phones worldwide –High spatial coverage –High-performance processor Cars equipped with sensors –650 million cars on the road –Abundance of power and space –Have >100 embedded sensors What system architecture is best suited for mobile, wide-area sensing?

6. Data collection servers Clients Open Wi-Fi GPRS Collect / Process Deliver Visualize CarTel System Overview Portal Cabernet / QuickWifi (Carry & Forward Network) ICEDB (Intermittently Connected & Embedded DB) Per-node sensing & computation hardware

7. Roadmap Overview CarTel Components –Portal –IceDB –Cabernet Managing missing and uncertain data Case Studies –Traffic Analysis –WiFi Mapping –Potholes

8. Portal Web-based visualization framework Apps retrieve sensor data by issuing queries to ICEDB Visualize sensor data using map overlays Continuous queries to direct sensing –Pushed to remote nodes using Cabernet Local ad-hoc queries read streaming results Visualize / Analyze Web server Portal Applications TrafficWi-Fi ICEDB server CQ Rel. DB Cabernet Portal Data Viz. Streaming sensor data Cont. queries

9. Visualize / Analyze Data Collection Demo

10. ICEDB: Intermittently Connected Embedded DB Relational model is a convenient abstraction for data collection ICEDB: –Queries written in extended SQL –Continuous query processor –Distributed Bandwidth is variable 1.Buffer query and DDL commands 2.Buffer query results 3.Support “drill down” queries 4.Prioritize results Collect / Process SELECT img FROM camera, gps WHERE gps.pos in [x,y] AND camera.time= gps.time SAMPLE 1s

11. ICEDB Query Processing DB CQ Ad-hoc Query Processor Output Buffers Cabernet ADAPTERSADAPTERS sensor Collect / Process Remote Node 2 data paths to cope with limited BW ICEDB Server ICEDB Remote results queries

12. Inter-query Prioritization Remote node BISECT Portal SELECT lat, lon FROM gps WHERE insert_time > cqtime – 5 EVERY 5 seconds BUFFER IN gpsbuf DELIVERY ORDER bisect(gpsbuf) Portal Remote node FIFO Portal t=1t=2t=3 SELECT lat, lon FROM gps WHERE insert_time > cqtime – 5 EVERY 5 seconds BUFFER IN gpsbuf DELIVERY ORDER fifo(gpsbuf) Collect / Process

13. Global Prioritization Collect / Process to ICEDB Server to ICEDB Server SELECT lat, lon, image FROM camera WHERE insert_time > cqtime – 5 EVERY 5 seconds BUFFER IN cambuf SUMMARIZE AS SELECT floor(lat/100), floor(lon/100) FROM cambuf GROUP BY floor(lat/100), floor(lon/100)

14. Global Prioritization Collect / Process ICEDB Server 1 2 2 2 1 1

15. Specifying Priorities Three SQL language extensions: 1.Global prioritization: SUMMARIZE AS 2.Inter-query prioritization: PRIORITY 3.Intra-query prioritization: DELIVERY ORDER BY For more details: ICDE ’06 Collect / Process

16. Cabernet Disconnection-tolerant transport layer Buffer data until connectivity becomes available –Not connection oriented QuickWifi: Fast connection establishment CTP: Unlike TCP, wireless losses ≠ congestion losses Deliver QuickWifi Jakob Eriksson et al – Mobicom 08

17. Challenge: Connection Establishment is Slow Process –Scan 11 channels (600 ms average), receive beacon –Authenticate (1 round trip, 3s timeout) –Associate (1 round trip, 3s timeout) –DHCP Discovery (1 round trip, 3s timeout) –DHCP Request (1 round trip, 3s timeout) –ARP Request (1 round trip, 2s timeout) Many messages, loss rates high, retries often needed Default Linux stack took 13s to associate, on average (whole connection lasts only 19s avg!) Deliver

18. Connection Optimizations Scan more popular channels first (1,6,11) Since authentication in open networks always succeeds, do it in parallel with association Set timeouts to 100 ms Mean conn. time: 370 ms DHCP via broadcast addr

19. Roadmap Overview CarTel Components –Portal –Cabernet Managing missing and uncertain data Case Studies –Traffic Analysis –WiFi Mapping –Potholes

20. Noisy Data Challenge Challenge: how to store and query all of this data? Discrete points don’t work well Most apps don’t actually want raw data! –Prefer trajectories, fields, fit functions –Idea: support these as first class objects inside the DBMS

21. Model-Based Views Proposed in MauveDB [SIGMOD 06] –Models can be queried like database views –Basic idea: compute model grid modeled area use model to compute values at each cell of grid answer queries using grid FunctionDB: efficient implementation for an important class of models –Continuous functions of one or more variables

22. Benefits Declarative Queries –No need to write procedures to manipulate models, or re-implement or re-optimize each new query View provides intuitive SQL-like interface If data is already in database, do not need to move data to/from math package like MATLAB

23. FunctionDB: Key Idea Database system that fits continuous functions to data MauveDB-like query Interface (SQL + Grids) Algebraic query processor Raw data (temp readings) Query: Report when temp crosses threshold SELECT time WHERE temp = thresh Regression Function temp(t) Solve equation temp(t) = thresh temp time

24. FunctionDB: System Architecture Raw Data “Function Table” (Storing Piecewise Model) XY 11.1 33.1 43.9 …… StartEndSlop e Intercep t 1810 8122-8 User fits or imports functions Algebraic Query Processor (Operates on algebraic representation) User QueryQuery Result

25. Query Results Grid semantics: all queries yield discrete points sampled at user-specified interval (“grid size”) SELECT x,y WHERE temp < 20 GRID x 8, y 8 8 8 temp < 20

26. Algebraic Execution: Overview FunctionDB supports efficient algebraic execution for piecewise polynomial functions Restriction helps achieve grid semantics even when no closed form solutions are available E.g., Non-linear polynomials, multi-variate polynomials, complex constraints X^2 + Y^3 < 25

27. SELECT * WHERE Temp < 20 GRID X 8, Y 8 Efficient Algebraic Implementation User X:[0,20), Y:[0,20), Temp = X+Y X:[0,20), Y:[0,20), Temp = X+Y, Temp < 20 Substitute Approx X:[0,20), Y:[0,20), Temp = X+Y, X+Y-20 < 0 Grid Symbolic Filter (Temp < 20) Function Table Temp = F(X,Y) Gridded Result Hypercubes+Boundary Points X+Y < 20

28. Hypercube Approximation Use technique from graphics rasterization (Taubin’s test) Tests if polynomial F(X,Y, …) can have zeros within a given hypercube H (i.e., F lies on boundary of H) Hypercube H Z = F(X,Y) Z = 0 Y X Zero of F Allows efficient pruning by testing corners of hypercube

29. Subdivision Algorithm Predicate Bounding Box - - - + Check centers of all grid cells smaller than grid size + + + +

30. More Algebraic Operators Solver for single-variable equations Function Inference E.g., X+Y = 20  X = 20-Y (Eliminates independent variables) Continuous aggregates E.g., Average  Integration

31. Evaluation: Temperature Data Data: 54 sensors, 10 days of data, 1 million raw data points Fit regression model temp = F(X,Y,time) Degree-2 piecewise polynomial with 22 pieces Evaluation queries: Find regions where temp < threshold (Filter) Area of region where temp < threshold (Aggregate) Compared algebraic approach to two baselines: Evaluating all grid points Pruning search using a precomputed B-Tree

32. Result 1: Benefits Of Algebraic Execution Depend On Selectivity

33. Result 2: Algebraic Execution Wins Significantly For Aggregate Queries Wider grid size is faster, but too wide a grid size results in low accuracy owing to discretization error

34. Roadmap Overview CarTel Components –Portal –Cabernet Managing missing and uncertain data Case Studies –Traffic Analysis –WiFi Mapping –Potholes

35. Route Planning Match traces to map Compute Gaussian delay for each segment –Assume independence Minimize 3 Objectives –Distance Google Maps –Expected delay –Pr(missing time goal)

36. Max. Probability Planning Travel time of each edge is a Gaussian –If indepdendent, travel time of a path is also Gaussian Goal: find path with max. probability of reaching destination by deadline Unlike standard shortest paths, no optimal substructure –If AxCyB is best path from A to B, AxC is not necessarily the best path from A to C Implies cannot use A* or Dijkstra 2 ABC 1 3 Lim et al. “Stochastic Motion Planning and Applications to Traffic.” To Appear, Workshop on Algorithmic Foundations of Robotics, 2008

37. Finding Potholes

38. Classification-based Approach Classifier differentiates between several types of anomalies Window data, compute features per window Variety of features: –Range of X,Y,Z accel –Energy in certain frequency bands –Car speed –… See Erikkson et al, MobiSys 2008

39. WiFi As A Sensor Is WiFi feasible as an uplink from cars? –Study with taxis; 30,000 distinct APs Answer: –Significant connectivity “in the wild” ~ 200 kB / minute Even at normal driving speeds –Connections last about 20 seconds –See an access point about every 20 seconds! See MOBICOM 2006

40. Measuring Urban Wi-Fi SFSeattle Chicago What are the performance properties of Wi-Fi networks from moving cars? Images from WiGLE.net

41. Experimental Setup 9 cars with CarTel software –Driving normally in Boston area –~32K access points (APs) –290 km of roads –~300 drive hours Fast scanning of WiFi access points, caching of AP parameters to speed up connection establishment Careful TCP data transfers to measure throughput, latency, loss rates Track performance statistics: connection durations, throughput distributions, etc. Area shown: ~20x15 km

42. Experimental Method scan associate get ip No access points found open AP found success e2e ping success local AP ping small tcp test upload ping success success 3 seconds of lost pings 9 cars with CarTel software Driving normally in Boston area ~32K access points (APs) 290 km of roads ~300 drive hours 9 cars with CarTel software Driving normally in Boston area ~32K access points (APs) 290 km of roads ~300 drive hours

43. Association Duration scanassociateget ipAP pingloss association duration time Last AP ping received 1 st AP ping received

44. Association Duration Distribution Several second associations - Across a range of speeds - Including fast scanning optimizations

45. Impact of Car’s Speed

46. Connections establish at a range of speeds Very little data at higher speeds

47. Estimating AP Coverage Procedure: 1.Note locations 2.Find bounding box 3.Report diagonal 200 ft 100 m location at the time of connection

48. Coverage Region Median coverage about 80 meters, tail surprisingly long

49. Disconnection Duration time scan associate get ip scan … … loss scan associate get ip tcp test upload delay between join attempts delay between end-to-end connections … …

50. Disconnection Duration Distribution Mean end-to-end disconnection time 260 s Mean time between assoc. attempts 23 s Will improve with higher openness Intriguing possibility for vehicular access

51. Bytes Transferred Per Connection Fraction of connections Data received (Kbytes) Non-trivial number of bytes per connection (mean=600 kB) Throughput consistent with home broadband access

52. WiFi as Sensor Part 2: vTrack Goal: Eliminate GPS –Adds cost (+$50 on $40 device) –Complicates deployment Requires line of site to sky Difficult in unattended phone use Approach –estimate position using WiFi + GPS logs –use road map and speed constraints to constrain estimates –match to roads Wifi Access Point

53. Raw WiFi

54. Demo, Result Summary Many tricky issues: –Moving APs –Looping routes –Crazy roads 2 Approaches –Iterative shortest paths –Particle-filters Good road and speed accuracy –~80% road segment accuracy

55. Roadmap Overview CarTel Components –Portal –ICEDB –Cabernet Deployments and Case Studies –User and cab deployment –Traffic Analysis –Wi-Fi Measurements Managing missing and uncertain data

56. Related Work Mobility in sensornets [Princeton/Zebranet,UCLA/NIMS,MIT/Underwater] DTN protocols: primarily focused on routing, but cf. [Rutgers/infostations,Intel/DTN-stack,Toronto/DTN-stack,CMU/DOT] Dist. query & stream processing: much work, but little when connectivity is intermittent [TinyDB,Cougar,Olston,Borealis] Back-end DBMS (geo-spatial, moving object databases) Traffic monitoring [JamBayes,traffic.com,smartraveler.com,inrix, Rutgers/TrafficLab,Umass/Diesel] –Focused on sensors on roads and/or real-time –CarTel instruments cars, mostly historic modeling Wireless / Wi-Fi monitoring –Very few previous studies of vehicular Wi-Fi

57. Conclusion Mobile and participatory sensornets can sense at much higher scale over larger areas than static networks Applications: traffic, fleet management, automotive diagnostics, wireless network monitoring, civil/environmental monitoring, traffic planning,… CarTel technologies: Portal, IceDB, Cabernet/QuickWifi, FunctionDB Platform enabled research results: Pothole Patrol, Traffic, WiFi as Uplink For more info: http://cartel.csail.mit.eduhttp://cartel.csail.mit.edu