Did You See Bob? Human Localization using Mobile Phones Ionut Constandache Duke University Presented by: Di Zhou Slides modified from Nichole Stockman

The Issue Finding someone in a public place can be difficult Might not know their location Maps and floor plans are not always available Might be unfamiliar with the area

Hypothetical Scenario Mobicom conference – in a big hotel Alice wants to meet her colleague Bob… – Can walk around – Can ask “Did you see Bob?” But these can take a long time and he may be moving – Can call him But he may be in a meeting already Best to have someone escort you… How?

Current Solutions GPS – Drains the battery (and not very precise) – Outdoor scheme WiFi/GSM schemes – Not enough accuracy than GPS – Require special infrastructure, RF transmitter or war-driving

Escort Guide a user to the vicinity of a desired person – mobile phone sensors – Opportunistic encounters – Client/server model Does not require: - Physical coordination – GPS/Wi-Fi – War-driving – Maps or floor plans Can be easily installed and provides localization service when GPS/WiFi not available

Outline Motivation Recap: – We need efficient and accurate software for people localization: Escort Outline – Basic Design of Escort – Challenges – Solutions – Experiment Specifics – Results – Future Work

Basic Design Escort consists of two parts : Navigation - Get directions to the person Visual Identification - End to end human localization

Part 1 - Navigation Clients report “Movement trail” periodically – A(t), B(t), C(t) – – Use accelerometer and compass measurements – Displacement = # of steps multiply step size – Direction read from compass Also report “Encounters” – T_AC, T_BC – – Definition ->Audio signals in inaudible frequencies within 5m Server builds a virtual graph Global view of users’ positions and paths ) A(t) C(t) B(t) T_AC T_BC ) B(t) ) C(t)

Challenges 1.Accelerometers & Compasses are noisy – Measured path error over time 2.No global reference frame – How to correct errors? 3.Even if correct user position, difficult to correct entire trail - Yet is necessary! (routing) 4. Trail graph grows over time

Solutions Accelerometers & Compasses are noisy – Use: (step size) * (step count) Displacement error using double integration Signature from up and down bounce of human body while walking

Solutions Accelerometers & Compasses are noisy – Use: (step size) * (step count) Error with step count method (avg 4%) Take into account varying step size Take into account varying step size (Vary step size with error factor drawn from Gaussian distribution centered on 0 and standard deviation 0.15m)

Solutions No global reference frame – Use a fixed beacon transmitter Beacon Transmitter – Location is origin of a virtual coordinate system – Location diffusion ( single point updates ) – Drift cancellation ( path correction )  sol’n 3

Solutions - 3 Drift Cancellation – Amortize the correction vector over time – Assume that user’s projected path deviates from the true path linearly over time Solid Line: actual path Dotted Line: user-computed trail Dashed Line: corrected user trail User encounters beacon at t_r1, and another beacon or recently updated user at t_r2.

Solutions 4 Graph computation done by Server – Pruning heuristic -> eliminate duplicates – Floyd-Warshall algorithm -> shortest paths Graph – 4 users, 10 min After pruning After Floyd-Warshall alg

Basic Design Escort consists of two parts : 1. Navigation 2. Visual Identification Help you identify the person if she is someone you have not met before (e.g. first-time meeting with a professional colleague at a conference)

Basic Design – Visual Identification Perhaps Alice has not met Bob before… Picture of face not enough In a trusted environment like Mobicom, can Take many photos of user to generate a “fingerprint” Alice takes photos of surroundings. Image processing is done to identify Bob from the photos Totally works in theory! Currently only implemented offline and requires user input…

Experiment Specifics Parking Lot Experiment – 4 users, 13 min, phones in hand, 40 routing exp’mts – Used parking spot lines & markers for ground truth (GPS not fine-grained enough) Indoor Experiment – 2 users, 6 min, 10 routing exp’mts

Results – Parking Lot Instantaneous location error over time

Results – Parking Lot Instantaneous location error < 10m: Intertial ~6% of cases Beacon & Encounter ~ 68% of cases Drift Cancellation ~ 84% of cases Final destination distance error 8.2m on average

Results – Indoor Final destination distance error was 7m on average Instantaneous location error: Better overall (due to indoor structure and user guesswork) Intertial (N/A because no GPS indoors) Beacon & Encounter ~ 85% of cases Drift Cancellation ~ 90% of cases

Results – Visual Identification Is this a good result? – 80% accurate with 8 people in surroundings

Future Work “Escort is not designed for energy efficiency” – Turn off sensors (except accelerometer) when user not moving – Less audio signaling when many users or beacons nearby – Less frequent uploading of data to server Routing through physical obstacles – People are smart – Visual representation of path may help Long routing paths – Include true- direction arrow to Bob’s location

Future Work Routing instructions under low location accuracy – If update too far in past, prompt user to approach beacon – Update path as more info becomes available Phone placement – Need advancement of phone gyroscopes to infer orientation Behavior under heavy user load – Scalability needs to be explored but should improve performance (more beacons/users = more updated info, etc)

Thank You Any Questions?