Accurate Indoor Localization With Zero Start-up Cost Swarun Kumar Dina Katabi Stephanie Gil Daniela Rus Prepared by Brian Alberto Ignacio Reyes 115030990049

Outline Background on Indoor Localization and Ubicarse Motivation Results Translation Resilient SAR SAR and its limitations Angle of arrival estimation Ubicarse SAR Model Tablet Localization Accurate Device Localization Object Geotagging Application to object geotagging Refined Tablet localization using vision algorithms

Background on Indoor Localization and Ubicarse Motivation Related work and theory applied in Ubicarse’s design.

Indoor RF Localization Infrastructure based Fingerprinting based

Object Geo-tagging Stereo-imaging and 3D-Reconstruction based on vision algorithms Localize object relative to known camera location

Motivation for Ubicarse An accurate indoor localization system with No specialized infrastructure No fingerprinting How? Emulate a large antenna array to identify the spatial direction of RF Signals. Use of Synthetic Aperture Radar (SAR) to mimic a large antenna ray, but is infeasible in low- power handheld devices.

Synthetic Aperture Radar Assume accurate knowledge of position of the device at different points in time (Polar coordinates (r,φi) must be known)

Limitations on SAR approach Must rely on handheld motion sensors which Unusable translation at fine granularity measurement Accuracy on orientation is reduced over time due to integration shift Challengues of Ubicarse Perform SAR using MIMO capabilities of modern Wireless cards and take only the orientation information of the device

Ubicarse SAR Design Transilient SAR Model Ubicarse Device Localization Ubicarse Object Localization

Translation Resilient SAR Ubicarse takes advantage of modern Wireless cards MIMO capability. As the user twist the device, the 2 antennas samples the Wireless channel at each point of device’s trajectory. This samples forms a virtual array of antennas. But how to use it with SAR? Solution: Relative Wireless channel quantity into SAR Circular rotation of the device. (Distance between 2 antennas is fixed) Measure a sample of Wireless channel ℎ 𝑖 Find relative Wireless channel at i-th snapshot ℎ 𝑖 = ℎ 2,𝑖 ℎ 1,𝑖 ∗ Since the antenna never translate relative to each other, we can emulate a circular array. The complexity of our algorithm is O(n*log(m)). Use of Channel State Information (CSI). Extendable to OFDM by average power profile.

Multipath Considerations In the multipath scenario, the peaks add up constructively when 𝛼= 𝛼 𝑇 and destructively as 𝛼 deviates from 𝛼 𝑇 . For Multipath scenarios there exists m distinct paths 𝑑 1 , 𝑑 2 ,…, 𝑑 𝑚 with directions 𝛼 1 , 𝛼 2 ,…, 𝛼 𝑚 and the relative channel power sample consider the following assumptions If device translation remains constant between snapshots, the part of the relative channel power related to translation becomes constant multiplier. If device translation varies, this is interpreted as noise which drops after summing large number of snapshots. Although the model is based in 2D, the 3D components just add a multiple (using polar coordinar) sin𝛽 which resembles 2D model and hereby similar math.

Active Shift Compensation The error in orientation reported by motion sensor accumulates gradually, because gyroscope measure angular velocity with error which is integrated to larger error in location. Although this drift can be approximated by unknown linear shift, which Ubicarse solves by observing that a linear drift in orientation leads to constant shift in SAR multipath profiles. This is corrected using phase correlation, a technique used in computer graphics to estimate shift in 2 noise images.

Accurate Device Localization Ubicarse ask user to twist device around vertical axis Issues beacon frames to multiple neighboring AP Perform SAR to these AP and generate multipath profiles Apply standard triangulation to locate itself relative to AP Localization under multipath: Compare resulting multipath profiles to eliminate peak that doesn’t persist. Select the set of peaks to access points that best fit in optimization problem (geometrically). Device orientation estimate using Angle of arrival signal from AP, known direction of AP relative to global x axis, then global device orientation is obtained by difference of these 2 angles.

Application to object geotagging Visual toolkits: Use of overlapping photos from camera in different vantage points to produce 3D point clouds or relative 3D position in the local coordinate frame of the camera. Ubicarse locate the global position of device’s camera. So combining both we can the global position of an object. Vision Algorithms 3D Reconstruction of the object Relative camera position and orientation Identify salient features (Corners, textures) Identify position of objects between images Create 3D pointcloud Ubicarse can leverage the relative camera location output by vision algorithms to refine device location. This improves accuracy by 86 % Transformation to map objects camera related location to global reference frame

Point cloud of library shelving and VSFM Reconstruction

Results Implementation details and results

Implementation details Test Device HP SplitX2 on Ubuntu (Linux) with Intel 5300 Wireless card Yei Technology motion sensor (Accelerometer, gyroscope and compass) Wireless Channel measurement built over 802.11 CS Tool in [13] (See Paper) Ubicarse’s SAR built over C++ and MATLAB® transmit beacon packets 10 times/ second Ubicarse’s object localization uses VisualSFM (VSFM) toolkit and 5.7 MegaPixels tablet camera. Experiment conducted in MIT University Library with 5 802.11n WiFi AP on 5GHz Object localization: Multiple perspective images + VSFM+ Ubicarse device localization. Baseline: Ubicarse vs Angle-of-arrival scheme.

Library floorplan and test bed AP: Red blocks Book Shelves: hashed rectangles

Results Validating Translation Resilient of Ubicarse’s SAR for different trajectories Computing Angle of Arrival in 3D Localizing devices in 3D Object geotagging Refined Tablet localization using vision

1. Validating Translation Resilient of Ubicarse’s SAR for different trajectories 3.3, 3.4 and 3.3 degrees mean error for ii, iii and iv. Correctly identifies error to 1.6, 1.7 and 1.6 degree on average compared to In place.

2.Computing Angle of Arrival in 3D 3.2 degrees median in Azimuth orientation and 3.6 in polar angle. Inaccuracy of 18 𝜑 29 𝜃 𝑡𝑜 48 𝜑 67 𝜃 without compesation

3. Localizing devices in 3D Median Device localization error of 39cm (22 cm in x, 28 cm in y, 18cm in z) and 6.09degrees in global device localization In NLOS additional median device localization error of (10 cm in x, 18 cm in y, 2cm in z) and 7.7degrees in global device localization

4. Object geotagging Ubicarse + VSFM: Median error of 17cm ( 5cm in x, 15cm in y,4cm in z) Up to 3m inaccuracy. Solution: Combine Ubicarse and VSFM for Localization Localization Refinement

5. Refined Tablet localization using Vision Median error of ground 15 cm Improvements 66% in X 16% in Y 86% in Z CDF

Comments on Ubicarse The advantage of the system is no infrastructure and low overhead to obtain indoor localization. The cost? High density computation on visual algorithms for mobile devices, which are the main reason of accuracy of the system. Improvements? Since indoor application is mostly restricted to 1 or some building, define PointCloud database and computation of object localization on cloud or server to reduce computation. New version of library shelf ? User agrees to computation, send this to data to cloud (or server) and updates differences. Ideas for improvement?

Thank you for your attention.