1. 1 Distributed localization of networked cameras Stanislav Funiak Carlos Guestrin Carnegie Mellon University Mark Paskin Stanford University Rahul Sukthankar Intel Research

2. 2 Distributed Localization of Cameras Place wireless cameras around an environment Need to know locations Costly to measure locations

3. 3 Distributed Localization of Cameras If camera 1 sees person, then camera 2 sees person, learn about relative positions of cameras As person moves around, estimate positions of all cameras

4. 4 Localization from pairwise distances Ihler et al., IPSN 2004 Whitehouse, Culler, ACM WSNA 02 Pollefeys, IJCV 2004 Soatto, Perona, IEEE PAMI 1998 Montemerlo et al., AAAI 2002 Paskin, IJCAI 2003 Simultaneous localization and mapping Structure from motion Simultaneous calibration and tracking Rahimi et al., CVPR 2004 Prior Work

5. 5 Distributed Localization of Cameras If camera 1 sees person, then camera 2 sees person, learn about relative positions of cameras As person moves around, estimate positions of all cameras Want a solution: online distributed represents uncertainty about estimated locations e.g. for active control

6. 6 Tracking with Kalman Filter: Estimation prior distribution Observation model: 1 2 object location camera poses prior distribution over object location: uncertain posterior distribution: more certain observation likelihood posterior distribution posterior distribution: even more certain image camera at known pose previous observations

7. 7 Tracking with Kalman Filter: Prediction t t+1 motion model Motion model: posterior distribution predicted distribution (prior at t+1)

8. 8 Camera Localization: Estimation d unknown camera pose prior distribution Start with wide prior on C Observe person at dist. d –Camera could be anywhere in a ring object location posterior distribution observation likelihood Posterior distribution in absolute parameters camera angle

9. 9 Kalman Filter uses a linear representation… Exact non-Gaussian posteriorGaussian approximation ? Far from ground truth Overconfident Exact posterior in absolute parameters Problem structure lost with Gaussian approximation ground truth estimate

10. 10 Ring distribution in polar coordinates – Almost Gaussian!!! Relative Over-Parameterization (ROP) Intuition: a ring structure can be represented with polar coordinates Not enough: Camera does not view person head on Relative over-parameterization – position relative to location of person 1.Distance u, angle  2.Lateral displacement v 3.The center – the unknown location of object u  (m x, m y ) ROP v  u -- ++

11. 11 Comparing parameterizations true posterior: best Gaussian approx. in x,y,  : Standard parameterization u  (m x, m y ) v ROP best Gaussian approx. in ROP:

12. 12 Test run on Tower Scenario standard parameterizationROP with further improvements (see paper)

13. 13 Donuts and Bananas on real data – Network of 5 cameras

14. 14 Distributed Localization of Cameras Goal: each camera estimates the location of itself and the object ROP lets us use a single Gaussian Challenges? 12 3 4 56 7 8 Want algorithm: efficient robust to message loss, node loss

15. 15 Motion model introduces dependencies tt + 1 Estimation at t : Prediction: Estimation at t +1 : Motion model introduces dependencies among distant cameras communication and computation inefficiency

16. 16 12 3 4 56 7 8 M t, C 1, C 2 M t, C 3, C 4 M t, C 4, C 5 M t, C 5, C 6 M t, C 6, C 7 M t, C 7, C 8 M t, C 2, C 3 Assumed density filtering C 1, C 2 C 2, C 3 C 3, C 4 C 4, C 5 C 5, C 6 C 6, C 7 C 7, C 8 Intuition: only capture strong dependencies among cameras based on [Boyen Koller 1998] Each clique containsEach clique contains: Camera and its neighbor Object location

17. 17 1.Assign each clique to one or more nodes can give clique to > 1 node for robustness 2.The nodes build a network junction tree [Paskin et al. 2005] build a routing tree ensure the flow of information Distributed Filtering: Initialization 12 3 4 56 7 8 12 3 4 56 7 8 M t, C 1, C 2 M t, C 3, C 4 M t, C 4, C 5 M t, C 5, C 6 M t, C 6, C 7 M t, C 2, C 3 M t, C 2, C 3, C 4 M t, C 7, C 8 M t, C 4, C 5, C 6 M t, C 6, C 7, C 8

18. 18 1.Each node starts with prior over its clique 2.Nodes make observations 3.Nodes communicate relevant likelihoods & priors neighbors 4.At convergence: condition on all measurements made in the network Distributed Filtering: Estimation 12 3 4 56 7 8 12 3 56 7 8 M t, C 1, C 2 M t, C 5, C 6 M t, C 6, C 7 M t, C 2, C 3 M t, C 2, C 3, C 4 M t, C 7, C 8 M t, C 4, C 5, C 6 M t, C 6, C 7, C 8 Instance of Robust Distributed Inference [Paskin Guestrin, UAI 2004]

19. 19 weak indirect dependence Prediction Revisited motion model posterior distribution prediction t t+1 How to implement the prediction step distributedly? How to prune weak dependencies? strong direct dependencies

20. 20 Distributed Filtering: Prediction Want the best approximation (minimizing KL divergence): captures short-range dependencies drops long-range dependencies Sufficient to compute the marginals over cliques [Boyen, Koller 1998]

21. 21 Summary of our approach 12 3 4 56 7 8 1.Each node maintains a clique marginal 2.Nodes build communication structure, network junction tree [Paskin et al. 2005] 3.Estimation: nodes condition on observations [Paskin & Guestrin UAI 04] 4.Prediction: best approximation computed locally M t, C 1, C 2 M t, C 2, C 3 M t, C 3, C 4 M t, C 4, C 5 M t, C 5, C 6 M t, C 6, C 7 M t, C 7, C 8

22. 22 Results: 44 simulated side-facing cameras

23. 23 Results: 44 simulated side-facing cameras

24. 24 Network of 25 cameras at Intel Research Pittsburgh

25. 25 Network of 25 cameras at Intel Research Pittsburgh

26. 26 Results: Model Complexity vs. Accuracy RMS error better pruning all dependencies dependencies among neighbors keeping all dependencies (exact solution) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

27. 27 Comparison with Rahimi et al., CVPR 2004 RMS error better pruning all dependencies dependencies among neighbors keeping all dependencies (exact solution) Rahimi et al. CVPR 2004 Our approach: distributed online estimates uncertainty

28. 28 Results: Communication vs. Accuracy RMS error better centralized solution 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 35101520 epochs per time step

29. 29 Conclusion Accurate camera localization with only a single Gaussian!!! –ROP – parameterization accurately representing ring-like distributions –Effective technique for incorporating nonlinear observations Distributed online algorithm for camera localization that represents uncertainty Algorithm for distributed filtering for general dynamic models Evaluated on network of 25 real cameras