1. 1 AM3 Task 1.3 Navigation Using Spatio- Temporal Gaussian Processes Songhwai Oh (Presented by Sam Burden) MAST Annual Review University of Pennsylvania March 8-9, 2010

2. 2 Overview Songhwai Oh – Professor, Seoul National University – In collaboration with Prof. Shankar Sastry (UC Berkeley) Goal: control and navigation in unstructured and uncertain environments – Model environment as a Gaussian Process (GP) – Efficiently incorporate new data – Use GP for control and navigation Expected Results at End of Fiscal Year: multiple agents learn GP model, navigate new environments Navigation Using Spatio-Temporal Gaussian Processes

3. 3 Technical Relevance Current techniques in GP regression – Offline Global gradient descent for parameter tuning Incorporating new observations requires full computation – Centralized Estimation decoupled from control and navigation No natural way to integrate data from multiple agents Our approach to GPs – Real-time – Distributed – Integrate estimation with navigation

4. 4 Relevance to MAST Our approach to GPs – Real-time – Distributed – Integrate estimation with navigation Enables MAST platforms to navigate in unstructured and uncertain environments

5. 5 1Q09 (A-09-1.3a): Spatio-temporal GPs: done 2Q09 (A-09-1.3b): Integrate GP with MPC: ongoing 3Q09 (A-09-1.3c): Assumed-Density GP – Developed a distributed GP learning algorithm instead 4Q09 (A-09-1.3d): Assumed-Density GP with MPC – Ongoing; developed bio-inspired navigation strategies [2] 1Q10 (A-10-1.3a): Approximation – Developed rigorous approximation for truncated Gaussian Process 2Q10 (A-10-1.3b): Multi-Agent Integration – Share observations among agents, coordinate navigation 3Q10 (A-10-1.3c): Learning GP Kernel 4Q10 (A-10-1.3d): Implement on MAST Platform 1Q09 (A-09-1.3a): Spatio-temporal GPs: 2Q09 (A-09-1.3b): Integrate GP with MPC: 3Q09 (A-09-1.3c): Assumed-Density GP 4Q09 (A-09-1.3d): Assumed-Density GP with MPC 1Q10 (A-10-1.3a): Approximation 2Q10 (A-10-1.3b): Multi-Agent Integration Technical Accomplishments

6. 6 Gaussian Process Estimation Unknown / uncertain environment modeled as a random process specified by mean and covariance Gaussian Process Regression provides straightforward but costly way to estimate process How can we reduce GP computation without degrading performance?

7. 7 Truncated Gaussian Process Idea: recent measurements are more informative Formalize, provide bounds to quantify tradeoff between accuracy and computation Approximation Error Truncation Size

8. 8 Collaborations and Future Plans Prof. Herbert Tanner (Autonomy, UDelaware) – Model Predictive Navigation using GPs – Helps avoid local minima Possible new collaborative efforts / experiments – Use GPs to map occupancy and wireless signal strength Ideas going forward – Nonstationary GP kernels for mapping – Efficient estimation of GP parameters – Mixture-of-GPs to estimate multiscale phenomena

9. 9 Collaborations and Future Plans Prof. Herbert Tanner (Autonomy, UDelaware) – Model Predictive Navigation using GPs – Helps avoid local minima Possible new collaborative efforts / experiments – Use GPs to map occupancy and wireless signal strength Ideas going forward – Nonstationary GP kernels for mapping – Efficient estimation of GP parameters – Mixture-of-GPs to estimate multiscale phenomena

10. 10 Metrics [1] Jongeun Choi, Songhwai Oh, and Roberto Horowitz, Distributed Learning and Cooperative Control for Multi-Agent Systems. Automatica, vol. 45, no. 12, pp. 2802-2814, Dec. 2009. [2] Jongeun Choi, Joonho Lee, and Songhwai Oh, Navigation Strategies for Swarm Intelligence using Spatio-Temporal Gaussian Processes. Submitted to Neural Computing and Applications. [3] Yunfei Xu, Jongeun Choi and Songhwai Oh, Mobile Sensor Network Coordination Using Gaussian Processes with Truncated Observations. Submitted to IEEE Transactions on Robotics.

11. 11 AM3 Task 1.4 Stochastic Hybrid Models for Aerial and Ground Vehicles Sam Burden MAST Annual Review University of Pennsylvania March 8-9, 2010

12. 12 Overview Sam Burden – Ph.D. Student, UC Berkeley – Advised by Prof. Shankar Sastry Collaborators: – Aaron Hoover (Prof. Ronald Fearing, Micromechanics, UC Berkeley) – Prof. Robert Full (Micromechanics, UC Berkeley) Goal: develop general principles for hybrid system identification, apply to MAST platforms – High-fidelity 3D motion capture system – Theoretical principles for identification of stochastic hybrid systems – Apply model to aid control and improve design of the platforms Expected Results at End of Fiscal Year: model for terrestrial platform, controller to execute beacon following behavior Stochastic Hybrid Models for Aerial and Ground Vehicles

13. 13 High-fidelity 3D Motion Capture System Identification State Estimation – Camera calibration via bundle adjustment (Argyros, 2009) – Open source library developed with Python – Linear systems (Ljung, 1987) – Continuum dynamics / PDEs (Tomlin, 2006) – Piecewise affine hybrid systems (Niessen, 2005) – Stochastic hybrid systems (Lygeros, 2008) – Linear systems (Kalman Filter (KF); Rauch, Tung, Striebel, 1965) – Nonlinear Systems (Unscented KF; Julier and Uhlmann, 1995) We are using nonlinear geometric theory to identify stochastic hybrid dynamics Technical Relevance

14. 14 Relevance to MAST Experimental tools ( http://eecs.berkeley.edu/~sburden/python ) – High-fidelity 3D motion capture – State estimation for stochastic nonlinear and hybrid systems Theoretical / Modeling tools – Unified theoretical framework for stochastic systems – System identification in this general framework Experimental outcomes – Develop empirically-validated model for terrestrial platform – Execute useful low-level behavior, e.g. beacon following

15. 15 1Q10 (A-10-1.4a): Experimentation – Evaluated fidelity of VICON; inadequate for present application – Developed high-speed camera calibration software suite – Tools available at http://eecs.berkeley.edu/~sburden/python 2Q10 (A-10-1.4b): Hybrid System Identification – Creating geometric framework for ID problem – Applying framework to abstract mathematical models – Implementing estimation and identification 3Q10 (A-10-1.4c): Characterize System Noise 4Q10 (A-10-1.4d): Control MAST Platform 1Q10 (A-10-1.4a): Experimentation – Evaluated fidelity of VICON; inadequate for present application – Software for camera calibration and nonlinear state estimation – Tools available at http://eecs.berkeley.edu/~sburden/python 2Q10 (A-10-1.4b): Hybrid System Identification – Creating geometric framework for ID problem – Applying framework to abstract models for locomotion – Implementation for empirical robot data 3Q10 (A-10-1.4c): Characterize System Noise 4Q10 (A-10-1.4d): Control MAST Platform Technical Accomplishments

16. 16 HS: Intuitive Picture Natural abstraction for running, flapping, & climbing robots

17. 17 HS ID: Illustrative Example Inelastic Bouncing Ball Velocity vector `jumps’ discontinuously Kalman filter, particle filter will fail to estimate state

18. 18 HS ID: Illustrative Example Inelastic Bouncing Ball How can we identify the dynamics? We can solve this problem in a geometric framework – i.e. for running, flapping, climbing robots

19. 19 Collaborations Weekly Meetings – Aaron Hoover (Prof. Ronald Fearing, Micromechanics, UC Berkeley) Design of terrestrial platform & control of platform Empirical evaluation of dynamical abstractions – Prof. Robert Full (Micromechanics, UC Berkeley) Experiment design and resources (force platform, HS cameras) HS ID for running cockroaches Monthly Meetings: Berkeley MAST Group – Autonomy, Micromechanics, Integration, Microelectronics Possible new collaborative efforts / experiments – Identification for other MAST platforms with hybrid dynamics – Implement controllers, improve design using identified dynamics

20. 20 Future Plans 3Q—4Q 2010: Plans – Develop, validate open-loop model for terrestrial robot – Characterize dynamical uncertainty with respect to model – Design control scheme using stochastic hybrid model 2011—2013: Ideas, Goals – Improve estimation and identification for hybrid systems using e.g. particle filters in abstract geometric spaces – Model the effect of varying terrain and morphology esp. as a means to decrease dynamical uncertainty – Closed-loop model for robot dynamics by explicitly considering dynamical effect of control effort

21. 21 Future Plans 3Q—4Q 2010: Plans – Develop, validate open-loop model for terrestrial robot – Characterize dynamical uncertainty with respect to model – Design control scheme using stochastic hybrid model 2011—2013: Ideas, Goals – Improve estimation and identification for hybrid systems using e.g. particle filters in abstract geometric spaces – Model the effect of varying terrain and morphology esp. as a means to decrease dynamical uncertainty – Closed-loop model for robot dynamics by explicitly considering dynamical effect of control effort

22. 22 Discussion & Questions Metrics – Presenting Hybrid System ID work at HSCC in April, 2010 (HSCC: Hybrid Systems, Computation, and Control) Thank you for your time Collaborators – Aaron Hoover (Prof. Ronald Fearing) – Prof. Robert Full Support – ARL MAST (Autonomy Center) – NSF GRF – Prof. Shankar Sastry Acknowledgements

23. 23 Technical Slides Hybrid System Formal Framework HS Identification Problem Statement HS ID Intuitive Example HS ID Recap

24. 24 HS: Formal Framework Consider hybrid dynamical systems 1 H := (Q, D, F, R) : 1. Bernardo et. al. 2007 Nice properties – Determinism – Existence & Uniqueness – Structural Stability

25. 25 Given output from the discrete-time stochastic model estimate the state. HS ID: Problem Statement

26. 26 HS ID: Illustrative Example Inelastic Bouncing Ball Velocity is discontinuous when ball bounces Kalman filter, Particle filter will give poor estimates after the bounce We can solve this problem We can identify general hybrid dynamical systems

27. 27 HS ID: Recap We consider a general class of hybrid systems We wish to estimate the state in the presence of uncertainty and noisy measurements We can solve this problem in general – i.e. for walking, flapping, climbing robots ex:

28. 28 Technical Slides Gaussian Processes Spatio-Temporal Conditional Distribution Dynamics and Sensing Navigation Strategies Path Planning Switched Path Planning Truncated Gaussian Processes Navigation using Truncated Observations

29. 29 Gaussian Process A Gaussian process (GP) is a stochastic process. Any finite number of samples from a GP has a Gaussian distribution Regression using Gaussian processes: – Widely used in geostatistics (Kriging), statistics, machine learning GP can be used to estimate a field such as – mapping 1 – radio signal strength (WiFi-SLAM 2 ) – danger level or stealthiness – others: temperature, lighting, noise-level, etc. 1 [O'Callaghan, Ramos, Durrant-Whyte, 2009] 2 [Ferris, Fox, Lawrence, 2007]

30. 30 Recap: Spatio-Temporal Gaussian Processes

31. 31 Recap: Conditional Distribution

32. 32 Recap: Agent Dynamics and Sensing

33. 33 Recap: Navigation Strategies

34. 34 Path Planning 1/25/2010 Flocking Consensus Navigation Goal

35. 35 Switching Path Planning Assumption: Stationary field, ¾ max 2 > k(s,s) Algorithm: 1.Starts with exploration strategy 2.If (max(prediction error) < ² ) Switch to exploitation (tracing) strategy Theorem [2]: Under some smoothness conditions, with probability at least 1- ¾ max 2 / ² 2, |z(s,t)-z*(s, ¿ |t)| t+1, for all s. 35

36. 36 Switching Path Planning 36 Field to be estimated Trajectories of agents and estimated field

37. 37 Switching Path Planning 37 RMS over simulation time (100 Monte Carlo runs) Predicted variance, RMS, maximum errors of agents performing switching path planning.

38. 38 Truncated Gaussian Processes 38 Motivation Keeping all measurements up to time t requires large memory and computation time. Most recent measurements are more informative. Kernel function of spatio-temporal Kalman filter Difference in prediction error variance as a function of the truncation size. (Blue: ¾ t 2 = 5, red: ¾ t 2 = 10.)

39. 39 Truncated Gaussian Processes 39 Theorem [3]: