1. Behrouz Haji Soleimani Dr. Moradi

2. Outline What is uncertainty? Some examples Solutions to uncertainty Ignoring uncertainty Markov Decision Process (MDP) Stochastic Motion Roadmap A detailed example Conclusion

3. What is uncertainty? Uncertainty in sensing the current state of the robot and workspace is not known with certainty Predictability the future state of the robot and workspace cannot be deterministically predicted even when the current state and future actions are known

4. Uncertainty in sensing It is not the world that is imperfect, it is our knowledge of it

5. Predictability Uncertainty in workspace Uncertainty in goal location Dynamic environments with moving obstacles Uncertainty in robot’s motion

6. Uncertainty example A robot with imperfect sensing must reach a goal location among moving obstacles (dynamic world)

7. Uncertainty example Robot created at Stanford’s ARL Lab to study issues in robot control and planning in no-gravity space environment air thrusters gas tank air bearing

8. Uncertainty in motion

10. Markov Decision Process (MDP) MDP is a general approach to considering uncertainty Determines model of the environment Descretizes state space Requires explicitly defining transition probabilities between states We can use dynamic programming to solve the MDP

11. Stochastic Motion Roadmap Combines a roadmap representation of configuration space with the theory of MDP’s Maximizes the probability of success Uses sampling to learn the configuration space (represented as states) learn the stochastic motion model (represented as state transition probabilities) Discretizes state space Discretizes actions

12. Stochastic Motion Roadmap Learning Phase Selecting random sample of discrete states Sample the robot’s motion model to build a Stochastic Motion Roadmap (SMR) Calculating transition probabilities for each action Query Phase Specify initial and goal states Roadmap is used to find a feasible path Possibly optimizing some criteria such as minimum length

13. Building the roadmap

15. Maximizing probability of success build an n × n transition probability matrix P(u) for each u U For each tuple (s, t, p), we set equals the probability of transitioning from state s to state t given that action u is performed

16. Maximizing probability of success

17. It is an MDP and has the form of the Bellman equation Where and It can be optimally solved using infinite horizon dynamic programming

18. A detailed example

22. Conclusion Uncertainty has a great effect on successfully reaching the goal MDP can consider uncertainty in the model SMR combines PRM and MDP to handle uncertainty SMR maximizes the probability of success SMR makes balance between path safety and minimum length Continuous actions in SMR is still an open question

23. References [1] R. Alterovitz, T. Simeon, and K. Goldberg, “The Stochastic Motion Roadmap: A Sampling Framework for Planning with Markov Motion Uncertainty” 2007 [2] R. Alterovitz, M. Branicky, and K. Goldberg, “Constant-curvature motion planning under uncertainty with applications in image-guided medical needle steering,” in Workshop on the Algorithmic Foundations of Robotics, July 2006. [3] R. Alterovitz, A. Lim, K. Goldberg, G. S. Chirikjian, and A. M. Okamura, “Steering flexible needles under Markov motion uncertainty,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), Aug. 2005, pp. 120–125. [4] B. Bouilly, T. Simeon, and R. Alami, “A numerical technique for planning motion strategies of a mobile robot in presence of uncertainty,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Nagoya, Japan, May 1995.

24. Questions ?

25. Thank you