Behrouz Haji Soleimani Dr. Moradi

Outline What is uncertainty? Some examples Solutions to uncertainty Ignoring uncertainty Markov Decision Process (MDP) Stochastic Motion Roadmap A detailed example Conclusion

What is uncertainty? Uncertainty in sensing the current state of the robot and workspace is not known with certainty Predictability the future state of the robot and workspace cannot be deterministically predicted even when the current state and future actions are known

Uncertainty in sensing It is not the world that is imperfect, it is our knowledge of it

Predictability Uncertainty in workspace Uncertainty in goal location Dynamic environments with moving obstacles Uncertainty in robot’s motion

Uncertainty example A robot with imperfect sensing must reach a goal location among moving obstacles (dynamic world)

Uncertainty example Robot created at Stanford’s ARL Lab to study issues in robot control and planning in no-gravity space environment air thrusters gas tank air bearing

Uncertainty in motion

Markov Decision Process (MDP) MDP is a general approach to considering uncertainty Determines model of the environment Descretizes state space Requires explicitly defining transition probabilities between states We can use dynamic programming to solve the MDP

Stochastic Motion Roadmap Combines a roadmap representation of configuration space with the theory of MDP’s Maximizes the probability of success Uses sampling to learn the configuration space (represented as states) learn the stochastic motion model (represented as state transition probabilities) Discretizes state space Discretizes actions

Stochastic Motion Roadmap Learning Phase Selecting random sample of discrete states Sample the robot’s motion model to build a Stochastic Motion Roadmap (SMR) Calculating transition probabilities for each action Query Phase Specify initial and goal states Roadmap is used to find a feasible path Possibly optimizing some criteria such as minimum length

Building the roadmap

Maximizing probability of success build an n × n transition probability matrix P(u) for each u U For each tuple (s, t, p), we set equals the probability of transitioning from state s to state t given that action u is performed

Maximizing probability of success

It is an MDP and has the form of the Bellman equation Where and It can be optimally solved using infinite horizon dynamic programming

A detailed example

Conclusion Uncertainty has a great effect on successfully reaching the goal MDP can consider uncertainty in the model SMR combines PRM and MDP to handle uncertainty SMR maximizes the probability of success SMR makes balance between path safety and minimum length Continuous actions in SMR is still an open question

References [1] R. Alterovitz, T. Simeon, and K. Goldberg, “The Stochastic Motion Roadmap: A Sampling Framework for Planning with Markov Motion Uncertainty” 2007 [2] R. Alterovitz, M. Branicky, and K. Goldberg, “Constant-curvature motion planning under uncertainty with applications in image-guided medical needle steering,” in Workshop on the Algorithmic Foundations of Robotics, July [3] R. Alterovitz, A. Lim, K. Goldberg, G. S. Chirikjian, and A. M. Okamura, “Steering flexible needles under Markov motion uncertainty,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), Aug. 2005, pp. 120–125. [4] B. Bouilly, T. Simeon, and R. Alami, “A numerical technique for planning motion strategies of a mobile robot in presence of uncertainty,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Nagoya, Japan, May 1995.

Questions ?

Thank you