2. SA-1 Probabilistic Robotics Tutorial AAAI-2000 Sebastian Thrun Computer Science and Robotics Carnegie Mellon University

3. SA-1 © Sebastian Thrun, CMU, 20002 Recommended Readings Probabilistic Algorithms in Robotics (basic survey, 95 references) AI Magazine (to appear Dec 2000) Also: Tech Report: CMU-CS-00-126 http://www.cs.cmu.edu/~thrun/papers/thrun.probrob.html

4. SA-1 © Sebastian Thrun, CMU, 20003 Collaborators and Funding Anita Arendra Michael Beetz Maren Bennewitz Eric Bauer Joachim Buhmann Wolfram Burgard Armin B. Cremers Frank Dellaert Dieter Fox Dirk Hähnel John Langford Gerhard Lakemeyer Dimitris Margaritis Michael Montemerlo Sungwoo Park Frank Pfenning Joelle Pineau Martha Pollack Charles Rosenberg Nicholas Roy Jamieson Schulte Reid Simmons Dirk Schulz Wolli Steiner Special thanks: Kurt Konolige, John Leonard, Andrew Moore, Reid Simmons Sponsored by: DARPA (TMR, CoABS, MARS), NSF (CAREER, IIS, LIS), EC, Daimler Benz, Microsoft + others

5. SA-1 © Sebastian Thrun, CMU, 20004 Tutorial Goal To familiarize you with probabilistic paradigm in robotics Basic techniques Advantages Pitfalls and limitations Successful Applications Open research issues

6. SA-1 © Sebastian Thrun, CMU, 20005 Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

7. SA-1 © Sebastian Thrun, CMU, 20006 Robotics Yesterday

8. SA-1 © Sebastian Thrun, CMU, 20007 Robotics Today

9. SA-1 © Sebastian Thrun, CMU, 20008 Robotics Tomorrow?

10. SA-1 © Sebastian Thrun, CMU, 20009 Current Trends in Robotics Robots are moving away from factory floors to Entertainment, toys Personal services Medical, surgery Industrial automatization (mining, harvesting, …) Hazardous environments (space, underwater)

11. SA-1 © Sebastian Thrun, CMU, 200010 Robots are Inherently Uncertain  Uncertainty arises from four major factors: Environment stochastic, unpredictable Robot stochastic Sensor limited, noisy Models inaccurate

12. SA-1 © Sebastian Thrun, CMU, 200011 Probabilistic Robotics

13. SA-1 © Sebastian Thrun, CMU, 200012 Probabilistic Robotics Key idea: Explicit representation of uncertainty (using the calculus of probability theory)  Perception = state estimation  Action = utility optimization

14. SA-1 © Sebastian Thrun, CMU, 200013 Advantages of Probabilistic Paradigm  Can accommodate inaccurate models  Can accommodate imperfect sensors  Robust in real-world applications  Best known approach to many hard robotics problems

15. SA-1 © Sebastian Thrun, CMU, 200014 Pitfalls  Computationally demanding  False assumptions  Approximate

16. SA-1 © Sebastian Thrun, CMU, 200015 Trends in Robotics Reactive Paradigm (mid-80’s) no models relies heavily on good sensing Probabilistic Robotics (since mid-90’s) seamless integration of models and sensing inaccurate models, inaccurate sensors Hybrids (since 90’s) model-based at higher levels reactive at lower levels Classical Robotics (mid-70’s) exact models no sensing necessary

17. SA-1 © Sebastian Thrun, CMU, 200016 Example: Museum Tour-Guides Robots Rhino, 1997Minerva, 1998

18. SA-1 © Sebastian Thrun, CMU, 200017 Rhino (Univ. Bonn + CMU, 1997) W. Burgard, A.B. Cremers, D. Fox, D. Hähnel, G. Lakemeyer, D. Schulz, W. Steiner, S. Thrun

19. SA-1 © Sebastian Thrun, CMU, 200018 Minerva (CMU + Univ. Bonn, 1998) Minerva S. Thrun, M. Beetz, M. Bennewitz, W. Burgard, A.B. Cremers, F. Dellaert, D. Fox, D. Hähnel, C. Rosenberg, N. Roy, J. Schulte, D. Schulz

20. SA-1 © Sebastian Thrun, CMU, 200019 “How Intelligent Is Minerva?” fishdogmonkeyhumanamoeba 5.7% 29.5% 25.4% 36.9% 2.5%

21. SA-1 © Sebastian Thrun, CMU, 200020 “Is Minerva Alive?" undecidednoyes 3.2% 27.0% 69.8%

22. SA-1 © Sebastian Thrun, CMU, 200021 “Is Minerva Alive?" undecidednoyes 3.2% 27.0% 69.8% “Are You Under 10 Years of Age?”

23. SA-1 © Sebastian Thrun, CMU, 200022 Nature of Sensor Data Odometry Data Range Data

24. SA-1 © Sebastian Thrun, CMU, 200023 Technical Challenges  Navigation Environment crowded, unpredictable Environment unmodified “Invisible” hazards Walking speed or faster High failure costs  Interaction Individuals and crowds Museum visitors’ first encounter Age 2 through 99 Spend less than 15 minutes

25. SA-1 © Sebastian Thrun, CMU, 200024

26. SA-1 © Sebastian Thrun, CMU, 200025 Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

27. SA-1 © Sebastian Thrun, CMU, 200026 The Localization Problem  Estimate robot’s coordinates s=(x,y,  ) from sensor data Position tracking (error bounded) Global localization (unbounded error) Kidnapping (recovery from failure) Ingemar Cox (1991): “Using sensory information to locate the robot in its environment is the most fundamental problem to provide a mobile robot with autonomous capabilities.” see also [Borenstein et al, 96]

28. SA-1 © Sebastian Thrun, CMU, 200027 s p(s)p(s) Probabilistic Localization [Simmons/Koenig 95] [Kaelbling et al 96] [Burgard et al 96]

29. SA-1 © Sebastian Thrun, CMU, 200028 Bayes Filters Bayes Markov [Kalman 60, Rabiner 85] d = data o = observation a = action t = time s = state Markov

30. SA-1 © Sebastian Thrun, CMU, 200029 Bayes Filters are Familiar to AI!  Kalman filters  Hidden Markov Models  Dynamic Bayes networks  Partially Observable Markov Decision Processes (POMDPs)

31. SA-1 © Sebastian Thrun, CMU, 200030 Markov Assumption used above Knowledge of current state renders past, future independent: “Static World Assumption” “Independent Noise Assumption”

32. SA-1 © Sebastian Thrun, CMU, 200031 Localization With Bayes Filters map m s’ a p(s|a,s’,m) a s’ laser datap(o|s,m) observation o

33. SA-1 © Sebastian Thrun, CMU, 200032 Xavier: (R. Simmons, S. Koenig, CMU 1996) Markov localization in a topological map

34. SA-1 © Sebastian Thrun, CMU, 200033 Markov Localization in Grid Map [Burgard et al 96] [Fox 99]

35. SA-1 © Sebastian Thrun, CMU, 200034 What is the Right Representation? Kalman filter [Schiele et al. 94], [Weiß et al. 94], [Borenstein 96], [Gutmann et al. 96, 98], [Arras 98] Piecewise constant (metric, topological) [Nourbakhsh et al. 95], [Simmons et al. 95], [Kaelbling et al. 96], [Burgard et al. 96], [Konolige et al. 99] Variable resolution (eg, trees) [Burgard et al. 98] Multi-hypothesis [Weckesser et al. 98], [Jensfelt et al. 99]

36. SA-1 © Sebastian Thrun, CMU, 200035 Idea: Represent Belief Through Samples Particle filters [Doucet 98, deFreitas 98] Condensation algorithm [Isard/Blake 98] Monte Carlo localization [Fox/Dellaert/Burgard/Thrun 99]

37. Monte Carlo Localization (MCL)

38. MCL: Importance Sampling

39. MCL: Robot Motion motion

40. MCL: Importance Sampling

41. SA-1 © Sebastian Thrun, CMU, 200040 Particle Filters draw s (i) t  1 from b ( s t  1 ) draw s (i) t from p ( s t | s (i) t  1,a t  1,m ) Represents b ( s t ) by set of weighted particles {s (i) t,w (i) t } Importance factor for s (i) t :

42. SA-1 © Sebastian Thrun, CMU, 200041 Monte Carlo Localization

43. SA-1 © Sebastian Thrun, CMU, 200042 Monte Carlo Localization, cont’d

44. SA-1 © Sebastian Thrun, CMU, 200043 Performance Comparison Monte Carlo localizationMarkov localization (grids)

45. SA-1 © Sebastian Thrun, CMU, 200044 Monte Carlo Localization  Approximate Bayes Estimation/Filtering Full posterior estimation Converges in O(1/  #samples) [Tanner’93] Robust: multiple hypothesis with degree of belief Efficient: focuses computation where needed Any-time: by varying number of samples Easy to implement

46. SA-1 © Sebastian Thrun, CMU, 200045 Pitfall: The World is not Markov! [Fox et al 1998] Distance filters:

47. SA-1 © Sebastian Thrun, CMU, 200046 Avoiding Collisions with Invisible Hazards Raw sensorsVirtual sensors added

48. SA-1 © Sebastian Thrun, CMU, 200047 Tracking People [Schulz et al, 2000]

49. SA-1 © Sebastian Thrun, CMU, 200048 Tracking People [Schulz et al, 2000]

50. SA-1 © Sebastian Thrun, CMU, 200049 Multi-Robot Localization Robots can detect each other (using cameras) [Fox et al, 1999]

51. SA-1 © Sebastian Thrun, CMU, 200050 Probabilistic Localization: Lessons Learned  Probabilistic Localization = Bayes filters  Particle filters: Approximate posterior by random samples  Extensions: Filter for dynamic environments Safe avoidance of invisible hazards People tracking Multi-robot localization Recovery from total failures [eg Lenser et al, 00, Thrun et al 00]

52. SA-1 © Sebastian Thrun, CMU, 200051 Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

53. SA-1 © Sebastian Thrun, CMU, 200052 The Problem: Concurrent Mapping and Localization 70 m

54. SA-1 © Sebastian Thrun, CMU, 200053 The Problem: Concurrent Mapping and Localization

55. SA-1 © Sebastian Thrun, CMU, 200054 On-Line Mapping with Rhino

56. SA-1 © Sebastian Thrun, CMU, 200055 Concurrent Mapping and Localization  Is a chicken-and-egg problem Mapping with known poses is “simple” Localization with known map is “simple” But in combination, the problem is hard!  Today’s best solutions are all probabilistic!

57. SA-1 © Sebastian Thrun, CMU, 200056 Mapping: Outline Posterior estimation with known poses: Occupancy grids Posterior estimation with known poses: Occupancy grids Maximum likelihood: ML* Maximum likelihood: ML* Maximum likelihood: EM Maximum likelihood: EM Posterior estimation: EKF (SLAM) Posterior estimation: EKF (SLAM)

58. SA-1 © Sebastian Thrun, CMU, 200057 Mapping as Posterior Estimation Assume static map [Smith, Self, Cheeseman 90, Chatila et al 91, Durrant-Whyte et al 92-00, Leonard et al. 92-00]

59. SA-1 © Sebastian Thrun, CMU, 200058 Kalman Filters  N-dimensional Gaussian  Can handle hundreds of dimensions

60. SA-1 © Sebastian Thrun, CMU, 200059 Underwater Mapping By: Louis L. Whitcomb, Johns Hopkins University

61. SA-1 © Sebastian Thrun, CMU, 200060 Underwater Mapping - Example “Autonomous Underwater Vehicle Navigation,” John Leonard et al, 1998

62. SA-1 © Sebastian Thrun, CMU, 200061 Mapping with Extended Kalman Filters Courtesy of [Leonard et al 1998]

63. SA-1 © Sebastian Thrun, CMU, 200062 The Key Assumption  Inverse sensor model p(s t |o t,m) must be Gaussian.  Main problem: Data association Posterior multi-modal  Undistinguishable features  In practice: Extract small set of highly distinguishable features from sensor data Discard all other data If ambiguous, take best guess for landmark identity Posterior uni-modal Distinguishable features

64. SA-1 © Sebastian Thrun, CMU, 200063 Mapping Algorithms - Comparison SLAM (Kalman) OutputPosterior ConvergenceStrong Local minimaNo Real timeYes Odom. ErrorUnbounded Sensor NoiseGaussian # Features10 3 Feature uniqYes Raw dataNo

65. SA-1 © Sebastian Thrun, CMU, 200064 Mapping: Outline Posterior estimation with known poses: Occupancy grids Posterior estimation with known poses: Occupancy grids Maximum likelihood: ML* Maximum likelihood: ML* Maximum likelihood: EM Maximum likelihood: EM Posterior estimation: EKF (SLAM) Posterior estimation: EKF (SLAM)

66. SA-1 © Sebastian Thrun, CMU, 200065 Mapping with Expectation Maximization Idea: maximum likelihood (with unknown data association) EM: Maximize log-likelihood by iterating E-step: M-step: [Dempster et al. 77]  Mapping with known poses  Markov localization (bi-directional) [Thrun et al. 98]

67. SA-1 © Sebastian Thrun, CMU, 200066 map(1)

68. SA-1 © Sebastian Thrun, CMU, 200067 backward forward map(2) map(1)

69. SA-1 © Sebastian Thrun, CMU, 200068 backward forward map(10)

70. SA-1 © Sebastian Thrun, CMU, 200069 CMU’s Wean Hall (80 x 25 meters) 15 landmarks 16 landmarks 17 landmarks27 landmarks

71. SA-1 © Sebastian Thrun, CMU, 200070 EM Mapping, Example (width 45 m)

72. SA-1 © Sebastian Thrun, CMU, 200071 Mapping Algorithms - Comparison SLAM (Kalman) EM OutputPosteriorML/MAP ConvergenceStrongWeak? Local minimaNoYes Real timeYesNo Odom. ErrorUnbounded Sensor NoiseGaussianAny # Features10 3  Feature uniqYesNo Raw dataNoYes

73. SA-1 © Sebastian Thrun, CMU, 200072 Mapping: Outline Posterior estimation with known poses: Occupancy grids Posterior estimation with known poses: Occupancy grids Maximum likelihood: ML* Maximum likelihood: ML* Maximum likelihood: EM Maximum likelihood: EM Posterior estimation: EKF (SLAM) Posterior estimation: EKF (SLAM)

74. SA-1 © Sebastian Thrun, CMU, 200073 Incremental ML Mapping, Online Idea: step-wise maximum likelihood Incremental ML estimate:

75. SA-1 © Sebastian Thrun, CMU, 200074 Incremental ML: Not A Good Idea path robot mismatch

76. SA-1 © Sebastian Thrun, CMU, 200075 ML* Mapping, Online Idea: step-wise maximum likelihood 2. Posterior: [Gutmann/Konolige 00, Thrun et al. 00] 1. Incremental ML estimate:

77. SA-1 © Sebastian Thrun, CMU, 200076 ML* Mapping, Online Courtesy of Kurt Konolige, SRI [Gutmann & Konolige, 00]

78. SA-1 © Sebastian Thrun, CMU, 200077 ML* Mapping, Online Yellow flashes: artificially distorted map (30 deg, 50 cm) [Thrun et al. 00]

79. SA-1 © Sebastian Thrun, CMU, 200078 Mapping with Poor Odometry map and exploration path raw data DARPA Urban Robot

80. SA-1 © Sebastian Thrun, CMU, 200079 Mapping Without(!) Odometry mapraw data (no odometry)

81. SA-1 © Sebastian Thrun, CMU, 200080 Localization in Multi-Robot Mapping

82. SA-1 © Sebastian Thrun, CMU, 200081 Localization in Multi-Robot Mapping Courtesy of Kurt Konolige, SRI [Gutmann & Konolige, 00]

83. SA-1 © Sebastian Thrun, CMU, 200082 3D Mapping two laser range finders

84. SA-1 © Sebastian Thrun, CMU, 200083 3D Structure Mapping (Real-Time)

85. SA-1 © Sebastian Thrun, CMU, 200084 3D Texture Mapping raw image sequencepanoramic camera

86. SA-1 © Sebastian Thrun, CMU, 200085 3D Texture Mapping

87. SA-1 © Sebastian Thrun, CMU, 200086 Mapping Algorithms - Comparison SLAM (Kalman) EMML* OutputPosteriorML/MAP ConvergenceStrongWeak?No Local minimaNoYes Real timeYesNoYes Odom. ErrorUnbounded Sensor NoiseGaussianAny # Features10 3  Feature uniqYesNo Raw dataNoYes

88. SA-1 © Sebastian Thrun, CMU, 200087 Mapping: Outline Posterior estimation with known poses: Occupancy grids Posterior estimation with known poses: Occupancy grids Maximum likelihood: ML* Maximum likelihood: ML* Maximum likelihood: EM Maximum likelihood: EM Posterior estimation: EKF (SLAM) Posterior estimation: EKF (SLAM)

89. SA-1 © Sebastian Thrun, CMU, 200088 Occupancy Grids: From scans to maps

90. SA-1 © Sebastian Thrun, CMU, 200089 Occupancy Grid Maps Assumptions: poses known, occupancy binary, independent [Elfes/Moravec 88] Assume

91. SA-1 © Sebastian Thrun, CMU, 200090 Example CAD map occupancy grid map The Tech Museum, San Jose

92. SA-1 © Sebastian Thrun, CMU, 200091 Mapping Algorithms - Comparison SLAM (Kalman) EMML*Occupan. Grids OutputPosteriorML/MAP Posterior ConvergenceStrongWeak?NoStrong Local minimaNoYes No Real timeYesNoYes Odom. ErrorUnbounded None Sensor NoiseGaussianAny # Features10 3  Feature uniqYesNo Raw dataNoYes

93. SA-1 © Sebastian Thrun, CMU, 200092 Mapping: Lessons Learned  Concurrent mapping and localization: hard robotics problem  Best known algorithms are probabilistic 1.EKF/SLAM: Full posterior estimation, but restrictive assumptions (data association) 2.EM: Maximum Likelihood, solves data association 3.ML*: less robust but online 4.Occupancy grids: Binary Bayes filter, assumes known poses (= much easier)

94. SA-1 © Sebastian Thrun, CMU, 200093 Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

95. SA-1 © Sebastian Thrun, CMU, 200094 The Decision Making Problem  Central Question: What should a robot do next?  Embraces control (short term, tight feedback) planning (longer term, looser feedback)  Probabilistic Paradigm: Considers uncertainty current future

96. SA-1 © Sebastian Thrun, CMU, 200095 Planning under Uncertainty EnvironmentStateModel Classical Planning deterministicobservableDeterministic, accurate MDP, universal plans stochasticobservablestochastic, accurate POMDPsstochasticpartially observable stochastic, inaccurate

97. SA-1 © Sebastian Thrun, CMU, 200096 Classical Situation hellheaven World deterministic State observable

98. SA-1 © Sebastian Thrun, CMU, 200097 MDP-Style Planning hellheaven World stochastic State observable [Koditschek 87, Barto et al. 89] Policy Universal Plan Navigation function

99. SA-1 © Sebastian Thrun, CMU, 200098 Stochastic, Partially Observable sign hell?heaven? [Sondik 72] [Littman/Cassandra/Kaelbling 97]

100. SA-1 © Sebastian Thrun, CMU, 200099 Stochastic, Partially Observable sign hellheaven sign heavenhell

101. SA-1 © Sebastian Thrun, CMU, 2000100 Stochastic, Partially Observable sign heavenhell sign ?? hellheaven start 50%

102. SA-1 © Sebastian Thrun, CMU, 2000101 Outline Deterministic, fully observable Stochastic, fully observable, discrete states/actions (MDPs) Stochastic, partially observable, discrete (POMDPs, Augmented MDPs) Stochastic, partially observable, continuous (Monte Carlo POMDPs)

103. SA-1 © Sebastian Thrun, CMU, 2000102 Robot Planning Frameworks Classical AI/robot planning State/actionsdiscrete & continuous Stateobservable Environmentdeterministic PlansSequences of actions CompletenessYes OptimalityRarely State space size Huge, often continuous, 6 dimensions Computational Complexity varies

104. SA-1 © Sebastian Thrun, CMU, 2000103 MDP-Style Planning hellheaven World stochastic State observable [Koditschek 87, Barto et al. 89] Policy Universal Plan Navigation function

105. SA-1 © Sebastian Thrun, CMU, 2000104 Markov Decision Process (discrete) s2s2 s3s3 s4s4 s5s5 s1s1 0.7 0.3 0.9 0.1 0.3 0.4 0.99 0.1 0.2 0.8 r=  10 r=  0  r=0 r=1 r=0 [Bellman 57] [Howard 60] [Sutton/Barto 98]

106. SA-1 © Sebastian Thrun, CMU, 2000105 Value Iteration  Value function of policy   Bellman equation for optimal value function  Value iteration: recursively estimating value function  Greedy policy: [Bellman 57] [Howard 60] [Sutton/Barto 98]

107. SA-1 © Sebastian Thrun, CMU, 2000106 Value Iteration for Motion Planning (assumes knowledge of robot’s location)

108. SA-1 © Sebastian Thrun, CMU, 2000107 Continuous Environments From: A Moore & C.G. Atkeson “The Parti-Game Algorithm for Variable Resolution Reinforcement Learning in Continuous State spaces,” Machine Learning 1995

109. SA-1 © Sebastian Thrun, CMU, 2000108 Approximate Cell Decomposition [Latombe 91] From: A Moore & C.G. Atkeson “The Parti-Game Algorithm for Variable Resolution Reinforcement Learning in Continuous State spaces,” Machine Learning 1995

110. SA-1 © Sebastian Thrun, CMU, 2000109 Parti-Game [Moore 96] From: A Moore & C.G. Atkeson “The Parti-Game Algorithm for Variable Resolution Reinforcement Learning in Continuous State spaces,” Machine Learning 1995

111. SA-1 © Sebastian Thrun, CMU, 2000110 Robot Planning Frameworks Classical AI/robot planning Value Iteration in MDPs Parti-Game State/actionsdiscrete & continuous discretecontinuous Stateobservable Environmentdeterministicstochastic PlansSequences of actions policy CompletenessYes OptimalityRarelyYesNo State space size Huge, often continuous, 6 dimensions millionsn/a Computational Complexity variesquadraticn/a

112. SA-1 © Sebastian Thrun, CMU, 2000111 Stochastic, Partially Observable sign ?? start sign heavenhell sign hellheaven 50% sign ?? start

113. SA-1 © Sebastian Thrun, CMU, 2000112 A Quiz  -dim continuous* stochastic 1-dim continuous stochastic actions# statessize belief space?sensors 3: s 1, s 2, s 3 deterministic3perfect 3: s 1, s 2, s 3 stochastic3perfect 2 3 -1: s 1, s 2, s 3, s 12, s 13, s 23, s 123 deterministic3 abstract states deterministic3stochastic 2-dim continuous*: p ( S=s 1 ), p ( S=s 2 ) stochastic3none 2-dim continuous*: p ( S=s 1 ), p ( S=s 2 ) *) countable, but for all practical purposes  -dim continuous* deterministic 1-dim continuous stochastic aargh!stochastic  -dim continuous stochastic

114. SA-1 © Sebastian Thrun, CMU, 2000113 Introduction to POMDPs 80  100 ba  0 ba  40 s2s2 s1s1 action a action b p(s1)p(s1) [Sondik 72, Littman, Kaelbling, Cassandra ‘97] s2s2 s1s1  100 0 100 action aaction b  Value function (finite horizon): Piecewise linear, convex Most efficient algorithm today: Witness algorithm

115. SA-1 © Sebastian Thrun, CMU, 2000114 Value Iteration in POMDPs  Value function of policy   Bellman equation for optimal value function  Value iteration: recursively estimating value function  Greedy policy: Substitute b for s

116. SA-1 © Sebastian Thrun, CMU, 2000115 Missing Terms: Belief Space  Expected reward:  Next state density: Bayes filters! (Dirac distribution)

117. SA-1 © Sebastian Thrun, CMU, 2000116 Value Iteration in Belief Space.... next belief state b’ observation o.... belief state b max Q(b’, a) next state s’, reward r’state s Q(b, a) value function

118. SA-1 © Sebastian Thrun, CMU, 2000117 Why is This So Complex? State Space Planning (no state uncertainty) Belief Space Planning (full state uncertainties) ?

119. SA-1 © Sebastian Thrun, CMU, 2000118 Augmented MDPs: [Roy et al, 98/99] conventional state space uncertainty (entropy)

120. SA-1 © Sebastian Thrun, CMU, 2000119 Path Planning with Augmented MDPs information gainConventional plannerProbabilistic Planner [Roy et al, 98/99]

121. SA-1 © Sebastian Thrun, CMU, 2000120 Robot Planning Frameworks Classical AI/robot planning Value Iteration in MDPs Parti-GamePOMDPAugmented MDP State/actionsdiscrete & continuous discretecontinuousdiscrete Stateobservable partially observable Environmentdeterministicstochastic PlansSequences of actions policy CompletenessYes No OptimalityRarelyYesNoYesNo State space size Huge, often continuous, 6 dimensions millionsn/adozensthousands Computational Complexity variesquadraticn/aexponentialO(N 4 )

122. SA-1 © Sebastian Thrun, CMU, 2000121 Decision Making: Lessons Learned  Four sources of uncertainty Environment unpredictable Robot wear and tear Sensors limitations Models inaccurate  Two implications Need policy instead of simple (open-loop) plan Policy must be conditioned on belief state  Approaches MDP: Only works with perfect sensors, models POMDPs: general framework, but scaling limitations Augmented MDPs: lower computation, but approximate

123. SA-1 © Sebastian Thrun, CMU, 2000122 Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

124. SA-1 © Sebastian Thrun, CMU, 2000123 Exploration: Maximize Knowledge Gain Pick action a that maximizes knowledge gain.  Constant time actions:  Variable time actions: [Thrun 93] [Yamauchi 96] [Burgard et al 00] + many others entropy of map

125. SA-1 © Sebastian Thrun, CMU, 2000124 Practical Implementation  For each location estimate number of cells robot can sense estimate costs of getting there (value iteration) [Simmons et al 00]

126. SA-1 © Sebastian Thrun, CMU, 2000125 Real-Time Exploration

127. SA-1 © Sebastian Thrun, CMU, 2000126 Coordinated Multi-Robot Exploration  Robots place “bids” for target areas  Greedy assignment of robots to areas  Exploration strategies and assignments continuously re-evaluated while robots in motion [Burgard et al 00] [Simmons et al 00]

128. SA-1 © Sebastian Thrun, CMU, 2000127 Collaborative Exploration and Mapping

129. SA-1 © Sebastian Thrun, CMU, 2000128 San Antonio Results

130. SA-1 © Sebastian Thrun, CMU, 2000129 Benefit of Cooperation [Burgard et al 00]

131. SA-1 © Sebastian Thrun, CMU, 2000130 Exploration: Lessons Learned  Exploration = greedily maximize knowledge gain  Greedy methods can be very effective  Facilitates multi-robot coordination

132. SA-1 © Sebastian Thrun, CMU, 2000131 Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

133. SA-1 © Sebastian Thrun, CMU, 2000132 Problem Summary  In Robotics, there is no such thing as A perfect sensor A deterministic environment A deterministic robot An accurate model  Therefore: Uncertainty inherent in robotics

134. SA-1 © Sebastian Thrun, CMU, 2000133 Key Idea  Probabilistic Robotics: Represents and reasons with uncertainty, represented explicitly Perception = posterior estimation Action = optimization of expected utility

135. SA-1 © Sebastian Thrun, CMU, 2000134 Examples Covered Today  Localization  Mapping  Planning  Exploration  Multi-robot

136. SA-1 © Sebastian Thrun, CMU, 2000135 Successful Applications of Probabilistic Robotics  Industrial outdoor navigation [Durrant-Whyte, 95]  Underwater vehicles [Leonard et al, 98]  Coal Mining [Singh 98]  Missile Guidance  Indoor navigation [Simmons et al, 97]  Robo-Soccer [Lenser et al, 00]  Museum Tour-Guides [Burgard et al, 98, Thrun 99]  + many others

137. SA-1 © Sebastian Thrun, CMU, 2000136 Relation to AI  Probabilistic methods highly successful in a range of sub-fields of AI Speech recognition Language processing Expert systems Computer vision Data Mining (and many others)

138. SA-1 © Sebastian Thrun, CMU, 2000137 Open Research Issues  Better representations, faster algorithms  Learning with domain knowledge (eg, models, behaviors)  High-level reasoning and robot programming using probabilistic paradigm  Theory: eg, surpassing the Markov assumption  Frameworks for probabilistic programming  Innovative applications