SA-1 Probabilistic Robotics Tutorial AAAI-2000 Sebastian Thrun Computer Science and Robotics Carnegie Mellon University

SA-1 © Sebastian Thrun, CMU, Recommended Readings Probabilistic Algorithms in Robotics (basic survey, 95 references) AI Magazine (to appear Dec 2000) Also: Tech Report: CMU-CS

SA-1 © Sebastian Thrun, CMU, Collaborators and Funding Anita Arendra Michael Beetz Maren Bennewitz Eric Bauer Joachim Buhmann Wolfram Burgard Armin B. Cremers Frank Dellaert Dieter Fox Dirk Hähnel John Langford Gerhard Lakemeyer Dimitris Margaritis Michael Montemerlo Sungwoo Park Frank Pfenning Joelle Pineau Martha Pollack Charles Rosenberg Nicholas Roy Jamieson Schulte Reid Simmons Dirk Schulz Wolli Steiner Special thanks: Kurt Konolige, John Leonard, Andrew Moore, Reid Simmons Sponsored by: DARPA (TMR, CoABS, MARS), NSF (CAREER, IIS, LIS), EC, Daimler Benz, Microsoft + others

SA-1 © Sebastian Thrun, CMU, Tutorial Goal To familiarize you with probabilistic paradigm in robotics Basic techniques Advantages Pitfalls and limitations Successful Applications Open research issues

SA-1 © Sebastian Thrun, CMU, Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

SA-1 © Sebastian Thrun, CMU, Robotics Yesterday

SA-1 © Sebastian Thrun, CMU, Robotics Today

SA-1 © Sebastian Thrun, CMU, Robotics Tomorrow?

SA-1 © Sebastian Thrun, CMU, Current Trends in Robotics Robots are moving away from factory floors to Entertainment, toys Personal services Medical, surgery Industrial automatization (mining, harvesting, …) Hazardous environments (space, underwater)

SA-1 © Sebastian Thrun, CMU, Robots are Inherently Uncertain  Uncertainty arises from four major factors: Environment stochastic, unpredictable Robot stochastic Sensor limited, noisy Models inaccurate

SA-1 © Sebastian Thrun, CMU, Probabilistic Robotics

SA-1 © Sebastian Thrun, CMU, Probabilistic Robotics Key idea: Explicit representation of uncertainty (using the calculus of probability theory)  Perception = state estimation  Action = utility optimization

SA-1 © Sebastian Thrun, CMU, Advantages of Probabilistic Paradigm  Can accommodate inaccurate models  Can accommodate imperfect sensors  Robust in real-world applications  Best known approach to many hard robotics problems

SA-1 © Sebastian Thrun, CMU, Pitfalls  Computationally demanding  False assumptions  Approximate

SA-1 © Sebastian Thrun, CMU, Trends in Robotics Reactive Paradigm (mid-80’s) no models relies heavily on good sensing Probabilistic Robotics (since mid-90’s) seamless integration of models and sensing inaccurate models, inaccurate sensors Hybrids (since 90’s) model-based at higher levels reactive at lower levels Classical Robotics (mid-70’s) exact models no sensing necessary

SA-1 © Sebastian Thrun, CMU, Example: Museum Tour-Guides Robots Rhino, 1997Minerva, 1998

SA-1 © Sebastian Thrun, CMU, Rhino (Univ. Bonn + CMU, 1997) W. Burgard, A.B. Cremers, D. Fox, D. Hähnel, G. Lakemeyer, D. Schulz, W. Steiner, S. Thrun

SA-1 © Sebastian Thrun, CMU, Minerva (CMU + Univ. Bonn, 1998) Minerva S. Thrun, M. Beetz, M. Bennewitz, W. Burgard, A.B. Cremers, F. Dellaert, D. Fox, D. Hähnel, C. Rosenberg, N. Roy, J. Schulte, D. Schulz

SA-1 © Sebastian Thrun, CMU, “How Intelligent Is Minerva?” fishdogmonkeyhumanamoeba 5.7% 29.5% 25.4% 36.9% 2.5%

SA-1 © Sebastian Thrun, CMU, “Is Minerva Alive?" undecidednoyes 3.2% 27.0% 69.8%

SA-1 © Sebastian Thrun, CMU, “Is Minerva Alive?" undecidednoyes 3.2% 27.0% 69.8% “Are You Under 10 Years of Age?”

SA-1 © Sebastian Thrun, CMU, Nature of Sensor Data Odometry Data Range Data

SA-1 © Sebastian Thrun, CMU, Technical Challenges  Navigation Environment crowded, unpredictable Environment unmodified “Invisible” hazards Walking speed or faster High failure costs  Interaction Individuals and crowds Museum visitors’ first encounter Age 2 through 99 Spend less than 15 minutes

SA-1 © Sebastian Thrun, CMU,

SA-1 © Sebastian Thrun, CMU, Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

SA-1 © Sebastian Thrun, CMU, The Localization Problem  Estimate robot’s coordinates s=(x,y,  ) from sensor data Position tracking (error bounded) Global localization (unbounded error) Kidnapping (recovery from failure) Ingemar Cox (1991): “Using sensory information to locate the robot in its environment is the most fundamental problem to provide a mobile robot with autonomous capabilities.” see also [Borenstein et al, 96]

SA-1 © Sebastian Thrun, CMU, s p(s)p(s) Probabilistic Localization [Simmons/Koenig 95] [Kaelbling et al 96] [Burgard et al 96]

SA-1 © Sebastian Thrun, CMU, Bayes Filters Bayes Markov [Kalman 60, Rabiner 85] d = data o = observation a = action t = time s = state Markov

SA-1 © Sebastian Thrun, CMU, Bayes Filters are Familiar to AI!  Kalman filters  Hidden Markov Models  Dynamic Bayes networks  Partially Observable Markov Decision Processes (POMDPs)

SA-1 © Sebastian Thrun, CMU, Markov Assumption used above Knowledge of current state renders past, future independent: “Static World Assumption” “Independent Noise Assumption”

SA-1 © Sebastian Thrun, CMU, Localization With Bayes Filters map m s’ a p(s|a,s’,m) a s’ laser datap(o|s,m) observation o

SA-1 © Sebastian Thrun, CMU, Xavier: (R. Simmons, S. Koenig, CMU 1996) Markov localization in a topological map

SA-1 © Sebastian Thrun, CMU, Markov Localization in Grid Map [Burgard et al 96] [Fox 99]

SA-1 © Sebastian Thrun, CMU, What is the Right Representation? Kalman filter [Schiele et al. 94], [Weiß et al. 94], [Borenstein 96], [Gutmann et al. 96, 98], [Arras 98] Piecewise constant (metric, topological) [Nourbakhsh et al. 95], [Simmons et al. 95], [Kaelbling et al. 96], [Burgard et al. 96], [Konolige et al. 99] Variable resolution (eg, trees) [Burgard et al. 98] Multi-hypothesis [Weckesser et al. 98], [Jensfelt et al. 99]

SA-1 © Sebastian Thrun, CMU, Idea: Represent Belief Through Samples Particle filters [Doucet 98, deFreitas 98] Condensation algorithm [Isard/Blake 98] Monte Carlo localization [Fox/Dellaert/Burgard/Thrun 99]

Monte Carlo Localization (MCL)

MCL: Importance Sampling

MCL: Robot Motion motion

MCL: Importance Sampling

SA-1 © Sebastian Thrun, CMU, Particle Filters draw s (i) t  1 from b ( s t  1 ) draw s (i) t from p ( s t | s (i) t  1,a t  1,m ) Represents b ( s t ) by set of weighted particles {s (i) t,w (i) t } Importance factor for s (i) t :

SA-1 © Sebastian Thrun, CMU, Monte Carlo Localization

SA-1 © Sebastian Thrun, CMU, Monte Carlo Localization, cont’d

SA-1 © Sebastian Thrun, CMU, Performance Comparison Monte Carlo localizationMarkov localization (grids)

SA-1 © Sebastian Thrun, CMU, Monte Carlo Localization  Approximate Bayes Estimation/Filtering Full posterior estimation Converges in O(1/  #samples) [Tanner’93] Robust: multiple hypothesis with degree of belief Efficient: focuses computation where needed Any-time: by varying number of samples Easy to implement

SA-1 © Sebastian Thrun, CMU, Pitfall: The World is not Markov! [Fox et al 1998] Distance filters:

SA-1 © Sebastian Thrun, CMU, Avoiding Collisions with Invisible Hazards Raw sensorsVirtual sensors added

SA-1 © Sebastian Thrun, CMU, Tracking People [Schulz et al, 2000]

SA-1 © Sebastian Thrun, CMU, Tracking People [Schulz et al, 2000]

SA-1 © Sebastian Thrun, CMU, Multi-Robot Localization Robots can detect each other (using cameras) [Fox et al, 1999]

SA-1 © Sebastian Thrun, CMU, Probabilistic Localization: Lessons Learned  Probabilistic Localization = Bayes filters  Particle filters: Approximate posterior by random samples  Extensions: Filter for dynamic environments Safe avoidance of invisible hazards People tracking Multi-robot localization Recovery from total failures [eg Lenser et al, 00, Thrun et al 00]

SA-1 © Sebastian Thrun, CMU, Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

SA-1 © Sebastian Thrun, CMU, The Problem: Concurrent Mapping and Localization 70 m

SA-1 © Sebastian Thrun, CMU, The Problem: Concurrent Mapping and Localization

SA-1 © Sebastian Thrun, CMU, On-Line Mapping with Rhino

SA-1 © Sebastian Thrun, CMU, Concurrent Mapping and Localization  Is a chicken-and-egg problem Mapping with known poses is “simple” Localization with known map is “simple” But in combination, the problem is hard!  Today’s best solutions are all probabilistic!

SA-1 © Sebastian Thrun, CMU, Mapping: Outline Posterior estimation with known poses: Occupancy grids Posterior estimation with known poses: Occupancy grids Maximum likelihood: ML* Maximum likelihood: ML* Maximum likelihood: EM Maximum likelihood: EM Posterior estimation: EKF (SLAM) Posterior estimation: EKF (SLAM)

SA-1 © Sebastian Thrun, CMU, Mapping as Posterior Estimation Assume static map [Smith, Self, Cheeseman 90, Chatila et al 91, Durrant-Whyte et al 92-00, Leonard et al ]

SA-1 © Sebastian Thrun, CMU, Kalman Filters  N-dimensional Gaussian  Can handle hundreds of dimensions

SA-1 © Sebastian Thrun, CMU, Underwater Mapping By: Louis L. Whitcomb, Johns Hopkins University

SA-1 © Sebastian Thrun, CMU, Underwater Mapping - Example “Autonomous Underwater Vehicle Navigation,” John Leonard et al, 1998

SA-1 © Sebastian Thrun, CMU, Mapping with Extended Kalman Filters Courtesy of [Leonard et al 1998]

SA-1 © Sebastian Thrun, CMU, The Key Assumption  Inverse sensor model p(s t |o t,m) must be Gaussian.  Main problem: Data association Posterior multi-modal  Undistinguishable features  In practice: Extract small set of highly distinguishable features from sensor data Discard all other data If ambiguous, take best guess for landmark identity Posterior uni-modal Distinguishable features

SA-1 © Sebastian Thrun, CMU, Mapping Algorithms - Comparison SLAM (Kalman) OutputPosterior ConvergenceStrong Local minimaNo Real timeYes Odom. ErrorUnbounded Sensor NoiseGaussian # Features10 3 Feature uniqYes Raw dataNo

SA-1 © Sebastian Thrun, CMU, Mapping: Outline Posterior estimation with known poses: Occupancy grids Posterior estimation with known poses: Occupancy grids Maximum likelihood: ML* Maximum likelihood: ML* Maximum likelihood: EM Maximum likelihood: EM Posterior estimation: EKF (SLAM) Posterior estimation: EKF (SLAM)

SA-1 © Sebastian Thrun, CMU, Mapping with Expectation Maximization Idea: maximum likelihood (with unknown data association) EM: Maximize log-likelihood by iterating E-step: M-step: [Dempster et al. 77]  Mapping with known poses  Markov localization (bi-directional) [Thrun et al. 98]

SA-1 © Sebastian Thrun, CMU, map(1)

SA-1 © Sebastian Thrun, CMU, backward forward map(2) map(1)

SA-1 © Sebastian Thrun, CMU, backward forward map(10)

SA-1 © Sebastian Thrun, CMU, CMU’s Wean Hall (80 x 25 meters) 15 landmarks 16 landmarks 17 landmarks27 landmarks

SA-1 © Sebastian Thrun, CMU, EM Mapping, Example (width 45 m)

SA-1 © Sebastian Thrun, CMU, Mapping Algorithms - Comparison SLAM (Kalman) EM OutputPosteriorML/MAP ConvergenceStrongWeak? Local minimaNoYes Real timeYesNo Odom. ErrorUnbounded Sensor NoiseGaussianAny # Features10 3  Feature uniqYesNo Raw dataNoYes

SA-1 © Sebastian Thrun, CMU, Mapping: Outline Posterior estimation with known poses: Occupancy grids Posterior estimation with known poses: Occupancy grids Maximum likelihood: ML* Maximum likelihood: ML* Maximum likelihood: EM Maximum likelihood: EM Posterior estimation: EKF (SLAM) Posterior estimation: EKF (SLAM)

SA-1 © Sebastian Thrun, CMU, Incremental ML Mapping, Online Idea: step-wise maximum likelihood Incremental ML estimate:

SA-1 © Sebastian Thrun, CMU, Incremental ML: Not A Good Idea path robot mismatch

SA-1 © Sebastian Thrun, CMU, ML* Mapping, Online Idea: step-wise maximum likelihood 2. Posterior: [Gutmann/Konolige 00, Thrun et al. 00] 1. Incremental ML estimate:

SA-1 © Sebastian Thrun, CMU, ML* Mapping, Online Courtesy of Kurt Konolige, SRI [Gutmann & Konolige, 00]

SA-1 © Sebastian Thrun, CMU, ML* Mapping, Online Yellow flashes: artificially distorted map (30 deg, 50 cm) [Thrun et al. 00]

SA-1 © Sebastian Thrun, CMU, Mapping with Poor Odometry map and exploration path raw data DARPA Urban Robot

SA-1 © Sebastian Thrun, CMU, Mapping Without(!) Odometry mapraw data (no odometry)

SA-1 © Sebastian Thrun, CMU, Localization in Multi-Robot Mapping

SA-1 © Sebastian Thrun, CMU, Localization in Multi-Robot Mapping Courtesy of Kurt Konolige, SRI [Gutmann & Konolige, 00]

SA-1 © Sebastian Thrun, CMU, D Mapping two laser range finders

SA-1 © Sebastian Thrun, CMU, D Structure Mapping (Real-Time)

SA-1 © Sebastian Thrun, CMU, D Texture Mapping raw image sequencepanoramic camera

SA-1 © Sebastian Thrun, CMU, D Texture Mapping

SA-1 © Sebastian Thrun, CMU, Mapping Algorithms - Comparison SLAM (Kalman) EMML* OutputPosteriorML/MAP ConvergenceStrongWeak?No Local minimaNoYes Real timeYesNoYes Odom. ErrorUnbounded Sensor NoiseGaussianAny # Features10 3  Feature uniqYesNo Raw dataNoYes

SA-1 © Sebastian Thrun, CMU, Mapping: Outline Posterior estimation with known poses: Occupancy grids Posterior estimation with known poses: Occupancy grids Maximum likelihood: ML* Maximum likelihood: ML* Maximum likelihood: EM Maximum likelihood: EM Posterior estimation: EKF (SLAM) Posterior estimation: EKF (SLAM)

SA-1 © Sebastian Thrun, CMU, Occupancy Grids: From scans to maps

SA-1 © Sebastian Thrun, CMU, Occupancy Grid Maps Assumptions: poses known, occupancy binary, independent [Elfes/Moravec 88] Assume

SA-1 © Sebastian Thrun, CMU, Example CAD map occupancy grid map The Tech Museum, San Jose

SA-1 © Sebastian Thrun, CMU, Mapping Algorithms - Comparison SLAM (Kalman) EMML*Occupan. Grids OutputPosteriorML/MAP Posterior ConvergenceStrongWeak?NoStrong Local minimaNoYes No Real timeYesNoYes Odom. ErrorUnbounded None Sensor NoiseGaussianAny # Features10 3  Feature uniqYesNo Raw dataNoYes

SA-1 © Sebastian Thrun, CMU, Mapping: Lessons Learned  Concurrent mapping and localization: hard robotics problem  Best known algorithms are probabilistic 1.EKF/SLAM: Full posterior estimation, but restrictive assumptions (data association) 2.EM: Maximum Likelihood, solves data association 3.ML*: less robust but online 4.Occupancy grids: Binary Bayes filter, assumes known poses (= much easier)

SA-1 © Sebastian Thrun, CMU, Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

SA-1 © Sebastian Thrun, CMU, The Decision Making Problem  Central Question: What should a robot do next?  Embraces control (short term, tight feedback) planning (longer term, looser feedback)  Probabilistic Paradigm: Considers uncertainty current future

SA-1 © Sebastian Thrun, CMU, Planning under Uncertainty EnvironmentStateModel Classical Planning deterministicobservableDeterministic, accurate MDP, universal plans stochasticobservablestochastic, accurate POMDPsstochasticpartially observable stochastic, inaccurate

SA-1 © Sebastian Thrun, CMU, Classical Situation hellheaven World deterministic State observable

SA-1 © Sebastian Thrun, CMU, MDP-Style Planning hellheaven World stochastic State observable [Koditschek 87, Barto et al. 89] Policy Universal Plan Navigation function

SA-1 © Sebastian Thrun, CMU, Stochastic, Partially Observable sign hell?heaven? [Sondik 72] [Littman/Cassandra/Kaelbling 97]

SA-1 © Sebastian Thrun, CMU, Stochastic, Partially Observable sign hellheaven sign heavenhell

SA-1 © Sebastian Thrun, CMU, Stochastic, Partially Observable sign heavenhell sign ?? hellheaven start 50%

SA-1 © Sebastian Thrun, CMU, Outline Deterministic, fully observable Stochastic, fully observable, discrete states/actions (MDPs) Stochastic, partially observable, discrete (POMDPs, Augmented MDPs) Stochastic, partially observable, continuous (Monte Carlo POMDPs)

SA-1 © Sebastian Thrun, CMU, Robot Planning Frameworks Classical AI/robot planning State/actionsdiscrete & continuous Stateobservable Environmentdeterministic PlansSequences of actions CompletenessYes OptimalityRarely State space size Huge, often continuous, 6 dimensions Computational Complexity varies

SA-1 © Sebastian Thrun, CMU, MDP-Style Planning hellheaven World stochastic State observable [Koditschek 87, Barto et al. 89] Policy Universal Plan Navigation function

SA-1 © Sebastian Thrun, CMU, Markov Decision Process (discrete) s2s2 s3s3 s4s4 s5s5 s1s r=  10 r=  0  r=0 r=1 r=0 [Bellman 57] [Howard 60] [Sutton/Barto 98]

SA-1 © Sebastian Thrun, CMU, Value Iteration  Value function of policy   Bellman equation for optimal value function  Value iteration: recursively estimating value function  Greedy policy: [Bellman 57] [Howard 60] [Sutton/Barto 98]

SA-1 © Sebastian Thrun, CMU, Value Iteration for Motion Planning (assumes knowledge of robot’s location)

SA-1 © Sebastian Thrun, CMU, Continuous Environments From: A Moore & C.G. Atkeson “The Parti-Game Algorithm for Variable Resolution Reinforcement Learning in Continuous State spaces,” Machine Learning 1995

SA-1 © Sebastian Thrun, CMU, Approximate Cell Decomposition [Latombe 91] From: A Moore & C.G. Atkeson “The Parti-Game Algorithm for Variable Resolution Reinforcement Learning in Continuous State spaces,” Machine Learning 1995

SA-1 © Sebastian Thrun, CMU, Parti-Game [Moore 96] From: A Moore & C.G. Atkeson “The Parti-Game Algorithm for Variable Resolution Reinforcement Learning in Continuous State spaces,” Machine Learning 1995

SA-1 © Sebastian Thrun, CMU, Robot Planning Frameworks Classical AI/robot planning Value Iteration in MDPs Parti-Game State/actionsdiscrete & continuous discretecontinuous Stateobservable Environmentdeterministicstochastic PlansSequences of actions policy CompletenessYes OptimalityRarelyYesNo State space size Huge, often continuous, 6 dimensions millionsn/a Computational Complexity variesquadraticn/a

SA-1 © Sebastian Thrun, CMU, Stochastic, Partially Observable sign ?? start sign heavenhell sign hellheaven 50% sign ?? start

SA-1 © Sebastian Thrun, CMU, A Quiz  -dim continuous* stochastic 1-dim continuous stochastic actions# statessize belief space?sensors 3: s 1, s 2, s 3 deterministic3perfect 3: s 1, s 2, s 3 stochastic3perfect : s 1, s 2, s 3, s 12, s 13, s 23, s 123 deterministic3 abstract states deterministic3stochastic 2-dim continuous*: p ( S=s 1 ), p ( S=s 2 ) stochastic3none 2-dim continuous*: p ( S=s 1 ), p ( S=s 2 ) *) countable, but for all practical purposes  -dim continuous* deterministic 1-dim continuous stochastic aargh!stochastic  -dim continuous stochastic

SA-1 © Sebastian Thrun, CMU, Introduction to POMDPs 80  100 ba  0 ba  40 s2s2 s1s1 action a action b p(s1)p(s1) [Sondik 72, Littman, Kaelbling, Cassandra ‘97] s2s2 s1s1  action aaction b  Value function (finite horizon): Piecewise linear, convex Most efficient algorithm today: Witness algorithm

SA-1 © Sebastian Thrun, CMU, Value Iteration in POMDPs  Value function of policy   Bellman equation for optimal value function  Value iteration: recursively estimating value function  Greedy policy: Substitute b for s

SA-1 © Sebastian Thrun, CMU, Missing Terms: Belief Space  Expected reward:  Next state density: Bayes filters! (Dirac distribution)

SA-1 © Sebastian Thrun, CMU, Value Iteration in Belief Space.... next belief state b’ observation o.... belief state b max Q(b’, a) next state s’, reward r’state s Q(b, a) value function

SA-1 © Sebastian Thrun, CMU, Why is This So Complex? State Space Planning (no state uncertainty) Belief Space Planning (full state uncertainties) ?

SA-1 © Sebastian Thrun, CMU, Augmented MDPs: [Roy et al, 98/99] conventional state space uncertainty (entropy)

SA-1 © Sebastian Thrun, CMU, Path Planning with Augmented MDPs information gainConventional plannerProbabilistic Planner [Roy et al, 98/99]

SA-1 © Sebastian Thrun, CMU, Robot Planning Frameworks Classical AI/robot planning Value Iteration in MDPs Parti-GamePOMDPAugmented MDP State/actionsdiscrete & continuous discretecontinuousdiscrete Stateobservable partially observable Environmentdeterministicstochastic PlansSequences of actions policy CompletenessYes No OptimalityRarelyYesNoYesNo State space size Huge, often continuous, 6 dimensions millionsn/adozensthousands Computational Complexity variesquadraticn/aexponentialO(N 4 )

SA-1 © Sebastian Thrun, CMU, Decision Making: Lessons Learned  Four sources of uncertainty Environment unpredictable Robot wear and tear Sensors limitations Models inaccurate  Two implications Need policy instead of simple (open-loop) plan Policy must be conditioned on belief state  Approaches MDP: Only works with perfect sensors, models POMDPs: general framework, but scaling limitations Augmented MDPs: lower computation, but approximate

SA-1 © Sebastian Thrun, CMU, Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

SA-1 © Sebastian Thrun, CMU, Exploration: Maximize Knowledge Gain Pick action a that maximizes knowledge gain.  Constant time actions:  Variable time actions: [Thrun 93] [Yamauchi 96] [Burgard et al 00] + many others entropy of map

SA-1 © Sebastian Thrun, CMU, Practical Implementation  For each location estimate number of cells robot can sense estimate costs of getting there (value iteration) [Simmons et al 00]

SA-1 © Sebastian Thrun, CMU, Real-Time Exploration

SA-1 © Sebastian Thrun, CMU, Coordinated Multi-Robot Exploration  Robots place “bids” for target areas  Greedy assignment of robots to areas  Exploration strategies and assignments continuously re-evaluated while robots in motion [Burgard et al 00] [Simmons et al 00]

SA-1 © Sebastian Thrun, CMU, Collaborative Exploration and Mapping

SA-1 © Sebastian Thrun, CMU, San Antonio Results

SA-1 © Sebastian Thrun, CMU, Benefit of Cooperation [Burgard et al 00]

SA-1 © Sebastian Thrun, CMU, Exploration: Lessons Learned  Exploration = greedily maximize knowledge gain  Greedy methods can be very effective  Facilitates multi-robot coordination

SA-1 © Sebastian Thrun, CMU, Tutorial Outline  Introduction  Probabilistic State Estimation Localization Mapping  Probabilistic Decision Making Planning Exploration  Conclusion

SA-1 © Sebastian Thrun, CMU, Problem Summary  In Robotics, there is no such thing as A perfect sensor A deterministic environment A deterministic robot An accurate model  Therefore: Uncertainty inherent in robotics

SA-1 © Sebastian Thrun, CMU, Key Idea  Probabilistic Robotics: Represents and reasons with uncertainty, represented explicitly Perception = posterior estimation Action = optimization of expected utility

SA-1 © Sebastian Thrun, CMU, Examples Covered Today  Localization  Mapping  Planning  Exploration  Multi-robot

SA-1 © Sebastian Thrun, CMU, Successful Applications of Probabilistic Robotics  Industrial outdoor navigation [Durrant-Whyte, 95]  Underwater vehicles [Leonard et al, 98]  Coal Mining [Singh 98]  Missile Guidance  Indoor navigation [Simmons et al, 97]  Robo-Soccer [Lenser et al, 00]  Museum Tour-Guides [Burgard et al, 98, Thrun 99]  + many others

SA-1 © Sebastian Thrun, CMU, Relation to AI  Probabilistic methods highly successful in a range of sub-fields of AI Speech recognition Language processing Expert systems Computer vision Data Mining (and many others)

SA-1 © Sebastian Thrun, CMU, Open Research Issues  Better representations, faster algorithms  Learning with domain knowledge (eg, models, behaviors)  High-level reasoning and robot programming using probabilistic paradigm  Theory: eg, surpassing the Markov assumption  Frameworks for probabilistic programming  Innovative applications