1. Trey Smith, Robotics Institute, Carnegie Mellon University
Rover Science Autonomy Probabilistic Planning for Science-Aware Exploration
Thesis Proposal
Trey Smith, Robotics Institute, Carnegie Mellon University
14 June 2004

2. Robotic Planetary Science

3. Mars Rover Operations Today

4. Mars Rover Operations Today

5. Mars Rover Operations Today

6. Over-the-Horizon Science
60 m

7. Over-the-Horizon Science
60 m
6 km

8. Science Autonomy
The opportunity: Long-distance mobility gives the rover access to new sites
The challenge: Do useful science in over-the- horizon situations
The method: Enable the rover to reason about science goals and the science data it collects

9. New Operational Modes !

10. Sensing Uncertainty
type 4
type 7
novel

11. Probabilistic Sensor Planning
contact sensing
spectrometer
keep moving
keep moving

12. Probabilistic Sensor Planning
contact sensing
spectrometer
keep moving
keep moving

13. POMDP Model
Principled method for reasoning about information gain and future beliefs

14. POMDP Model
Principled method for reasoning about information gain and future beliefs
Horribly, horribly intractable

15. POMDP Model
Principled method for reasoning about information gain and future beliefs
Horribly, horribly intractable
Develop approximation techniques that take advantage of structure in the SA domain to speed up planning

16. Thesis Statement
Enabling a robot to reason about science goals and the science data it collects will significantly improve its exploration efficiency,

17. Thesis Statement
Enabling a robot to reason about science goals and the science data it collects will significantly improve its exploration efficiency, and POMDP methods can significantly improve the quality of science autonomy plans.

18. Outline Motivation Background and related work Technical approach
Preliminary work
Proposed research

19. Science Autonomy Related Work
Image and spectral analysis [Gulick et al., 2001] [Gazis and Roush, 2001]
Balancing exploration priorities [Moorehead, 2001]
Sensor planning to detect meteorites [Wagner et al., 2001]
Orbiter reacting to science events [Chien et al., 2003]
OASIS project: closed-loop SA system [Estlin et al., 2003]

20. POMDP Introduction Recall: Great for sensor planning
Tend to be massively intractable

21. Example Problem: RockSample

22. Example Problem: RockSample

23. Example Problem: RockSample
exit

24. Example Problem: RockSample
exit
check

25. Example Problem: RockSample
exit
check
good
bad
(+20 or +0)
(+10)
= 0.9

26. History and Belief initially 50/50 check good check good 0.5 0.7 0.9
bad
good
Pr(good) = 0.5
Pr(good) = 0.7
Pr(good) = 0.9
check
good
check
good

27. Policy exit check sample sample check bad good good 0.38 0.64 0.5 bad
0.7
bad

28. Value Function: Horizon 1
exit
sample 20 10
check
bad
good

29. Value Function: Horizon 1
exit
sample 20 10
bad
good

30. Value Function: Horizon 2
exit
sample 20
check
bad
good 10
exit
sample
bad
good

31. Computing the Policy exit check sample 20 10 exit sample check bad
good 10
exit
sample
exit
check
sample

32. optimal value function V*
Value Iteration
Horizon 2
optimal value function V*
Horizon 1
Horizon 0

33. POMDP Related Work
Policy iteration [Hansen, 1998] [Poupart and Boutilier, 2003]
Gradient ascent policy search [Baxter and Bartlett, 2000]
Value iteration [Sondik, 1971]

34. Value Iteration Variants
Belief Space

35. Value Iteration Variants
Exact value iteration
[Littman, 1994] [Cassandra et al., 1997]

36. Value Iteration Variants
Exact value iteration
[Littman, 1994] [Cassandra et al., 1997]

37. Value Iteration Variants

38. Value Iteration Variants
Grid-based
[Brafman, 1997] [Hauskrecht, 2000]

39. Value Iteration Variants
What next?

40. Value Iteration Variants
Focused value iteration

41. Value Iteration Variants
Initial Belief

42. Value Iteration Variants
Reachable Beliefs

43. Value Iteration Variants
Relevant Beliefs

44. Value Iteration Variants

45. Value Iteration Variants
Point-based (PBVI)
[Pineau et al., 2003]

46. Value Iteration Variants

47. Value Iteration Variants
Heuristic Search (HSVI)
[Smith and Simmons, 2004]

48. Outline Motivation Background and related work Technical approach
Preliminary work
Proposed research

49. Intelligent Site Survey

50. Intelligent Site Survey

51. Intelligent Site Survey

52. Intelligent Site Survey!

53. Intelligent Site Survey

54. System Architecture
planner
executive
functional
layer
environment

55. System Architecture planner executive functional layer science
observer
environment

56. System Architecture planner domain planners executive functional layer
science
observer
environment

57. System Architecture off-board onboard planner domain planners science
interface
executive
off-board
planner
functional
layer
science
observer
environment

58. Pushing POMDP Technology
Heuristic search
Combine heuristic search with efficient value function representations
Factored state
Do better when the state is mostly observable
Data structures that leverage independence
Continuous planning
Fast replanning

59. Outline Motivation Background and related work Technical approach
Preliminary work
Proposed research

60. Heuristic Search Value Iteration
Solves POMDPs
Fast anytime algorithm with provable bounds on plan quality
On some benchmark problems, >100x speedup
V * V

61. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

62. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

63. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

64. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

65. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

66. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

67. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

68. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

69. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

70. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

71. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

72. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

73. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

74. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

75. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

76. Heuristic Search Value Iteration
Focus on reachable beliefs
Avoid considering foolish actions and unlikely observations

77. Bounds Update at b b

78. HSVI Performance
Tag (870s 5a 30o)

79. Multi-Algorithm Comparison

80. Comparison with PBVI [Pineau et al., 2003]
optimal value
HSVI
PBVI
solution quality
wallclock time

81. Comparison with PBVI [Pineau et al., 2003]
optimal value vc
HSVI
PBVI
reported times
HSVI
PBVI

82. Comparison with PBVI [Pineau et al., 2003]

83. HSVI Related Work

84. Outline Motivation Background and related work Technical approach
Preliminary work
Proposed research

85. Research Goals
Develop and field test an SA system, includes developing
Problem definition and performance criteria
Overall architecture
Planning module
Extend POMDP state-of-art to meet the needs of the SA problem

86. Performance Criteria: SA
Overall performance: did the science team characterize the site accurately?
Subgoals with quantitative metrics
Coverage, key signatures, anomalies, representative sampling
Compare to baseline operational modes

87. Baseline Operational Modes

88. Baseline Operational Modes
directed sampling

89. Baseline Operational Modes

90. Baseline Operational Modes
periodic sampling

91. Performance Criteria: POMDP
Well-established metrics for POMDP planning performance
Primarily test in simulation
Compare to baseline planning techniques
Ignoring informational value (MDP)
Greedy or heuristic techniques (e.g. Q- MDP)

92. Life in the Atacama
Robotic astrobiology: work under Mars-relevant operational constraints and do meaningful science on Earth.

93. Science Mission
The biodiversity and distribution of habitats in the Atacama is not yet measured. Where does life survive and where does it not?
Coastal Range
Interior Desert

94. Planner Implementations
Atacama 2004
Make it work: deterministic world model, heuristic search, quick development
Technology research
A series of quickly developed planners that focus on specific POMDP techniques, tested in simulation
Atacama 2005
Integrate lessons from earlier versions: optimize for stability, extensibility
POMDP technology may not be ready

95. Pushing POMDP Technology
Heuristic search
Combine heuristic search with efficient value function representations
Factored state
Do better when the state is mostly observable
Data structures that leverage independence
Continuous planning
Fast replanning

96. Contributions
Lessons learned from developing and field testing one of the first science autonomy systems
Distinguished by “keep moving” operational strategy and novel intelligent site survey operational mode
Extensions to the POMDP state-of-art that apply both to SA and other domains

97. Schedule POMDP technology research (ongoing)
Initial SA development (Summer 04)
LITA Expedition 04 (Fall 04)
POMDP SA model development (Spring 05)
LITA Expedition 05 (Fall 05)
Thesis writing (Spring-Summer 06)

98. Questions?

99. NASA Mars Agenda 2005 Mars Reconnaissance Orbiter
2007 Mars Scout 1: Phoenix Lander
2009 Mobile Lab
2011 Mars Scout 2
2015 Mars Sample Return
...
~2025 Human Landing

100. OASIS Project Broad agreement on goals and approach
For instance, ideas about preferences (key signature, anomaly, etc.)
Distinctions of the proposed work
Operational strategy: keep moving
Kilometer-scale traverses, multiple sites per day
Novel intelligent site survey concept

101. LITA Science Autonomy 2004 September 2004 Expedition
Early versions of all modules present
Test science observer/planner closed loop in very simple situations
Teleoperate rover to simulate SA system
Gain experience with architecture and domain modeling
Get feedback from science team

102. LITA Science Autonomy 2005 September 2005 Expedition
Test full-up SA operational modes
Learn about integration into overall operations strategy
Characterize performance

103. Sensing Uncertainty
type 4
type 7
novel

104. Sensing Uncertainty
type 4
type 7
novel
type 4
type 7
novel

105. Sensing Uncertainty type 4 type 7 novel type 4 type 7 novel type 4

106. Sensing Uncertainty type 4 type 7 novel type 4 type 7 novel type 4

107. Science on the Fly

108. Science on the Fly

109. Science on the Fly

110. Science on the Fly

111. Science on the Fly !

112. Science on the Fly

113. Science on the Fly
carbonate!

114. Science on the Fly

115. Science on the Fly

116. Science on the Fly

117. Science on the Fly

118. Science on the Fly