Spatio-Temporal Case-Based Reasoning for Behavioral Selection Maxim Likhachev and Ronald Arkin Mobile Robot Laboratory Georgia Tech

Maxim Likhachev and Ronald Arkin Broad Picture of the Work Part of Multi-Level Learning in Hybrid Deliberative/Reactive Mobile Robot Architectural Software Systems project at Georgia Tech Sponsored by the DARPA MARS program Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Motivation Constant parameterization of robotic behavior results in inefficient robot performance Manual selection of “right” parameters is difficult and tedious work Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Motivation (cont’d) Use of Case-Based Reasoning methodology for an automatic selection of optimal parameters in run- time Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Evaluated on: Simulations Real robot –ATRV-JR in outdoor environment –Nomad 150 in indoor environment Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Related Work ACBARR, SINS and KINS systems –use of case-based reasoning and reinforcement learning for the optimization of behavioral parameters –contribute to some ideas behind the present algorithm Automatic optimization of parameters –genetic programming –reinforcement learning Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Behavioral Control and CBR Module CBR Module controls: Weights for each behavior BiasMove Vector Noise PersistenceObstacle Sphere Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Input Features for Case Selection Vector of spatial characteristics of environment –D - distance to the goal – - degree of obstruction and distance to the most obstructing cluster of obstacles for each of K angular regions around the robot Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Input Features for Case Selection Vector of temporal characteristics of environment –R s - short term robot movement –R l - long term robot movement Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Computation of Traversability Vector F F: –represents traversability of each region –approximates obstacle density function around the robot –independent of goal distance –smoothed over time: Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Input Features: Example Spatio-Temporal Case-Based Reasoning for Behavioral Selection f 0 =0.92 f 1 =0.58 f 2 =1.0 f 3 =0.68 f 0 =0.02 f 1 =0.22 f 2 =0.63 f 3 =0.02 V temporal ShortTerm: R s =1.0 LongTerm: R l =0.7 V temporal ShortTerm: R s =0.01 LongTerm: R l =1.0

Maxim Likhachev and Ronald Arkin High Level Structure of CBR Module Spatio-Temporal Case-Based Reasoning for Behavioral Selection Current environment Feature Identification Spatial Features & Temporal Features vectors Spatial Features Vector Matching (1st stage of Case Selection) Temporal Features Vector Matching (2nd stage of Case Selection) Set of Spatially Matching cases Set of Spatially and Temporally Matching cases Case switching Decision tree Case Adaptation Case Library All the cases in the library Best Matching or currently used case Case Application Case ready for application Case Output Parameters (Behavioral Assemblage Parameters) Random Selection Process (3rd stage of Case Selection) Best Matching case

Maxim Likhachev and Ronald Arkin Case Example I Spatio-Temporal Case-Based Reasoning for Behavioral Selection CLEARGOAL Spatial Vector: D (goal distance) = 5 density distance Region 0: σ 0 = 0.00; r 0 = 0.00 Region 1: σ 1 = 0.00; r 1 = 0.00 Region 2: σ 2 = 0.00; r 2 = 0.00 Region 3: σ 3 = 0.00; r 3 = 0.00 Temporal Vector: (0 - min, 1 - max) ShortTerm_Motion R s = LongTerm_Motion R l = Case Output Parameters: MoveToGoal_Gain = 2.00 Noise_Gain = 0.00 Noise_Persistence = 10 Obstacle_Gain = 2.00 Obstacle_Sphere = 0.50 Bias_Vector_X = 0.00 Bias_Vector_Y = 0.00 Bias_Vector_Gain = 0.00 CaseTime = 3.0

Maxim Likhachev and Ronald Arkin Case Example II Spatio-Temporal Case-Based Reasoning for Behavioral Selection FRONTOBSTRUCTED_SHORTTERM Spatial Vector: D (goal distance) = 5 density distance Region 0: σ 0 = 1.00; r 0 = 1.00 Region 1: σ 1 = 0.80; r 1 = 1.00 Region 2: σ 2 = 0.00; r 2 = 1.00 Region 3: σ 3 = 0.80; r 3 = 1.00 Temporal Vector: (0 - min, 1 - max) ShortTerm_Motion R s = LongTerm_Motion R l = Case Output Parameters: MoveToGoal_Gain = 0.10 Noise_Gain = 0.02 Noise_Persistence = 10 Obstacle_Gain = 0.80 Obstacle_Sphere = 1.50 Bias_Vector_X = Bias_Vector_Y = 0.70 Bias_Vector_Gain = 0.70 CaseTime = 2.0

Maxim Likhachev and Ronald Arkin Results Spatio-Temporal Case-Based Reasoning for Behavioral Selection Average travel distanceMission success rate Simulations: ATRV-JR: 12% average performance improvement in time steps ( based on 10 runs for each system in outdoor environment)

Maxim Likhachev and Ronald Arkin Simulations & real robot experiments: Performance improvement as a function of obstacle density Spatio-Temporal Case-Based Reasoning for Behavioral Selection SimulationsNomad 150 Based on 10 runs for each system in indoor environment

Maxim Likhachev and Ronald Arkin Real Robot Run with CBR Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Real Robot Run without CBR Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Maxim Likhachev and Ronald Arkin Trajectories of the robot Spatio-Temporal Case-Based Reasoning for Behavioral Selection Robot with CBR moduleRobot without CBR module 11% less travel distance

Maxim Likhachev and Ronald Arkin Conclusions Automatic selection of optimal behavioral parameters results in robot performance improvement (based on simulations and real robot experiments) Careful manual selection of behavioral parameters is no longer required from a user Future Work –Automatic learning of cases: identifying when to create a new case applying reinforcement learning techniques in finding optimal parameters for existing cases –Integration with other adaptation & learning methods (e.g., Learning Momentum) Spatio-Temporal Case-Based Reasoning for Behavioral Selection