Presentation is loading. Please wait.

Presentation is loading. Please wait.

Learning to Coordinate Behaviors Pattie Maes & Rodney A. Brooks Presented by: Javier Martinez.

Similar presentations

Presentation on theme: "Learning to Coordinate Behaviors Pattie Maes & Rodney A. Brooks Presented by: Javier Martinez."— Presentation transcript:

1 Learning to Coordinate Behaviors Pattie Maes & Rodney A. Brooks Presented by: Javier Martinez

2 Introduction Behavior-based system Learning using positive and negative feedback Behaviors decide when is time to activate Distributed algorithm Test the concept in a robot

3 Motivation Behavior control is a weak point initial Behavior-based systems Behavior control has to be prewired This approach doesn’t scale too well

4 New Ideas Behavior control is learned through experience Learning algorithm completely distributed Each behavior learns when to become active The solution maximizes positive feedback and minimizes negative feedback

5 The Learning Task What is needed: Vector of binary perceptual conditions Set of behaviors Positive feedback generator Negative feedback generator

6 The Learning Task The task: Change the precondition list from each behavior to maximize relevance and reliability

7 The Learning Task Constraints: Relevance: behavior correlated to positive feedback, not correlated with negative feedback Reliability: behavior receives consistent feedback

8 The Learning Task More constraints: Algorithm should deal with noise, Perform in real time, Support readaptation

9 The Learning Task Assumptions: At least one combination of preconditions is bounded Feedback is immediate Only combinations of conditions can be learned

10 Algorithm Measure: Number of times a positive/negative feedback did/didn’t happen when a behavior was/wasn’t active Calculate the correlation between positive/negative feedback and the status of the behavior

11 Algorithm Measure: Express relevance and reliability in terms of this correlation Relevance controls whether a behavior should be active or not Reliability decides whether the behavior should try to improve itself

12 Algorithm Measure: Improvement is done by monitoring a perceptual condition If reliability increases, the behavior is added to the list of preconditions Keep monitoring in a circle until reaching the threshold

13 Genghis Six-legged robot that walks forward 12 behaviors, 6 conditions, 8742 nodes 4 eight-bit microprocessors, 32 KB memory The challenge is to learn how to coordinate the legs to produce a forward movement

14 Results Convergence time Non-intelligent search during the monitoring stage: 10 minutes Intelligent search: 1min 45sec A “tripod” gait emerged which is common among six-legged insects

15 Conclusions A learning algorithm was developed which allows a behavior-based robot to learn when its behaviors should become active using positive and negative feedback

16 Comments + Impressive results + Global behavior (walking) emerges from coordinated Behaviors + Simple idea, powerful consequences. Robot learned how to walk, wasn’t taught

17 Comments – Dead behaviors don’t revive. They might be useful in other situations ? How to deal with concurrent actions? (i.e. walking and following a target)

Download ppt "Learning to Coordinate Behaviors Pattie Maes & Rodney A. Brooks Presented by: Javier Martinez."

Similar presentations

Ads by Google