1. Data Structure for Behavior-Based Machine Learning/Path Planning Zachary Dawson 16-264 A: Humanoids 5/7/13

2. Goals  Develop a method in which a robot’s past experience can be applied to new situations, creating a finite number of behaviors  Create a data structure and algorithm that utilize this method  Implement this data structure and algorithm in Python

3. The Method  Get desired final state (target)  Check known behaviors  Execute behavior — OR—  Make behavior  Select known behavior that advances closest to target  Add to list of steps in new behavior  Repeat until target is achieved  Add new behavior to list of behaviors  Execute new behavior

4. Data Structure – Behavior  Parameters:  Instructions (array of other behaviors or primitives)  Primary functions:  __init__  makeBehavior  predict  execute

5. Data Structure – The Model  Parameters:  Canvas (Python)  State  Draw function  Validity check function  To do list  Functions:  __init__  draw  check  copymodel

6. Data Structure – Client Implementation  drawRobot  stateChecker (Ensures model’s state is valid)  *checkExisting (Checks to see if existing behaviors accomplish task. Not actually required for implementation to run)  stateCompareFn (Compares two states to see if they are the same)  Primitives  One for each controlled joint  Have maximum displacement in 1 time-step  Account for all physical limits of robot

7. Example of hierarchy of behaviors Move to target Move right/left Slide Adjust arm Rotate lower Rotate upper

8. Example instance of behavior class Behavior: move arm to far away target [move right, adjust arm, move right, adjust arm, move left] [slide, slide][rotate upper, rotate lower][slide, skip]

9. Algorithmic Implementation 1.Receive target (assume no known behavior gets to target) 2.Loop through existing behaviors and get predicted result 3.Select behavior with the predicted result closest to target 4.Add that behavior to the instructions for the new behavior 5.Repeat steps 2 through 4 until target is achieved 6.If program hangs (indicates that the distance from the target is the same in two iterations), pick a random behavior from the behavior list that does NOT have the closest predicted value 7.Once new behavior is made, add it to the list and execute

10. Algorithmic Implementation  On execute, instructions from behaviors are recursively added to a to-do queue in the model class so that the queue becomes a list of primitives and their target states. This enables animation by calling instructions from this queue at set time intervals  If an instruction is predicted to cause no change, it is skipped on the to-do list

11. Demonstration term.py

12. Assessment and Further Development  Developed new behaviors, to a finite extent  Reused behaviors where appropriate  With proper implementation of primitives, could accomplish any task  Terminated over-complex behaviors early  Reduce runtime of predict  Introduce automatic loop creation  Test on system with more parameters  Use sensor based system (do not assume location of everything is exactly known)

13. Questions?