1. Behavior design CS 395: Behavior-based systems Ian Horswill

2. Previously  Programs  Policies  Determine action from world state rather than from internal program counter  Building simple control loops from feedback  Compare measured state and desired state  Action is f(state error)  Linear feedback is when f is multiplication by a constant (or constant matrix)

3. Outline  Behaviors and naïve behavior-based systems  Simulating a frog  Motor schemas and potential fields

4. Composing policies from behaviors  Behavior = policy + trigger  Naïve behavior-based system  Run lots of behaviors in parallel  Mutually exclusive triggers  Use policy of currently triggered behavior

5. Common types of behaviors  Orientation behaviors  Tropisms  Followers  Avoidance behaviors  Ballistic actions (fixed action patterns)

6. GRL code for behaviors  Behavior data type  Activation level  The trigger Boolean or numerical  Motor-vector  Implementation- dependent  drive-base modified  Uses the policy of the leftmost currently active behavior Constructor: (behavior activation-level motor-vector )  behavior Accessors: (activation-level behavior )  behavior’s trigger (motor-vector behavior )  output of behavior’s policy (drive-base behavior behavior …)

7. Outline  Behaviors and naïve behavior-based systems  Simulating a frog  Motor schemas and potential fields

8. Frog feeding behaviors  Three separate neural circuits  Orientation behavior  Triggered by black dot moving on a white background (i.e. fly)  Turns body toward prey  Pouncing  Moves body within range of prey  Eating  Ballistic motion of the tongue

9. Simplified robot frog  Motor vector  Rotation/translation control  Eat? fires the tongue  Tongue is not steerable  (we won’t simulate hopping)  Sensors  Detect flies using motion on the retina  Doesn’t work when frog is moving!  Vestibula (inner ear)  Measures rotation like a gyroscope (frog-motor-vector motion eat? )  control info for “base” fly-motion  direction of fly, if visible doesn’t work when frog moves see-prey?  true, if fly is visible rotational-velocity  current rotation speed of frog

10. Simulated feeding behaviors  Eat  Fires the tongue when facing prey  Face prey  Steers toward prey when visible  Inactive when already facing prey  (no pouncing) (drive-base eat face-prey)

11. The eat behavior (define-signal eat (behavior (and see-prey? facing-prey?) (frog-motor-vector sit-still #t))) (define-signal facing-prey? (< (abs fly-motion) facing-prey-tolerance))) (define-signal sit-still (rt-vector 0 0)) Activation level (trigger): Wait until prey in sights Control policy: fire tongue while standing still

12. The face-prey behavior (define-signal face-prey (behavior (and see-prey? (not facing-prey?)) (frog-motor-vector (rt-vector (* fly-motion face-prey-gain) 0) #f))) (define-signal facing-prey? (< (abs fly-motion) facing-prey-tolerance))) (define-signal sit-still (rt-vector 0 0)) P-controller

13. Problem (define-signal face-prey (behavior (and see-prey? (not facing-prey?)) (frog-motor-vector (rt-vector (* fly-motion face-prey-gain) 0) #f))) Fly-motion sensor detects relative motion of frog and fly Stops responding as soon as the frog moves  Need to remember the fly’s position somehow

14. Solution: dead reckoning (define-signal face-prey (behavior (and see-prey? (not facing-prey?)) (frog-motor-vector (rt-vector (* prey-heading face-prey-gain) 0) #f))) (define-signal frog-orientation (integrate rotational-velocity)) (define-signal prey-orientation (latch (+ frog-orientation fly-motion) (zero? rotational-velocity))) (define-signal prey-heading (subtract-angle-degrees frog-orientation prey-orientation))

15. Outline  Behaviors and naïve behavior-based systems  Simulating a frog  Motor schemas and potential fields

16. Designing behaviors using potential fields  Suppose:  Your state is your position in space  Your action is a velocity vector  General technique for constructing policies:  Assign every point in space x a “value” V(x)  Use the gradient  V(x) as your policy  Analogy to physics   V(x) is like a force field  V(x) is its potential Approach behavior (tropism) Avoidance behavior

17. Advantages of potential field methods  Simple to compute  Can combine policies by adding value functions (same as adding gradients)

18. “Motor schemas”  Like potential fields but we program directly in terms of the vectors – no value function  One schema (vector field) per behavior  Sum outputs of behaviors

19. Example: sonar-based obstacle avoidance  First, compute obstacle positions  16 equally spaced sonars  So sonar n is at orientation 2  n/16  So obstacle is at: r(cos 2  n/16, sin 2  n/16)  where r is the distance reading for the sonar (define-signal sonar-count 16) (define-signal ( sonar-direction s-num) (/ (* 2 pi s-num) sonar-count)) (define-signal ( unit-vector direction) (xy-vector (cos direction) (sin direction))) (define-signal ( obstacle-vector s-num reading) (* (unit-vector (sonar-direction s-num)) reading)) X-axis and direction of motion positive rotation

20. Vectors in GRL  Like arrays in C  Constructors  (vector a b c)  [a b c]  (make-vector k elt)  [elt elt … elt] k times  (index-generator i)  [0 1 2 … i]  Accessors  (vector-ref v i)  i’th element of v  (vector-length v)  number of elements in v

21. Implicit mapping in GRL  Scheme and C++ STL let you map a function over a list:  (map f (list a b c d))  (list (f a) (f b) (f c) (f d))  GRL automatically maps functions over groups and vectors:  (f (vector a b c d))  (vector (f a) (f b) (f c) (f d))  (* 2 (vector 1 2 1 3))  (vector 2 4 2 6)  (* (rt-vector a b) (rt-vector c d))  (rt-vector (* a c) (* b d))  Exceptions: length, vector-ref, etc.

22. Example: sonar-based obstacle avoidance  Each obstacle (i.e. each sonar reading) generates a force  Opposite in direction to the obstacle  Magnitude is inverse square of distance  Like electrostatic force (define-signal ( obstacle-force ob-vector) (- (/ ob-vector (cube (magnitude ob-vector))))) (define-signal obstacle-vectors (obstacle-vector (index-generator sonar-count))) (define-signal obstacle-forces (obstacle-force obstacle-vectors)) (define-signal total-force (vector-reduce + 0 obstacle-forces))

23. Following the xy-vector  Problem  Policy generates Cartesian (xy) velocities  Base controlled with polar (rotate/translate) velocities  Most xy vectors can’t be executed directly by the base  Need to transform Cartesian policy into a polar policy  Lots of techniques possible. Here are two. ;; CMU policy (Stone et al.) (define-signal ( follow-xy vec) (rt-vector (y-of vec) (x-of vec)) ;; GATech policy (Arkin et al.) (define-signal ( follow-xy vec) (if (< (abs (y-of vec)) threshold) (rt-vector 0 speed) (rt-vector (* (y-of vec) r-gain) 0)))

24. Local minima  Different forces from different objects can exactly cancel, leaving the robot stuck  Work-arounds  Unwedging  Noise  Add random vectors to motion  Avoid-recent behavior  Remember recent locations  Generate repulsive force away from them  Path planning  Run a search algorithm to find a path through freespace