1. Evolving Cutting Horse and Sheepdog Behavior with a Simulated Flock Chris Beacham Computer Systems Research Lab 2009

2. Table of Contents Focus Applications Previous Research Explanation of Problem Flocking Designing the Project Running the Project Results Future Research Possibilities Evaluation of Success Questions

3. Focus The focus of this project is attempting to evolve the behavior of a single agent or small group of agents so that they can effectively direct the movement of a much larger group which displays flocking behavior. In English: Trying to produce a computer program that can herd like a sheepdog

4. Applications Two main applications: Animation – Control direction of flocks in animations, currently no good method exists Livestock Herding: Could be used by a robotic agent to actually herd livestock. Robotic agent could reduce livestock stress.

5. Previous Research There has been little development in this specific field. Flocking algorithm is well established, but has no method to control movement. Robotic Sheepdog Project: Proof of Concept for robotic handling of livestock

6. Explanation of Problem: Sheepdog and Cutting Horse The sheepdog directs the movements of sheep, ducks, cows, goats and other herd stock. It accomplishes this by moving around the herd or flock in specific ways. The cutting horse is used with herds of cattle. It’s job is to single out a single cow, and “cut” it off from the herd, thus allowing people to access it.

7. Flocking The herd will be simulated with a flocking algorithm. Three algorithms make up flocking behavior: Separation Cohesion Alignment Assumption: Herds obey the same rules as flocks, but are stable

8. Designing the Project: Genetic Algorithms Genetic Algorithms mimic evolution. They provide unexpected and non-intuitive solutions. Capable of searching massive search space. Require four things: –Gene Pool –Heuristic (scoring system) –Breeding Algorithm –Time

9. Designing the Project: Genetic Algorithms 1000 agents per generation Each agent is evaluated 500 agents are placed into the breeding pool –15 copies each of the 10 best agents –5 copies each of agents in places 11-50 –1 copy of each agent in places 51-200 500 new agents are bred randomly from the breeding pool All new agents and breeding pool agents are mutated. These comprise the next generation. Repeat

10. Designing the Project: Gene Pool Which behavior determination system to use? Neural Networks Cellular Automaton Weighting Pros and Cons for all options

11. Designing the Project: Gene Pool Weighting method chosen as best behavior determination system. Genetic code adds weight to different vectors that determine behavior. Pros Many strategies can be produced Should be reproducible in other situations Cons Nowhere near the possibilities for innovation of neural nets.

12. Designing the Project: Heuristic 2 different methods – Time trials and Distance trials Both types evaluate a single agent Time Trials –Create a goal, agent and flock –Stop trial when flock is within certain distance of goal –Score is the time taken to reach that distance –Should create fast solutions Distance Trials –Create a goal, agent and flock –Stop simulation after a certain amount of time –Score is the distance from the flock to the goal –Should create exact solutions

13. Running the Project: Creating the Simulation

17. Running the Project: Now We Wait Lots of time need for repeated trials Monopolized a server round-the-clock for several weeks 14 simultaneous populations Over one million agents created and evaluated

18. Results Two strong successes Herding behavior is surprisingly simple, relies on only two vectors to determine movement. Demonstration.

19. Results: Genetic Algorithms 2 – Fixed Error 5 – Automated Testing, increased field size

20. Results: Genetic Algorithms

21. Results Essential difference between flocking and herding behavior – Specificity Flocking behavior is non-specific, and thus hard to judge. Herding behavior is goal oriented and specific. It requires tuning to get good results

22. Evaluation of Success: Are any Strategies Viable? Two applications: Animations and Livestock Livestock – Strategy would likely work. Needs feedback mechanisms and real life trials. Animation – depends on application. Evolving a specific agent would take too long, but a trial and error implementation could give good results.

23. Future Research Future research should focus on these areas: –Feedback mechanisms and fine tuning –Option of other vectors to determine behavior –Accuracy of Herd simulation –Actual livestock responses and trials

24. Questions?

25. Third Quarter Breeding program and Automated testing program produced. Breeding program designed for very heavy selection pressure Agents evaluated on how close they get the flock to the goal.

26. Third Quarter Several strategies predominate “Bubble” strategy “Slow” strategy “Southeast” strategy “Spinning wheel” strategy “Wall-bounce” strategy – suspect disconnect between graphical output and nongraphical output

27. Second Quarter First generation sheepdog behavior is randomly generated. Movement is very chaotic.

28. First Quarter The first quarter was spent creating a flocking behavior simulation, and debugging it. This is a screenshot of a flock flying to the right.

29. Second Quarter Herding behavior established. Same algorithm as flocking.

30. Second Quarter Herd reacting to ‘predator’ Sheepdog agent very simple at this point