1. Exploring Altruism in Emergent Behavior of Evolving Cooperative Robots
Anthony Groves
Advisor: Dr. C. David Shaffer

2. Introduction Teams of smaller robots are cheaper and more effective.
Cooperative multi-agent systems (MAS)
Altruism: a specific action that allows the agents to help its teammates at a cost to itself.
Are cooperative agents more effective at performing a task if they are encouraged to be altruistic?
Use genetic algorithms (GA) to evolve teams of simulated robots to perform a task cooperatively.
Compare two different types of teams:
Forced altruism. Genetic algorithm encourages altruism
No forced altruism.

3. Genetic Algorithms
Natural selection: A population of chromosomes evolves, creating next generation’s population by selecting and changing some of the most fit chromosomes in current population.
Some randomness and mating causes new strategies to evolve.
Many different selection, mutation, and crossover strategies.

4. Previous work All experiments simulated a MAS using GAs
T. Haynes (1995)
Teams of 4 predators to catch single prey
Temporally and spatially discrete simulation
Brad Patton (2005)
Coevolved competitive teams that played freeze tag against each other in a pursuer-evader system.
Showed emergent altruistic behavior

5. Task
Used GA to evolve teams of Lions to perform the task of capturing Gazelle.
Task (Gazelle capturing) requires more than 1 Lion
An altruistic action that allows a Lion to benefit other Lions at a cost to itself.
Used two different types of teams (two different fitness functions):
No encouraged altruism. Fitness function only credits team for how many gazelle are captured
Encouraged altruistic behavior: team receives higher fitness for more altruistic actions
Same genotypes and breeding strategies
Effectiveness: how many gazelle are captured in allotted time.
Hypothesis

6. Simulation
Temporally and spatially discrete, 10 x 10 grid where an agent can occupy one cell.
Agents move one neighboring cell each time step.
100 steps
Fixed number of each types of agents.
Simulation ran to evaluate a chromosome’s fitness.

7. Agents: Hunters and Gazelle
Pursuers of the lions, do not interact with gazelle.
“Traps” a lion when it attempts to move into a cell occupied by a lion. Trapped lions are frozen (unable to move) until freed.
Moves towards nearest Lion every 2 time steps.
Gazelle
Prey of the lions, do not interact with hunters.
Can only move orthogonally (unlike hunters and lions).
Random movement

8. Cooperative Agents: Lions
Predators of gazelle, captures when multiple lions are neighboring gazelle orthogonally.
Must eat Gazelle to live, or else will die before simulation ends.
Evaders of hunters. Gets trapped or “frozen” when a hunter moves into its cell.
Able to “free” trapped teammates when adjacent.
Altruism: Freeing a frozen teammate when a hunter is near.

9. Inputs and Genetic Structure
A chromosome of binary bits.
Has 10 ‘blocks’. Each block contains a header section and a move section.
Header section: distance of Lion to the 4 types of sensors: nearest Gazelle, nearest Hunter, nearest trapped Lion, nearest non-trapped Lion.
Move section: picks movement and amount of time steps for movement (0-15 steps).
Starts with first block and iterates through each block until the header section of block is true, then executes movement section.

10. GA Configurations Population size: 64 Selection: Roulette wheel
Mutation: 4%, single bit flip.
Crossover: Exchange last 5 blocks with first 5 blocks.
Elitism: 4 most fit
All teams homogeneous
Fitness function:
Number of Gazelle captured * 1000
(Number Gazelle Captured*1000) + (Total Altruistic Points*10)
Altruistic Points: 1 or 2 pts awarded to Lion for each time step near a Frozen Lion and Hunter (2 pts if very close to Hunter)
Each chromosome evaluated based on its performance with a simulation.

11. Basic Results
Requiring Lions to completely surround Gazelle (typically 4 Lions) too difficult for GA to produce effective strategies.

12. Basic Results
Later generations of Lions cooperated more and captured more Gazelle.

13. Signs of Evolution
Most fit from Generation 100

14. Altruistic Team: Basic Results
Too much focus on altruism, not Gazelle capturing, causes less effective Lions.
No consistent increase in effectiveness as gens evolve.

15. Comparison & Conclusion: Altruistic vs Non-Altruistic
Graph of # of Gazelle captured, does not include Altruism Points.
Red – Encouraged altruism.
Orange – No encouraged altruism
Altruistic team: No consistent increase in effectiveness, even if started out slightly more effective.
Higher level of intelligence required to be effective when forced to be altruistic.

16. References
Dickerson, K. NASA’s Robot Army of ‘Swarmies’ Could Explore Other Planets, Space. Retrieved September 30, 2016, from Space:
Gentzel A. Exploring the Evolution of Cooperative Behaviors in Multi-Robot Systems. Westminster College Honors Thesis
Haynes T, Sen S, Schoenefeld D, Wainwright R. Evolving a Team. In Working Notes of the AAAI-95 Fall Symposium on Genetic Programming. Eric V. Siegel and John R. Koza, ed. AAAI Press
Holland J. Hidden Order: How Adaptation Builds Complexity. Basic Books, USA
Jeong IK, Lee J. Evolving cooperative mobile robots using a modified genetic algorithm. Robotics and Autonomous Systems 21,
Luke S, Hohn C, Farris J, Jackson G, Hendler J. Co-evolving soccer softbot team coordination with genetic programming. In Proceedings of the RoboCup-97 Workshop at the 15th international Joint Conference on Artificial Intelligence. H. Kitano, ed. IJCAI
Luke S, Spector L. Evolving teamwork and coordination with genetic programming. In Genetic Programming 1996: Proceedings of the First Annual Conference. MIT Press
Panait L, Luke S Cooperative Multi-Agent Learning: The State of the Art. Autonomous Agents and Multi-Agent Systems 11(3):
Patton B. Coevolving Teams for Robotic Freeze Tag. Westminster College Honors Thesis
Russell S, Norvig P. Artificial Intelligence: A Modern Approach. Pearson, India
Shadbolt, P. U.S. Navy unveils robotic firefighter, CNN. Retrieved September 30, 2016, from CNN: