Presentation on theme: "Evolving Cooperative Strategies in Multi-Agent Systems Using a Coevolutionary Algorithm Cesario C. Julaton III, Ramanathan S. Thinniyam, Una-May O’Reilly."— Presentation transcript:
Evolving Cooperative Strategies in Multi-Agent Systems Using a Coevolutionary Algorithm Cesario C. Julaton III, Ramanathan S. Thinniyam, Una-May O’Reilly Computer Science and Artifical Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA 94720 Coevolutionary AlgorithmBackground Results Acknowledgements: C. C. Julaton III acknowledges support from the MIT Summer Research Program Cooperative Coevolutionary Genetic Algorithms Used in search, optimization, and machine learning Solution parameters are encoded as genes in chromosomes Chromosomes are evaluated using a fitness function to see how well the corresponding solution answers the problem Highly fit chromosomes are selected and participate in crossover and mutation to give rise to new potential solutions. Relatively new subfield of genetic algorithms that can be broken down into subcomponents Goals Have a better understanding of coevolutionary algorithms Create strategies in fixed environments that succeed in modified environments Discussion Grid Sizes 10x10 grown: 60% capture rate on a 10x10 grid, but the rate approaches 0 as the grid size increases 50x50 grown: performs better in smaller grids than in larger grids 100x100 grown: higher capture rate in larger grid sizes, including grids larger than training size Adding/Removing a Predator Adding a predator to an evolved strategy that either moves directly towards the prey or remains stationary reduces the team’s ability to capture the prey Removing a predator from an evolved team significantly reduces the rate at which the strategy captures the prey Prey Behavior Using prey 1 during training gave rise to a strategy that performs at its best against prey 1 and performs better on prey 2 than on prey 3 The strategy obtained by training with prey 2 performs close to perfection when tested on prey 2 (capture percentage above 98%), but it performs poorly when tested on prey 1 and on prey 3 (capture percentages close to 0) The strategy grown on prey 3 does not have a capture percentage greater than 50% for prey 3, but it has capture percentages above 7% for prey 1 and prey 2 Future Research Allow the predators to follow different rulesets Introduce a prey that evolves during evolution Compare coevolutionary algorithm with other evolutionary algorithms Vary parameters affecting the simulation, such as the number of chromosomes in a subpopulation, number of generations, selection, crossover, mutation rate, etc. Problem Formulation Simulation Predators cooperatively work together to capture a prey in a square toroid Predators Move horizontally or vertically according to their relative positions from the prey Except one, every predator also moves according to one other predator’s relative position from the prey Prey Behavior Prey 1: move away from nearest predator Prey 2: move to maximize the mean distance from predators Prey 3: move in a position that is not adjacent to a predator Genotype Encodings 1 Predator: 8 genes Remaining Predators: 64 genes 1 1 1 1 3 3 1 3 Fitness = 1 if captured 0 otherwise Coevolutionary algorithm evolved a cohesive unit and the introduction of new, unseen agents in a team reduces the efficiency of the strategy We saw the emergence of behavioral patterns humans can relate to One predator waited at the bottom while others attacked from the top and from the left To exploit prey 2’s weakness, the majority of the predators remain in one portion of the grid while one approaches the prey.