1. Using Genetic Programming to Evolve Sumobots Shai Sharabi Dept. of Computer Science Ben-Gurion University, Israel

2. Overview Introduction Sumobot system description GP system description –Preparatory steps –Evolutionary process Results Conclusions

3. Sumo history Sumo has its roots in the Shinto religion (800 A.D). There are two principal ways to win a Sumo bout: –The first wrestler to touch the ground outside the circle loses –The first wrestler to touch the ground with any part of his body other than the soles of his feet loses A Sumo match (Ozeki Kaio vs. Tamanoshima in May 2005).

4. Sumobot rules Sumobot contests are hosted in Seattle www.robothon.org 2 robots try to push each other outside the arena boundaries

5. Sumobots The complexity of robot behavior Studies in Evolutionary Robotics (Nolfi & Floreano) Floreano and Mondada utilized a Khepera robot to validate a neural network system evolved using a genetic algorithm – for navigation and obstacle avoidance Liu & Zhang, Multi-Phase Genetic Programming: A Case Study in Sumo Maneuver Evolution

6. Sumobot System Description Two overhead web cameras, each connected to its own computer. Each computer transmits via its remote controller the maneuver commands to the respective sumobot, which then acts accordingly.

7. GP System Description Preparatory steps 1.Determining the set of terminals 2.Determining the set of functions 3.Determining the fitness measure 4.Determining the parameters for the run (population size, number of generations, minor parameters) 5.Determining the method for designating a result and the criterion for terminating a run

8. Functions & Terminals (steps 1-2)

10. Fitness Function (step 3) w1 … w7 – empirically derived weights count - number of iterations in one fight radius – distance between robot’s starting & farthest location sticky – rewards spending time close to the target closer – rewards approaching the target speed – rewards higher speed push – rewards pushing the opponent bonuspush – rewards faster wining programs exploring – rewards exploring the grounds bonustay - is a bonus added for staying in the arena w 1 radius(count) + w 2 sticky(count)+ w 3 closer(count) + w 4 speed(count)+ w 5 push(count) + w 6 bonuspush(count)+ w 7 exploring(count)+bonustay

11. Parameters (steps 4-5) Population size20 Generation count7 - 57 Selection methodrank Reproduction probability 0.2 Crossover probability0.8 Mutation probability0.05 - 0.1 Elitism group2

12. Linear Ranking Selection Based on sorting of individuals by decreasing fitness The probability to be extracted for the i th individual in the ranking is defined as where  can be interpreted as the expected sampling rate of the best individual

13. Various Linear Ranking Parameters

14. GP System Description Evolutionary Process 1.Creating random initial population using functions and terminals 2.Running and evaluating all the programs using the fitness measure 3.If termination criterion satisfied for the run stop the evolutionary process 4.Select programs and apply genetic operations to them 5.Goto step 2

15. GP flowchart

16. Initial Population Ramped half-and-half –Using md = 8 [max depth] –Divide the population evenly to md-1 bins –Half the population in each bin is created using the “Grow” method, the other half using the “Full” method. –Each bin is given a new md’ starting with md’=2 up to md’=md

17. Four Batches of Experiments

18. Typical Results of Batch A a)Evolved fighter rotates toward the target and approaches it if starting from a position where  < -11 b) Evolved fighter circles widely unconditionally c) Simplified program of robot (a) d) Simplified program of robot (b)

19. Typical Result of a Fight in Batch B a)Demonstration of a fight between two individuals from the 44th generation of Sumo1 and generation 39 of Sumo2 b) Sumo1's simplified code (bottom robot) c) Sumo2's simplified code (top robot)

20. Batch C Details At some point (criteria dependent * ) we continue evolution with a different, more demanding fitness function (like higher score for pushing and fast wining) Start with fitness function which “easily” evolves simple sumo strategies (like exploring the arena)

21. Changes of Dynamic Fitness in Batch C Dynamic fitness computation: After 10 generations the fitness weights were adjusted to assign each fitness component with a new range.

22. Typical Results of Batch C a) A fight at generation 3 b) A fight at generation 7 c) Left robot scored Yuko at generation 13. This is the highest possible score, given when one contender manages to push its opponent out of the arena

23. Typical Fitness Progress of One Experiment in Batch C After 10 generations the fitness computation changed

24. Typical Evolved Code in Batch C PPMMOTORSPEED manuver(int x1, int y1, int x2, int y2, int angle){ if(ifl( angle, 7 )) {if(ifl( negative_1( sdiv( negative_1( plus( x2, angle ) ), minus( mul( 0, y1 ), mul( y2, y1 ) ) ) ), minus( negative_1( abs_1( 7 ) ), x2 ) ) ) {if(ifl( abs_1( plus( minus( 0, angle ), negative_1( y2 ) ) ), abs_1( mul( mul( 0, 0 ), x2 ) ) ) ) {return spin( negative_1( mul( abs_1( x1 ), plus( x2, x2 ) ) ) );} else {if(ifl( x1, abs_1( negative_1( plus( y2, 0 ) ) ) )) {if(ifl( abs_1( y1 ), sdiv( x2, abs_1( angle ) ) )) {if(ifl( plus( angle, x2 ), x1 )) {if(ifl( y2, x1 )) {return moveLW( y1 ); }else {if(ifl( 0, angle )) {return spin( x2 ); }else {return moveFree_2( 0, y1 ); } } }else {return spin( 0 ); } }else {return moveRW( abs_1( y2 ) ); } }else {if(ifl( sdiv( sdiv( x2, x1 ), minus( y1, 0 ) ), negative_1( negative_1( y2 ) ) )) {return spin( minus( 0, x1 ) ); }else {return spin( minus( y2, angle ) ); } }} } else {return spin( negative_1( mul( abs_1( x1 ), plus( x2, angle ) ) ) ); }} else {return moveLW( x1 ); }}

25. Typical Results of Batch D a)Evolved bot fights avoid- contact opponent b)Evolved bot fights pushing opponent and scores a Yuko c)Evolved bot fights spinning opponent d)Fitness graph of a run that produced evolved sumobot. The drop of fitness in generation 12 was due to a mechanical failure in the sumo wheels. Nevertheless evolution overcame this problem and slowly yielded an adapted sumo program.

26. Batch A: Best Programs Fitness Progress of 10 Runs

27. Batch A: Average Fitness Progress of 10 Runs

28. Batch D: Best Programs Fitness progress of 9 runs

29. Batch D: Average Fitness Progress of 9 Runs

30. Movies Presentation from Batch C (co-evolution) Dancing Approach Riding the Enemy Wining Yuko

31. Conclusion GP can be utilized to evolve sumo fighter strategies for a simple robot with only 5 input terminals All 4 Batches yielded positive results Fitness oscillation Convergence Comparing results to others

32. Future Work Using more difficult to operate platforms Adding more specific domain terminal ( e.g., past location ) Using ADF