1. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP1 Experiments With Explicit For-Loops in Genetic Programming Vic Ciesielski, Xiang Li {vc, xiali}@cs.rmit.edu.au School of Computer Science and Information Technology RMIT University, Australia

2. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP2 Overview Introduction –Why do we experiment loops? Related works –Who have used loops in GP before? Syntax and Semantic of the For-Loops Experiments and Results –Santa Fe Ant Problem –Sorting Problem Conclusion –Use loops to achieve your goals quicker and better

3. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP3 Introduction Why do we experiment loops? –Powerful – provide a mechanism for repetition. Why loops are seldom used in GP? –Hard to evolve {start, end, body, update branch} –Avoid infinity –Large class of problems can be solved without loops –Implicit loops are used. {Robsoccer, Santa Fe Ant} Type of loops –For-loops and while-loops –The research is focusing on for-loops.

4. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP4 Loop Syntax & Expectations Two for-loop constructs are used –(FOR-LOOP1 NUM-INTERATIONS BODY) –(FOR-LOOP2 START END BODY) Two strategies for setting values for NUM- ITERATIONS, START and END –Set the value to a random integer (simple loops). –Set the value to a result of any computation (unrestricted loops). Investigation and Expectation –Convergence behavior, size and solutions –Simple loops are easier to evolve than unrestricted loops

5. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP5 Related Works [1] John R. Koza, Forrest H Bennet III, David Andre, and Martin A. Keane. Genetic Programming III; Darwinian invention and problem solving. Morgan Kaufmann, 1999. –ADL – automatically defined loops – pre-established maximum number of execution [2] Kenneth E. Kinnear, Jr. Generality and difficulty in genetic programming: Evolving a sort. In Stephanie Forrest, editor, Proceedings of the 5th International Conference on Genetic Algorithms, ICGA-93,pages 287–294, University of Illinois at Urbana- Champaign, 17-21 1993.Morgan Kaufmann. –A sorting algorithm – restricted total and individual loops times. – Inverse sized fitness measure [3] William B. Langdon. Data structures and genetic programming. In Peter J. Angeline and K. E. Kinnear, Jr., editors, Advances in Genetic Programming 2, pages 395–414. MIT Press, Cambridge, MA, USA, 1996. –A list data structure- for-while – no nested loops and maximum iteration 32 [4] Sidney R. Maxwell III. Experiments with a co-routine model for genetic programming. In Proceedings of the 1998 United Kingdom Automatic Control Council International Conference on Control (UKACC International Conference on Control ’98), University of Wales, volume 455, Swansea, UK, 1-4 1998. IEEE Press. –Santa Fe Ant problem, Partial fitness calculation based on time spend

6. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP6 Syntax and Semantics Strong Type GP –We use STGP. It simultaneously allows multiple data types and enforces closure by only generating parse trees which satisfy the type constraints. (FOR-LOOP1 NUM-ITERATIONS BODY) –Simple loops : NUM-ITERATIONS ::INTERGER (Between 1 and MAX-ITERATIONS) –Unrestricted loops: NUM-ITERATIONS could involve arithmetic functions {+, -} and allows nesting (FOR-LOOP2 START END BODY) –If start > end, body is not executed. –Others, same as above.

7. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP7 Exp.1 – Santa Fe Ant Problem (1) ORIGINAL Direct a robot ant to navigate through a twisting “Santa Fe Trail”, on a 32 x 32 grid. There are 89 pieces of food on the trail. The robot eats the food when it enters into a square. The goal is for the robot to eat all of the food in as few moves as possible. (Maximum number of steps is 600).

8. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP8 Exp.1 – Santa Fe Ant Problem (2) Original functions and terminals Move::Terminal The robot moves one square forward and it costs one step. TurnLeft::Terminal Turn the robot to the left direction of its current facing and cost one step. TurnRight::Terminal Turn the robot to the right direction of its current facing and cost one step. IfFoodAhead::Function Takes 2 arguments and executes the first argument if there is a food in front, else executes the second. Prog2::Function Takes 2 arguments and executes them sequentially. Prog3::Function Takes 3 arguments and executes them sequentially.

9. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP9 Exp.1 – Santa Fe Ant Problem (3) MODIFIED Grid size is changed to 20x20 and 108 pieces of food are placed on the grid in 3 blocks of 6x6. The regular placement of food is intended to encourage the evolution of loops. External loops are not allowed. Termination Criteria 2000 generations or all food (108 pieces) is found or 600 steps are reached.

10. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP10 Exp.1 – Santa Fe Ant Problem (4) Added Nodes RandTimes::Terminal Generates a random integer between 0-6 or 0-20 or 0-50. For-Loop1::Function Takes 2 arguments. The first argument indicates number of times the second argument is executed. It returns number pieces of food left after the execution of the loop body. Genetic variable values Population Size: 100Mutation Rate:0.28 Max. Tree Depth: 8 Crossover Rate:0.70 Min. Tree Depth:1Elitism Rate:0.02

11. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP11 Exp.1 – Santa Fe Ant Problem (5) Mean best program fitness of the modified ant problem using different approaches. Max iteration for loops is 100. Results are the average of 100 runs. Loops performs better than no-loops. Unrestricted is better than simple-loops.

12. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP12 Exp.1 – Santa Fe Ant Problem (6) Cumulative probability of success for the modified ant problem using different approaches. Max iteration for loops is 100. Results are the average of 100 runs. Loops performs better than no-loops. Unrestricted is better than simple-loops. Remember no external loops are allowed

13. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP13 Exp.1 – Santa Fe Ant Problem (7) Different max-iteration settings and favouring fewer loops. Mean best program fitness using different approaches. Results are the average of 100 runs. Favouring programs with fewer loops has a dramatic effect on fitness for simple loops, but has no effect on the unrestricted loops.

14. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP14 Exp.1 – Santa Fe Ant Problem (8) Different max-iteration settings and favouring fewer loops. Mean average program size. Results are the average of 100 runs. Quit a difference in program size if fewer loops are favoured. Notice the curves show an initial dramatic drop in program size.

15. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP15 Exp.1 – Santa Fe Ant Problem (9) All kinds of interesting solutions for runs using loops. A solution with loops with MAX-ITERATIONS = 20. It is a dummy solution, but works.

16. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP16 Exp.1 – Santa Fe Ant Problem (10) A solution evolved by favouring programs with fewer loops. (ForLoop1 times5 (IfFoodAhead ((Prog2 move turnRight) turnRight) (Prog2 turnLeft (IfFoodAhead (Prog2 move move) move))))))

17. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP17 Exp.1 – Santa Fe Ant Problem (11) A solution evolved without favouring programs with fewer loops. (ForLoop1 times5 (ForLoop1 times4 (Prog3 (ForLoop1 times4 move) (Prog3 (IfFoodAhead (Prog3 move move move) move) (Prog3 (Prog2 turnRight move) turnRight (IfFoodAhead (ForLoop1 times4 move) (Prog2 turnRight turnLeft))) (IfFoodAhead (Prog3 move move move) move)) (Prog3 (IfFoodAhead (IfFoodAhead (Prog3 turnLeft turnRight move) move) turnLeft) turnLeft (Prog2 move move)))))

18. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP18 Exp.1 – Santa Fe Ant Problem (12) A solution evolved without using loops. Out of 300 runs, only one solution is found. It contains more than 5000 nodes and takes 4 full A4 pages to print out. Remember no external loops are allowed

19. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP19 Exp. 2 – Sorting Problem (1) Sorting an array of numbers has natural looping characteristics. Sorting has been studied by [1] Kenneth E. Kinnear, Jr. Evolving a sort: Lessons in genetic programming. In Proceedings of the 1993 International Conference on Neural Networks, volume 2, pages 881–888, San Francisco, USA, 28 -1 1993. IEEE Press. [2] Kenneth E. Kinnear, Jr. Generality and difficulty in genetic programming: Evolving a sort. In Stephanie Forrest, editor, Proceedings of the 5th International Conference on Genetic Algorithms, ICGA-93, pages 287–294, University of Illinois at Urbana-Champaign, 17-21 1993. Morgan Kaufmann. [3] John R. Koza. Genetic Programming II: Automatic Discovery of Reusable Programs. MIT Press, 1994. Our focus: –Is on evolution of loops of different complexities and on the comparison of loop and non-loop solutions.

20. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP20 Exp. 2 – Sorting Problem (2) Functions and terminals POS::Terminal Random number in range 0..6 IfLessThanSwap::Function Takes two arguments. If arg1 is less than arg2 the positions are swapped and the position of the larger value is returned Prog2::Function Takes 2 arguments and executes them sequentially. ForLoop2::Function Takes 3 arguments, start position, end position and body. +, -, *, / with usual meanings

21. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP21 Exp. 2 – Sorting Problem (3) Genetic variable values Population Size: 100Mutation Rate:0.28 Max. Tree Depth: 8 Crossover Rate:0.70 Min. Tree Depth:1Elitism Rate:0.02 Algorithm for fitness calculation int calculateFitness(int length, int * array) { int i, result = 0; for( i=1; i <= length; i++) result += abs( array[i-1] - i ); return result; }

22. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP22 Exp. 2 – Sorting Problem (4) Mean best program fitness, averaged by 50 runs. Loops perform better than no-loops. Be careful to notice the scale in graph. Simple- fewer-loops does not get zero fitness in the beginning.

23. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP23 Exp. 2 – Sorting Problem (5) Cumulative probability of success. Loops perform better than no-loops. Unrestricted ones perform best.

24. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP24 Exp. 2 – Sorting Problem (6) Number of comparison made by the best individual, averages of 50 runs. no-loops has the smallest size, while unrestricted loops is in the middle. Reasons:. Quicker getting a solution in loops approach. Loop constructs overheads

25. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP25 Exp. 2 – Sorting Problem (7) Table of comparisons and swaps –7 element arrays, 5040 test cases MethodsComparisonsSwaps Bubble Sort Shell Sort Insertion Sort Selection Sort Quick Sort 21.00 16.50 17.09 21.00 44.42 10.50 16.50 15.50 6.00 10.19 No loops Simple loops Unrestricted loops 17.00 22.00 21.00 7.33 10.00 9.00

26. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP26 Exp. 2 – Sorting Problem (8) An example of solution in simple loops approach (Prog2 (Prog2 (Prog2 (ForLoop2 POS3 POS4 (ILETs i (i+1))) (ForLoop2 POS2 POS6 (ILETs i (i+1)))) (Prog2 (ForLoop2 POS3 POS4 (ILETs i (i+1))) (ForLoop2 POS4 POS5 (ILETs i (i+1))))) (Prog2 (ForLoop2 POS1 POS6 (ILETs i (i+1))) (Prog2 (ForLoop2 POS3 POS2 (ILETs i (i+1))) (ForLoop2 POS1 POS3 (ILETs i (i+1)))) (Prog2 (ForLoop2 POS1 POS3 (ILETs i (i+1))) (ForLoop2 POS0 POS6 (ILETs i (i+1)))))))

27. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP27 Conclusion By restricting the semantic complexity of for-loops, we have succeeded in evolving small, efficient and reasonable understandable solutions. Using fitness function to favour programs with fewer loops was very beneficial in our experiments. Unrestricted loops performs better, as unrestricted ones in a way are more restricted. Looping constructs are worth considering when the problem domain has some repetitive characteristics. Acknowledgement: This work was partially supported by grant EPPNRM054 from the Victorian Partnership for Advanced Computing.

28. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP28 Questions

29. RMIT UNIVERSITY CEC2004Experiments with Explicit For-Loops in GP29 Evolved Program with Loops which Solve the Original Santa Fe Ant Problem Program 1 (Max Depth 6, Favor Fewer Loops) (ForLoop (i+ (i+ times5 (i+ times48 (i+ times48 times14))) (i- times48 times14)) (IfFoodAhead move (Prog3 turnLeft (IfFoodAhead move turnRight) (Prog3 turnRight (IfFoodAhead move turnLeft) move)))) Program 2 (Max Depth 8) (ForLoop (ForLoop times13 (IfFoodAhead move (IfFoodAhead (IfFoodAhead turnRight turnLeft) turnRight))) (IfFoodAhead (Prog2 (Prog3 (Prog3 (IfFoodAhead (IfFoodAhead move turnRight) turnLeft) (Prog3 move move move) (Prog2 turnRight (IfFoodAhead turnLeft turnRight))) (IfFoodAhead move turnRight) (Prog2 (IfFoodAhead move turnRight) move)) (IfFoodAhead turnRight turnRight)) (Prog2 (IfFoodAhead (Prog2 (IfFoodAhead turnLeft turnLeft) (IfFoodAhead (Prog3 turnRight turnLeft turnLeft) turnRight)) (Prog2 (IfFoodAhead move (IfFoodAhead turnRight turnRight)) turnRight)) (Prog2 (Prog2 (IfFoodAhead move (IfFoodAhead turnRight turnRight)) move) (IfFoodAhead (Prog3 turnRight turnLeft (Prog2 turnRight turnRight)) turnRight))))) Program 3 (Max Depth 6) (i+ (ForLoop times50 (Prog3 move (IfFoodAhead (Prog2 move turnLeft) turnLeft) (Prog3 (IfFoodAhead turnLeft turnRight) turnRight (IfFoodAhead move turnLeft)))) (ForLoop times39 (Prog3 move (IfFoodAhead (Prog3 move move turnLeft) turnLeft) (Prog3 (IfFoodAhead move turnRight) turnRight (IfFoodAhead move turnLeft)))))