1. Geometric Interpretation of Crossover Alberto Moraglio amoragn@essex.ac.uk BCTCS 2005

2. Contents I – Quick Preliminaries II – Geometric Interpretation of Crossover Extremely quick overview of its implications: III – Unification of Major Representations IV – Crossover Principled Design V – Is Biological Recombination Geometric? VI – Unity of Evolutionary Search

3. I. Quick Preliminaries

4. Evolutionary Algorithms… Are function optimizers Mimic biological evolution Are robust, hence preferred for real world problems Have little theory to explain how and why they work There are various flavours

5. Evolutionary Algorithm Template Problem & representation independent

6. Standard representations & EAs flavours/dialects Binary strings (genetic algorithms, the classic) Real code vectors (evolution strategies, continuous optimization) Permutations (order-based GAs, combinatorial optimization) Parse trees (genetic programming, evolution of computer programs) Algorithmically irrelevant differences: name/authorship/solution interpretation/domain of application Algorithmically relevant differences: solution representation/genetic operators

7. What is crossover? Crossover Is there any common aspect ? Is it possible to give a representation- independent definition of crossover and mutation? 100000011101000 100111100011100 100110011101000 100001100011100

8. Mutation & Crossover for binary strings Mutation = bit flip at random position 101001  101101 Crossover = selection crossover point at random swap tails 1010|01  101000 1110|00  111001 1*10|0* 1*100* All offspring match the parent schema

9. II. Geometric Interpretation of Crossover

10. Genetic operators & Neighbourhood structure Forget the representation and consider the neighbourhood structure (= search space structure) Mutation: offspring are “close to” their parent  in the direct neighbourhood

11. Direct Neighbour Mutation 000001 010011 100101 111110 Representation: Binary String Move: Bit Flip Neighbourhood: Hamming Representation + Move = Neighbourhood ? Mutation: Offspring in the direct neighbourhood What is crossover?

12. Neighbourhood and Crossover Crossover idea: combining parents genotypes to get children genotypes “somewhere in between” them Topologically speaking, “somewhere in between” = somewhere on a shortest path Why on a shortest path?

13. Shortest Path Crossover 011001 010001011101011011 010101011111 010011 010111 D0 : P1 D2 : P2 D1 Parent1: 011101 Parent2: 010111 Children: 01*1*1 Children are on shortest paths More than one shortest path in general

14. Interpretation & Generalization Traditional mutation & crossover have a natural interpretation in the neighbourhood structure in terms of closeness and betweenness Given any representation plus a notion of neighbourhood (move), mutation & crossover operators are well-defined

15. From graphs to geometry Forget the neighbourhood structure and consider the metric space (= space with a notion of distance) The distance in the neighbourhood is the length of the shortest path connecting two solutions Mutation  Direct neighbourhood  Ball Crossover  All shortest paths  Line Segment

16. Balls & Segments In a metric space (S, d) the closed ball is the set of the form where x belongs to S and r is a positive real number called the radius of the ball. In a metric space (S, d) the line segment or closed interval is the set of the form where x and y belong to S and are called extremes of the segment and identify the segment.

17. Squared balls & Chunky segments 3 3 000001 010 011 100 101 111 110 B(000; 1) Hamming space 3 B((3, 3); 1) Euclidean space 3 B((3, 3); 1) Manhattan space Balls 1 2 1 2 000001 010011 100101 111 110 [000; 011] = [001; 010] 2 geodesics Hamming space 1 3 [(1, 1); (3, 2)] 1 geodesic Euclidean space 1 3 [(1, 1); (3, 2)] = [(1, 2); (3, 1)] infinitely many geodesics Manhattan space Line segments

18. Uniform Mutation & Uniform Crossover Uniform topological crossover: Uniform topological ε-mutation: Genetic operators have a geometric nature

19. Representation independent and rigorous definition of crossover and mutation in the neighbourhood seen as a geometric space…

20. This is cheating! I have generalized from a single example of solution representation!

21. III. Unification of Major Representations & Operators

22. Minkowski spaces – real vectors 2 2 2 B((2, 2); 1) Euclidean space 2 B((2, 2); 1) Manhattan space Balls 2 2 B((2, 2); 1) Chessboard space 1 2 1 2 1 3 [(1, 1); (3, 2)] 1 geodesic Euclidean space 1 3 [(1, 1); (3, 2)] = [(1, 2); (3, 1)] infinitely many geodesics Manhattan space Line segments 1 2 1 3 [(1, 1); (3, 2)] infinitely many geodesics Chessboard space Representation: real vectors Neighbourhoods: continuous (3 types) Distances: Minkowski distances Implementation: algebraic manipulation of real vector (equation of line passing through two points) Pre-existing recombination operators: - both blend crossovers and discrete crossovers fit geometric definition - extended blend crossovers do not fit

23. Hamming spaces – binary strings 00 01 02 10 11 12 20 21 22 00 01 02 10 11 12 20 21 22 B(00;1) Hamming space H(2,3) [00;11]=[01;10] 2 geodesics Hamming space H(2,3) 000001 010011 100101 111110 B(000; 1) Hamming space H(3,2) 000001 010011 100101 111 110 [000; 011] = [001; 010] 2 geodesics Hamming space H(3,2) Representation: binary/multary strings Neighbourhoods: bit-flip/site substitution Distances: Hamming distances Implementation: symbolic manipulation of multary strings (mask-based crossovers) Pre-existing recombination operators: - all binary crossovers fit the geometric definition

24. Cayley spaces - permutations Representation: permutations Neighbourhoods: adj. swap, swap, reversal, insertion Distances: corresponding distances Implementation: “minimal permutation sorting by X move” algorithms: - adj. swap = bubble sort - swap = selection sort - insertion = insertion sort - reversal = approximated MPS by reversals (NP-Hard)) Pre-existing recombination operators: various pre-existing crossover operators are sorting algorithm in disguise (because sorting permutations is easier than sorting vectors of other items) abc bac acb bca cab cba B(abc; 1) Adjacent swap space abc bac acb bca cab cba [abc; bca] 1 geodesic Adjacent swap space B(abc; 1) Swap space & Reversal space abc bac acb bca cab cba abc bac acb bca cab cba [abc; bca] 3 geodesics Swap space & Reversal space B(abc; 1) Insertion space [abc; bca] 1 geodesic Insertion space abc bac acb bca cab cba abc bac acb bca cab cba

25. Syntactic tree spaces Representation: syntactic tree (lisp expression) Neighbourhood: weighted sub-tree neighbourhood Distance: structural distance Implementation: - sub-tree swap crossover - common region mask based crossover Pre-existing recombination operators: - traditional crossover (non-geometric) - homologous crossover - the geometric framework can help to clarify what is the landscape and distance related to homologous crossover and a distance connected with a geometric crossover which traditional crossover is an approximation + sin + xxx * * * yx * yy Parent 1Parent 2 y + sin x * * yy x Alignment Crossover Point Swap * * yy + xx Offspring 1 Offspring 2

26. Significance of Unification Most of the pre-existing crossover operators for major representations fit geometric definition Established pre-existing operators have emerged from experimental work done by generations of practitioners over decades Geometric crossover compresses in a simple formula an empirical phenomenon

27. IV. Crossover Principled Design

28. Crossover Principled Design Domain specific solution representation is effective Problem: for non-standard representations it is not clear how crossover should look like But: given a combinatorial problem you may know already a good neighbourhood structure Geometric Interpretation of Crossover  Give me your neighbourhood definition and I give you a crossover definition

29. += ? Crossover Design Example

30. Non-labelled graph neighbourhood MOVE: Insert/remove an edge Fixed number of nodes 0 1 2 1 2 3

31. + Offspring

32. V. Is Biological Recombination Geometric?

33. Levenshtein spaces – sequences Representation: multary sequences (DNA/amino acids) Neighbourhood: insertion + deletion + substitution (compound edit move) Distance: Levenshtein distance Implementation: inexact sequence alignment (dynamic programming) and sites exchange (crossover mask) Pre-existing recombination operators: - none - it could be a good crossover for linear GP - it could be a better model of biological crossover to study molecular evolution because it keeps into account the inexact alignment due to molecular annealing of DNA strands that produces evolution of size variation Parent1=AGCACACA Parent2=ACACACTA best inexact alignment (with gaps): AGCA|CAC-A  Child1=AGCACACTA A-CA|CACTA  Child2=ACACACA

34. A simple model of (homologous) biological recombination fits the geometric definition under a DNA distance used in bioinformatics

35. VI. Unity of Evolutionary Search

36. Example of evolutionary search

37. Abstract convex evolutionary search Main result: an evolutionary algorithm using geometric crossover with any probability distribution, any kind of representation, any problem, any selection and replacement mechanism, does the same search: convex search Proof based on abstract convexity (axiomatic geodesic convexity) and axiomatization of search process (abstract search process)

38. …Nearly Over!

39. Future work THEORY: Generalizing and accommodating pre-existent theories into geometric framework (schema theorem, fitness landscapes, representation theories…) PRACTICE: Testing crossover principled design on important problems with non- standard representation (problem domain representation)

40. Questions?