1. 1 Basic frame of Genetic algorithm Initialise a population Evaluate a population While (termination condition not met) do Select sub-population based on fitness Produce offspring of the population using crossover Mutate offspring stochastically

2. 2 Evolution Runs Until: A perfect individual appears (if you know what the goal is), Or: improvement appears to be stalled, Or: you give up (your computing budget is exhausted).

3. 3 Simple GA Simulation Optimisation problem maximise the function f(x)=x 2 x: integer between 0 and 31 objective function as the fitness function code using binary representation 11011 2 = 19 10 1*2 4 +0*2 3 +0*2 2 +1*2 1 +1*2 0 0 is then 00000 31 is then 11111

4. 4 Simple GA Simulation: Initial Population Lets assume that we want to create a initial population of 4 random flip coin 20 times (4 population size * string of 5) String Initial No. Population 1 01101 2 11000 3 01000 4 10011

5. 5 Simple GA Simulation: Fitness Function String Initial f(x)Prob. = f i /Sum No. Population 1 01101 169 0.14 2 11000 576 0.49 3 01000 64 0.06 4 10011 361 0.31 Sum 1170 Average 293 Max 576

6. 6 Simple GA Simulation: Roulette Wheel String Initial f(x)ProbRoulette. No. PopulationWheel 1 01101 1690.14 1 2 11000 5760.49 2 3 01000 640.06 0 4 10011 3610.31 1 Sum 1170 Average 293 Max 576

7. 7 Simple GA Simulation: Mating Pool String Initial f(x)ProbRoulette. Mating No. PopulationWheel Pool 1 01101 1690.14 1 01101 2 11000 5760.49 2 11000 3 01000 640.06 0 11000 4 10011 3610.31 1 10011 Sum 1170 Average 293 Max 576

8. 8 Simple GA Simulation: Mate String Initial f(x)ProbRoulette. Mating Mate No. PopulationWheel Pool 1 01101 1690.14 1 01101 2 2 11000 5760.49 2 11000 1 3 01000 640.06 0 11000 4 4 10011 3610.31 1 10011 3 Sum 1170 Average 293 Mate is Randomly selected Max 576

9. 9 Simple GA Simulation: Crossover String Initial f(x) Prob R. Mating Mate Crossover No. Population W Pool 1 01101 169 0.14 1 01101 2 4 2 11000 576 0.49 2 11000 1 4 3 01000 64 0.06 0 11000 4 2 4 10011 361 0.31 1 10011 3 2 Sum 1170 Average 293 Crossover is Randomly selected Max 576

10. 10 Simple GA Simulation: New Population String Initial f(x) Prob R. Mating Mate Cross New No. Population W Pool over Pop. 1 01101 169 0.14 1 01101 2 401100 2 11000 576 0.49 2 11000 1 411001 3 01000 64 0.06 0 11000 4 211011 4 10011 361 0.31 1 10011 3 210000 Sum 1170 Average 293 Max 576

11. 11 Simple GA Simulation: Mutation String Initial f(x) Prob R. Mating Mate Cross New No. Population W Pool over Pop. 1 01101 169 0.14 1 01101 2 401100 2 11000 576 0.49 2 11000 1 411001 3 01000 64 0.06 0 11000 4 211011 4 10011 361 0.31 1 10011 3 210000 Sum 1170 Average 293 Max 576

12. 12 Simple GA Simulation: Mutation String Initial f(x) Prob R. Mating Mate Cross New No. Population W Pool over Pop. 1 01101 169 0.14 1 01101 2 401100 2 11000 576 0.49 2 11000 1 410001 3 01000 64 0.06 0 11000 4 211011 4 10011 361 0.31 1 10011 3 210000 Sum 1170 Average 293 Max 576

13. 13 Simple GA Simulation: Evaluation New Population String Initial f(x) ProbR. Mating Mate C New f(x) No. PopulationW Pool o Pop. 1 01101 169 0.141 01101 2 4 01100 144 2 11000 576 0.492 11000 1 4 10001 289 3 01000 64 0.060 11000 4 2 11011 729 4 10011 361 0.311 10011 3 2 10000 256 Sum 1170 1418 Average 293 354 Max 576 729

14. 14 Problem & Representations Chromosomes represent problems' solutions as genotypes They should be amenable to: Creation (spontaneous generation) Evaluation (fitness) via development of phenotypes Modification (mutation) Crossover (recombination)

15. 15 How GAs Represent Problems' Solutions: Genotypes Bit strings -- this is the most common method Strings on small alphabets (e.g., C, G, A, T) Permutations (Queens, Salesmen) Trees (Lisp programs). Genotypes must allow for: creation, modification, and crossover.

16. 16 Binary-Encoding for Genetic Algorithms The original formulation of genetic algorithms relied on a binary encoding of solutions, i.e., the chromosomes consist of a string of 0’s and 1’s. No restriction on the phenotype so long as a good method for encoding/decoding exists. Other methods of encoding will come later.

17. 17 Genetic Binary Encoding/Decoding of Integers Initially, single parameters Integer parameters: p is an integer parameter to be encoded. 3 distinct cases to consider: Case 1 p takes values from {0, 1, 2, …..2^N-1} for some N. In this case, p can be encoded by its equivalent binary representation.

18. 18 Genetic Binary Encoding/Decoding of Integers Case 2 p takes values from {M, M+1, …., M+2^N-1} for some M, N. In this case (p-M) can be encoded directly by its equivalent binary representation. Case 3 p takes values from {0, 1,.., L-1} for some L such that there exist no N for which L=2^N. There are two possibilities. Clipping and Scaling

19. 19 Clipping Take N = log(L)+1 and encode all parameter values 0<= p <= L-2 by their equivalent binary representation, letting all other n-bit strings serve as encodings of p = L-1. Example: p from {0,1,2,3,4,5}, i.e., L=6. Then N=log(6)+1= 3 p01234555 Code000001010011100101110111 Advantages: Easy to implement Disadvantages: Strong representational bias. All parameter values between 0 and L-2 have a single encoding, but the single value L-1 has 2^N-L+1 encodings

20. 20 Scaling Take N=log(L)+1 and encode p by the binary representation of the integer e such that p =  e  (L-1)/(2^N-1)  Example: p from {0,1,2,3,4,5} i.e. L=6, N=log(6)+1=3. p00122345 Code000001010011100101110 111 Advantages: Easy to implement. Smaller representational bias than clipping (at most double representations) Disadvantages: Small representational bias. More computation than clipping.

21. 21 Gray Coding Desired: points close to each other in representation space also close to each other in problem space This is not the case when binary numbers represent floating point values Binary gray code 000 000 001 001 010 011 011 010 100 110 101 111 110 101 111 100

22. 22 Gray Coding m is number of bits in representation Binary number b = (b1; b2;  ; bm) Gray code number g = (g1; g2;  ; gm)

23. 23 Gray Coding PROCEDURE Binary-To-Gray g 1 =b 1 for k=2 to m do g k =b k-1 XOR b k endfor

24. 24 Gray Coding PROCEDURE Gray-To-Binary value = g 1 b1 = value for k = 2 to m do if g k = 1 then value = NOT value end if b k =value end for

25. 25 Binary Encoding and Decoding of Real-valued parameters Can be encoded as: fixed-point numbers integers using scaling and quantisation If p ranges over [min,max] encode p using N bits by using the binary representation of the integer part of (2^N-1)(p-min)/(max - min)

26. 26 Multiple parameters Vectors of parameters are encoded on multi-gene chromosomes by combining the encodings of each individual parameter. Let e_i =[b_i0,….b_iN] be the encoding of the ith of M parameters. There are two ways of combining the e_i’s into a chromosome. Concatenating: Individual encodings simply follow one another in some predefined order e.g. [b_10,…b_1N,..,b_M0,…b_MN] Interleaving: the bits of each individual encoding are interleaved e.g. [b_10,…,b_M0,b_11,…,b_MN] The order of parameters in the vector (i.e. genes in the chromosome) is important, especially for concatenated encodings.

27. 27 Initialization Initialise a population Evaluate a population While (termination condition not met) do Select sub-population based on fitness Produce offspring of the population using crossover Mutate offspring stochastically

28. 28 Initialization init( ) { for( i = 0; i < POP_SIZE; i++ ) for( j = 0; j < N; j++ ) p[i][j] = random_int( 2 ); }

29. 29 GA Main Program Initialise a population Evaluate a population While (termination condition not met) do Select sub-population based on fitness Produce offspring of the population using crossover Mutate offspring stochastically

30. 30 GA Main Program for( trial = 0; trial < LOOPS; trial++ ) { selection( ) crossover( ) mutation( ) for( who = 0; who < POP_SIZE; who++ ) fitness[who] = fv(who); }

31. 31 Selection Initialise a population Evaluate a population While (termination condition not met) do Select sub-population based on fitness Produce offspring of the population using crossover Mutate offspring stochastically

32. 32 Selection Selects individuals for reproduction –randomly with a probability depending on the relative fitness of the individuals so that the best ones are more often chosen for reproduction rather than poor ones –Proportionate-based selection picks out individuals based upon their fitness values relative to the fitness of the other individuals in the population –Ordinal-based selection selects individuals based upon their rank within the population; independent of the fitness distribution

33. 33 Roulette Wheel Selection Here is a common technique: let F =  j=1 to popsize fitness j Select individual k with probability fitness k /F

34. 34 Roulette Wheel Selection assigns to each solution a sector of a roulette wheel whose size is proportional to the appropriate fitness measure chooses a random position on the wheel (spin the wheel) Fitness a:1 b:3 c:5 d:3 e:2 f:2 g:8 c d e f g a b

35. 35 Roulette Wheel Realization For each chromosome evaluate the fitness and the cumulative fitness For N times create a random number Select the chromosome where its cumulative fitness is the first value greater than the generated random number Individual Chromosome Fitness Cumulative x1 101100 20 20 x2 111000 7 27 x3 001110 6 33 x4 101010 10 43 x5 100011 12 55 x6 011011 9 64

36. 36 Roulette Wheel Example Individual Chromosome Fitness Cumulative Random Individual x1 101100 20 20 42.8 x2 111000 7 27 x3 001110 6 33 x4 101010 10 43 x5 100011 12 55 x6 011011 9 64

37. 37 Roulette Wheel Example Individual Chromosome Fitness Cumulative Random Individual x1 101100 20 20 42.8 x4 x2 111000 7 27 x3 001110 6 33 x4 101010 10 43 x5 100011 12 55 x6 011011 9 64

38. 38 Roulette Wheel Example Individual Chromosome Fitness Cumulative Random Individual x1 101100 20 20 42.8 x4 x2 111000 7 27 19.78 x3 001110 6 33 x4 101010 10 43 x5 100011 12 55 x6 011011 9 64

39. 39 Roulette Wheel Example Individual Chromosome Fitness Cumulative Random Individual x1 101100 20 20 42.8 x4 x2 111000 7 27 19.78 x1 x3 001110 6 33 x4 101010 10 43 x5 100011 12 55 x6 011011 9 64

40. 40 Roulette Wheel Example Individual Chromosome Fitness Cumulative Random Individual x1 101100 20 20 42.8 x4 x2 111000 7 27 19.78 x1 x3 001110 6 33 42.73 ? x4 101010 10 43 58.44 ? x5 100011 12 55 27.31 ? x6 011011 9 64 28.31 ?

41. 41 Roulette Wheel Example Individual Chromosome Fitness Cumulative Random Individual x1 101100 20 20 42.8 x4 x2 111000 7 27 19.78 x1 x3 001110 6 33 42.73 x4 x4 101010 10 43 58.44 x6 x5 100011 12 55 27.31 x3 x6 011011 9 64 28.31 x3

42. 42 Roulette Wheel Selection There are some problems here: fitnesses shouldn't be negative only useful for max problem probabilities should be “right” avoid skewing by super heros.

43. 43 Parent Selection: Rank Here is another technique. Order the individuals by fitness rank Worst individual has rank 1. Best individual has rank POP  SIZE Let F = 1 + 2 + 3 +  + POP_SIZE Select individual k to be a parent with probability rank k /F Benefits of rank selection: the probabilities are all positive can be used for max and min problems the probability distribution is “even”

44. 44 Parent Selection: Rank Power Yet another technique. Order the individuals by fitness rank Worst individual has rank 1. Best individual has rank POP_SIZE Let F = 1 s + 2 s + 3 s +  + POP_SIZE s Select individual k to be a parent with probability rank k s /F benefits: the probabilities are all positive can be used for max and min problems the probabilities can be skewed to use more “elitist” selection

45. 45 Tournament Selection Pick k members of the population at random select one of them in some manner that depends on fitness

46. 46 Tournament Selection void tournament(int *winner) { int size = tournament_size, i, winfit; for( i = 0; i < size; i++ ) { int j = random_int( POP_SIZE );; if( j==0 || fitness[j] > winfit ) winfit = fitness[j],*winner = j; }

47. 47 Truncation selection Choose the best k individuals as the offspring There is no randomness If the population size is N, some researchers suggest k/N=0.3

48. 48 Crossover Methods Initialise a population Evaluate a population While (termination condition not met) do Select sub-population based on fitness Produce offspring of the population using crossover Mutate offspring stochastically

49. 49 Crossover Methods Crossover is a primary tool of a GA. (The other main tool is selection.) CROSS_RATE: determine if the chromosome attend the crossover Common techniques for bit string representations: One-point crossover: Parents exchange a random prefix Two-point crossover: Parents exchange a random substring Uniform crossover: Each child bit comes arbitrarily from either parent (We need more clever methods for permutations & trees.)

50. 50 1-point Crossover Suppose we have 2 strings a and b, each consisting of 6 variables a1, a2, a3, a4, a5, a6 b1, b2, b3, b4, b5, b6 representing two solutions to a problem a crossover point is chosen at random and a new solution is produced by combining the pieces of the original solutions if crossover point was 2 a1, a2, b3, b4, b5, b6 b1, b2, a3, a4, a5, a6

51. 51 1-point Crossover ParentsChildren

52. 52 2-point Crossover With one-point crossover the head and the tail of one chromosome cannot be passed together to the offspring If both the head and the tail of a chromosome conatin good generic information, none of the offsprings obtained directly with one-point crossover will share the two good features A 2-point crossover avoids such a drawback ParentsChildren

53. 53 Uniform Crossover Each gene in the offspring is created by copying the corresponding gene from one or the other parent chosen according to a random generated binary crossover mask of the same length as the chromosomes where there is a 1 in the crossover mask the gene is copied from the first parent and where there is a 0 in the mask the gene is copied from the second parent a new crossover mask is randomly generated for each pair of parents

54. 54 Uniform Crossover Parents Child 10010110 Crossover Mask

55. 55 Uniform Crossover make_children(int p1, p2, c1, c2) { int i, j; for( i = 0; i < N; i++ ) { if( random_int(2) ) p[c1][i] = p[p1][i],p[c2][i] = p[p2][i]; else p[c1][i] = p[p2][i],p[c2][i] = p[p1][i]; }

56. 56 Another Clever Crossover Select three individuals, A, B, and C. Suppose A has the highest fitness and C the lowest. Create a child like this. for(i = 0; i < length; i++ ) { if( A[i] == B[i] ) child[i] = A[i]; else child[i] = 1 - C[i]; } We just suppose C is a “bad example.”

57. 57 Crossover Methods & Schemas Crossovers try to combine good schemas in the good parents. The schemas are the good genes, building blocks to gather. The simplest schemas are substrings. 1-point & 2-point crossovers preserve short substring schemas. Uniform crossover is uniformly hostile to all kinds of schemas.

58. 58 Limit Consistency of Crossover Operator

59. 59 Crossover for Permutations (A Tricky Issue) Small-alphabet techniques fail. Some common methods are: OX: ordered crossover PMX: partially matched crossover CX: cycle crossover We will address these and others later.

60. 60 Crossover for Trees These trees often represent computer programs. Think Lisp Interchange randomly chosen subtrees of parents.

61. 61 Mutation: Preserve Genetic Diversity Initialise a population Evaluate a population While (termination condition not met) do Select sub-population based on fitness Produce offspring of the population using crossover Mutate offspring stochastically

62. 62 Mutation: Preserve Genetic Diversity Mutation is a minor GA tool. Provides the opportunity to reach parts of the search space which perhaps cannot be reached by crossover alone. Without mutation we may get premature convergence to a population of identical clones –mutation helps for the exploration of the whole search space by maintaining genetic diversity in the population –each gene of a string is examined in turn and with a small probability its current value is changed –011001 could become 010011 if the 3 rd and 5 th genes are mutated

63. 63 Mutate Strings & Permutations Bit strings (or small alphabets) Flip some bits Reverse a substring (nature does this) Permutations Transpose some pairs Reverse a substring Trees...

64. 64 Mutation Mutate each bit with probability MUT_ RATE mutate(int who) { int i,j; for(i=0;i<Population_size;i++) for( j = 0; j < N; j++ ) { if( MUT_RATE > random_float( ) ) p[who][j] = 1-p[who][j]; }

65. 65 Mutation Mutation rate determines the probability that a mutation will occur. Mutation is employed to give new information to the population and also prevents the population from becoming saturated with similar chromosomes Large mutation rates increase the probability that good schemata will be destroyed, but increase population diversity. The best mutation rate is application dependent but for most applications is between 0.001 and 0.1.

66. 66 Mutation Some researchers have published "rules of thumb" for choosing the best mutation rate based on the length of the chromosome and the population size. DeJong suggested that the mutation rate be inversely proportional to the population size. (1/L) Hessner and Manner suggest that the optimal mutation rate is approximately (M * L 1/2 ) -1 where M is the population size and L is the length of the chromosome.

67. 67 Crossover vs Mutation Crossover modifications depend on the whole population decreasing effects with convergence exploitation operator GA emphasize crossover Mutation mandatory to escape local optima exploration operator ES and EP emphasize mutation

68. 68 Replacement A method to determine which of the current members of the population, if any, should be replaced by the new solutions. Generational updates Steady state updates

69. 69 Generational Updates Replacement produce N children from a population of size N to form the population at the next time step and this new population of children completely replaces the parent selection a derived generational update scheme can also be used –( +  )-update and (,  )-update  is the parent population is the number of children produced of size  –the  best individuals from either the offspring population or the combined parent and offspring populations form the next generation

70. 70 Steady state replacement New individuals are inserted in the population as soon as they are created by replacing an existing member of the population –the worst or the oldest member –tournament replacement as tournament selection but this time the less good solutions are picked more often than the good ones –the most similar member –elitism never replace the best individuals in the population with inferior solution, so best solution is always available for reproduction –harder to escape from a local optimum

71. 71 Basic frame of Genetic algorithm Initialise a population Evaluate a population While (termination condition not met) do Select sub-population based on fitness Produce offspring of the population using crossover Mutate offspring stochastically

72. 72 Another Function to Optimize Find the max. of the “peaks” function z = f(x, y) = 3*(1-x)^2*exp(-(x^2) - (y+1)^2) - 10*(x/5 - x^3 - y^5)*exp(-x^2-y^2) -1/3*exp(-(x+1)^2 - y^2).

73. 73 Derivatives of the “peaks” function dz/dx = -6*(1-x)*exp(-x^2-(y+1)^2) - 6*(1-x)^2*x*exp(-x^2-(y+1)^2) - 10*(1/5-3*x^2)*exp(-x^2-y^2) + 20*(1/5*x-x^3-y^5)*x*exp(- x^2-y^2) - 1/3*(-2*x-2)*exp(-(x+1)^2-y^2) dz/dy = 3*(1-x)^2*(-2*y-2)*exp(-x^2-(y+1)^2) + 50*y^4*exp(-x^2- y^2) + 20*(1/5*x-x^3-y^5)*y*exp(-x^2-y^2) + 2/3*y*exp(- (x+1)^2-y^2) d(dz/dx)/dx = 36*x*exp(-x^2-(y+1)^2) - 18*x^2*exp(-x^2-(y+1)^2) - 24*x^3*exp(-x^2-(y+1)^2) + 12*x^4*exp(-x^2-(y+1)^2) + 72*x*exp(-x^2-y^2) - 148*x^3*exp(-x^2-y^2) - 20*y^5*exp(-x^2- y^2) + 40*x^5*exp(-x^2-y^2) + 40*x^2*exp(-x^2-y^2)*y^5 - 2/3*exp(-(x+1)^2-y^2) - 4/3*exp(-(x+1)^2-y^2)*x^2 -8/3*exp(- (x+1)^2-y^2)*x d(dz/dy)/dy = -6*(1-x)^2*exp(-x^2-(y+1)^2) + 3*(1-x)^2*(-2*y- 2)^2*exp(-x^2-(y+1)^2) + 200*y^3*exp(-x^2-y^2)-200*y^5*exp(- x^2-y^2) + 20*(1/5*x-x^3-y^5)*exp(-x^2-y^2) - 40*(1/5*x-x^3- y^5)*y^2*exp(-x^2-y^2) + 2/3*exp(-(x+1)^2-y^2)-4/3*y^2*exp(- (x+1)^2-y^2)

74. 74 GA process 10th generation5th generation Initial population

75. 75 Performance profile

76. 76 Setting the Parameters How do we choose the “params”? There is a statistical discipline: experimental design For the present, we will treat this issue informally. Try different setting and get a problem to work. Then systematically try various settings.

77. 77 Probability of applying operations Probability of Crossover (Pc) Pc is normally chosen from range {0.5,…0.8} This means that Crossover is applied to individuals with a probability of Pc and cloning with a probability of 1- Pc Probability of Mutation (Pm) This is the probability of flipping any individual bit In GA’s Pm is normally kept very low, generally in the range{0.001,0.1}

78. 78 Pop Sizes 5, 10, 100, 300, 1000 For Problem 1 5: too much exploitation. 1000: too much exploration.

79. 79 Mutation Rates 0.001, 0.005, 0.007, 0.01 For Problem 1 0.01: too much exploration

80. 80 Improving Performance Although GAs are conceptually simple, to the newcomer the number of configuration choices can be overwhelming. Once the scientist using a GA for the first time receives less than satisfactory results there are several steps which can be taken to improve search performance.

81. 81 Improving Performance The first approach to improving search performance is to simply use different values for mutation rates, population size, etc.. Many times, search performance can be improved by making the optimization more stochastic or becoming more hill- climbing in nature. This trial and error approach, although time-consuming, will usually result in improved search performance. If changing the configuration parameters has no effect on search performance a more fundamental problem may be the cause.

82. 82 Mapping Quality Functions to Fitness Functions So far, we have assumed that there exists a known quality measure Q >= 0 for the solutions of the problem and that finding a solution can be achieved by maximising Q. Under this assumption, a chromosome's fitness is taken to be the quality measure of the individual it encodes. When this assumption is not valid, adjustments must be made for fitness-proportionate selection to be used. Of course, one may use a different selection mechanism. ` We will look at some of the problems that may arise with quality measures and suggest ways how these can be mapped into fitness functions that allow FPS to be applied.

83. 83 Negative-valued Quality Measure In some problems, Q may take on negative values for some of the solutions, hence cannot be used directly as a fitness function with FPS (it would produce negative probabilities!). One solution is to use an offset and a threshold We can do this by defining fitness as follows: f(s) = Q(s) - C_min if Q(s) - C_min > 0, and 0 otherwise C_min (the offset) is taken to be one of the following: 1) The minimum value Q may take (when known) 2) The minimum value of Q in the current and/or last k generations 3) A function of the variance of Q in the current population, e.g. Mean(Q)- 2Sqrt(Variance(Q))

84. 84 Cost-based or Error-based Quality Measure In some problems, the natural measure of quality is actually a cost or an error E, and finding a solution consists of minimising E (rather than maximising Q). In this case, there is a straightforward solution, which consists of taking -E as the raw fitness, and then using an offset and a threshold (as before) to avoid negative values, if FPS is to be applied. f(s) = C_max - E(s) if C_max > E(s) and 0 otherwise C_max (the offset) is taken to be one of the following: 1) The maximum value E may take (when known) 2) The maximum value of E in the current and/or last k generations 3) A function of the variance of Q in the current population e.g. Mean(E)+2 Sqrt(Variance(E))

85. 85 Stagnation-prone Quality Measure If Q ranges over [Q_min, Q_max] where Q_min >> 0 and Q_max - Q_min !>> 0 then FPS can lead to stagnation even at the beginning of a run. In this case, one solution is to use an offset and threshold as follows: f(s) = Q(s) - C_min if Q(s) - C_min > 0 and 0 otherwise, where C_min = Q_min, so that f ranges over [0, Q_max - Q_min]