Metaheuristics Byung-Hyun Ha Population Methods Metaheuristics Byung-Hyun Ha R1

Outline Introduction Evolution Strategies Genetic Algorithm Exploitative Variations Differential Evolution Particle Swarm Optimization Summary

Introduction Population-based methods Keeping around a sample of candidate solutions rather than a single candidate solution Candidate solutions affect how other candidates will hill-climb Poor solutions to be rejected and new ones created, by good solutions Solutions to be Tweaked in the direction of the better solutions c.f., different from a parallel hill-climber Stealing concepts from biology Evolutionary Computation, broader concept Evolutionary Algorithm (EA) Generational algorithms e.g., Genetic Algorithm (GA) Steady-state algorithms e.g., Evolution Strategies (ES)

U' = {1, 4} (include item 1 and 4) Introduction Common terms used in Evolutionary Computation Individual and population A candidate solution and set of candidate solutions Genotype or genome An individual’s data structure, as used during breeding Chromosome -- a genotype in the form of a fixed-length vector Gene -- particular slot position in a chromosome Phenotype How the individual operates during fitness assessment 1 genotype U' = {1, 4} (include item 1 and 4) phenotype knapsack problem artificial ant (Ch. 4)

Introduction Common terms used in Evolutionary Computation (cont’d) Child and parent A child is the Tweaked copy of a candidate solution (its parent) Mutation Plain Tweaking (asexual breeding) Crossover or recombination A Tweak which takes two parents (sexual breeding) Fitness and fitness landscape Quality and quality function Selection Picking individuals based on their fitness Breeding Producing children from a population of parents by selection and Tweaking Generation One cycle of fitness assessment, breeding, and population reassembly; or The population produced each such cycle

Introduction Common terms used in Evolutionary Computation (cont’d) ith population 1 parents selection 1 1 1 mutation crossover 1 1 1 1 children 1 1 1 1 ith generation (i + 1)th generation (i + 1)th population

Introduction Abstract form of EA Population initialization Biased to good ones vs. uniform randomness?

Evolution Strategies Truncation selection and mutation only (, ) and ( + ) algorithm

Evolution Strategies (, ) algorithm Effect of parameters (/ children) for each of  selected parents  ( in population) e.g., (5, 20) Evolution Strategy Effect of parameters Size of , size of , degree to which Mutation is performed

Evolution Strategies ( + ) algorithm ( selected parents) + (/ children of each selection)  next population More exploitative than (, ), because parents compete with children

Evolution Strategies Mutation rate, 2 Mutation of vector of real numbers Gaussian Convolution with 2 Controlling exploration and exploitation Specifying mutation rate Static Adaptive Good starting point -- one-fifth rule if more than 1/5 children are fitter than parents: increase 2 else: decrease 2 c.f., degeneracy of Evolution Strategy (1 + 1) -- plain Hill-Climbing (1 + ) -- Steepest Ascent Hill-Climbing (1, ) -- Steepest Ascent Hill-Climbing with Replacement

Genetic Algorithm John Holland, 1970s Originally for fixed-length vectors of boolean values, e.g., 001101001

Genetic Algorithm Mutation (Tweaking) Examples For binary-valued vectors Bit-flip mutation For permutations (Adjaceint) pairwise interchange, insertion, inversion For tour of TSP 2-opt or 3-opt For tree For real-valued vectors Requirements (Talbi, 2009) Ergodicity Validity Locality 2-opt

Genetic Algorithm Crossover Examples One-point Weakness -- Pr(v1 and vl to be broken) > Pr(v1 and v2 to be broken) If v1 and vl simultaneously work for high fitness? Two-point -- treating v1 and vl fairly Uniform -- treating vi and vj fairly

Genetic Algorithm Crossover (cont’d) Convergence by successive crossover of individuals Possibly premature Then, why we use it? Premise of building blocks for high fitness e.g., 1011 of 10110101 c.f., schema theory i.e., linkage between genes for high fitness One- and two-point crossover assuming highly-linked genes are located near to one another Moral Check your problem. ;-)

Genetic Algorithm More crossover For real-valued vectors Line recombination Intermediate recombination

Genetic Algorithm More crossover (cont’d) (Talbi, 2009) (convergence?) For permutations Order crossover (OX) Partially mapped crossover (PMX) For tour of TSP Cycle crossover (CX) 1 2 3 4 5 6 7 8 9 1 9 3 4 5 6 2 7 8 8 4 1 5 9 3 6 2 7 8 4 3 5 6 2 7 1 2 3 4 5 6 7 8 9 8 5 3 4 6 2 7 8 4 1 5 9 3 6 2 7 8 9 3 4 5 6 1 2 7 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 3 7 4 2 6 5 8 9 9 3 7 8 2 6 5 1 4 9 3 7 8 2 6 5 1 4 9 2 3 8 5 6 7 1 4

Genetic Algorithm Selection c.f., Evolution Strategy  Truncation Selection Roulette Selection (Fitness-Proportionate Selection) Scheme Let s = i fi , where is the fitness of ith individual. Select ith individual with probability (fi / s) Selection pressure problem How about minimization problems? Just to use fi' = 1/ fi ? Or, if difference of fi and fj is too large?

Genetic Algorithm Selection (cont’d) Stochastic Universal Sampling

Genetic Algorithm Selection (cont’d) Tournament Selection Primary selection technique used for GA Recall selection pressure problem of previous methods. How to scale fitness appropriately? A solution -- non-parametric selection algorithm Tournament size t 1 (random search), very large (Truncation Selection with  = 1) 2 (most popular), ...

Exploitative Variations Trend in new algorithms -- more exploitative! Elitism Similar to ( + ) Injecting directly fittest individuals (elites) of previous population into the next Preventing possible premature convergence Increasing mutation and crossover noise Weakening selection pressure Reducing number of elites Steady-State Genetic Algorithm Updating population, incrementally Introducing a few children (possibly one or two) Give death to the same number of selected individuals Preventing premature convergence Selecting individuals randomly for death + Approaches in Elitism c.f., Generation Gap Algorithms -- using large value of new children

Exploitative Variations Tree-Style Genetic Programming Pipeline Hybrid Evolutionary and Hill-Climbing Algorithms Revising and replacing each individual with some hill-climbing Adjusting degree of exploitation -- t A.k.a. memetic algorithms

Exploitative Variations Scatter Search with Path Relinking Combining exploitative mechanisms (hybrid methods, steady-state evolution) explicit attempt to force diversity (exploration) Schema Initial population Seeds provided + random individuals that are very different from one another and from the seeds Applying hill-climbing to each seed Loop Selecting some very fit individuals and some very diverse individuals (much less than population) Generating new seeds by crossover Path Relinking, here Add new ones to population after applying hill-climb Required -- measure of diversity Then, how to generate diverse individuals?

Exploitative Variations Scatter Search with Path Relinking (cont’d) Path Relinking Original path (solid) and one possible relinked path (dotted)

Differential Evolution Mutation by vector addition and subtraction Working in metric vector spaces (booleans, metric integer spaces, reals) Generating children For each parent i, Selecting randomly individuals, a, b, c Generating child by crossover of i and a + (b – c) Survival selection Child is thrown away if it is worse than parent Adaptive but not much global Determining the size of mutation largely based on current variance in the population Children are entirely based on the existing parents

Particle Swarm Optimization Idea based on swarming and flocking behaviors in animals maintaining a single static population Tweaking in response to new discoveries about the space Particle representation Location x, velocity v = x(t) – x(t–1) where x(t) is location at time t c.f., working in multidimensional metric (usually real-valued) spaces Update of location x, x+, x! -- best of x, best of any of informant of x, best of anyone, so far v  v + b(x* – x) + c(x+ – x) + d(x! – x) x  x + v c.f., candidate is mutated towards the best discovered solutions so far. Effect of parameters

Summary Population methods Mutation and crossover Selection Resampling techniques (EA) and directed mutation (PSO) Generational algorithms and steady-state algorithms Mutation and crossover Chromosomes in real- and boolean-valued vectors, permutations, tours Selection Roulette-wheel with fitness scaling and tournament Adaptive mutation algorithm Particle Swarm Optimization