0. Metaheuristics Byung-Hyun Ha
Population Methods
Metaheuristics
Byung-Hyun Ha R1

1. Outline Introduction Evolution Strategies Genetic Algorithm
Exploitative Variations
Differential Evolution
Particle Swarm Optimization
Summary

2. Introduction Population-based methods
Keeping around a sample of candidate solutions rather than a single candidate solution
Candidate solutions affect how other candidates will hill-climb
Poor solutions to be rejected and new ones created, by good solutions
Solutions to be Tweaked in the direction of the better solutions
c.f., different from a parallel hill-climber
Stealing concepts from biology
Evolutionary Computation, broader concept
Evolutionary Algorithm (EA)
Generational algorithms
e.g., Genetic Algorithm (GA)
Steady-state algorithms
e.g., Evolution Strategies (ES)

3. U' = {1, 4} (include item 1 and 4)
Introduction
Common terms used in Evolutionary Computation
Individual and population
A candidate solution and set of candidate solutions
Genotype or genome
An individual’s data structure, as used during breeding
Chromosome -- a genotype in the form of a fixed-length vector
Gene -- particular slot position in a chromosome
Phenotype
How the individual operates during fitness assessment 1
genotype
U' = {1, 4} (include item 1 and 4)
phenotype
knapsack problem
artificial ant (Ch. 4)

4. Introduction Common terms used in Evolutionary Computation (cont’d)
Child and parent
A child is the Tweaked copy of a candidate solution (its parent)
Mutation
Plain Tweaking (asexual breeding)
Crossover or recombination
A Tweak which takes two parents (sexual breeding)
Fitness and fitness landscape
Quality and quality function
Selection
Picking individuals based on their fitness
Breeding
Producing children from a population of parents by selection and Tweaking
Generation
One cycle of fitness assessment, breeding, and population reassembly; or
The population produced each such cycle

5. Introduction Common terms used in Evolutionary Computation (cont’d)
ith population 1
parents
selection 1 1 1
mutation
crossover 1 1 1 1
children 1 1 1 1
ith generation
(i + 1)th generation
(i + 1)th population

6. Introduction Abstract form of EA Population initialization
Biased to good ones vs. uniform randomness?

7. Evolution Strategies Truncation selection and mutation only
(, ) and ( + ) algorithm

8. Evolution Strategies (, ) algorithm Effect of parameters
(/ children) for each of  selected parents  ( in population)
e.g., (5, 20) Evolution Strategy
Effect of parameters
Size of , size of , degree to which Mutation is performed

9. Evolution Strategies ( + ) algorithm
( selected parents) + (/ children of each selection)  next population
More exploitative than (, ), because parents compete with children

10. Evolution Strategies Mutation rate, 2
Mutation of vector of real numbers
Gaussian Convolution with 2
Controlling exploration and exploitation
Specifying mutation rate
Static
Adaptive
Good starting point -- one-fifth rule
if more than 1/5 children are fitter than parents: increase 2
else: decrease 2
c.f., degeneracy of Evolution Strategy
(1 + 1) -- plain Hill-Climbing
(1 + ) -- Steepest Ascent Hill-Climbing
(1, ) -- Steepest Ascent Hill-Climbing with Replacement

11. Genetic Algorithm John Holland, 1970s
Originally for fixed-length vectors of boolean values, e.g.,

12. Genetic Algorithm Mutation (Tweaking) Examples
For binary-valued vectors
Bit-flip mutation
For permutations
(Adjaceint) pairwise interchange, insertion, inversion
For tour of TSP
2-opt or 3-opt
For tree
For real-valued vectors
Requirements (Talbi, 2009)
Ergodicity
Validity
Locality
2-opt

13. Genetic Algorithm Crossover Examples One-point
Weakness -- Pr(v1 and vl to be broken) > Pr(v1 and v2 to be broken)
If v1 and vl simultaneously work for high fitness?
Two-point -- treating v1 and vl fairly
Uniform -- treating vi and vj fairly

14. Genetic Algorithm Crossover (cont’d)
Convergence by successive crossover of individuals
Possibly premature
Then, why we use it?
Premise of building blocks for high fitness
e.g., 1011 of
c.f., schema theory
i.e., linkage between genes for high fitness
One- and two-point crossover
assuming highly-linked genes are located near to one another
Moral
Check your problem. ;-)

15. Genetic Algorithm More crossover For real-valued vectors
Line recombination
Intermediate recombination

16. Genetic Algorithm More crossover (cont’d) (Talbi, 2009) (convergence?)
For permutations
Order crossover (OX)
Partially mapped crossover (PMX)
For tour of TSP
Cycle crossover (CX) 1 2 3 4 5 6 7 8 9 1 9 3 4 5 6 2 7 8 8 4 1 5 9 3 6 2 7 8 4 3 5 6 2 7 1 2 3 4 5 6 7 8 9 8 5 3 4 6 2 7 8 4 1 5 9 3 6 2 7 8 9 3 4 5 6 1 2 7 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 3 7 4 2 6 5 8 9 9 3 7 8 2 6 5 1 4 9 3 7 8 2 6 5 1 4 9 2 3 8 5 6 7 1 4

17. Genetic Algorithm Selection
c.f., Evolution Strategy  Truncation Selection
Roulette Selection (Fitness-Proportionate Selection)
Scheme
Let s = i fi , where is the fitness of ith individual.
Select ith individual with probability (fi / s)
Selection pressure problem
How about minimization problems?
Just to use fi' = 1/ fi ?
Or, if difference of fi and fj is too large?

18. Genetic Algorithm
Selection (cont’d)
Stochastic Universal Sampling

19. Genetic Algorithm Selection (cont’d) Tournament Selection
Primary selection technique used for GA
Recall selection pressure problem of previous methods.
How to scale fitness appropriately?
A solution -- non-parametric selection algorithm
Tournament size t
1 (random search), very large (Truncation Selection with  = 1)
2 (most popular), ...

20. Exploitative Variations
Trend in new algorithms -- more exploitative!
Elitism
Similar to ( + )
Injecting directly fittest individuals (elites) of previous population into the next
Preventing possible premature convergence
Increasing mutation and crossover noise
Weakening selection pressure
Reducing number of elites
Steady-State Genetic Algorithm
Updating population, incrementally
Introducing a few children (possibly one or two)
Give death to the same number of selected individuals
Preventing premature convergence
Selecting individuals randomly for death
+ Approaches in Elitism
c.f., Generation Gap Algorithms -- using large value of new children

21. Exploitative Variations
Tree-Style Genetic Programming Pipeline
Hybrid Evolutionary and Hill-Climbing Algorithms
Revising and replacing each individual with some hill-climbing
Adjusting degree of exploitation -- t
A.k.a. memetic algorithms

22. Exploitative Variations
Scatter Search with Path Relinking
Combining
exploitative mechanisms (hybrid methods, steady-state evolution)
explicit attempt to force diversity (exploration)
Schema
Initial population
Seeds provided + random individuals that are very different from one another and from the seeds
Applying hill-climbing to each seed
Loop
Selecting some very fit individuals and some very diverse individuals
(much less than population)
Generating new seeds by crossover
Path Relinking, here
Add new ones to population after applying hill-climb
Required -- measure of diversity
Then, how to generate diverse individuals?

23. Exploitative Variations
Scatter Search with Path Relinking (cont’d)
Path Relinking
Original path (solid) and one possible relinked path (dotted)

24. Differential Evolution
Mutation by vector addition and subtraction
Working in metric vector spaces (booleans, metric integer spaces, reals)
Generating children
For each parent i,
Selecting randomly individuals, a, b, c
Generating child by crossover of i and a + (b – c)
Survival selection
Child is thrown away if it is worse than parent
Adaptive but not much global
Determining the size of mutation largely based on current variance in the population
Children are entirely based on the existing parents

25. Particle Swarm Optimization
Idea based on
swarming and flocking behaviors in animals
maintaining a single static population
Tweaking in response to new discoveries about the space
Particle representation
Location x, velocity v = x(t) – x(t–1)
where x(t) is location at time t
c.f., working in multidimensional metric (usually real-valued) spaces
Update of location
x, x+, x! -- best of x, best of any of informant of x, best of anyone, so far
v  v + b(x* – x) + c(x+ – x) + d(x! – x)
x  x + v
c.f., candidate is mutated towards the best discovered solutions so far.
Effect of parameters

26. Summary Population methods Mutation and crossover Selection
Resampling techniques (EA) and directed mutation (PSO)
Generational algorithms and steady-state algorithms
Mutation and crossover
Chromosomes in real- and boolean-valued vectors, permutations, tours
Selection
Roulette-wheel with fitness scaling and tournament
Adaptive mutation algorithm
Particle Swarm Optimization