1. A Parallel Genetic Algorithm with Distributed Environment Scheme
M. Kaneko
M. Miki
T. Hiroyasu
Now I talk about our study “ A parallel genetic algorithm with distributed environment scheme”.
Doshisha University, Kyoto, Japan

2. Background GAs(Genetic Algorithms) Disadvantage
Stochastic search algorithms based on the mechanics of natural selection and natural genetics
Disadvantage
A huge amount of computational resource is required.
The performance of GAs depends on a choice for the rates of parameters. However, it is difficult to choose proper rates of parameters.
Parallel Distributed GA (PDGA)
GAs(Genetic Algorithm) are stochastic search algorithms based on the mechanics of natural selection and natural genetics.
These disadvantage are well known.
The first one is that a huge amount of computational resources is required. Because GAs require many generations and individuals in the population.
Therefore, there are a lot of research efforts to impliment GAs on parallel computers. Research in the area of Parallel GAs can be separated into some categories. One of them, island model, is effective and we call it Parallel Distributed GA, PDGA.
The second one is that the performance of GAs depends on a choice for the rates of parameters. However, it is difficult to choose proper rates of parameters.
To make the task of choosing them easier, we proposed PDGA with Distributed Environment.
PDGA with
Distributed Environment

3. Parallel Distributed GA
Single Population GA
(SPGA)
Parallel Distributed GA
(PDGA)
Subpopulation
Population
Individual
Migration
I explain Parallel Distributed GAs.
This figure illustrates canonical GAs. Many individuals in a population search solution. We call this GA Single Population GA, SPGA.
This figure illustrates Parallel Distributed GA, PDGA. In PDGA, a population is divided into several subpopulations, and some GAs are performed in multiple subpopulations. Then, migration, exchange of individuals among subpopulations, are executed.
Some GAs are performed in multiple subpopulations.
Migration: Exchange of individuals among subpopulations

4. Crossover and Mutation
To perform direct information exchange between individuals
Mutation
To avoid stagnation in evolution
0.6 　 DeJong (1975)
0.95 　 Grefenstette (1986)
0.75~ Bäck (1996)
parent A
parent B
child A
child B
0.001 　 DeJong (1975)
0.01 　 Grefenstette (1986)
0.005~0.01 Schaffer (1989)
1/L Bäck (1996)
L: Coromosome Length
In GAs, the roles of crossover and mutation are significantly important.
Crossover is employed to perform direct information exchange between individuals. The performance of each GA depends on making a good choice for the crossover rate, and many studies have been performed to know the best crossover rate.
Mutation is employed to avoid stagnation in evolution. Making a good choice of mutation rate is also important, and many studies have been performed.

5. Test Functions Rastrigin Schwefel Griewank Rosenbrock Name Functions
Epistasis
Chromosome
length (bit)
Rastrigin
none
100
(10bits×10variables)
Schwefel
none
100
(10bits×10variables)
Griewank
weak
100
(10bits×10variables)
Rosenbrock
strong
120
(12bits×10variables)
Rastrigin
Schwefel
Griewank
Rosenbrlck
The effect of crossover and mutation rates is examined with these standard test functions, Rastrigin, Schwefel, Griewank, and Rosenbrock functions.
All the functions are 10 dimensional while their 2-dimensional shapes are shown these figures.

6. Procedures of Experiments
Mutation Rate
Number of Subpopulations
Subpopulation size
Total Population size
Migration Interval
Migration Rate
Max Generations 9
20, 180
180,1620 20
0.3
1000
0.1/L
1/L
10/L
0.3
0.1/L
1/L
10/L
0.3
0.3
0.3
0.6
0.1/L
1/L
10/L
Crossover Rate
0.6
0.6
0.6
Roulette selection
Conservation of elite
One point crossover
The average of 10 trials out of 12 trials omitting the highest and lowest values
1.0
0.1/L
1/L
1.0
10/L
1.0
1.0
L：Chromosome length
To examine the effect of crossover and mutation rates, we use these combination of the crossover rates of 0.3, 0.6 and 1.0 and the mutation rates of 0.1/L, 1/L and 10/L, where L is the length of the chromosome.
In PDGA, number of subpopulations is 9, and subpopulation size is 20 or 180. When subpopulation size is 20, total population size is 180. Migration interval is 5 generations and migration, and migration rate is 0.3. Max generations are 1000.
We use roulette selection, conservation of elite and one point crossover.
Then the result are represented by the average of 10 trials out of 12 trials omitting the highest and lowest values.
The parallel computer we use is nCUBE2 with 64 processors. Its processor network is hypercube. One processor is assigned to one subpopulation.
nCUBE2 with 64 processors
Processor network : Hypercube
One processor is assigned to one subpopulation.

7. History of Fitness (SPGA)
Rastrigin
Pop. Size 180
Pc 1.0
0.6
0.3
Fitness value
These figures show the histories of the fitness values for the Rastrigin function when population size is 180 in SPGA.
X-axis shows generations and Y-axis shows the fitness values. The fitness values which have higher shows better solutions.
The red line shows the results when crossover rates, Pc is 1.0, the orange line shows them when Pc is 0.6 and the yellow line shows them when Pc is 0.3.
In each figure, the mutation rate is constant, then the effect of crossover rates under a constant mutation rate can be seen in each figure.
When the mutation rate, Pm, is 0.1/L, the better results are obtained with the high crossover rates.
When Pm, is 10/L, the results are opposite to the case with 0.1/L. The better results are obtained with the low crossover rates.
When Pm is 1/L, at the beginning, 1.0 is appropriate, but 0.6 exceeds 1.0, then 0.3 exceeds 0.6.
That is, the appropriate crossover rate varies with the mutation rates, and sometimes it changes along with the generations.
Pm = 0.1/L
Pm = 1/L
Pm = 10/L

8. The Effect of Crossover and Mutation Rates
(SPGA)
Pc - Pm
This shows that the comparison of the function values at 1000 generations among the combinations of crossover and mutation rates in SPGA.
X-axis shows the population sizes and functions. Y-axis shows the function values in logarithmic scale, and the function values which have lower shows better solutions.
For all population sizes and functions, the function values vary with the combinations of crossover and mutation rates.
For example, for the Rastrigin function, at population size of 180, the best combination of Pc and Pm is 0.3 and 1/L, and function values vary with the combination of Pc and Pm. While at population size of 1620, the best combination of Pc and Pm is 1.0 and 0.1/L, which is different from that of the case of 180.
For all functions, the function values vary with population sizes.
Then, the best combination of crossover and mutation rates are different.
Therefore, it takes a lot of pre-experiments to find the best combination of crossover and mutation rates for tuning GAs.

9. History of Fitness (PDGA)
Rastrigin
Pop. Size 180
Pc 1.0
0.6
0.3
Fitness value
These figures show the histories of the fitness values for the Rastrigin function when population size is 180 in PDGA
When Pm, is 0.1/L and 1/L, the better results are obtained with the high crossover rates.
When Pm, is 10/L, the best crossover rate is not clear.
In PDGAs, the crossover plays not only a role of global search, but also a role of mating between migrants and native individuals.
Therefore, in PDGAs, there is a different tendency with that in SPGAs.
Pm = 0.1/L
Pm = 1/L
Pm = 10/L

10. The Effect of Crossover and Mutation Rates
(PDGA)
This shows that the comparison of the function values at 1000 generations among the combinations of crossover and mutation rates in PDGA.
Like the result of SPGA, for all population sizes and functions, the function values vary with the combinations of crossover and mutation rates.
Then, the best combination of crossover and mutation rates are different.
Therefore, it takes a lot of pre-experiments to find the best combination of crossover and mutation rates for tuning PDGAs, too.

11. Comparison of the performance
(SPGA and PDGA)
Pop. Size 180
1.0E-05
1.0E-04
1.0E-03
1.0E-02
1.0E-01
1.0E+00
1.0E+01
1.0E+02
1.0E+03
SPGA
PDGA
Function value /L /L /L /L /L /L
0.3-10/L
0.6-10/L
1.0-10/L
1.0E-14
1.0E-15 ～
Rastrigin Schwefel Griewank Rosenbrock
This shows that the comparison of the function values at 1000 generations between SPGA and PDGA with population size of 180.
For all functions, the function values are compared between SPGA and PDGA.
PDGA outperforms SPGA except for one combinations for the Rosenbrock function. The Roesnbrock function has a strong interaction between its variables, and some results in PDGA do not provide better results than those in SPGA.
But the advantage of PDGA is clear from these results.

12. PDGA/DE (Distributed Environment)
Different crossover rates
Different mutation rates
PDGA/CE
(Constant Environment)
From these results, it was concluded that to obtain good results appropriate combination of the crossover rate and mutation rate must be used. But, the determination of appropriate crossover rate and mutation rate is a time consuming task.
To overcome this problem, we proposed a PDGA with distributed Environment scheme, PDGA/DE. In PDGA/DE, the mutation rates and the crossover rates in each subpopulation are different from each other.
This figure illustrates PDGA/DE. The thermometers represent the mutation rates and the heart symbols represent the crossover rates. High temperature means a high mutation rate and a big heart symbol means a high crossover rate.
The turning of the GA parameters is not necessary with the PDGA/DE since many combinations of such GA parameters occur in many subpopulations.
Consequently, it can be expected that a global optimum can be easily obtained without any pre-experiments with the PDGA/DE.
To demonstrate the effectiveness of PDGA/DE, PDGA/DE compared with PDGA/CE, where PDGA/CE have constant a crossover rate and a constant mutation rate in the entire subpopulations.
A Constant crossover rate
A Constant mutation rate
Mutation rate
Crossover rate

13. Effectiveness of PDGA/DE
Pop. Size 180
To demonstrate the effectiveness PDGA/DE, PDGA/DE with 9 subpopulations was performed. The parameters used for the PDGA/DE are the combination of 3 crossover rates and 3 mutation rates used before.
This shows that the comparison of the function values at 1000 generations with population size of 180.
At first, the performance of PDGA/DE are compared with SPGA. PDGA/DE outperforms SPGA except for two combination of the Rosenbrock function.
Secondly, the performance of PDGA/DE and PDGA/CE are compared. It is clear that the performance of PDGA/DE relatively very high although it is not the best.
Therefore the superiority of PDGA/DE can be recognized from this figure.

14. PDGA/DE vs. SPGA (with the best combination)
Speedup
PDGA/DE vs. SPGA (with the best combination)
1000 generations
same quality of solutions
(at 1000 generations in PDGA/DE)
Pop. Size = 450
Number of Subpopulations = 9 (9PEs)
Ideal speedup
In order to find the efficiency of parallel processing for the PDGA/DE, the calculation time in the PDGA/DE is compared to the one in a SPGA which have the best combination of the crossover rate and mutation rate. Tthe total population size is 450, and number of subpopulations is 9.
This figure shows the speed up for the four test functions. (1) is obtained at 1000 generations, and (2) is obtained at the same quality of solutions at 1000 generations in PDGA/DE.
At first, for all functions, the vales of the speedup for the case of (1) are approximately 8.6 and they are similar to the ideal speedup which is 9. It can be recognized that the PDGA/DE can provide good parallel efficiency since the inter processor communication occurs only when the migration is performed.
On he other hand, the values of the speedup for the case of (2) are between 22 and 25 except for the Rosenbrock function. This surprising speedup is due to the increase in the performance of the PDGA/DE, that is the PDGA/DE provides good solutions between 2.6 to 2.9 times faster than the SPGA.
(1) 8.6 (similar to the ideal speedup)
(2) between 22 and 25 (except for the Rosenbrock function)
PDGA/DE provides solution 2.6 to 2.9 times faster than SPGA

15. Conclusions
The optimum crossover and mutation rates vary according to the population size and the problem to be solved.
A parallel distributed GA with distributed environment(PDGA/DE) is proposed, and the superiority of this scheme is experimentally proved.
PDGA/DE is the fastest way to gain the best solution under uncertainty of the appropriate crossover and mutation rates.
I’d like to conclude as follows.
The proposed PDGA/DE is the fastest way to gain the best solution under uncertainty of the appropriate crossover and mutation rates.