1. Parallel Simulated Annealing using Genetic Crossover Tomoyuki Hiroyasu Mitsunori Miki Maki Ogura November 09, 2000 Doshisha University, Kyoto, Japan

2. Doshisha University, Japan Background Simulated Annealing (SA) SA is based on the simulation of the physical process of annealing”. SA is one of the emergent calculation algorithms. 1. Moving a solution 2. Change of the energy; E Decreased The movement is always accepted Increased The movement is accepted in a certain probability; P = 3. Temperature; T is decreased Algorithm － (E next － E current ) T exp SA is an effective technique for solving combination optimization problems.

3. Doshisha University, Japan Background SA requires a huge calculation cost. Specifically, it takes much time in finding a optimum solution in continuous problems. Simulated Annealing (SA) SA can derive a global solution. local minimum global minimum high temperature low temperature Parallel SA models, Hybrid SA models

4. Doshisha University, Japan Background Parallel SA model Temperature Parallel Simulated Annealing (Konishi, Taki et al. 1995) Hybrid SA models Parallel Recombinative Simulated Annealing (Mahfoud, Goldberg et al. 1992) Thermodynamical Genetic Algorithm (Mori, Kita et al. 1996)

5. Doshisha University, Japan Purpose of this study Development of algorithm that can derive a good solution with a low calculation cost. Genetic Crossover (GA operation) Parallel SA + Genetic Algorithm (GA) is good at searching a solution in a wider area. (global search) SA is good at searching a solution in a narrower area. (local search) Hybrid method of SA with GA operation is good at not only locally but also globally

6. Doshisha University, Japan Purpose of this study We propose a new model of parallel SA. Parallel Simulated Annealing using Genetic Crossover (PSA/GAc) To discuss the effectiveness of the proposed model, we apply this model to some test functions Searching points : “Individuals” Total number of SA searching points : “Population size” Annealing steps : “Number of generations”

7. Doshisha University, Japan Parallel Simulated Annealing using Genetic Crossover n:crossover interval high temperature SA nn ・・・ low temperature n Genetic Crossover is used to exchange the information of PSA

8. Doshisha University, Japan Parallel Simulated Annealing using Genetic Crossover rank 2 1 3 4 evaluation -2.0 -1.1 parent1 parent2 x1x1 x2x2 x3x3 x1x1 x2x2 x3x3 -1.8 -1.3 crossover child1 child2 x1x1 x2x2 x3x3 x1x1 x2x2 x3x3 next searching points x1x1 x2x2 x3x3 x1x1 x2x2 x3x3 e.g. continuous optimization problem(3 dimensions )

9. Doshisha University, Japan Experimental study Test functions: Continuous optimization problems Comparison between PSA/GAc and SA using other GA operations Is Crossover operation best for exchanging the information of GA operations ? Searching ability of PSA/GAc Comparison of searching ability with Sequential SA and Distributed GA

10. Doshisha University, Japan Comparison between PSA/GAc and SA using other GA operations PSA using Elite Selection (elitePSA) PSA using Roulette Selection (roulettePSA) PSA using Roulette Selection including Elite (e-roulettePSA)

11. Doshisha University, Japan Comparison between PSA/GAc and SA using other GA operations 20, 200 5, 10, 20, 30 0.93 32 parametervalue Population size Initial temperature Cooling rate Communication interval Terminal of simulation When the value of the solution does not change more than 1.0e-4 for 100 generations Parameters

12. Doshisha University, Japan Test functions Rastrigin function Griewank function Rastrigin function [2dimensions] Griewank function [2dimensions] Many local minima in global area Many local minima in local area

13. Doshisha University, Japan Experimental result [Rastrigin] [20population, Average of 10 trials]

14. Doshisha University, Japan Experimental result [Griewank] [200population, Average of 10 trials]

15. Doshisha University, Japan Searching ability of PSA/GAc 400 10 0.93 32 parametervalue Population size Initial temperature Cooling rate Communication interval Parameters of PSA/GAc

16. Doshisha University, Japan Searching ability of PSA/GAc Sequential SA -long (SSA-long) Sequential SA -short (SSA-short) Distributed GA (DGA) In PSA/GAc, the solution was derived with 8000 generations with 400 populations SSA for 3200000 (8000 generations * 400 populations) SSA for 8000 generations for 400 times DGA for 8000 generation * 400 populations (20 individuals * 20 islands)

17. Doshisha University, Japan Test functions Rastrigin function [10, 30dimensions] Griewank function [10, 30dimensions] Rosenbrock function Rosenbrock function [10, 30dimensions] Many local minima in global area Many local minima in local area

18. Doshisha University, Japan Experimental result [Rastrigin] 10 ( 0.0000 ) 10 ( 0.0000 ) 3,034,881773,940 0 ( 0.9950 ) 1 ( 0.0000 ) 3,117,6413,181,940 0 ( 12.85 ) 0 ( 21.21 ) 3,200,000 0 ( 199.9 ) 0 ( 190.6 ) 3,200,000 SSA-shortSSA-longPSA/GAcDGA 10 dimensions 30 dimensions Number of good solutions [Out of 10 trials] Evaluations [Average of 10 trials]

19. Doshisha University, Japan Experimental result [Griewank] SSA-shortSSA-longPSA/GAcDGA 10 dimensions 30 dimensions 9 ( 0.0000 ) 0 ( 0.0074 ) 10 ( 0.0000 ) 7 ( 0.0000 ) 0 ( 1.642 ) 0 ( 0.1512 ) 0 ( 0.3994 ) 0 ( 0.4666 ) 3,008,2013,200,4003,200,000 3,118,0412,922,1203,200,000 Number of good solutions [Out of 10 trials] Evaluations [Average of 10 trials]

20. Doshisha University, Japan Experimental result [Rosenbrock] SSA-shortSSA-longPSA/GAcDGA 10 dimensions 30 dimensions 10 ( 0.0000 ) 0 ( 0.0004 ) 10 ( 0.0000 ) 0 ( 18.79 ) 1 ( 0.0000 ) 1 ( 0.0000 ) 0 ( 0.0005 ) 0 ( 0.0001 ) 2,750,7213,200,400 2,723,4413,200,400 3,200,000 Number of good solutions [Out of 10 trials] Evaluations [Average of 10 trials]

21. Doshisha University, Japan Conclusion In this study, We proposed a new model of parallel SA: Parallel Simulated Annealing using Genetic Crossover (PSA/GAc) PSA is good at searching a solution locally and Genetic Crossover (GA operation) is good at searching a solution globally. To apply this model to test functions, the performance of PSA/GAc was evaluated. SA’s searching ability was expanded

22. Doshisha University, Japan Conclusion Comparison between PSA/GAc and other algorithms in continuous optimization problems… PSAs using other GA operation PSA/GAc was superior to other methods in all problems. Sequential SA Distributed GA PSA/GAc was superior to DGA in problems that are not suited to GAs PSA using Genetic Crossover is effective for continuous problems.

23. Appendix Parallel Simulated Annealing using Genetic Crossover Doshisha University, Japan

24. Corana’s SA m, m’: Neighborhood range g(p) : Coefficient for adjusting neighborhood range p : Rate of acceptance p = n / N N : Interval for adjusting neighborhood range n : Number of acceptance in interval N 0.4 0.6 1/(c+1) c+1 1 p g(p) c=2, N=8

25. Doshisha University, Japan Background Simulated Annealing (SA) Based on the simulation of the physical process “annealing”