Download presentation

Published byJimena Lippincott Modified over 3 years ago

1
**USING GENETIC ALGORITHM WITH ADAPTIVE MUTATION MECHANISM FOR NEURAL NETWORKS DESIGN AND TRAINING**

Yuri R. Tsoy, Vladimir G. Spitsyn, Department of Computer Engineering Tomsk Polytechnic University Good afternoon, My name is Yuri Tsoy, I represent Computer Science Department of Tomsk Polytechnic University. Let me introduce a report “USING GENETIC ALGORITHM WITH ADAPTIVE MUTATION MECHANISM FOR NEURAL NETWORKS DESIGN AND TRAINING” 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

2
**Report contents 1. Introduction 2. Description of the algorithm**

3. Adaptive mutation mechanism 4. Results of experiments 5. Implementation 6. Conclusion My report consists of 6 sections. You can see their titles on the slide. So let us proceed to the first Section: Introduction 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

3
**1.Genetic algorithms and neural networks**

Genetic algorithms (GAs) use evolutionary concept (heredity, mutability and natural selection) to solve optimization tasks. The idea of Artificial neural networks (ANNs) is inspired by functionality of human brain. ANNs are often used to solve classification and approximation tasks Genetic algorithm (GA) uses evolutionary concept to solve optimization tasks. Due to mechanisms of heredity, mutability and selective pressure, GAs show great adaptive abilities. The idea of artificial neural networks (ANN) is inspired by functionality of human brain and ANNs are often used to solve classification and approximation problems. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

4
**+ + 1. Use of neural networks Preparation of Training Data Definition**

of Structure (Design) Tuning of Weights (Training) + + Genetic Algortihm Use of neural networks can be roughly divided in 3 stages: 1. Preparation of training data 2. Definition of structure of neural network (so-called design of neural net) 3. Tuning of weights of connections (also known as training of neural network) Although genetic algorithm can be applied to all 3 stages we will consider only the 2nd and the 3rd stages: design and training. Use of genetic algorithms for design and training of neural networks is often referred as neuroevolution. Neuroevolution 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

5
**2. Description of the algorithm**

NEvA – NeuroEvolutionary Algorithm 1. Simultaneous design and tuning of neural network 2. 1 individual = 1 neural network 3. Population = set of neural networks The developed algorithm (NEvA) models evolution of the neural networks population. Genotype of each organism contains information about connections and their weights of the network corresponding to this organism. An example is given on the [slide]. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

6
**2. Description of the algorithm**

Initial population: Networks without hidden neurons. Weights of connections are encoded with 19-bit within range [-26,2144; 26,2144] with precision 0,0001. Growth of the complexity of networks during the process of evolution. Neurons with log-sigmoid activation function: There are some changes in comparison with the description in published paper because we’ve found better encoding parameters during our experiments. The weight of connection is encoded with 19 bits and its value lies within [-26,2144; 26,2144] range with precision of 0,0001. So obtained results differ from results in paper. In initial population all organisms represent networks without hidden neurons. Connections weights are initialized randomly in range [-0,5; 0,5]. During the process of evolution networks “grow” and become more complex. That is new neurons and new connections appear. We used neurons with log-sigmoid activation function. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

7
**3. Adaptive mechanism of mutation**

Types of mutation: Addition of random connection Deletion of random connection Addition of random neuron Deletion of random neuron Change of weight of random connection Mutation mechanism allows to add and to delete neurons and connections and also to change connection weight for the random value. Selection of type of mutation performed with respect to the neurons number and number of connections. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

8
**3. Adaptive mechanism of mutation**

fC percentage of implemented connections fN depends on number of hidden neurons FC = fC2 FN = FC * fN2 Rnd is a random number NH is a number of hidden neurons Rnd > FC Delete random neuron Add connection NH > 0 no Rnd > FN yes Rnd > FC & NH > 0 Schematically the algorithm of selection of the applicable mutation type is shown in [slide]. Some notation is also presented, where fC and fN are heuristics that allow us to control a speed of growth of the neural network. To simplify, the adaptive mutation algorithm can be divided into two branches (by the first conditional transition) according to the influence on “linkage” degree presented by the fC coefficient. These branches are: 1) Branch of fC decrease (the rightmost branch). 2) Branch of fC increase (the leftmost branch). Since neuron deletion can lead either to decrease as well as to increase of fC (it depends on the number of connections associated with this neuron) then this type of mutation exists in both branches. Thus the main factor for regulation of the network structure is its “linkage” degree. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

9
**3. Adaptive mechanism of mutation**

Example: 1 3 4 2 x1 x2 y1 Rnd > FC Delete random neuron Add connection NH > 0 no Rnd > FN yes Rnd > FC & NH > 0 fC = 6 /(0,5*(5*4 – 2*1 – 1*0)) = 0,667 fN = 3/5 = 0,6 Let’s look at an example of mutation type selection with respect to the introduced scheme. Let the network on the [slide] is undergoing mutation. It is necessary to define which type of mutation change will be applied to this network. To estimate values of the probabilities of different mutation types the coefficients FC and FN should be calculated first (results are shown on the [slide]). After the coefficients are calculated we can estimate what type of mutation is likely to be applied. As it can be seen the probability that a random hidden neuron will be deleted is higher than the other probabilities although the probability to add a connection is also high. In fact we don’t calculate these probabilities in implementation of our neuroevolutionary algorithm, we just follow the scheme (showing the scheme) of mutation type selection instead. FC = fC2 = 0,445, FN = FC * fN2 = 0,160 p1 = ( 1 - FC )(1-FC+FC) = 0,308 p2 = 0,454, p3 = 0,071, p4 = 0,166 p1 p2 p3 p4 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

10
**3. Adaptive mechanism of mutation**

Probability of mutation: “Decrease of probability of mutation during the evolution improves performance of the genetic algorithm” (Schаffer, Caruana, Eshelman, Das, 1989, Goldberg, 1989, Eiben, Hinterding, Michalewicz,1999, Igel, Kreutz, 2003 ) NC – number of connections in neural network represented by individual. Mutation event is “gambled” NI+NO times for each individual, where NI and NO – number of inputs and outputs in networks respectively. Probability and type of mutation are defined individually for each network. Not only a mutation type, but also a mutation operator probability denoted as Pm (showing) is defined adaptively. It’s been noted by many researchers that decrease of the probability of mutation during the evolution improves performance of the genetic algorithm. So then, since we use a concept of “growing” neural networks, it is possible to say that mutation effect decreases as evolution goes. In other words the larger networks we evolve the lesser mutations occur and the more combinations of the existing networks as a result of crossing are used. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

11
**4. Results of experiments**

Exclusive OR Average number of object function calculations NH NC Population size NEAT1 4755 2,35 7,48 150 NEAT2 6612 3,12 11,72 CGA 13165,63 4 17 50 BP 5338,28 - BPM 828,32 NEvA (no adaptive Pm) 6026,26 4,8 16,22 13 NEvA (19-bit) 6063,4 4,78 15,86 To test the introduced algorithm the following two problems are chosen: exclusive OR problem (classification task) and inverted pendulum problem by R.Sutton (approximation task). We first introduce results for XOR problem ([slide]). The performance of NEvA algorithm is compared with that of canonical genetic algorithm (CGA) from, NEAT and two variants of back-propagation learning algorithm: simple (BP) and with momentum (BPM). We also made a comparison of NEvA with and without adaptive mutation probability (Pm). The performance is estimated with use of average number of object function calculations, i.e. how many times a training data set was introduced (the less, the better). Here NH is a number of hidden neurons and NC is a number of connections in the network. Although both NEvA and NEAT lose to gradient methods, we should note that the task was much more complex for neuroevolutionary algorithms, since not only weights but also network structures were tuned. And also NEvA outperforms CGA as many as 2 times and shows better performance than NEAT2, although NEAT networks are much simpler (the number of hidden neurons and total number of connections is much less). NEvA (19-bit encoding) with adaptive mutation probability is slightly worse than algorithm without such ability, and networks in average are little “better” than networks obtained by NEvA algorithm without adaptive Pm. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

12
**4. Results of experiments**

1 pole balancing Number of trials NH NC Mean Best Worst GENITOR 1846 272 7052 5 35 SANE 535 70 1910 8 40 ESP 285 11 1326 30 NEvA (no adaptive Pm) 451 37 3872 0,58 5,22 NEvA (19-bit) 358,2 1670 1,18 6,5 For the single pole balancing benchmark with discrete force impact we compared NEvA with other evolutionary algorithms such as GENITOR, SANE and ESP [9] algorithms. The goal is to try to balance 1 or 2 poles on the cart for approximately 30 minutes of the modeling time, moving this cart in the horizontal direction straightforward or backwards. Number of trials, before successful network is found, was estimated (the less, the better). Results averaged over 50 runs are presented in [slide]. By obtained results NEvA loses only to ESP algorithm and performs better than all the other algorithms. Neural networks, obtained by NEvA, are much better then networks for ESP, SANE and GENITOR in all cases. Note that results of NEvA with adaptive mutation rate have better characteristics than NEvA with fixed Pm since the difference between the best and the worst results is much less. This means that NEvA with fully adaptive mutation operator showed more stable results for 1-pole balancing problem. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

13
**4. Results of experiments**

2 poles balancing Average number of trials Population size NH NC Evolutionary programming 307200 2048 N/A SANE 12600 200 ESP 3800 5 35 NEAT 3578 150 (0–4) (6–15) NEvA (no adaptive Pm) 3777 59 1,4 7,24 NEvA (19-bits) 1338,92 32 0,58 7,2 For the 2 poles balancing the goal is the same as for previous problem, but now there are 2 poles of different length that should be balanced on the cart and also force impact takes continuous values. Number of trials, before successful network is found, was also estimated (the less, the better). Results averaged over 50 runs are presented in [slide]. Results for 2-pole problem with continuous force impact and also information about obtained neural networks are shown on the next [slide].We compared results for NEvA with results of evolutionary programming, SANE, ESP and NEAT. The performance of NEvA without adaptive Pm is comparable with that of ESP and NEAT. The results for NEvA with fully adaptive mutation mechanism surpass the results for all the other algorithms. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

14
4. Future plans “Parameterless” variant of NEvA with adaptive mutation mechanism and adaptive population sizing. Some preliminary results: XOR (NH : NC) 1-pole balancing 2-pole balancing NEvA (adaptive mutation) 6063,4 (4,78 : 15,86) 358,2 (1,18 : 6,5) 1338,92 (0,58 : 7,22) NEvA (adaptive mutation + adaptive population sizing) 6608,84 (5,14 : 17,26) 634,22 (1,26 : 6,74) 1464,54 (0,62 : 7,32) For now we developed “parameterless” variant of NEvA, i.e. we have adaptive mutation and adaptive population sizing strategy, whereas selection and crossover parameters are fixed as a result of numerous experiments. The only parameter that is to be set by user is desired training error. Although fully adaptive algorithm is very simple to use, its results are some worse than results for NEvA without adaptive sizing ([slide]), but we should note that difference is not critical. We’ve also tested fully adaptive NEvA on Proben1 benchmarking test set including medicine and industry real-world data and obtained encouraging results. Nevertheless we’re still working on improvement of sizing strategy to achieve better performance. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

15
5. Implementation The introduced algorithm is implemented to comply with the architecture of the software environment “GA Workshop” General scheme of “GA Workshop” The introduced algorithm is implemented to comply with the architecture of the software environment “GA Workshop”. You can see the structure of this environment on the [slide]. The notion of the “GA Workshop” is to separate solving algorithm from the problem solved and the processing of run data with use of design patterns. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

16
5. “GA Workshop” At the time “GA Workshop” is under construction (there is no GUI), although it is already possible to use it for researches. The following researches have been done: Experiments with NEvA. Study of quasi-species model by M. Eugen. Study of majoring model by V.G. Red’ko. Investigation of simple population sizing techniques. Numerical optimization with use genetic algorithm. Experiments with compensatory genetic algorithm. At the time “GA Workshop” is under construction (there is no GUI yet), although it is already possible to use it for researches. Nevertheless we have already used this environment for several researches on different evolutionary algorithms and problems (details on slide). 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

17
**5. “GA Workshop” Features:**

3 different variants of genetic algorithm, including “standard” GA, compensatory GA and NEvA; 5 different crossover operators for the binary encoded GA and 4 different crossover operators for NEvA; 4 selection strategies; 13 benchmark problems; 3 different population sizing strategies; For now the following features presented on the slide are implemented. Most of these parameters can be combined with parameters of genetic algorithm (such as population size, genetic operators type and probabilities, different stop criteria). This allows examination of the performance of GA with different characteristics possible. Some of the parameters are algorithm-independent. For example, implemented variants of selection as well as population sizing strategies can be used without any changes with different variants of genetic algorithms. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

18
**5. “GA Workshop” Data available for analysis:**

Data that describes each generation of GA (fitness distribution for each generation averaged over multiple launches). Data that describes GA behavior (dynamics of averaged mean, the best and the worst fitness value for each generation, dynamics of averaged deviation of fitness, time per launch in milliseconds). Data that describes obtained solutions (number of object function calculations until solution is found, time until the first solution is found in milliseconds. All solutions and some data describing additional properties of the solutions are output into separate file for further analysis and use) The data that is available for analysis of results of multiple independent launches of genetic algorithm is presented on this slide. It is data that characterizes evolutionary process, fitness distributions for each generation of genetic algorithm, statistics about solutions or candidate-solutions. Some additional parameters depending on the peculiarities of the evolutionary algorithm or the task solved can also be written. All data can be extracted for further statistical analysis and drawing of plots, charts and so on.. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

19
Conclusion Results of experiments showed that NEvA performance is comparable and in some cases surpass results of other algorithms for the reviewed problems (XOR problem and full information pole balancing) Adaptive mutation rate caused increase of performance in comparison with NEvA with fixed Pm (up to 40% for 2-pole balancing task), although resulting networks were slightly worse for 1-pole balancing problems. Results of experiments showed that NEvA performance is comparable and in some cases surpass results of other algorithms for the reviewed problems (XOR problem and full information pole balancing). Due to adaptive selection of the type of mutation there is no need in direct global limiting variables or deterministic rules for network structure evolution, because addition or deletion of the new elements performed depending on individual peculiarity of each phenotype. Adaptive mutation rate caused increase of performance in comparison with NEvA with fixed Pm (up to 40% for 2-pole balancing task), although resulting networks were slightly worse for 1-pole balancing problems. Use of mutation that deletes connections can help to get rid of insignificant network inputs thus decreasing a search space complexity. The NEvA algorithms is implemented in “GA Workshop” software environment. 9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

20
**Thank you for your attention!**

9th Korea-Russia International Symposium on Science and Technology - KORUS 2005 June 26 - July 2, 2005, Novosibirsk State Technical University, Novosibirsk

Similar presentations

Presentation is loading. Please wait....

OK

Genetic Algorithms By: Anna Scheuler and Aaron Smittle.

Genetic Algorithms By: Anna Scheuler and Aaron Smittle.

© 2018 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google