1. Multiobjective Optimization for Locating Multiple Optimal Solutions of Nonlinear Equation Systems and Multimodal Optimization Problems Yong Wang School of Information Science and Engineering Central South University Changsha 410083, China ywang@csu.edu.cn http://ist.csu.edu.cn/YongWang.htm

2. 2  Part I: Multiobjective Optimization for Locating Multiple Optimal Solutions of Nonlinear Equation systems (MONES)  Part II: Multiobjective Optimization for Locating Multiple Optimal Solutions of Multimodal Optimization Problems (MOMMOP)  Future Work Outline of My Talk

3. 3  Part I: Multiobjective Optimization for Locating Multiple Optimal Solutions of Nonlinear Equation systems (MONES)  Part II: Multiobjective Optimization for Locating Multiple Optimal Solutions of Multimodal Optimization Problems (MOMMOP)  Future Work Outline of My Talk

4. 4 Nonlinear Equation Systems (NESs) (1/2) NESs arise in many science and engineering areas such as chemical processes, robotics, electronic circuits, engineered materials, and physics. The formulation of a NES

5. Nonlinear Equation Systems (NESs) (2/2) An example 5 the optimal solutions A NES may contain multiple optimal solutions

6. 6 Solving NESs by Evolutionary Algorithms (1/4) The aim of solving NESs by evolutionary algorithms (EAs) –Locate all the optimal solutions in a single run The principle At present, there are three kinds of methods –Single-objective optimization based methods –Constrained optimization based methods –Multiobjective optimization based methods

7. 7 Solving NESs by Evolutionary Algorithms (2/4) Single-objective optimization based methods The main drawback –Usually, only one optimal solution can be found in a single run or

8. Solving NESs by Evolutionary Algorithms (3/4) Constrained optimization based methods The main drawbacks –Similar to the first kind of method, this kind of methods can only locate one optimal solution in a single run –Additional constraint-handling techniques should be integrated 8 or

9. Solving NESs by Evolutionary Algorithms (4/4) Multiobjective optimization based methods (CA method) The main drawbacks –It may suffer from the “curse of dimensionality” (i.e., many- objective) –Maybe only one solution can be found in a single run 9 C. Grosan and A. Abraham, “A new approach for solving nonlinear equation systems,” IEEE Transactions on Systems Man and Cybernetics - Part A, vol. 38, no. 3, pp. 698- 714, 2008.

10. 10 MONES: Multiobjective Optimization for NESs (1/9) The main motivation –When solving a NES by EAs, it is expected to locate multiple optimal solutions in a single run –Obviously, the above process is similar to that of the solution of multiobjective optimization problems by EAs –A question arises naturally is whether a NES can be transformed into a multiobjective optimization problem and, as a result, multiobjective EAs can be used to solve the transformed problem W. Song, Y. Wang, H.-X. Li, and Z. Cai, “Locating multiple optimal solutions of nonlinear equation systems based on multiobjective optimization,” IEEE Transactions on Evolutionary Computation, Accepted.

11. MONES: Multiobjective Optimization for NESs (2/9) Multiobjective optimization problems –Pareto dominance –Pareto optimal solutions The set of all the nondominated solutions –Pareto front The images of the Pareto optimal solutions in the objective space ≤ < Pareto dominates minimize ≤≤ ≤ f1f1 f2f2 11

12. 12 MONES: Multiobjective Optimization for NESs (3/9) The main idea ① ② minimize

13. The images of the optimal solutions of the first term in the objective space are located on the line segment defined by y=1-x 13 MONES: Multiobjective Optimization for NESs (4/9) The principle of the first term minimize Each decision vector in the decision space of a NES is a Pareto optimal solution of the first term

14. 14 MONES: Multiobjective Optimization for NESs (5/9) The principle of the second term minimize

15. 15 MONES: Multiobjective Optimization for NESs (6/9) The principle of the first term plus the second term The images of the optimal solutions of a NES in the objective space are located on the line segment defined by y=1-x minimize Pareto Front 0 1 1

16. MONES: Multiobjective Optimization for NESs (7/9) Summary –In MONES, a NES has been transformed into a biobjective optimization problem –There are some very good properties for our transformation technique –Multiobjective EAs (such as NSGA-II) can be easily used to solve the transformed biobjective optimization problem 16

17. 17 MONES: Multiobjective Optimization for NESs (8/9) The differences between CA and MONES CAMONES

18. 18 MONES: Multiobjective Optimization for NESs (9/9) The differences between CA and MONES CA MONES

19. The Experimental Results (1/4) Test instances 19

20. The Experimental Results (2/4) IGD indicator: Inverted Generational Distance 20 IGD The images of the best solutions found

21. The Experimental Results (3/4) 21 NOF indicator: Number of the Optimal Solutions Found

22. The Experimental Results (4/4) 22 F1 F4 F5 Convergence behavior in a typical run provided by CA in the decision space F1 F4 F5 Convergence behavior in a typical run provided by MONES in the decision space

23. 23  Part I: Multiobjective Optimization for Locating Multiple Optimal Solutions of Nonlinear Equation systems (MONES)  Part II: Multiobjective Optimization for Locating Multiple Optimal Solutions of Multimodal Optimization Problems (MOMMOP)  Future Work Outline of My Talk

24. Multimodal Optimization Problems (MMOPs) (1/2) Many optimization problems in the real-world applications exhibit multimodal property, i.e., multiple optimal solutions may coexist. The formulation of multimodal optimization problems (MMOPs) 24

25. Multimodal Optimization Problems (MMOPs) (2/2) Several examples 25

26. 26 The Previous Work (1/2) Niching methods –The first niching method The preselection method suggested by Cavicchio in 1970 –The current popular niching methods Clearing (Pétrowski, ICEC, 1996) Fitness sharing (Goldberg and Richardson, ICGA, 1987) Crowding (De Jong, PhD dissertation, 1975) Restricted tournament selection (Harik, ICGA, 1995) Speciation (Li et al., ECJ, 2002) The disadvantages –Some problem-dependent niching parameters are required

27. The Previous Work (2/2) Multiobjective optimization based methods, usually two objectives are considered: –The first objective: the original multimodal function –The second objective: the distance information (Das et al., IEEE TEVC, 2013) or the gradient information (Deb and Saha, ECJ, 2012) The disadvantages –It cannot guarantee that the two objectives in the transformed problem totally conflict with each other –The relationship between the optimal solutions of the original problems and the Pareto optimal solutions of the transformed problems is difficult to be verified theoretically. 27

28. 28 MOMMOP: Multiobjective Optimization for MMOPs (1/5) The main motivation minimize Y. Wang, H.-X. Li, G. G. Gary, and W. Song, “MOMMOP: Multiobjective optimization for locating multiple optimal solutions of multimodal optimization problems,” IEEE Transactions on Cybernetics, Accepted.

29. 29 MOMMOP: Multiobjective Optimization for MMOPs (2/5) The main idea –Convert an MMOP into a biobjective optimization problem ① ② minimize

30. 30 MOMMOP: Multiobjective Optimization for MMOPs (3/5) The principle of the second term the objective function value of the best individual found during the evolution the objective function value of the worst individual found during the evolution the objective function value of the current individual the range of the first variable Remark: the aim is to make the first term and the second term have the same scale the scaling factor For the optimal solutions of the original multimodal optimization problems, the values of the second term are equal to zero.

31. 31 MOMMOP: Multiobjective Optimization for MMOPs (4/5) The principle of the first term plus the second term The images of the optimal solutions of an MMOP in the objective space are located on the line segment defined by y=1-x minimize Pareto Front 0 1 1

32. 32 MOMMOP: Multiobjective Optimization for MMOPs (5/5) Why does MOMMOP work? –MOMMOP is an implicit niching method x f(x)f(x) xbxb xaxa xcxc (0.1, 1) (0.15, 0.8) (0.6, 0.8) f1f1 f2f2 x f(x)f(x) 0 1 1 1 1 0 0 1 1 1 1 1 1 (0.0, 0.9) (0.35, 1.05) (0.8, 0.6) xaxa xbxb xcxc

33. 33 Two issues in MOMMOP (1/3) The first issue –Some optimal solutions may have the same value in one or many decision variables

34. 34 Two issues in MOMMOP (2/3) The second issue –In some basins of attraction, maybe there are few individuals –Meanwhile, some individuals in the same basin may be quite similar to each other

35. Two issues in MOMMOP (3/3) 35 if or When compare two individuals

36. Test Instances 36 20 benchmark test functions developed for the IEEE CEC2013 special session and competition on niching methods for multimodal function optimization

37. The Experimental Results (1/3) Comparison with four recent methods in IEEE CEC2013 37

38. The Experimental Results (2/3) Comparison with four state-of-the-art single-objective optimization based approaches 38

39. The Experimental Results (3/3) Comparison with two state-of-the-art multiobjective optimization based approaches 39

40. 40  Part I: Multiobjective Optimization for Locating Multiple Optimal Solutions of Nonlinear Equation systems (MONES)  Part II: Multiobjective Optimization for Locating Multiple Optimal Solutions of Multimodal Optimization Problems (MOMMOP)  Future Work Outline of My Talk

41. 41 Future Work We have proposed a multiobjective optimization based framework for nonlinear equation systems and multimodal optimization problems, respectively, however –The principle should be analyzed in depth in the future –The rationality should be further verified –The framework could be further improved This generic framework could be generalized into solve other kinds of optimization problems The source codes of MONES and MOMMOP can be downloaded from: http://ist.csu.edu.cn/YongWang.htm