1. 1 Copyright notices This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder. Concerning copyrighted figures and tables with copyright, the following apply ©2008, 2009 IEEE ©2010 Applied Energy. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE /Applied Energy.

2. 2 Non-revisiting Stochastic Search Shiu Yin Yuen, Kelvin Department of Electronic Engineering, City University of Hong Kong

3. 3 Outline of Talk Introduction: need for stochastic search; evolutionary computation; no-free-lunch theorems; known problems Non-revisiting Stochastic Search: the philosophy; non-revisiting genetic algorithm; experimental results; theoretical results Other Non-revisiting Stochastic Searches Conclusions

4. 4 Need for Stochastic Search  Many real world optimization problems are multi- modal; the landscape is non-smooth; no well defined gradient. They present great difficulties to conventional optimization techniques  Scissors-paper-stone: the “foil an adversary argument”  The nature argument: nature inspired algorithm is suitable for real world problems

5. 5 Evolutionary Computation (EC)  One popular type of stochastic search  Idea inspired by nature  Can apply to multi-modal, non-smooth, ill defined gradient situation  Simple to learn and give reasonably good result  Famous examples: genetic algorithm, evolutionary strategies, evolutionary programming, genetic programming (natural genetics) simulated annealing (cooling of hot objects) particle swarm optimization, ant colony, cultural algorithm (social behavior of a group)

6. 6 No-Free-Lunch Theorems  (Wolpert and Macready 1997) The average performance of any black box algorithm, when averaged over the set of problems, assumed to be uniformly distributed, is equal  Assume that all algorithms are non-revisiting: a visited search position will not be re-visited  Such a non-revisiting algorithm has never been reported before!

7. 7 Known Problems in EC  The algorithms are revisiting: throws away all past history; only keeps the current population  Premature convergence: the whole population consists of a single type of individual  Parameter control: you need to fine tune your evolutionary algorithm by selecting good parameters; the good parameters may vary over time  Which algorithm for which problem?  Does anybody know what it’s actually doing?

8. 8 Non-revisiting (Nr) Stochastic Search  Basic Idea 1) Remember every past experience 2) Use 1) to make intelligent future search decisions based on the entire previous search history  Nr search solves the first three problems  Main Objection Remember everything will take up too much memory  Answer Memory is cheap in IT age and obeys Moore’s law Vast majority of engineering problems involves expensive function evaluations

9. 9 Genetic Algorithm (GA)  Standard Algorithm 1. Randomly initialize a population of  individuals 2. Select two individuals by a selection operator 3. Crossover the two individuals using a crossover operator to generate an offspring 4. Mutate the offspring using a mutation operator 5. Repeat steps 2-4 until offspring have been generated 6. Use a replacement policy involving the  parents and offspring to generate the next generation of  individuals 7. Repeat steps 2-6 until the stopping criterion is satisfied

10. 10 Non-revisiting Genetic Algorithm (NrGA)  GA is a standard one but without mutation  New solution is stored in a binary space partitioning (BSP) structure  When there is a revisit, mutate randomly within the nearest unvisited “subspace”; prune the “subspace” if full (The figure appears in reference [4],  2009 IEEE)

11. 11 An Illustrative Example  Search space [3, 9] x [3, 6]  axis resolution is 3 and 2 (i.e. possible values of the first gene is 3, 6, 9; possible values of the second gene is 3, 6)  GA randomly generates sq = (s(1), s(2), s(3), s(4)) = (s1 = (9,6), s2 = (6,6), s3 = (9,3) and s4 = (9,3)). s4 is a revisit

12. 12 s1 s2 s3, s4 (The figure is adapted from reference [4],  2009 IEEE)

13. 13 Pruning (The figure is adapted from reference [4],  2009 IEEE)

14. 14 RevisitParameter-less adaptively mutate (The figure is adapted from reference [4],  2009 IEEE)

15. 15 Experiment 1  15 EC benchmark functions with Dimension = 30, 40 and 4 functions with Dimension = 2  Compare with  Standard GA: GA (CGA), Real coded GA (RC- GA)  GA with diversity mechanism: Div-GA  Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES)  Three Improved PSOs: DPSO, SEPSO, PSOMS

16. 16 NrGA ranks 1st (or joint 1st) in 39 and is 2nd in 13 out of a total of 64 cases) (Statistically significant via t test) (The table appears in reference [4],  2009 IEEE)

17. 17 Experiment 2  Heating Ventilating and Air Conditioning (HVAC) Engineering  Determine optimal (x1, x2), where x1 is chilled H 2 0 supply temperature, x2 is supply air temperature  Simulate using TRNSYS software  10 sec. per fitness evaluation (PC 3.2 GHz)

18. 18 This figure appears in reference [10], ©2010 Applied Energy

19. 19 Reference [10], ©2010 Applied Energy

20. 20 This figure appears in reference [10], ©2010 Applied Energy

21. 21 Conventional gradient descent based method, e.g. BFGS, perform very poorly on such landscape This figure appears in reference [10], ©2010 Applied Energy

22. 22 Comparison of GA and NrGA This figure appears in reference [10], ©2010 Applied Energy

23. 23 Optimal Settings This table appears in reference [10], ©2010 Applied Energy

24. 24 Theoretical Results  On 1Max-0Spike problem, Progressive Randomized local search – a simplified NrGA - solve it in expected time O(nlogn), n is the problem size  (1+1) EA solve it in  ((n/2) n )  For some problem, a search algorithm can change the expected time complexity from exponential to polynomial by storing visited solutions and disallows revisits

25. 25 Other Non-revisiting Stochastic Search  NrGA has been modified to work on a combinatorial optimization problem: Traveling Salesman Problem. It outperforms GA (compare using wall clock time) AM stands for parameter-less adaptive mutation used in NrGA, EN, EX, SI and DP stand for other famous mutation used in TSP problems Note that standalone AM is better than other famous TSP mutation operators (The table appears in reference [5],  2008 IEEE)

26. 26 NrSA outperforms adaptive simulated annealing (ASA)  NrSA is better than ASA in 20 out of 24 cases (The table appears in reference [6],  2008 IEEE)

27. 27 NrPSO outperforms three improved PSOs  NrPSO ranks 1 st (or joint 1 st ) in 10 out of 11 cases (The table appears in reference [7],  2009 IEEE)

28. 28 NrGP outperforms the conventional GP  Symbolic regression (function estimation)  Logic circuit synthesis (The tables appear in reference [3],  2009 IEEE)

29. 29 Conclusions Introduce a new search method: Non-revisiting stochastic search It is a useful add-on to existing evolutionary algorithms NrX; X = GA, SA, PSO, GP shows improvement Previous EA method throws away valuable information; remembering all past information and use them intelligently will significantly enhance search performance The method is “parameter-less”. It obviates the difficult “parameter control” problem in EA.

30. 30 Acknowledgments  Dr. Chi Kin Chow (EE) – co-inventer of NrSS  Dr. Square Fong (Division of Building Science and Techology) – collaborator in HVAC engineering  Dr. Albert Sung (EE) – collaborator in theoretical analysis of evolutionary algorithm with memory  Mr. Leung Shing Wa – collaborator in NrGP project  All published results can be found in my web page

31. 31 前事不忘，後事之師 《戰國策．趙策一》 Past experience, if not forgotten, will serve as a teacher in future decisions

32. 32 References [1] Yuen, S.Y., Chow, C.K., Continuous Non-revisiting Genetic Algorithm, Proc. IEEE Congress on Evolutionary Computation (CEC) (May 2009) 1896-1903. [2] Yuen, S.Y., Chow, C.K., A Study of Operator and Parameter Choices in Non-revisiting Genetic Algorithm, Proc. IEEE Congress on Evolutionary Computation (CEC) (May 2009) 2977-2984. [3] Yuen, S.Y., Leung, S.W., Genetic Programming that Ensures Programs are Original, Proc. IEEE Congress on Evolutionary Computation (CEC) (May 2009) 860-866. [4] Yuen, S.Y., Chow, C.K., A Genetic Algorithm that Adaptively Mutates and Never Revisits, IEEE Transactions on Evolutionary Computation, Vol 13(2) (April 2009) 454-472. [5] Yuen, S.Y., Chow, C.K., Applying Non-revisiting Genetic Algorithm to Traveling Salesman Problem, Proc. IEEE Congress on Evolutionary Computation (CEC) (June 2008) 2217-2224. [6] Yuen, S.Y., Chow, C.K., A Non-revisiting Simulated Annealing Algorithm, Proc. IEEE Congress on Evolutionary Computation (CEC) (June 2008) 1886-1892. [7] Chow, C.K., Yuen, S.Y., A Non-revisiting Particle Swarm Optimization, Proc. IEEE Congress on Evolutionary Computation (CEC) (June 2008) 1879-1885. [8] Sung, C.W., Yuen, S.Y., On the analysis of the (1+1) Evolutionary Algorithm with Short- term Memory, Proc. IEEE Congress on Evolutionary Computation (CEC) (June 2008) 235-241. [9] Yuen S.Y., Chow, C.K., A Non-revisiting Genetic Algorithm, Proc. IEEE Congress on Evolutionary Computation, Singapore (CEC) (Sept. 2007) 4583-4590. [10] K.F. Fong, S.Y. Yuen, C.K. Chow, S.W. Leung, Energy Management and Design of Centralized Air-conditioning Systems Through the Non-revisiting Strategy for Heuristic Optimization Methods, Applied Energy, Vol. 87(11), (Nov. 2010) 3494- 3506. (doi:10.1016/j.apenergy.2010.05.002)doi:10.1016/j.apenergy.2010.05.002