1. Hierarchical Allelic Pairwise Independent Function by DAVID ICLĂNZAN Present by Tsung-Yu Ho At Teilab, 2011.08.08

2. Reference Paper Hierarchical Allelic Pairwise Independent Functions Author : David Iclănzan Department of Electrical Engineering, Sapientia Hungarian University of Transylvania, Romania GECCO 2011, Best Paper Award Winners in Estimation of Distribution Algorithms

3. Abstract Current multivariate EDAs rely on computationally efficient pairwise linkage detection mechanisms to identify higher order linkage blocks. Historical attempts to exemplify the potential disadvantage of this computational shortcut were scarcely successful. In this paper we introduce a new class of test functions to exemplify the inevitable weakness of the simplified linkage learning techniques. Specifically, we show that presently employed EDAs are not able to efficiently mix and decide between building-blocks with pairwise allelic independent components. These problems can be solved by EDAs only at the expense of exploring a vastly larger search space of multivariable linkages.

4. Outline Basic knowledge for 碩一 Genetic algorithm (GA) Estimation of distribution algorithm (EDA) What’s the problem? Motivation Goal of this paper Testing allelic pairwise ibdependant function Concatenated parity function (CPF) Concatenated parity / trap function (CP/TF) Proposed Function Difference between Teil’s work and this paper Conclusion

5. Basic Knowledge (1) Genetic algorithm (GA), or evolutionary algorithm (EA) Stochastic search for optimization Operate on the population, numerous solutions (search space) Evolve solutions generation by generation A GA flow in one Generation Evaluation Selection Crossover Mutation Replacement Convergence on expected condition

6. Basic Knowledge (2) Estimation of distribution algorithms (EDAs) Or probabilistic model-building genetic algorithm (PMBGAs) Extend of traditional GAs GAs with linkage learning and model building (not exactly) Main idea Extract information from promising solutions Exploit information to build probabilistic model Generate next population from building model.

7. Basic Knowledge (3) Classes of EDAs based on linkage learning mechanisms Univariate No linkage between any variables cGA, PBIL, UMDA Bivariate Most two variables with linkage BMDA, MIMIC Multivariate K variables with linkage Detect from 2 order to k order. ECGA, BOA, hBOA, EBNA, DSMGA

8. Basic Knowledge (4) cGA (univariate EDA) Vector [P1,P2,P3,P4] = [0.5, 0.5,0.5,0.5] Generate two individuals 0 1 1 1 (f=3) 1 0 0 1 (f=2) Update Vector=[0.25,0.75,0.75,0.5] ECGA (multivariate EDA) [0] [1] [2] [3] -> [0,1] [2] [3] -> [0,2] [1] [3] -> [0,2,1] [3] [0,2] [1] [3] [0,2,3] [1] [0,3] [1] [2]

9. Motivation (1) 1 2 4 5 31 2 4 5 3

10. Motivation (2) Innovation time Crossover operator achieve a solution better than any solutions at this point. Takeover time Selection operator converge a solution If innovation time < takeover time New innovation will be generated If innovation time > takeover time Results in premature convergence. Innovation time is heavily affected by linkage model (1) (2)

11. Motivation (3) The precision of the linkage model Model Accuracy Overfitting Spurious linkage Underfiiting Missing important linkage (???) Even without important linkage, EDA can solve problem under polynomial time What is necessary linkage ? Define linkage by Nfe (number of function evaluation) The bonding will be probabilistic? Sometime link, or sometime not If there exists a nearly decomposable function to make inaccuracy linkage Parity Function?

12. CPF Function Concatenated Parity Function 0010 1101 1100 0001 1001 Length = 20, k=4, m=5 2 3 22 33 3 CPF(X) = X =

13. CP/TF Function Concatenated Parity / Trap Function 011 001 110 110 000 000 0 2 3 2 1 CPF(X) = X = 3 2 111 110 3 0 001 000 1 2 1 1 1 0 0 0 2

14. Wash Function

15. Goal of this paper (1) Coffin and Smith found parity functions, where variables appear to be independent when observing only two of them, to fail EDAs. It was believed to be difficult for EDAs. Chen and Yu found that cGA and ECGA can solve parity functions in the polynomial time. cGA, without linkage learning, can solve parity function. The author think that parity function can’t present the allelic pairwise independent functions.

16. Goal of this paper (2) Author says, “This paper focuses on settling the open question, whether or not the heavy reliance on pairwise exploitation in present EDAs implies a weakness on linkage learning for some nearly decomposable problems --- a class of problems for which EDAs are considered well-suited.” The author think “CPF” is too easy to be the typical problems. Therefore, his goal is to propose a new allelic pairwise independent functions to fail current EDAs

17. From Teil’s View True EDAs(hBOA) cannot solve CPF efficiently. cGA and ECGA can solve CPF efficiently. Give correct linkage model, EDAs can solve CPF efficiently (???) [Author’s view] CPF is easy, because cGA can solve it. Find a new functions to fail ECGA (or other EDAs). ECGA with pairwise linkage learning must fail. Must give correct linkage, then ECGA can solve. (???) [Teil’s View] CPF is easy, because cGA can solve it. Other EDAs can’t solve CPF? Why? => Find the reason. We found the replacement mechanism.

18. The summary of this paper’s flow EDAs Pairwise Linkage Detect Computational must CPF without pairwise linkage Uni- variae Multi- variae (2)Easy (2)Hard (?) Some famous nearly decomposable problems New Function (3)CPF Is easy Nearly Decomposable Problems (1)Well (4)Hard (5) Weakness

19. Hierarchical Pairwise Allelic Independent Function (1) 11 -> 1 00 -> 0 Other -> -

20. Hierarchical Pairwise Allelic Independent Function (2) 0010 1101 0011 0001 1001 0 0 101 --- 1 0

21. CPF and HPAIF

22. Experiment

23. Conclusion EDAs on some parity function may scale exponentially. cGA and ECGA can scale CPF on polynomial time. CPF, CP/TF, Walsh Function may be easy on allelic pairwise independent function The author proposed a new function to detect if EDAs with pairwise detection has weakness. ECGA failed with it. Current EDAs with pairwise linkage learning has weakness on some nearly decomposable function. He propose a new allelic pairwise independent function to support this.

24. Discussion There is no need to design a new testing function that most EDAs cannot solve. Moreover, the only way to solve this testing function is to give EDAs the information of linkage. The author said that CPF is easy testing problem. Hence, ECGA can solve CPF. However, no explain on why hBOA failed on it. If we believe that CPF is an easy testing function. We should first clear why hBOA failed on CPF