Evolutionary Computational Intelligence Lecture 10a: Surrogate Assisted Ferrante Neri University of Jyväskylä

Computationally expensive problems Optimization problems can be computationally expensive because of two reasons: high cardinality decision space (usually combinatorial) computationally expensive fitness function (e.g. design of on-line electric drives)

High Cardinality Decision Space Under such conditions it should be tried, on the basis of the application, to reduce the cardinality by means of an ”a priori” analysis or an heuristic to detect a promising region of the decision space Memetic approach (e.g. intelligent initial sampling) can be beneficial

Computationally expensive fitness It might happen that the fitness function evaluation requires by itself a lot of computational effort (e.g. in online PMSM drive design each fitness evaluation requires 8 s) In such conditions it should be found a way to reduce the numer of fitness evaluations and still reach the optimum

Surrogate Assisted Algorithms Surrogate Assisted Algorithms employ approximated models of the fitness function (cheap) alternatively with the real fitness (expensive) One of the crucial problems is the model to be employed and how to arrange such a combination

Global vs Local Surrogate models There are two complementary and contrasting algorithmic philosophy: – Global Surrogate Models: attempt of finding an approximated model of the landscape over all the decision space – Local Surrogate Models: attempt of approximating locally the landscape over the neighborhood of a certain point

Comparison between the two philosophies Global models assume that a wide knowledge of the decision space allows to build up an accurate model that can be employed as a cheap alternative of the real fitness Local models assume that a huge amount of information does not help in determining an accurate model and thus it is preferable to build up models that approximate only locally the behavior of the landscape Global models employ one very complex model, Local models employ many simple approximated functions

Coordination of models/real fitness The right way to perform the coordination is very problem dependent, both deterministic and stocastic rules have been implemented Models can be ”installed” in both evolutionary framework and local searchers

Surrogate Assisted Hooke-Jeeves Algorithm Surrogate Assisted Hooke Jeeves Algorithm (SAHJA): deterministic scheme for coordinating real fitness and a linear model obtained by least square method Computes N+1 points and generates a local linear model for calculating the remaining N points (Cost of exploratory move is thus kept constant) Check every directional move, by calculating the real fitness if a surrogate was prevously calculated (does not allow search directions by means of surrogate points)

SAHJA

SAHJA Results Very promising algorithmic performance Noise filtering

Evolutionary Computational Intelligence Lecture 10b: Experimentalism Ferrante Neri University of Jyväskylä

Goals To propose a research protocol in order to execute a fair experimental comparison which allows us to check whether the newly proposed algorithm outperforms the methods existing in literature In other words, if I designed a novel algorithm how can I be sure that my work outperform (for a certain problem) the state of the art?

Towards Performance Comparison If I designed a novel algorithm B how can I prove that B outperforms the benchmark algorithm A? How can I thus have a confirmation that the novel algorithmic component is really effective? Performance is an abstract concept not related to a specific machine. It is the capability of an algorithm of reaching a good performing solution in a certain time interval. The time trigger is the number of fitness (functional) evaluations

Experimental Setup For both A and B, a certain number n of runs must be performed The average best fitness values (e.g. at the end of each generation) must be saved N.B. for making the trends comparable an interpolation can be necessary Standard deviation bars can also be included

Two Possible kinds of outperforming Case 1: A and B converge on different final values Case 2: A and B converge on the same final values but with different convergence velocities

…Case 1 The data define two Tolerance Intervals (TIs) It is fixed a desired confidence level δ The proportion γ of a set of data which falls within a given interval with a given confidence level δ are determined by: γ =1 −a/n where n is the number of available samples and a is the positive root of the equation (1 + a) − (1 − δ) · e a = 0

…Case 2 A threshold value f thr is fixed. If during an experiment f best < f thr then the algorithm ”almost converged” For the n experiments, the number of fitness evaluations necessary to verify the inequality f best < f thr defines a TI The probability γ that an algorithm requires no more fitness evaluations than the most unlucky case is given by: γ =1 −d/n where n is the number of the available experiments. d is given by d = −ln(1 − δ)

How to conclude In both the cases, if the tolerance intervals are not separated it is impossible to establish that B outperforms A in all the cases. In this case it is possible only to state that B outperforms A in average