Managing Director / CTO NuTech Solutions GmbH / Inc. Martin-Schmeißer-Weg 15 D – Dortmund Tel.: +49 (0) 231 / Fax: +49 (0) 231 / Thomas Bäck October 11, 2004 Evolution Strategies: A Different Type of EC and its Applications Natural Computing Leiden Institute for Advanced Computer Science (LIACS) Niels Bohrweg 1 NL-2333 CA Leiden Tel.: +31 (0) Fax: +31 (0)

2 AlgorithmsTheoryExamples Overview Other Background I Daniel Dannett: Biology = Engineering

3 AlgorithmsTheoryExamples Overview Other Background II Realistic Scenario....

4 AlgorithmsTheoryExamples Overview Other Background III Phenotypic evolution... and genotypic

5 AlgorithmsTheoryExamples Overview Other Optimization E.g. costs (min), quality (max), error (min), stability (max), profit (max),... Difficulties: High-dimensional Nonlinear, non-quadratic: Multimodal Noisy, dynamic, discontinuous Evolutionary landscapes are like that !

6 AlgorithmsTheoryExamples Overview Other Overview  Evolutionary Algorithm Applications: Examples  Evolutionary Algorithms: Some Algorithmic Details  Genetic Algorithms  Evolution Strategies  Some Theory of EAs  Convergence Velocity Issues  Other Examples  Drug Design  Inverse Design of Cas  Summary

7 AlgorithmsTheoryExamples Overview Other Modeling – Simulation - Optimization !!! ??? !!! Simulation Modeling / Data Mining Optimization !!! ??? !!!

8 AlgorithmsTheoryExamples Overview Other General Aspects Evaluation EA-OptimizerBusiness Process Model Simulation FunctionModel from Data ExperimentSubjectiveFunction(s)

9 AlgorithmsTheoryExamples Overview Other Examples I: Inflatable Knee Bolster Optimization Support plate FEM #4 Initial position of knee bag modeldeployed knee bag (unit only) Volume of 14L Load distribution plate Tether Support plate Vent hole MAZDA Picture Load distribution plate FEM #3 Tether FEM #5 Knee bag FEM #2 Straps are defined in knee bag(FEM #2) Low Cost ES: GA (Ford): 0.72 Hooke Jeeves DoE: 0.88 Low Cost ES: GA (Ford): 0.72 Hooke Jeeves DoE: 0.88

10 AlgorithmsTheoryExamples Overview Other IKB: Previous Designs

11 AlgorithmsTheoryExamples Overview Other Objective:Min PtotalSubject to: Left Femur load <= 7000 Right Femur load <= 7000 IKB: Problem Statement

12 AlgorithmsTheoryExamples Overview Other Quality: Simulations: 160 IKB Results I: Hooke-Jeeves

13 AlgorithmsTheoryExamples Overview Other Quality: Simulations: 122 IKB Results II: (1+1)-ES

14 AlgorithmsTheoryExamples Overview Other Optical Coatings: Design Optimization  Nonlinear mixed-integer problem, variable dimensionality.  Minimize deviation from desired reflection behaviour.  Excellent synthesis method; robust and reliable results.

15 AlgorithmsTheoryExamples Overview Other  Dielectric filter design.  n=40 layers assumed.  Layer thicknesses x i in [0.01, 10.0].  Quality function: Sum of quadratic penalty terms.  Penalty terms = 0 iff constraints satisfied. Client: Corning, Inc., Corning, NY Dielectric Filter Design Problem

16 AlgorithmsTheoryExamples Overview Other Benchmark Results: Overview of Runs  Factor 2 in quality.  Factor 10 in effort.  Reliable, repeatable results.

17 AlgorithmsTheoryExamples Overview Other  Grid evaluation for 2 variables.  Close to the optimum (from vector of quality ).  Global view (left), vs. Local view (right). Problem Topology Analysis: An Attempt

18 AlgorithmsTheoryExamples Overview Other 18 Speed Variables (continuous) for Casting Schedule Turbine Blade after Casting  FE mesh of 1/3 geometry: nodes, tetrahedrons, radiation surfaces large problem:  run time varies: 16 h 30 min to 32 h (SGI, Origin, R12000, 400 MHz)  at each run: 38,3 GB of view factors (49 positions) are treated! Examples II: Bridgman Casting Process

19 AlgorithmsTheoryExamples Overview Other Quality Comparison of the Initial and Optimized Configurations Initial (DoE) GCM(Commercial Gradient Based Method) Evolution Strategy Global Quality Turbine Blade after Casting Examples II: Bridgman Casting Process

20 AlgorithmsTheoryExamples Overview Other  Generates green times for next switching schedule.  Minimization of total delay / number of stops.  Better results (3 – 5%) / higher flexibility than with traditional controllers.  Dynamic optimization, depending on actual traffic (measured by control loops). Client: Dutch Ministry of Traffic Rotterdam, NL Examples IV: Traffic Light Control

21 AlgorithmsTheoryExamples Overview Other  Minimization of passenger waiting times.  Better results (3 – 5%) / higher flexibility than with traditional controllers.  Dynamic optimization, depending on actual traffic. Client: Fujitec Co. Ltd., Osaka, Japan Examples V: Elevator Control

22 AlgorithmsTheoryExamples Overview Other  Minimization of defects in the produced parts.  Optimization on geometric parameters and forces.  Fast algorithm; finds very good results. Client: AutoForm Engineering GmbH, Dortmund Examples VI: Metal Stamping Process

23 AlgorithmsTheoryExamples Overview Other  Minimization of end-to-end-blockings under service constraints.  Optimization of routing tables for existing, hard-wired networks.  10%-1000% improvement. Client: SIEMENS AG, München Examples VII: Network Routing

24 AlgorithmsTheoryExamples Overview Other  Minimization of total costs.  Creates new fuel assembly reload patterns.  Clear improvements (1%-5%) of existing expert solutions.  Huge cost saving. Client: SIEMENS AG, München Examples VIII: Nuclear Reactor Refueling

25 AlgorithmsTheoryExamples Overview Other Experimental design optimisation: Optimise efficieny. Initial design Final design: 32% improvement in efficieny.... evolves... Two-Phase Nozzle Design (Experimental)

26 AlgorithmsTheoryExamples Overview Other Multipoint Airfoil Optimization (1) High Lift! Low Drag! Start Cruise Client: 22 design parameters.

27 AlgorithmsTheoryExamples Overview Other Multipoint Airfoil Optimization (2) Pareto set after 1000 Simulations Three compromise wing designs Find pressure profiles that are a compromise between two given target pressure distributions under two given flow conditions!

28 AlgorithmsTheoryExamples Overview Other Evolutionary Algorithms: Some Algorithmic Details

29 AlgorithmsTheoryExamples Overview Other Unifying Evolutionary Algorithm t := 0; initialize(P(t)); evaluate(P(t)); while not terminate do P‘(t) := mating_selection(P(t)); P‘‘(t) := variation(P‘(t)); evaluate(P‘‘(t)); P(t+1) := environmental_selection(P‘‘(t) u Q); t := t+1; od

30 AlgorithmsTheoryExamples Overview Other Evolutionary Algorithm Taxonomy Evolution Strategies Genetic Algorithms Genetic Programming Evolutionary Programming Classifier Systems Many mixed forms; agent-based systems, swarm systems, A-life systems,...

31 AlgorithmsTheoryExamples Overview Other  Real-valued representation  Normally distributed mutations  Fixed recombination rate (= 1)  Deterministic selection  Creation of offspring surplus  Self-adaptation of strategy parameters: Variance(s), Covariances  Binary representation  Fixed mutation rate p m (= 1/n)  Fixed crossover rate p c  Probabilistic selection  Identical population size  No self-adaptation Genetic Algorithm Evolution Strategies Genetic Algorithms vs. Evolution Strategies

32 AlgorithmsTheoryExamples Overview Other Genetic Algorithms  Often binary representation.  Mutation by bit inversion with probability p m.  Various types of crossover, with probability p c.  k -point crossover.  Uniform crossover.  Probabilistic selection operators.  Proportional selection.  Tournament selection.  Parent and offspring population size identical.  Constant strategy parameters.

33 AlgorithmsTheoryExamples Overview Other Mutation  Mutation by bit inversion with probability p m.  p m identical for all bits.  p m small (e.g., p m = 1/l ).

34 AlgorithmsTheoryExamples Overview Other Crossover  Crossover applied with probability p c.  p c identical for all individuals.  k -point crossover: k points chosen randomly.  Example: 2-point crossover.

35 AlgorithmsTheoryExamples Overview Other Selection  Fitness proportional:  f fitness  population size  Tournament selection:  Randomly select q << individuals.  Copy best of these q into next generation.  Repeat times.  q is the tournament size (often: q = 2 ).

36 AlgorithmsTheoryExamples Overview Other Evolution Strategies  Real-valued representation.  Normally distributed mutations.  Various types of recombination.  Discrete (exchange of variables).  Intermediate (averaging).  Involving two or more parents.  Deterministic selection, offspring surplus .  Elitist: (  )  Non-elitist: (  )  Self-Adaptation of strategy parameters.

37 AlgorithmsTheoryExamples Overview Other Mutation  -adaptation by means of –1/5-success rule. –Self-adaptation. Creation of a new solution: Convergence speed:  Ca. 10  n down to 5  n is possible. More complex / powerful strategies: –Individual step sizes  i. –Covariances.

38 AlgorithmsTheoryExamples Overview Other Self-Adaptation  Learning while searching: Intelligent Method.  Different algorithmic approaches, e.g: Pure self-adaptation: Mutational step size control MSC: Derandomized step size adaptation Covariance adaptation

39 AlgorithmsTheoryExamples Overview Other Self-Adaptive Mutation n = 2, n  = 1, n  = 0 n = 2, n  = 2, n  = 0 n = 2, n  = 2, n  = 1

40 AlgorithmsTheoryExamples Overview Other Self-Adaptation:  Motivation: General search algorithm  Geometric convergence: Arbitrarily slow, if s wrongly controlled !  No deterministic / adaptive scheme for arbitrary functions exists.  Self-adaptation: On-line evolution of strategy parameters.  Various schemes:  Schwefel one , n , covariances; Rechenberg MSA.  Ostermeier, Hansen: Derandomized, Covariance Matrix Adaptation.  EP variants (meta EP, Rmeta EP).  Bäck: Application to p in GAs. Step size Direction

41 AlgorithmsTheoryExamples Overview Other Self-Adaptation: Dynamic Sphere  Optimum  :  Transition time proportionate to n.  Optimum  learned by self- adaptation.

42 AlgorithmsTheoryExamples Overview Other Selection (  ) (  )

43 AlgorithmsTheoryExamples Overview Other Possible Selection Operators  (1+1)-strategy: one parent, one offspring.  (1, )-strategies: one parent, offspring. Example: (1,10)-strategy. Derandomized / self-adaptive / mutative step size control.  ( , )-strategies:  >1 parents,  >  offspring Example: (2,15)-strategy. Includes recombination. Can overcome local optima.  (  + )-strategies: elitist strategies.

44 AlgorithmsTheoryExamples Overview Other Advantages of Evolution Strategies  Self-Adaptation of strategy parameters.  Direct, global optimizers !  Faster than GAs !  Extremely good in solution quality.  Very small number of function evaluations.  Dynamical optimization problems.  Design optimization problems.  Discrete or mixed-integer problems.  Experimental design optimisation.  Combination with Meta-Modeling techniques.

45 AlgorithmsTheoryExamples Overview Other Some Theory of EAs

46 AlgorithmsTheoryExamples Overview Other Robust vs. Fast:  Global convergence with probability one: General, but for practical purposes useless.  Convergence velocity: Local analysis only, specific functions only.

47 AlgorithmsTheoryExamples Overview Other Convergence Velocity Analysis, ES:  A convex function („sphere model“).  Simplest case: (  )-ES  Illustration: (1,4)-ES

48 AlgorithmsTheoryExamples Overview Other Convergence Velocity Analysis, ES:  Order statistics:  p  (z) denotes the p.d.f. of Z   Idea: Best offspring has smallest r / largest z‘.  The following holds from geometric considerations:  One gets:

49 AlgorithmsTheoryExamples Overview Other Convergence Velocity Analysis, ES:  Using one finally gets  One gets: (dimensionless)

50 AlgorithmsTheoryExamples Overview Other Convergence Velocity Analysis, ES:  Convergence velocity, (  )-ES and (  )-ES:

51 AlgorithmsTheoryExamples Overview Other Convergence Velocity Analysis, GA:  (1+1)-GA, (1, )-GA, (1+ )-GA.  For counting ones function:  Convergence velocity:  Mutation rate p, q = 1 – p, k max = l – f a.

52 AlgorithmsTheoryExamples Overview Other Convergence Velocity Analysis, GA:  Optimum mutation rate ?  Absorption times from transition matrix in block form, usingwhere

53 AlgorithmsTheoryExamples Overview Other Convergence Velocity Analysis, GA:  p too large: Exponential  p too small: Almost constant.  Optimal: O(l ln l). p

54 AlgorithmsTheoryExamples Overview Other Convergence Velocity Analysis, GA:  (1, )-GA ( k min = -f a ), (1+ )-GA ( k min = 0 ) :

55 AlgorithmsTheoryExamples Overview Other Convergence Velocity Analysis, EA: (1, )-GA, (1+ )-GA:(1, )-ES, (1+ )-ES: Conclusion: Unifying, search-space independent theory !?

56 AlgorithmsTheoryExamples Overview Other Current Drug Targets: GPCR

57 AlgorithmsTheoryExamples Overview Other Goals (in Cooperation with LACDR):  CI Methods:  Automatic knowledge extraction from biological databases – fuzzy rules.  Automatic optimisation of structures – evolution strategies.  Exploration for  Drug Discovery,  De novo Drug Design. Charge distribution on VdW surface of CGS15943 “Fingerprint” New derivative with good receptor affinity. Initialisation Final (optimized)

58 AlgorithmsTheoryExamples Overview Other Evolutionary DNA-Computing (with IMB):  DNA-Molecule = Solution candidate !  Potential Advantage: > candidate solutions in parallel.  Biological operators:  Cutting, Splicing.  Ligating.  Amplification.  Mutation.  Current approaches very limited.  Our approach:  Suitable NP-complete problem.  Modern technology.  Scalability (n > 30).

59 AlgorithmsTheoryExamples Overview Other UP of CAs (= Inverse Design of CAs)  1D CAs: Earlier work by Mitchell et al., Koza,...  Transition rule: Assigns each neighborhood configuration a new state.  One rule can be expressed by bits.  There are rules for a binary 1D CA Neighborhood (radius r = 2)

60 AlgorithmsTheoryExamples Overview Other UP of CAs (rule encoding)  Assume r=1: Rule length is 8 bits  Corresponding neighborhoods

61 AlgorithmsTheoryExamples Overview Other Inverse Design of CAs: 1D  Time evolution diagram:

62 AlgorithmsTheoryExamples Overview Other Inverse Design of CAs: 1D  Majority problem:  Particle-based rules.  Fitness values: 0.76, 0.75, 0.76, 0.73

63 AlgorithmsTheoryExamples Overview Other Inverse Design of CAs: 1D Don‘t care about initial state rules Block expanding rules Particle communication based rules

64 AlgorithmsTheoryExamples Overview Other Inverse Design of CAs: 1D Majority Records  Gacs, Kurdyumov, Levin 1978 (hand-written):81.6%  Davis 1995 (hand-written):81.8%  Das 1995 (hand-written):82.178%  David, Forrest, Koza 1996 (GP):82.326%

65 AlgorithmsTheoryExamples Overview Other Inverse Design of Cas: 2D  Generalization to 2D (nD) CAs ?  Von Neumann vs. Moore neighborhood (r = 1)  Generalization to r > 1 possible (straightforward)  Search space size for a GA: vs

66 AlgorithmsTheoryExamples Overview Other Inverse Design of CAs  Learning an AND rule.  Input boxes are defined.  Some evolution plots:

67 AlgorithmsTheoryExamples Overview Other Inverse Design of CAs  Learning an XOR rule.  Input boxes are defined.  Some evolution plots:

68 AlgorithmsTheoryExamples Overview Other Inverse Design of CAs  Learning the majority task.  84/169 in a), 85/169 in b).  Fitness value: 0.715

69 AlgorithmsTheoryExamples Overview Other Inverse Design of CAs  Learning pattern compression tasks.

70 AlgorithmsTheoryExamples Overview Other Evolution = Computation ? Yes Search & Optimization are fundamental problems / tasks in many applications (learning, engineering,...).

71 AlgorithmsTheoryExamples Overview Other Summary  Explicative models based on Fuzzy Rules.  Descriptive models based on e.g. Kriging method.  Few data points necessary, high modeling accuracy.  Used in product design, quality control, management decision support, prediction and optimization.  Optimization based on Evolution Strategies (and traditional methods).  Few function evaluations necessary.  Robust, widely usable, excellent solution quality.  Self-adaptivity (easy to use !).  Patents: US no. 5,826,251; Germany no , ,

72 AlgorithmsTheoryExamples Overview Other Questions ? Thank you very much for your time !