1. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 1 Optimization in Dynamic Environments Ernesto Costa DEI/CISUC ernesto@dei.uc.pt http://www.dei.uc.pt/~ernesto/

2. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 2 Summary Agents, Problems and Environments Agents: Natural Selection and Genetics Problems:Optimization Environments: Dynamic Optimization and Dynamic Environments State of the Art The Challenge / Problem

3. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 3 Agents, Problems and Environments Agent Environment Problem Behavior Performance

4. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 4 Agents and Evolutionary Computation Darwin Evolution by Natural selection Mendel Genetics and Inheritance procedure EC t = 0; inicialization P(t); evaluation P(t); while not stop_condition do t = t+1; P 1 (t) = selection (P(t-1)); P 2 (t) = op_modification (P 1 (t)); evaluation (P 2 (t)); P(t) = combine (P 2 (t),P(t-1)); End_do; return_best (P(t)); end_proc.

5. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 5 Problems that: have a finite number, F, of feasible solutions each solution has na associate cost, c. goal: a solution f in F that minimizes c Examples: knapsack minimum spanning tree bin packing set covering vehicle routing... Problems: Combinatorial Optimization

6. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 6 W[i], P[i] w 1, p 1 w 2, p 2 w 3, p 3. w n, p n Items 0/1 Knapsack Choose the items that maximize your profit ans such that the total weight is less that some given limit (knapsack capacity)! Combinatorial Optimization Binary representation: vector x=(x 1,...,x n )

7. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 7 Minimum Spanning Tree Given G=(V,E): a connected weighted undirected graph V={v 1,...,v n } E={e 1,...,e m } W={w 1,...,w m }: weight or cost of each edge Find a subgraph S of G : S contains all the vertices of G S is connected and contains no cycles S has minimum cost A minimum spaning tree (MST) Combinatorial Optimization (2a) 5 4 4 5 5 4 3 3 33 6 2

8. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 8 Problems: Function Optimization Rastringin Function n=2, A=10

9. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 9 Environment: Dynamic Changes in the environment: Restrictions: Knapsack capacity C Goal: Rastringin Parameter A Problem Instance: MST #V, #E, W A different, time dependent, fitness landscape!

10. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 10 Environment: Dynamic Types of dynamics Discrete vs Continuous Periodic vs Non-Periodic cycle length Dimension of change small vs big Predictability of change

11. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 11 Change detection Explicitly known Average or best fitness drop Reevaluating a set of individuals every generation Keep a model of the environment (model and real ≠) Does the EA change (e.g. representation)? Environment: Dynamic Further Aspects

12. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 12 State of the Art The problem Standard EA loose diversity (converge to an optimum) No memory of the past Solution Start from scratch??? New optimization algorithm (new Agent) Kind of open-ended evolution Using past information Diversity Memory The challenge!!!

13. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 13 State of the Art Promoting Diversity Hypermutation Maintaining Diversity Avoid convergence Random immigrants Use of Memory Redundant Representations Multiploidy Explicit Memory Interplay between memory and the evolving population Multiple Populations Self-Adaptive Memory

14. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 14 The Challenge Choose a problem Modify the standard Genetic Algorithm Diversity mechanisms Memory mechanism Make Experiments with (some) previous approaches Analyse Results Propose New Solutions New Results?!

15. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 15 Moving Parabola The problem: Benchmarks

16. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 16 The problem: Benchmarks Having several peaks Position Height Width Changes One or several parameters Possible to test different dynamics A C-version available (Jurgen Branke) A Matlab version (R. Morrison) Moving Peaks

17. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 17 The problem: Measuring Performance On-line performance The average of the averages so far Off-line performance The average of the best so far Best-of-generation averages for many runs on the same problem Question: we want to measure the performance of the EA across the entire range of the fitness landscape dynamics

18. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 18 The IS is a complex system that includes cells, molecules and organs that constitutes an identification mechanism capable of perceiving and combating: dysfunction of our own cells (infectious self) action of exogenous infectious microorganisms (infectious non-self) The IS insures the integrity of the self! Immune System Other Ideas

19. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 19 Invasion Detection Reaction Maturation Memorizing Immune System How it works? Other Ideas Challenge : can we use it for dynamic environments???

20. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 20 Let’s Work! References Evolutionary Optimization in Dynamic Environments, Jürgen Branke, Kluwer Academic Publishers,2002. Evolutionary Algorithms for Dynamic optimization Problems (EvoDOP 2003) in GECCO 2003, Jürgen Branke (Organizer) http://www.aifb.uni-karlsruhe.de/~jbr/MovPeaks/

21. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 21 0/1 Knapsack Environment: Dynamic Restrictions: changing the knapsack capacity, C

22. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 22 5 4 4 5 5 4 3 3 33 6 2 Minimum Spanning Tree Environment: Dynamic 4 8 11 7 12 2 6 4 7 9 10 14 8 Problem Instance: different vertices, edges and weights

23. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 23 A=1..9 Rastringin Function Environment: Dynamic Goal: different Max

24. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 24 The problem: Measuring Performance Question: we want to measure the performance of the EA across the entire range of the fitness landscape dynamics Adapting the offline performance Moment of changes are known Using the error Optimum is known Current error Offline error Dynamic Environments

25. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 25 The problem: Measuring Performance Accuracy: recovery capacity Adaptability: speed of recovery K= # changes during the run r= # generations between two consecutive changes Err i,j = difference between current best at generation j after change #i and the optimum Dynamic Environments

26. DEI/CISUC Evonet Summer School - Parma © 2003 Ernesto Costa 26 Minimum Spanning Tree Formally Any subgraph S can be represented by a binary vector x={x 1,...x m }, with x i = 1 if e i is in S If T is the set of all spanning trees in G then the MST is defined by: Combinatorial Optimization (2b)