1. Andrea G. B. Tettamanzi, 2002 Evolutionary Algorithms Andrea G. B. Tettamanzi

2. Andrea G. B. Tettamanzi, 2002 Contents of the Lectures Taxonomy and History; Evolutionary Algorithms basics; Theoretical Background; Outline of the various techniques: plain genetic algorithms, evolutionary programming, evolution strategies, genetic programming; Practical implementation issues; Evolutionary algorithms and soft computing; Selected applications from the biological and medical area; Summary and Conclusions.

3. Andrea G. B. Tettamanzi, 2002 Bibliography  Th. Bäck. Evolutionary Algorithms in Theory and Practice. Oxford University Press, 1996  L. Davis. The Handbook of Genetic Algorithms. Van Nostrand & Reinhold, 1991  D.B. Fogel. Evolutionary Computation. IEEE Press, 1995  D.E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, 1989  J. Koza. Genetic Programming. MIT Press, 1992  Z. Michalewicz. Genetic Algorithms + Data Structures = Evolution Programs. Springer Verlag, 3rd ed., 1996  H.-P. Schwefel. Evolution and Optimum Seeking. Wiley & Sons, 1995  J. Holland. Adaptation in Natural and Artificial Systems. MIT Press 1995

4. Andrea G. B. Tettamanzi, 2002 Taxonomy (1) Genetic Algorithms Evolutionary Algorithms Evolutionary Programming Evolution Strategies Genetic Programming Simulated Annealing Taboo Search Monte Carlo methods Stochastic optimization methods

5. Andrea G. B. Tettamanzi, 2002 Taxonomy (2) Distinctive features of Evolutionary Algorithms: operate on appropriate encoding of solutions; population search; no regularity conditions requested; probabilistic transitions.

6. Andrea G. B. Tettamanzi, 2002 History (1) I. Rechenberg, H.-P. Schwefel TU Berlin, ‘60s John H. Holland University of Michigan, Ann Arbor, ‘60s L. Fogel UC S. Diego, ‘60s John Koza Stanford University ‘80s

7. Andrea G. B. Tettamanzi, 2002 History (2) 1859 Charles Darwin: inheritance, variation, natural selection 1957 G. E. P. Box: random mutation & selection for optimization 1958 Fraser, Bremermann: computer simulation of evolution 1964 Rechenberg, Schwefel: mutation & selection 1966 Fogel et al.: evolving automata - “evolutionary programming” 1975 Holland: crossover, mutation & selection - “reproductive plan” 1975 De Jong: parameter optimization - “genetic algorithm” 1989 Goldberg: first textbook 1991 Davis: first handbook 1993 Koza: evolving LISP programs - “genetic programming”

8. Andrea G. B. Tettamanzi, 2002 Evolutionary Algorithms Basics what an EA is (the Metaphor) object problem and fitness the Ingredients schemata implicit parallelism the Schema Theorem the building blocks hypothesis deception

9. Andrea G. B. Tettamanzi, 2002 The Metaphor EnvironmentObject problem Individual Fitness Candidate solution Quality EVOLUTIONPROBLEM SOLVING

10. Andrea G. B. Tettamanzi, 2002 Object problem and Fitness genotype solution M object problem s f fitness

11. Andrea G. B. Tettamanzi, 2002 The Ingredients t t + 1 mutation recombination reproduction selection

12. Andrea G. B. Tettamanzi, 2002 The Evolutionary Cycle Recombination Mutation Population OffspringParents Selection Replacement Reproduction

13. Andrea G. B. Tettamanzi, 2002 Pseudocode generation = 0; SeedPopulation(popSize);// at random or from a file while(!TerminationCondition()) { generation = generation + 1; CalculateFitness();//... of new genotypes Selection();// select genotypes that will reproduce Crossover(p cross );// mate p cross of them on average Mutation(p mut );// mutate all the offspring with Bernoulli // probability p mut over genes }

14. Andrea G. B. Tettamanzi, 2002 A Sample Genetic Algorithm The MAXONE problem Genotypes are bit strings Fitness-proportionate selection One-point crossover Flip mutation (transcription error)

15. Andrea G. B. Tettamanzi, 2002 The MAXONE Problem Problem instance: a string of l binary cells,  l  : Objective: maximize the number of ones in the string. Fitness:

16. Andrea G. B. Tettamanzi, 2002 Fitness Proportionate Selection Implementation: “Roulette Wheel” Probability of  being selected: 

17. Andrea G. B. Tettamanzi, 2002 One Point Crossover 00000111110110101010 crossover point 01000010100010111111 parentsoffspring

18. Andrea G. B. Tettamanzi, 2002 Mutation 1110101010 p mut 0110101011 independent Bernoulli transcription errors

19. Andrea G. B. Tettamanzi, 2002 Example: Selection 0111011011f = 7Cf = 7P = 0.125 1011011101f = 7Cf = 14P = 0.125 1101100010f = 5Cf = 19P = 0.089 0100101100f = 4Cf = 23P = 0.071 1100110011f = 6Cf = 29P = 0.107 1111001000f = 5Cf = 34P = 0.089 0110001010f = 4Cf = 38P = 0.071 1101011011f = 7Cf = 45P = 0.125 0110110000f = 4Cf = 49P = 0.071 0011111101f = 7Cf = 56P = 0.125 Random sequence: 43, 1, 19, 35, 15, 22, 24, 38, 44, 2

20. Andrea G. B. Tettamanzi, 2002 Example: Recombination & Mutation 0111011011  0111011011  0111111011f = 8 0111011011  0111011011  0111011011f = 7 110|1100010  1100101100  1100101100f = 5 010|0101100  0101100010  0101100010f = 4 1|100110011  1100110011  1100110011f = 6 1|100110011  1100110011  1000110011f = 5 0110001010  0110001010  0110001010f = 4 1101011011  1101011011  1101011011f = 7 011000|1010  0110001011  0110001011f = 5 110101|1011  1101011010  1101011010f = 6 TOTAL = 57

21. Andrea G. B. Tettamanzi, 2002 Schemata Don’t care symbol:   1  10 a schema S matches 2 l - o(S) strings a string of length l is matched by 2 l schemata order of a schema: o(S) = # fixed positions defining length  (S) = distance between first and last fixed position

22. Andrea G. B. Tettamanzi, 2002 Implicit Parallelism In a population of n individuals of length l 2 l  # schemata processed  n2 l n 3 of which are processed usefully (Holland 1989) (i.e. are not disrupted by crossover and mutation) But see Bertoni & Dorigo (1993) “Implicit Parallelism in Genetic Algorithms” Artificial Intelligence 61(2), p. 307  314

23. Andrea G. B. Tettamanzi, 2002 Fitness of a schema f(  ): fitness of string  q x (  ): fraction of strings equal to  in population x q x (S): fraction of strings matched by S in population x

24. Andrea G. B. Tettamanzi, 2002 The Schema Theorem {X t } t=0,1,... populations at times t suppose thatis constant i.e. above-average individuals increase exponentially!

25. Andrea G. B. Tettamanzi, 2002 The Schema Theorem (proof)

26. Andrea G. B. Tettamanzi, 2002 The Building Blocks Hypothesis ‘‘An evolutionary algorithm seeks near-optimal performance through the juxtaposition of short, low-order, high-performance schemata — the building blocks’’

27. Andrea G. B. Tettamanzi, 2002 Deception i.e. when the building block hypothesis does not hold: butfor some schema S, Example:  * = 1111111111 S 1 = 111******* S 2 = ********11 S = 111*****11 S = 000*****00

28. Andrea G. B. Tettamanzi, 2002 Remedies to deception Prior knowledge of the objective function Non-deceptive encoding Inversion Semantics of genes not positional “Messy Genetic Algorithms” Underspecification & overspecification

29. Andrea G. B. Tettamanzi, 2002 Theoretical Background Theory of random processes; Convergence in probability; Open question: rate of convergence.

30. Andrea G. B. Tettamanzi, 2002 Events  Sample space A B  D

31. Andrea G. B. Tettamanzi, 2002 Random Variables   0 X

32. Andrea G. B. Tettamanzi, 2002 Stochastic Processes A sequence of r.v.’s Each with its own probability distribution. Notation:

33. Andrea G. B. Tettamanzi, 2002 EAs as Random Processes a sample of size n probability space “random numbers” trajectory evolutionary process

34. Andrea G. B. Tettamanzi, 2002 Markov Chains A stochastic process Is a Markov chain iff, for all t, ABC 0.4 0.6 0.3 0.7 0.25 0.75

35. Andrea G. B. Tettamanzi, 2002 Abstract Evolutionary Algorithm select:  (n)   cross:      mutate:    mate:      insert:    XtXt X t+1 select cross mate insertmutate Stochastic functions: Transition function:

36. Andrea G. B. Tettamanzi, 2002 Convergence to Optimum Theorem: if {X t (  )} t = 0, 1,... is monotone, homogeneous, x 0 is given,  y in reach (x 0 )    (n) O reachable, then Theorem: if select, mutate are generous, the neighborhood structure is connective, transition functions T t (  ), t = 0, 1,... are i.i.d. and elitist, then

37. Andrea G. B. Tettamanzi, 2002 Outline of various techniques Plain Genetic Algorithms Evolutionary Programming Evolution Strategies Genetic Programming

38. Andrea G. B. Tettamanzi, 2002 Plain Genetic Algorithms Individuals are bit strings Mutation as transcription error Recombination is crossover Fitness proportionate selection

39. Andrea G. B. Tettamanzi, 2002 Evolutionary Programming Individuals are finite-state automata Used to solve prediction tasks State-transition table modified by uniform random mutation No recombination Fitness depends on the number of correct predictions Truncation selection

40. Andrea G. B. Tettamanzi, 2002 Evolutionary Programming: Individuals Finite-state automaton: ( Q, q 0, A, ,  ) set of states Q ; initial state q 0 ; set of accepting states A ; alphabet of symbols  ; transition function  : Q   Q ; output mapping function  : Q   ; q0q0 q1q1 q2q2 a b c state input q0q0 q0q0 q0q0 q1q1 q1q1 q1q1 q2q2 q2q2 q2q2 q1q1 q0q0 q2q2 b/cc/b a/b c/c a/b b/c a/a c/ab/a a cc c a ab bb

41. Andrea G. B. Tettamanzi, 2002 Evolutionary Programming: Fitness abcabcab b =? no yes f(  ) = f(  ) + 1 individual  prediction

42. Andrea G. B. Tettamanzi, 2002 Evolutionary Programming: Selection Variant of stochastic q-tournament selection:  11 22 qq... score(  ) = #{  i | f(  ) > f(  i ) } Order individuals by decreasing score Select first half (Truncation selection)

43. Andrea G. B. Tettamanzi, 2002 Evolution Strategies Individuals are n-dimensional vectors of reals Fitness is the objective function Mutation distribution can be part of the genotype (standard deviations and covariances evolve with solutions) Multi-parent recombination Deterministic selection (truncation selection)

44. Andrea G. B. Tettamanzi, 2002 Evolution Strategies: Individuals candidate solution rotation angles standard deviations

45. Andrea G. B. Tettamanzi, 2002 Evolution Strategies: Mutation Hans-Paul Schwefel suggests: self-adaptation

46. Andrea G. B. Tettamanzi, 2002 Genetic Programming Program induction LISP (historically), math expressions, machine language,... Applications: –optimal control; –planning; –sequence induction; –symbolic regression; –modelling and forecasting; –symbolic integration and differentiation; –inverse problems

47. Andrea G. B. Tettamanzi, 2002 Genetic Programming: The Individuals subset of LISP S-expressions (OR (AND (NOT d0) (NOT d1)) (AND d0 d1)) OR AND NOT d0 NOT d1 AND d0d1

48. Andrea G. B. Tettamanzi, 2002 Genetic Programming: Initialization ORANDNOT d0 NOT d1 ANDd0d1OR AND OR AND OR AND NOT

49. Andrea G. B. Tettamanzi, 2002 Genetic Programming: Crossover OR ANDNOT d0 d1 OR AND d1NOT d0 d1 OR AND NOT d0 d1 OR AND d1NOT d0 d1

50. Andrea G. B. Tettamanzi, 2002 Genetic Programming: Other Operators Mutation: replace a terminal with a subtree Permutation: change the order of arguments to a function Editing: simplify S-expressions, e.g. (AND X X)  X Encapsulation: define a new function using a subtree Decimation: throw away most of the population

51. Andrea G. B. Tettamanzi, 2002 Genetic Programming: Fitness Fitness cases:j = 1,..., N e “Raw” fitness: “Standardized” fitness:s(  )  [0, +  ) “Adjusted” fitness:

52. Andrea G. B. Tettamanzi, 2002 Sample Application: Myoelectric Prosthesis Control Control of an upper arm prosthesis Genetic Programming application Recognize thumb flection, extension and abduction patterns

53. Andrea G. B. Tettamanzi, 2002 Prosthesis Control: The Context human arm myoelectric signals measure raw myo-measurements preprocess myo-signal features deduce intentions map into goal human motion robot motion convert actuator commands robot arm 150 ms 2 electrodes

54. Andrea G. B. Tettamanzi, 2002 Prosthesis Control: Terminals Features for electrodes 1, 2: Mean absolute value (MAV) Mean absolute value slope (MAVS) Number of zero crossings (ZC) Number of slope sign changes (SC) Waveform length (LEN) Average value (AVG) Up slope (UP) Down slope (DOWN) MAV1/MAV2, MAV2/MAV1 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 0.01, -1.0

55. Andrea G. B. Tettamanzi, 2002 Prosthesis Control: Function Set Additionx + y Subtractionx - y Multiplicationx * y Divisionx / y(protected for y=0) Square rootsqrt(|x|) Sinesin x Cosinecos x Tangenttan x (protected for x=  /2) Natural logarithmln |x| (protected for x=0) Common logarithmlog |x| (protected for x=0) Exponentialexp x Power functionx ^ y Reciprocal1/x (protected for x=0) Absolute value|x| Integer or truncateint(x) Signsign(x)

56. Andrea G. B. Tettamanzi, 2002 Prosthesis Control: Fitness type 1 type 2type 3 undefined   separation spread 22 signals per motion result

57. Andrea G. B. Tettamanzi, 2002 Myoelectric Prosthesis Control Reference Jaime J. Fernandez, Kristin A. Farry and John B. Cheatham. “Waveform Recognition Using Genetic Programming: The Myoelectric Signal Recognition Problem. GP ‘96, The MIT Press, pp. 63–71

58. Andrea G. B. Tettamanzi, 2002 Classifier Systems (Michigan approach) IF X = A AND Y = B THEN Z = D individual: IF... THEN... where number of attributes in antecedent part

59. Andrea G. B. Tettamanzi, 2002 Practical Implementation Issues from elegant academia to not so elegant but robust and efficient real-world applications, evolution programs handling constraints hybridization parallel and distributed algorithms

60. Andrea G. B. Tettamanzi, 2002 Evolution Programs Slogan: Genetic Algorithms + Data Structures = Evolution Programs Key ideas: use a data structure as close as possible to object problem write appropriate genetic operators ensure that all genotypes correspond to feasible solutions ensure that genetic operators preserve feasibility

61. Andrea G. B. Tettamanzi, 2002 Encodings: “Pie” Problems 0–255 128329020 0–255 WXYZ X = 32/270 = 11.85%

62. Andrea G. B. Tettamanzi, 2002 Encodings: “Permutation” Problems Adjacency Representation Ordinal Representation (2, 4, 8, 3, 9, 7, 1, 5, 6) 1 - 2 - 4 - 3 - 8 - 5 - 9 - 6 - 7 (1, 1, 2, 1, 4, 1, 3, 1, 1) Path Representation (1, 2, 4, 3, 8, 5, 9, 6, 7) Matrix Representation 101 0 0 0 0 0 0 0 0 111111 01111111 00011111 00111111 00001111 00000111 00000011 10 0 000000 0000000 Sorting Representation (-23, -6, 2, 0, 19, 32, 85, 11, 25)

63. Andrea G. B. Tettamanzi, 2002 Handling Constraints Penalty functions Risk of spending most of the time evaluating unfeasible solutions, sticking with the first feasible solution found, or finding an unfeasible solution that scores better of feasible solutions Decoders or repair algorithms Computationally intensive, tailored to the particular application Appropriate data structures and specialized genetic operators All possible genotypes encode for feasible solutions

64. Andrea G. B. Tettamanzi, 2002 Penalty Functions S c P

65. Andrea G. B. Tettamanzi, 2002 Decoders / Repair Algorithms S c recombination mutation

66. Andrea G. B. Tettamanzi, 2002 Hybridization 2) Use local optimization algorithms as genetic operators (Lamarckian mutation) 1) Seed the population with solutions provided by some heuristics heuristicsinitial population 3) Encode parameters of a heuristics genotype heuristicscandidate solution

67. Andrea G. B. Tettamanzi, 2002 Sample Application: Unit Commitment Multiobjective optimization problem: cost VS emission Many linear and non-linear constraints Traditionally approached with dynamic programming Hybrid evolutionary/knowledge-based approach A flexible decision support system for planners Solution time increases linearly with the problem size

68. Andrea G. B. Tettamanzi, 2002 The Unit Commitment Problem EmissionsCost

69. Andrea G. B. Tettamanzi, 2002 Predicted Load Curve

70. Andrea G. B. Tettamanzi, 2002 Unit Commitment: Constraints Power balance requirement Spinning reserve requirement Unit maximum and minimum output limits Unit minimum up and down times Power rate limits Unit initial conditions Unit status restrictions Plant crew constraints...

71. Andrea G. B. Tettamanzi, 2002 Unit Commitment: Encoding Unit 1Unit 2Unit 3Unit 4Time 1.000:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 1.0 0.9 0.8 0.4 0.8 0.75 0.8 0.2 0.25 0.2 0.15 0.0 0.5 0.65 0.5 1.0 Fuzzy Knowledge Base

72. Andrea G. B. Tettamanzi, 2002 Unit Commitment: Solution Unit 1Unit 2Unit 3Unit 4Time 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 down hot-stand-by starting shutting down up

73. Andrea G. B. Tettamanzi, 2002 Unit Commitment: Selection cost ($) emission $507,762$516,511 213,489 £60,080 £ competitive selection:

74. Andrea G. B. Tettamanzi, 2002 Unit Commitment References D. Srinivasan, A. Tettamanzi. “An Integrated Framework for Devising Optimum Generation Schedules”. In Proceedings of the 1995 IEEE International Conference on Evolutionary Computing (ICEC ‘95), vol. 1, pp. 1-4. D. Srinivasan, A. Tettamanzi. A Heuristic-Guided Evolutionary Approach to Multiobjective Generation Scheduling. IEE Proceedings Part C - Generation, Transmission, and Distribution, 143(6):553-559, November 1996. D. Srinivasan, A. Tettamanzi. An Evolutionary Algorithm for Evauation of Emission Compliance Options in View of the Clean Air Act Amendments. IEEE Transactions on Power Systems, 12(1):336-341, February 1997.

75. Andrea G. B. Tettamanzi, 2002 Parallel Evolutionary Algorithms Algoritmo evolutivo standard enunciato come sequenziale... … ma gli algoritmi evolutivi sono intrinsecamente paralleli Vari modelli: –algoritmo evolutivo cellulare –algoritmo evolutivo parallelo a grana fine (griglia) –algoritmo evolutivo parallelo a grana grossa (isole) –algoritmo evolutivo sequenziale con calcolo della fitness parallelo (master - slave)

76. Andrea G. B. Tettamanzi, 2002 Terminology Panmictic Apomictic

77. Andrea G. B. Tettamanzi, 2002 Island Model

78. Andrea G. B. Tettamanzi, 2002 Selected Applications in Biology and Medical Science the protein folding problem, i.e. determining the tertiary structure of proteins using evolutionary algorithms; quantitative structure-activity relationship modeling for drug design; applications to medical diagnosis, like electroencephalogram (EEG) classification and automatic feature detection in medical imagery (PET, CAT, NMR, X-RAY, etc.); applications to radiotherapy treatment planning; applications to myoelectric prosthesis control.

79. Andrea G. B. Tettamanzi, 2002 Sample Application: Protein Folding Finding 3-D geometry of a protein to understand its functionality Very difficult: one of the “grand challenge problems” Standard GA approach Simplified protein model

80. Andrea G. B. Tettamanzi, 2002 Protein Folding: The Problem Much of a proteins function may be derived from its conformation (3-D geometry or “tertiary” structure). Magnetic resonance & X-ray crystallography are currently used to view the conformation of a protein: –expensive in terms of equipment, computation and time; –require isolation, purification and crystallization of protein. Prediction of the final folded conformation of a protein chain has been shown to be NP-hard. Current approaches: –molecular dynamics modelling (brute force simulation); –statistical prediction; –hill-climbing search techniques (simulated annealing).

81. Andrea G. B. Tettamanzi, 2002 Protein Folding: Simplified Model 90° lattice (6 degrees of freedom at each point); Peptides occupy intersections; No side chains; Hydrophobic or hydrophilic (no relative strengths) amino acids; Only hydrophobic/hydrophilic forces considered; Adjacency considered only in cardinal directions; Cross-chain hydrophobic contacts are the basis for evaluation.

82. Andrea G. B. Tettamanzi, 2002 Protein Folding: Representation preference order encoding: relative move encoding: UPDOWNFORWARDLEFTUPRIGHT UP LEFT RIGHT DOWN FORWARD DOWN LEFT UP FORWARD RIGHT FORWARD UP DOWN LEFT RIGHT LEFT DOWN FORWARD UP RIGHT...

83. Andrea G. B. Tettamanzi, 2002 Protein Folding: Fitness Decode: plot the course encoded by the genotype. Test each occupied cell: any collisions: -2; no collisions AND a hydrophobe in an adjacent cell: 1. Notes: for each contact: +2; adjacent hydrophobes not discounted in the scoring; multiple collisions (>1 peptides in one cell): -2; hydrophobe collisions imply an additional penalty (no contacts are scored).

84. Andrea G. B. Tettamanzi, 2002 Protein Folding: Experiments Preference ordering encoding; Two-point crossover with a rate of 95%; Bit mutation with a rate of 0.1%; Population size: 1000 individuals; crowding and incest reduction. Test sequences with known minimum configuration;

85. Andrea G. B. Tettamanzi, 2002 Protein Folding References S. Schulze-Kremer. “Genetic Algorithms for Protein Tertiary Structure Prediction”. PPSN 2, North-Holland 1992. R. Unger and J. Moult. “A Genetic Algorithm for 3D Protein Folding Simulations”. ICGA-5, 1993, pp. 581–588. Arnold L. Patton, W. F. Punch III and E. D. Goodman. “A Standard GA Approach to Native Protein Conformation Prediction”. ICGA 6, 1995, pp. 574–581.

86. Andrea G. B. Tettamanzi, 2002 Sample Application: Drug Design Purpose: given a chemical specification (activity), design a tertiary structure complying with it. Requirement: a quantitative structure-activity relationship model. Example: design ligands that can bind targets specifically and selectively. Complementary peptides.

87. Andrea G. B. Tettamanzi, 2002 Drug Design: Implementation NLHAFGLFKA amino acid (residue) individual name hydropathic value Operators: Hill-climbing Crossover Hill-climbing Mutation Reordering (no selection) implicit selection

88. Andrea G. B. Tettamanzi, 2002 Drug Design: Fitness target acomplement b moving average hydropathy hydropathy of residues k  s,..., n  s n : number of residues in target (lower Q = better complementarity)

89. Andrea G. B. Tettamanzi, 2002 Drug Design: Results

90. Andrea G. B. Tettamanzi, 2002 Drug Design References T. S. Lim. A Genetic Algorithms Approach for Drug Design. MS Dissertation, Oxford University, Computing Laboratory, 1995. A. L. Parrill. Evolutionary and Genetic Methods in Drug Design. Drug Discovery Today, Vol. 1, No. 12, Dec 1996, pp. 514–521.

91. Andrea G. B. Tettamanzi, 2002 Sample Application: Medical Diagnosis Classifier Systems application Learning by examples Lymphography –148 examples, 18 attributes, 4 diagnoses –estimated performance of a human expert: 85% correct Prognosis of breast cancer recurrence –288 examples, 10 attributes, 2 diagnoses –performance of human expert unknown Location of primary tumor –339 examples, 17 attributes, 22 diagnoses –estimated performance of a human expert: 42% correct

92. Andrea G. B. Tettamanzi, 2002 Medical Diagnosis Results Performance indistiguishable from humans Performance for breast cancer: about 75% In primary tumor, patients with identical symptoms have different diagnoses Symbolic (= comprehensible) diagnosis rules

93. Andrea G. B. Tettamanzi, 2002 Medical Diagnosis References Pierre Bonelli, Alexandre Parodi, “An Efficient Classifier System and its Experimental Comparison with two Representative learning methods on three medical domains”. ICGA 4, pp. 288–295. Tod A. Sedbrook, Haviland Wright, Richard Wright. “Application of a Genetic Classifier for Patient Triage”. ICGA 4, pp. 334–338. H. F. Gray, R. J. Maxwell, I. Martínez-Perez, C. Arús, S. Cerdán. “Genetic Programming Classification of Magnetic Resonance Data”. GP ‘96, p. 424. Alejandro Pazos, Julian Dorado, Antonio Santos. “Detection of Patterns in Radiographs using ANN Designed and Trained with GA”. GP ‘96, p. 432.

94. Andrea G. B. Tettamanzi, 2002 Sample Application: Radiotherapy Treatment Planning X-rays or electron beams for cancer treatment Conformal therapy: uniform dose over cancerous regions, spare healthy tissues Constrained optimization, inverse problem From dose specification to beam intensities Constraints: –beam intensities are positive –rate of intensity change is limited Conflicting objectives: Pareto-optimal set of solutions

95. Andrea G. B. Tettamanzi, 2002 RTP: The Problem plane of interest tretment area organ at risk head x y z beam TA:dose delivered to treatment area OAR:dose delivered to organs at risk OHT:dose delivered to other healty tissues TA = 100% OAR < 20% OHT < 30%

96. Andrea G. B. Tettamanzi, 2002 RTP: Fitness and Solutions |TA - TA*| |OAR - OAR*| A B C Pareto optimal set

97. Andrea G. B. Tettamanzi, 2002 Radiotherapy Treatment Planning References O. C. L. Haas, K. J. Burnham, M. H. Fisher, J. A. Mills. “Genetic Algorithm Applied to Radiotherapy Treatment Planning”. ICANNGA ‘95, pp. 432–435.

98. Andrea G. B. Tettamanzi, 2002 Evolutionary Algorithms and Soft Computing EAs FLNNs optimization monitoringfitness SC

99. Andrea G. B. Tettamanzi, 2002 Soft Computing Tolerant of imprecision, uncertainty, and partial truth Adaptive Methodologies: –Evolutionary Algorithms –Neural Networks –Bayesian and Probabilistic Networks –Fuzzy Logic –Rough Sets Bio-inspired: Natural Computing A Scientific Discipline? synergyMethodologies co-operate, do not compete (synergy)

100. Andrea G. B. Tettamanzi, 2002 Artificial Neural Networks axon dendritis synapsis 

101. Andrea G. B. Tettamanzi, 2002 Fuzzy Logic 1 0

102. Andrea G. B. Tettamanzi, 2002 EAs FLNNs optimization fitness

103. Andrea G. B. Tettamanzi, 2002 Neural Network Design and Optimization Evolving weights for a network of predefined structure Evolving network structure –direct encoding –indirect encoding Evolving learning rules Input data selection

104. Andrea G. B. Tettamanzi, 2002 Evoluzione dei pesi (struttura predefinita) 0.2 -0.3 0.60.7 -0.50.4 (0.2, -0.3, 0.6, -0.5, 0.4, 0.7)

105. Andrea G. B. Tettamanzi, 2002 Evolving the Structure: Direct Encoding 6 45 123 123456100011020001013000010400000150000016000000123456100011020001013000010400000150000016000000

106. Andrea G. B. Tettamanzi, 2002 Evoluzione pesi e struttura feed-forward codifica diretta (3, 2, 3) 3x3 3x22x33x1 W0W0 W1W1 W2W2 W3W3

107. Andrea G. B. Tettamanzi, 2002 Evoluzione pesi e struttura feed-forward codifica diretta Operatore di mutazione: –rimozione neurone: elimina colonna in W i - 1, riga in W i ; –duplicazione neurone: copia colonna in W i - 1, riga in W i ; –rimozione di un layer con un solo neurone: W T i - 1 W i ; –duplicazione di un layer: inserisci matrice identità; Operatore di semplificazione: –rimuovi neuroni con riga in W i di norma <  ; Operatore di incrocio: –scegli due punti di incrocio nei genitori; –scambia le code; –collega i pezzi con nuova matrice di pesi casuale

108. Andrea G. B. Tettamanzi, 2002 Structure Evolution: Direct Encoding Graph-generating Grammar (S: A, B, C, D || A: c, d, a, c || B: a, a, a, e || C: a, a, a, a ||... )

109. Andrea G. B. Tettamanzi, 2002 EAs FLNNs optimization SC monitoring

110. Andrea G. B. Tettamanzi, 2002 Evolutionary Algorithms and Fuzzy Logic Evolutionary Algorithm Fuzzy Sistem Fuzzy Government fuzzy fitness fuzzy operators 1 2 3

111. Andrea G. B. Tettamanzi, 2002 Representation Genetic operators Selection mechanism Example: Learning fuzzy classifiers Fuzzy System Design and Optimization

112. Andrea G. B. Tettamanzi, 2002 Fuzzy Rule-Based Systems

113. Andrea G. B. Tettamanzi, 2002 totally overlapping membership functions 1001100011011010 membership function genes c1c1 c2c2 c3c3 c4c4 00001010 max = N dom N input * N output rule genes of value (0... N dom ) inputoutput FA 1 FA 2 FA 3 FA 1 FA 2 FA 3 R1R1 R2R2 R max... genotype rules Representation of a Fuzzy Rulebase

114. Andrea G. B. Tettamanzi, 2002 Input membership functions Output MFs Rules IF x is A AND v is B THEN F is C IF a is D THEN F is E IF  is G AND x is H THEN F is C IF true THEN F is K A richer representation

115. Andrea G. B. Tettamanzi, 2002 Initialization Input variables Output variables Rules no. domains = 1 + exponential(3) minmaxC no. domains = 2 + exponential(3) no. rules = 2 + exponential(6) IFANDis ANDisANDisTHENis for each input variable, flip a coin to decide whether to include  abcd

116. Andrea G. B. Tettamanzi, 2002 Recombination IF x is A AND v is B THEN F is C something else IF true THEN F is K something else IF a is D THEN F is E IF  is G AND x is H THEN F is C something else IF x is A AND v is B THEN F is C IF a is D THEN F is E IF  is G AND x is H THEN F is C IF true THEN F is K A rule takes with it all the referred domains with their MFs

117. Andrea G. B. Tettamanzi, 2002 Mutation {add, remove, change} domain to {input, output} variable; {duplicate, remove} a rule; change a rule: {add, remove, change} a clause in the {antecedent, consequent} input MF perturbation: abcd

118. Andrea G. B. Tettamanzi, 2002 Esempio: “Learning fuzzy classifiers”

119. Andrea G. B. Tettamanzi, 2002 Motivation: –EAs easy to implement –little specific knowledge required –long computing time Features: –complex dynamics –non-binary conditions –“intuitive” knowledge available Controlling the Evolutionary Process

120. Andrea G. B. Tettamanzi, 2002 ALGORITHM statisticsvisualization KNOWLEDGE Knowledge Acquisition

121. Andrea G. B. Tettamanzi, 2002 Fuzzy fitness (objective function) Fuzzy encoding Fuzzy operators –recombination –mutation Population Statistics Fuzzfying Evolutionary Algorithms

122. Andrea G. B. Tettamanzi, 2002 Faster calculation Less precision Specific Selection Fuzzy Fitness

123. Andrea G. B. Tettamanzi, 2002 “Fuzzy rulebase for the dynamic control of an evolutionary algorithm” Population Statistics Parameters If D(X t ) is LOW then p mut is HIGH If f (X t ) is LOW and D(X t ) is HIGH then Emerg is NO... Fuzzy Government

124. Andrea G. B. Tettamanzi, 2002 EAs FLNNs integration

125. Andrea G. B. Tettamanzi, 2002 Neuro-Fuzzy Systems Fuzzy Neural Networks –fuzzy neurons (OR, AND, OR/AND) –learning algorithms (backpropagation-style) –NEFPROX –ANFIS Co-operative Neuro-Fuzzy Systems –Adaptive FAMs: differential competitive learning –Self-Organizing Feature Maps –Fuzzy ART and Fuzzy ARTMAP

126. Andrea G. B. Tettamanzi, 2002 Fuzzy Neural Networks AND OR AND

127. Andrea G. B. Tettamanzi, 2002 FAM Systems fuzzdefuzz 

128. Andrea G. B. Tettamanzi, 2002 EAs FLNNs optimization monitoringfitness SC integration A. Tettamanzi, M. Tomassini. Soft Computing. Springer-Verlag 2001

129. Andrea G. B. Tettamanzi, 2002 Summary and Conclusions