Presentation on theme: "Evolution: Games, dynamics and algorithms Karen Page Bioinformatics Unit Dept. of Computer Science, UCL."— Presentation transcript:
Evolution: Games, dynamics and algorithms Karen Page Bioinformatics Unit Dept. of Computer Science, UCL
Evolution zDarwinian evolution is based on three fundamental principles: reproduction, mutation and selection zConcepts like fitness and natural selection are best defined in terms of mathematical equations zWe show how many of the existing frameworks for the mathematical description of evolution may be derived from a single unifying framework
Summary of what will be discussed zGames, evolutionary game theory zKey frameworks of evolutionary dynamics zDeriving a unifying framework zAn application to Fishers Fundamental Theorem zRelationship with Genetic Algorithms
What is game theory? zFormal way to analyse interactions between agents who behave strategically zMathematics of decision making in conflict situations zUsual to assume players are rational zWidely applied to the study of economics, warfare, politics, animal behaviour, sociology, business, ecology and evolutionary biology
Assumptions of game theory zThe game consists of an interaction between two or more players zEach player can decide between two or more well- defined strategies zFor each set of specified choices, each player gets a given score (payoff)
The Prisoners Dilemma zProbably most studied of all games zNot enough evidence to convict two suspects of armed robbery, enough for theft of getaway car zBoth confess (4 years each), both stay quiet (2 years each), one tells (0 years) the other doesnt (5 years) zStay quiet= cooperate (C) ; confess = defect (D) zPayoff to player 1: R is REWARD for mutual cooperation =3 S SUCKERs payoff =0 T TEMPTATION to defect =5 P PUNISHMENT for mutual defection=1 with T>R>P>S
The problem of cooperation zWhat ever player 2 does, player 1 does better by defecting: zClassical game theory both players D zShame because theyd do better by both cooperating zCooperation is a very general problem in biology zEveryone benefits from being in cooperative group, but each can do better by exploiting cooperative efforts of others
Trade wars and cartels zImport tariffs - Should countries remove them? zPrice fixing- why not cheat?
Repeated games zIn many situations, typically players interact repeatedly- repeated Prisoners Dilemma zStrategies can involve memory, use reciprocity zTit-for-tat zPavlov
Game theory and a computer tournament zGame theory says it is rational to defect in single game or fixed number of rounds zAxelrods tournament- double victory for Tit- for-Tat
Evolutionary Game Theory So how can cooperation be explained?
Evolutionary games zJohn Maynard Smith- evolution of animal behaviour zBehaviour shaped by trial and error- adaptation through natural selection or individual learning zPlayers no longer have to be rational: follow instincts, procedures, habits rather than computing best strategy. zGames played in a population. Scores are summed. Strategies which do well against the population on average propagate. zPhenotypic approach to evolution zFrequency-dependent selection
Simple evolutionary game simulations zEveryone starts with a random strategy zEveryone population plays game against everyone else zThe payoffs are added up zThe total payoff determines the number of offspring (Selection) zOffspring inherit approximately the strategy of their parents (Mutation) z[Note similarity to genetic algorithms.] z[Nash equilibrium in a population setting- no other strategy can invade]
Evolution in the Prisoners Dilemma zStandard evolutionary game (random interactions) all Defect zModifications- spatial games: Interactions no longer random, but with spatial neighbours: zSum scores. Player with highest score of 9 shaded takes square (territory, food, mates) in next generation zSome degree of cooperation evolves!
Simulations of the spatial Prisoners Dilemma 75 generations Winner-takes-all selection No mutation Red=d(d last) Blue=c(c last) Yellow=d(c last) Green=c(d last)
Conclusions on Evolutionary Games zGame theory can be applied to studying animal and human behaviour (economics - evolutionary biology). zOften traditional game theorys assumption of rationality fails to describe human/ animal behaviour zInstead of working out the optimal strategy, assume that strategies are shaped by trial and error by a process of natural selection or learning. This can be modelled by evolutionary game theory. zSpace can matter
The replicator equation zReplicator equation describes evolution of frequencies of phenotypes within a population with fitness- proportionate selection zEg. game theory, replicators like Game of Life zFrequency of type i is and fitness of type i is then
The equivalence with Lotka Volterra equations zLotka Volterra systems of ecology describe the numbers of animals (eg. fish) of different species and are of the form: where is the abundance of species i, its fitness and there are n species in total. zOften these interacting species oscillate in abundance. zThere is a precise equivalence with the replicator system for (n+1) types given by the substitution
zSuppose there are errors in replicating. The probability of type j mutating to type i is. zWe obtain a replicator equation with mutation: zThe equivalent with numbers rather than frequencies of types is zWhen the fitnesses do not depend on frequencies, this is the quasispecies eqn. (Probably the case in most GAs?) Replicator equation with mutation and quasispecies
Quasispecies equation zDescribes molecular evolution (Eigen) zN biochemical sequences zBiochemical species i has frequency y i zReplication at rate f i is error-prone - mutation to type j at rate q ij
Adaptive dynamics framework zGame consists of a continuous space of strategies (eg.) zPopulation is assumed to be homogeneous- all players adopt same strategy zMutation generates variant strategies very close to the resident strategy zIf a mutant beats the resident players it takes over otherwise it is rejected zAdaptive dynamics illustrates the nature of evolutionary stable strategies
Adaptive dynamics equations zStrategies are described by continuous parameters : zExpected score of mutant against S is given by E(S,S) zThe adaptive dynamics flow in the direction which maximises the score:
We can derive Prices equation from replicator-mutator equation zPrices equation from population genetics describes any type of selection. zSuppose an individual of type i, frequency, has some trait p of value z, so using the replicator equation with mutation we obtain zThis applies when the values of are const. z[ p is the expected mutational change in p.]
Prices equation selection mutation
Prices equation gives rise to adaptive dynamics zIf we assume that the mutation is localised and symmetrical then we can neglect the second term in Prices eqn. zAssume population is almost homogeneous and fitness is differentiable then we can Taylor expand the fitness, obtaining zcf. adaptive dynamics:
Fishers fundamental theorem zSuppose fitnesses of genotypes constant. Can consider f as the trait p and obtain (for symmetric mutation): zFishers fundamental theorem of NS zIn general, fitnesses of genotypes depend on environment. In game theory context, depend on the frequencies of other genotypes. Fishers theorem doesnt apply- eg. PD
Generalized version zWe can use Prices equation to obtain a generalized version of Fishers fundamental theorem: where zThis applies when the s depend linearly on the frequencies of genotypes- normally the case in evolutionary game theory.
Fishers theorem and GAs zIn most GAs, fitnesses of particular solutions (chromosomes) probably fixed and so (except for the complication of recombination) Fishers theorem should hold: zSo for a GA with fitness-proportionate selection, no recombination and fixed fitness for a given solution, the average fitness of the population of solutions increases until there is no diversity left in the fitnesses.
Conclusions on unifying evolutionary dynamics zUnifying framework zDifferent frameworks for different problems. zWe derive from Prices equation a generalized version of Fishers Fundamental Theorem of Natural Selection. zThe Price – replicator framework can also be applied to discrete time formulations and to formulations with sexual reproduction.
Relationship to GAs
Evolutionary games and genetic algorithms zTwo-way interaction: 1) So far discussed computer simulations of evolutionary processes, eg. evolution of animal behaviour 2) Evolutionary computation, eg. genetic algorithms = computer science based on theory of biological evolution zEvolutionary games very like genetic algorithms- but 1) Population size is usually quite large and may be few phenotypes: space well searched but not v. efficient. 2) Usually no recombination 3) Fitnesses depend on interactions
Genetic Algorithms zEvolutionary models are computer algorithms which use evolutionary methods of optimisation to solve practical problems (cf. finding stable strategies in games rather than working out rational solution)- eg. Evolutionary programming, genetic algorithms zEvolutionary operations involved in genetic algorithms: selection, mutation, recombination :
How evolutionary dynamics relates to GAs zGAs evolve by selection and mutation their dynamics can be (to some extent) described by the replicator equation with mutation (cf. unifying framework). zThe replicator equation describes fitness-proportionate selection. zFicici, Melnik and Pollack (2000) - effects of different types of selection (eg. truncation) on the dynamics of the Hawk-Dove game + relevance for evolutionary algorithms. Can lead to different dynamics. zMust also consider the effects of recombination.
Incorporating recombination into the replicator framework zDo this by assuming that r jk;i = probability that when parent chromosome of type j combines with parent chromosome of type k, an offspring of type i is formed. zNo mutation, recombination after replication: z[NB discrete-time version]
Adding in mutation zAdd in mutation. Assume, as before, is probability type i mutates to form type j ( large). Assume this happens after recombination. zWhat we had before was zWhat we have now is
The diversity of the population and adaptive dynamics zFrom Fishers theorem, see that no diversity of fitness in population no further increase in average fitness. zHowever, because the variation in the parameters of the your system has become very small (population convergence), does not mean no further evolution. zIn the case of small variation, we can apply the adaptive dynamics framework which shows how the average values of traits (parameters) will change in time
Relationship: evolutionary games & GAs - Conclusions zOften evolution leads in the long run to optimal solutions, like Nash equilibria. zAbility of evolutionary processes to seek out optimal strategies has been exploited in computer science by the development of genetic algorithms and evolutionary computation for problem solving. zComparing with the use of computer simulations to study biological evolution, we see that there is a two- way interaction between biological evolutionary theory and computer science.
Relationship to GAs- Conclusions zFrameworks of evolutionary dynamics can be applied to GAs by modifying them to include recombination. zWhich framework is most informative depends on the individual problem, but we have shown they are equivalent. zEg. can look at detailed dynamics using the replicator- mutator framework zOr we can look at a converged population using the adaptive dynamics framework. zLooking further at the relationship between GAs and evolutionary dynamics could yield new solutions/ techniques for both.