1. cs301fs07a5 The Iterated Prisoner’s Dilemma

2. Overview Two criminals get caught and choose to cooperate or defect to alter their sentence Two criminals get caught and choose to cooperate or defect to alter their sentence Objective: Minimize sentence time Objective: Minimize sentence time

3. CoEA difficulties (1) Disengagement Occurs when one population evolves so much faster than the other that all individuals of the other are utterly defeated, making it impossible to differentiate between better and worse individuals without which there can be no evolution Occurs when one population evolves so much faster than the other that all individuals of the other are utterly defeated, making it impossible to differentiate between better and worse individuals without which there can be no evolution

4. CoEA difficulties (2) Cycling Occurs when populations have lost the genetic knowledge of how to defeat an earlier generation adversary and that adversary re-evolves Occurs when populations have lost the genetic knowledge of how to defeat an earlier generation adversary and that adversary re-evolves Potentially this can cause an infinite loop in which the populations continue to evolve but do not improve Potentially this can cause an infinite loop in which the populations continue to evolve but do not improve

5. CoEA difficulties (3) Suboptimal Equilibrium (aka Mediocre Stability) Occurs when the system stabilizes in a suboptimal equilibrium Occurs when the system stabilizes in a suboptimal equilibrium ?

6. Phenotype Function Function –Range = All possible memory states –Domain = {C,D} Genotype Expression Tree OR Expression Tree OR Array Array –Length = 2 6 +2 4 +2 2 +2 0 = 85 –Values = {C,D}

7. Memory to index For full memory convert to binary For full memory convert to binary –{(C,D),(C,C),(D,C)} = 00101101b = 45 For partial memory prepend w/(01) n 00 For partial memory prepend w/(01) n 00 –n = maxsize - |M| –{(C,D),(C,C)} = 1011b  (01) 1 001011b = 75 –{(C,D)} = 10  (01) 2 0010b = 82 –{} =  (01) 3 00b = 84

8. Fitness Analysis At the beginning of every generation At the beginning of every generation –Arrest all unique pairs of individuals p m and p n –Have them play k games in a row maintaining a memory of the last three rounds and making decisions based on the memory –Fitness is negative the total time sentenced through all pairings

9. Survival Selection The offspring don’t start with a fitness so how can they compare? The offspring don’t start with a fitness so how can they compare? –Calculate fitness by pairing each offspring with every other individual in the population and offspring –OR  Generational  Choose individuals in the population to replace with a uniform random selection of the offspring –Truncation, Fitness proportional, Tournaments

10. TIT FOR TAT Quite possibly the ultimate strategy Quite possibly the ultimate strategy Choose to do whatever the partner did last round Choose to do whatever the partner did last round The genotype of TIT FOR TAT is: The genotype of TIT FOR TAT is:{D,C,D,C,D,C,D,…,D}

11. Bonus Logging What else is there to log about the EA? What else is there to log about the EA? –Compare to TIT FOR TAT –Check strategy statistics –And more…