Two-Player Repeated Game Symmetric model: 2 signal space dimensions Interference gains g Actions: Spectral power allocation x Metric: Average capacity C i =( k C i (k))/K w/ K large Strategy: Play sequence of powers x i (k)
Actions and Payoffs Moderate Interference + Low Noise = Water filling is BAD Reputation Matters Same player pool Players remember
Strategy Structure Players with a memory of 2 games Number of possible histories = 9x9 = 81 Each history is associated with an action in response Strategy = string of beads (actions)
Strategy Search HUGE Search Space Use genetic algorithms Basic Idea: Each possible solution is a ‘genome’. Generate populations and evaluate ‘fitness’. ‘Mate’ members to evolve new populations. Repeat until “good enough”
The Experiment Populations are collections of policy genomes Devise an evaluator set Completely random strategies Squatters, hoppers, avoiders No-learn fools (legacy?) Run tournaments, evaluate ‘fitness’ Result: Populations evolve effective strategies We distill essential features
Winning Strategy Performance When played among themselves (left graph) they negotiate to segregate in signal space to reach a ‘fair’ settlement When thrown into a population of random strategies, it outperforms the others
Winning Strategy Interactions When two of our evolved strategies interact, there is an initial probing stage followed by a mutually agreed segregation The intelligence of the opposition is necessary for attaining this ‘fair’ settlement. What are the key features of these strategies?
Identifying Traits Positive Traits: preference for an action Negative Traits: avoidance of an action Compose histogram and see what pops up
Examples of Useful Traits If the opponent doesn’t react to exploitation, continue to exploit Do not break a status-quo of segregation You push me, I push you back! Occasionally forgive to encourage cooperation Randomness to avoid repeated collision Avoid following the opponent in signal space
The Schema Skeleton Fix useful traits and choose the rest of the genome randomly Now test the fitness of this constructed strategy The performance is good! Simple strategy descriptions might be possible!
The Seeds of Evolution Strategy structure depends on the other players Spread-only strategies spawned by Single channel squatters Spread channel squatters Random action Adaptive strategies emerge from interactions with other adaptive policies You’re as good as the company you keep
Questions Might we be able to use distributed competitive strategies in cognitive radios? Deploy fixed strategies? evolve via user purchase decisions? Radios evolve strategies in real time? Extensible to multi-player games? Formal strategy description grows exponentially with strategy memory size What is marginal improvement w/ size?
The Golden Rule Be Polite Play Nice The REAL Golden Rule Be aware of your surroundings A sucker and his toys are soon parted Develop a good left hook Bruises are bad Forgiveness is divine (when you have a good left hook) Cognitive Radio Kindergarten