Evolutionary Game Algorithm for continuous parameter optimization Alireza Mirian

Evolutionary Computation presentation, 2012  A system in which a number of rational players make decision in a way that maximize their utility. 2  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms What is a Game?

Alireza Mirian Evolutionary Computation presentation, 2012  Each player (agents) has a set of possible actions (strategies) to choose from  Each player have their Utility Function that determines the profit/outcome of any decision  Agents are rational self-interested decision makers, i.e. they make decision upon their view of utility.  Players doesn’t have full control over outcome. That is, a person’s success is based upon the choices of others 3  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms What is a Game?

Alireza Mirian Evolutionary Computation presentation, 2012  Games have wide range, from parlor games (chess, poker, bridge) to various economic, political, military or biological situations. 4  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms What is a Game?

Alireza Mirian Evolutionary Computation presentation, 2012  Game theory: the study of mathematical models of games  John von Neumann & John Nash  Has lots of applications in economics, political science, and psychology, and other, more prescribed sciences, like logic or biology.  tries to find a “solution” for game 5  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms What is Game Theory?

Alireza Mirian Evolutionary Computation presentation, 2012  Decision Theory: A special case of Game with one player 6  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms What is Game Theory?

Alireza Mirian Evolutionary Computation presentation, 2012  In non-cooperative games the goal of each player is to achieve the largest possible individual gain (profit or payoff)  In cooperative games the action of players are directed to maximize the gain of “collectives” (coalitions) without subsequent subdivision of the gain among the players within the coalition 7  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Non-cooperative and cooperative games

Alireza Mirian Evolutionary Computation presentation, 2012  Non-cooperative: Two player Hokm  Cooperative: Four player Hokm 8  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Non-cooperative and cooperative games

Alireza Mirian Evolutionary Computation presentation, 2012  Let I denote the set of players  Let S i denote the set of all possible actions for player i (strategies of player i)  |S i | > 1 (why?)  At each “round” of the game, each player chooses a certain strategy s i S i  So, after each round: (s 1,s 2,…,s n ) = s is put together.  This system is called a situation  In each situation, each player gets a profit  S = S 1 ×…×S n = ∏ iI S i (strategy profile). 9  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Non-cooperative game

Alireza Mirian Evolutionary Computation presentation, 2012  Definition of Non-cooperative Game: G=[ I, { S i } iI, {U i } iI ]  I = {1,2, …, n} : set of players  S i : strategy set for player i (set of possible actions)  Ui : Utility function defined on set S=∏ iI S i 10  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Non-cooperative game

Alireza Mirian Evolutionary Computation presentation, 2012  Example: 4-barg!  I = {1,2}  S 1 = {,,, }  S 2 = {,,, }  U 1 ( s ) = U 1 ({, }) = 2  U 2 ( s ) = U 2 ({, }) = 0 11  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Non-cooperative game 2 1 s ={, }

Alireza Mirian Evolutionary Computation presentation, 2012  s = {s 1, …,s i-1, s i, s i+1, …, s n }  s || s ΄ i = {s 1, …,s i-1, s ΄ i, s i+1, …, s n }  That is, s || s ΄ i is a situation that differs from s, only in s i  Admissible situation: a situation s is called admissible for player i if any other strategy s ΄ i for this player we have: U i (s || s ΄ i ) ≤ U i (s) 12  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Admissible situation

Alireza Mirian Evolutionary Computation presentation, 2012  A situation s, which is admissible for all the players is called an equilibrium situation  That is, in a equilibrium situation, no player is interested to change their strategy. (why?)  Solution of a non-cooperative game: determination of an equilibrium situation 13  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Equilibrium point

Alireza Mirian Evolutionary Computation presentation, 2012  An optimization problem:  arg max f(x) x ∈ D where x = (x 1,x 2,...,x n ) ∈ R n, xi ∈ [x i l, x i u ], i = 1,2,...,n, is n-dimensional real vector, f(x) is the objective function, D = [x i l, x i u ] ⊆ R n defines the search space, and x ∗ that satisfies f(x ∗ )= max { f (x) | x ∈ D } is the optimal solution of problem 14  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Optimization problem

Alireza Mirian Evolutionary Computation presentation, 2012  In EGA the optimization problem maps into a non-cooperative  Optimum will find by exploring the equilibrium situations in corresponding game  Global convergence property of the algorithm is proofed 15  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Optimization problem and game

Alireza Mirian Evolutionary Computation presentation, 2012  x = (x 1,x 2,...,x n )G = ( I, { S i } iI, {U i } iI )  Variable x is mapped to strategy profile of game agents  Objective function f is mapped to game agents΄ utility function  N x :the number of agents that their strategy profile will represent a variable x i  |I| = n * n x | S i | = m  Size of strategy profile of n x agent: m n x -1  Precision of this mapping: (x i u – x i l )/(m n x -1) 16  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Mapping between strategy profile and x i

Alireza Mirian Evolutionary Computation presentation, 2012  Decoding function φ:  x i = φ(s i ) = x i l + decimal(s i ) × (x i u – x i l )/(m n x -1)  Example: f(x) = x 1 + x 2 where x i [-2.048, 2/048], i = 1,2  x n = 10, m = 2  overall strategy profile of n I = n × n x =20 agent is a binary string with length of 20:  S:  x 1 = decimal( ) 2 ×4.096/( )  x 2 = decimal( ) 2 ×4.096/( )  x 1 = , x 2 =  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Mapping between strategy profile and x i x1x1 x2x2

Alireza Mirian Evolutionary Computation presentation, 2012  All the agents have the same utility function which is just objective function  u = { u i (s) ≡ f(φ(s)), i є I} where I = {1, 2, 3, …, n I }  s is the strategy profile of n I = n × n x  In the previous example:  s = ( )  u(i) = f(φ(s)) = f(x 1, x 2 ) = x 1 + x 2 = i = 1, 2, 3, …,  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Utility function

Alireza Mirian Evolutionary Computation presentation, 2012  At the start of EGA each agent randomly selects a strategy from its strategy set {0, 1,..., m − 1} with a probability 1/m  After that, In each loop:  Random perturb: current strategy of each agent is replaced with a random strategy with a probability 1/m for each strategy  agents will do a deterministic process to reach an equilibrium point s e (t) 19  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Procedure of EGA

Alireza Mirian Evolutionary Computation presentation, 2012 Procedure EGA t = 0; randomly initialize s (0) and set it as current solution; while termination condition is not satisfied do perform a random perturb on current solution s (t) ; do a deterministic process to reach an equilibrium point s e (t) ; if utility of s e (t) ≥ utility of current solution current solution = s e (t) end t = t + 1; end 20  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Procedure of EGA

Alireza Mirian Evolutionary Computation presentation, 2012  How to reach the equilibrium point?  Coalition: n x agents that represent the same component x i of variable x are defined as one coalition  In out example: agent 1, 2,..., 10 that represent x 1 is a coalition, and agent 11, 12,..., 20 that represent x 2 is another coalition.  BRC: the strategy profile of a coalition that maximizes its utility while strategy profile of other coalitions are fixed is called the Best-Response Correspondence (BRC) of that coalition.  Process of reaching equilibrium:  While equilibrium point is not reached, all coalitions replace their strategy profile with their BRC in sequence 21  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Reaching equilibrium point

Alireza Mirian Evolutionary Computation presentation, 2012 Pseudo code of reaching equilibrium point: while equilibrium state is not achieved for agent coalition i = 1, 2,...,n agent coalition i replaces its strategy profile with its BRC; end Now two other thing:  How to decide whether an equilibrium point is achieved?  How does an agent coalition find out its BRC 22  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Reaching equilibrium point

Alireza Mirian Evolutionary Computation presentation, 2012  How to decide whether an equilibrium point is achieved?  when r (the number index of BRC rounds) reaches a predefined number R  the utility has not improved in d r consecutive rounds  How does an agent coalition find out its BRC?  Exact BRC ~> have to compute the utilities of all possible strategy profiles within its strategy profile space  Cardinality of the strategy profile set of a coalition ( = m n x ) usually is a very large number  inner level optimization is used to find an approximate BRC. 23  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms Two remaining problem

Alireza Mirian Evolutionary Computation presentation, 2012  Inner level optimization for approximating BRC has two phases:  first phase: with a perturb probability p d, the current strategy of each agent in a coalition is replaced with a new strategy with a probability 1/m for each strategy.  Second phase: each agent in the coalition replaces its current strategy with an optimal strategy selected from its strategy set { 0,1,...,m − 1 } which maximizes its utility in sequence.  inner level optimization process has the same structure as the main loop of EGA itself if we regard one agent as a coalition (except that the inner process only has one loop i.e. one BRC round) 24  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms inner level optimization

Alireza Mirian Evolutionary Computation presentation,  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms inner level optimization

Alireza Mirian Evolutionary Computation presentation,  What is Game Theory?  Non-cooperative and cooperative games  Equilibrium point  Evolutionary Game Algorithm  Mapping between strategy profile and x i  Procedure of EGA  Results and comparison with other algorithms inner level optimization

Alireza Mirian Evolutionary Computation presentation, 2012  Y. Jun a, L. Xiande, H. Lu, “Evolutionary game algorithm for continuous parameter optimization”, Information Processing Letters, 2004  N. N. Vorob’ev, “Game Theory Lectures for Economists and Systems Scientists”, Springer-verlag,1977  R. D. Luce, H. Raiffa, “Games and Decision”, J. Wiley & sons, 1957  R. Cressman, “The Stability Concept of Evolutionary Game Theory”, Springer-verlag, 1992  E. V. Damme, “non-cooperative Games” TILEC and CentER, Tilburg University, 2004  Y. Jun, L. Xiande, H. Lu, “Evolutionary game algorithm for multiple knapsack problem”, Proc. of 2003 IEEE/WIC International Conference on Intelligent Agent Technology,  Ross, Don, "Game Theory", The Stanford Encyclopedia of Philosophy (Fall 2011 Edition), Edward N. Zalta (ed.), 2011  D. K. Levine, “What is Game Theory?”, Department of Economics, UCLA 27 References

Alireza Mirian Evolutionary Computation presentation, 2012 Thanks for your attention :D 28