1. Multiagent Systems
Game Theory
© Manfred Huber 2018

2. Non-Cooperative Game Theory
Game Theory is the study of the interaction and decision making of rational, self-interested agents
Each agent has its own interests
While these are mostly distinct, non-cooperative does not preclude that agents have common interests
Agents are rational and only pursue their interests
The individual agent is the basic unit
Collaborative or coalitional game theory model teams as basic units
© Manfred Huber 2018

3. Game Theory TCP backoff example
Agents can use a correct or an incorrect TCP backoff mechanism
Correct mechanism backs off after package collisions
Incorrect mechanism does not increase backup over time
Collisions of packets from multiple agents can lead to delays due to retries
Correct implementation leads to an expected delay of 1
Incorrect implementation immediately retries, causing a correct implementation to back off further, leading to a delay of 4
Two incorrect implementations clash multiple times, leading to an expected delay of 3
TCP backoff game -1/-1 , -4/0, 0/-4, -3/-3
© Manfred Huber 2018

4. Game Theory
The interaction and decision process of the agents is modeled as a game with separate utility functions for each agent
What action should each player take ?
Will both agents behave the same ?
Do the precise numbers influence behavior ?
Would communication change behavior ?
Should actions change when a game is played repeatedly ?
Is it important that agents are assumed to be rational ?
© Manfred Huber 2018

5. Single Shot Games
Single shot (or single stage) games are games where multiple agents have to make a single action choice.
Past experiences have no influence on the current game and the agents’ decisions
Utility is equal to the expected reward
Finite n-person game
N is a finite set of n agents (players)
is the joint action set of the agents
is the joint utility function for the agent
© Manfred Huber 2018

6. Matrix Games
A finite single stage game can be written in the form of an n-dimensional matrix (called “normal form”)
Each dimension enumerates the actions of one agent
Entries in the matrix are n-tuples of utility values for each agent for outcome of the given action choices
The TCP backoff example
Correct
Incorrect
-1, -1
-4, 0
0, -4
-3, -3
© Manfred Huber 2018

7. Matrix Games The prisoners’ dilemma Cooperate Stay silent -5, -5
-1, -10
-10, -1
-3, -3
© Manfred Huber 2018

8. Canonical Games Pure competition games are 2-player constant sum games
Every utility gain for one agent is an equal utility loss for the other agents
Only one utility function is needed
Example: Matching Pennies
Heads
Tails
1, -1
-1, 1
© Manfred Huber 2018

9. Pure competition Games
Example: Rock, Paper, Scissors
Rock
Paper
Scissors
0, 0
-1, 1
1, -1
Compute utility under different assumptions
© Manfred Huber 2018

10. Canonical Games
Coordination games are games where players have exactly the same interests
All players obtain the same utilities
Only one utility function is needed
Note: this is still non-cooperative since each agent makes its own decision
Example: Driving on a side of the road
Left
Right
1, 1
-1, -1
© Manfred Huber 2018

11. Canonical Games
General games contain elements of cooperation and competition
Example: Battle of the sexes
What happens if they feel differently strongly about different options ?
Opera
Football
3, 1
0, 0
1, 3
© Manfred Huber 2018

12. Decision-Making: Analyzing Games
“Better” outcomes/strategies have to be defined
Utilities for different agents can not be compared
All agents are equally important
Pareto dominance
An outcome o Pareto-dominates an outcome o’ if for every agent outcome o is at least as good as outcome o’
© Manfred Huber 2018

13. Pareto Optimality
An outcome/strategy is Pareto optimal if it is not Pareto dominated by any other outcome
Every game has at least one Pareto optimal strategy
Every game has at least one Pareto optimal pure strategy
Games can have multiple Pareto optimal strategy
© Manfred Huber 2018

14. Pareto Optimality
Pareto Optimality allows to identify better strategies from an outside point of view.
Examples:
Heads
Tails
1, -1
-1, 1
Left
Right
1, 1
-1, -1
Cooperate
Stay silent
-5, -5
-1, -10
-10, -1
-3, -3
Opera
Football
3, 1
0, 0
1, 3
© Manfred Huber 2018

15. Decision-Making: Best Response
If an agent knows the strategies of all other agents it can easily pick its own action
Best response:
When the strategies of other agents are not known we have to look for “stable” strategies
A strategy is a (pure strategy) Nash equilibrium if for every agent its action is a best response to the actions of the other agents
© Manfred Huber 2018

16. Nash Equilibrium A Nash Equilibrium is a stable strategy
Any agent deviating from it will not increase its utility
An agent that is aware of the other agents’ strategies can not increase its utility using this knowledge (the equilibrium is its best strategy)
In an equilibrium it is possible to determine the other agents’ strategies based on the assumption of rationality
© Manfred Huber 2018

17. Nash Equilibrium
Nash equilibria allow agents to identify stable strategies that lead to “best” outcomes.
Examples:
Heads
Tails
1, -1
-1, 1
Left
Right
1, 1
-1, -1
Cooperate
Stay silent
-5, -5
-1, -10
-10, -1
-3, -3
Opera
Football
3, 1
0, 0
1, 3
© Manfred Huber 2018

18. Nash Equilibrium Many games do not have pure strategy equilibria
In competitive settings it is often a bad idea to always do the same
If other agents know your actions they can use this knowledge to utilize your weaknesses
© Manfred Huber 2018

19. Mixed Strategies Agents can follow different strategies
Pure strategies are strategies where agents select actions deterministically
Mixed strategies are strategies where actions are taken according to a probability distribution
Strategy profiles are the joint strategies of all the agents
© Manfred Huber 2018

20. Utility of Mixed Strategies
The utility of a mixed strategy profile can be computed as the expected value of the outcome lottery
© Manfred Huber 2018

21. Best Response and Nash Equilibrium
If an agent knows the strategies of all other agents it can pick its own (mixed) strategy
Best response:
When the strategies of other agents are not known we have to look for “stable” strategies
A strategy profile is a Nash equilibrium if for every agent its strategy is a best response to the strategies of the other agents
© Manfred Huber 2018

22. Nash Equilibrium
A Strict Nash equilibrium is a Nash equilibrium where for each agent the strategy is a strictly better (higher utility) response to the other agents’ strategies than any other strategy
A Weak Nash equilibrium is any Nash equilibrium that is not a Strict Nash equilibrium
All mixed strategy equilibria are Weak
© Manfred Huber 2018

23. Nash Equilibrium
Nash (1951): Every finite game has at least one Nash equilibrium
Computing Nash equilibria can be complex
It becomes easier if the support of the strategy (i.e. the set of actions with probabilities greater than 0) is known
If support is known then there is no regret for the pure actions involved – they have to have the same response value
Heads
Tails
1, -1
-1, 1
Opera
Football
3, 1
0, 0
1, 3
© Manfred Huber 2018

24. Solution Approaches
To address decision making in matrix games we need algorithms that can determine Nash equilibria
Maxmin and Minmax for constant sum games
Linear programming for (zero-sum) matrix games
Linear Complimentarity Problem (LPC) for general 2-player games
Lemke-Howson algorithm
© Manfred Huber 2018