Download presentation

Presentation is loading. Please wait.

Published byRay Blacker Modified over 2 years ago

1
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

9 Markov Chains and the Theory of Games Markov Chains Regular Markov Chains Absorbing Markov Chains Game Theory and Strictly Determined Games Games with Mixed Strategies Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

2
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Any square matrix that satisfies the properties: 2. The sum of the entries in each column of T is 1. can be referred to as a stochastic matrix. Ex. Stochastic, columns add to 1 Not stochastic, column 1: sum = 1.3 Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

3
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Markov Process (Chain) Stochastic Process in which the outcomes at any stage of the experiment depend only on the outcomes of the preceding stage. The state is the outcome at any stage. The outcome at the current stage is called the current state. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

4
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Ex. We are given a process with 2 choices: A and B. It is expected that if a person chooses A then that person has a 30% probability of choosing A the next time. If a person chooses B then that person has a 60% probability of choosing B the next time. 0.3 A 0.4 A A B Basically: 0.7 B B 0.6 This can be represented by a transition matrix Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

5
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

State 1 State 2 The probability that something in state 1 will be in state 1 in the next step is a11 = 0.3 The probability that something in state 2 will be in state 1 in the next step is a12 = 0.4 Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

6
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Transition Matrix A transition matrix associated with a Markov chain with n states is an n X n matrix T with entries aij Current State Next state Next state Current state 2. The sum of the entries in each column of T is 1. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

7
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Ex. It has been found that of the people that eat brand X cereal, 85% will eat brand X again the next time and the rest will switch to brand Y. Also, 90% of the people that eat brand Y will eat brand Y the next time with the rest switching to brand X. At present, 70% of the people will eat brand X and the rest will eat brand Y. What percent will eat brand X after 1 cycle? X Initial state Transition: Y One cycle: So 62.5% will eat brand X Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

8
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Notice from the previous example: is called a distribution vector In general, the probability distribution of the system after n observations is given by Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

9
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Regular Markov Chain A stochastic matrix T is a regular Markov chain if the sequence T, T2, T3,… approaches a steady-state matrix in which the rows of the limiting matrix are all equal and all the entries are positive. Ex. Regular: all entries positive Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

10
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

T is Regular: all entries of T2 positive Ex. Notice Ex. Not regular: entries of T to a power will never all be positive. Notice Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

11
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Steady–State Distribution Vector Ex. Given Notice (after some work) Tends toward The steady-state distribution vector is the limiting vector from the repeated application of the transition matrix to the distribution vector. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

12
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Finding the Steady-State Distribution Vector Let T be a regular stochastic matrix. Then the steady-state distribution vector X may be found by solving the vector equation together with the condition that the sum of the elements of the vector X be equal to 1. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

13
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Ex. Find the steady-state vector for the transition matrix: both are Also need: Which gives: So Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

14
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Absorbing Stochastic Matrix An absorbing stochastic matrix has the properties: There is at least one absorbing state. It is possible to go from any non-absorbing state to an absorbing state in one or more stages. Ex. Absorbing Matrix State 3 is an absorbing state and an object may go from state 2 or 1 (non-absorbing states) to state 3 Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

15
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Given an absorbing stochastic matrix it is possible to rewrite it in the form: Absorbing Nonabsorbing I: identity matrix O: zero matrix Ex. 1 2 3 4 3 2 4 1 Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

16
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Finding the Steady-State Matrix for an Absorbing Stochastic Matrix Suppose an absorbing stochastic matrix A has been partitioned into submatrices Then the steady-state matrix of A is given by Where the order of the identity matrix is chosen to have the same order as R. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

17
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Ex. Compute the steady-state matrix for the matrix from the previous example: Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

18
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

The steady-state matrix is: Original columns 3 2 4 1 Original rows This means that an object starting in state 3 will have a probability of of being in state 2 in the long term. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

19
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Game Theory A combination of matrix methods with the theory of probability to determine the optimal strategies to be used when opponents are competing to maximize gains (minimize losses). Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

20
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Ex. Rafe (row player) and Carley (column player) are playing a game where each holds out a red or black chip simultaneously (neither knows the other’s choice). The betting is summarized below Carley holds black Carley holds red Rafe holds black Carley pays Rafe $5 Rafe pays Carley $10 Rafe holds red Carley pays Rafe $2 Carley pays Rafe $3 Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

21
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

We can summarize the game as a payoff matrix for Rafe: C1 C2 R1 R2 The game is a zero-sum game since one person’s payoff is the same as the other person’s loss. Rafe basically picks a row and Carley picks a column. Since the matrix is a payoff for Rafe, he wants to maximize the entry while Carley wants to minimize the entry. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

22
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Rafe should look at the minima for each row, then pick the larger of the minima. This is called the Maximin strategy. C1 C2 minima R1 R2 –10 2 maxima Carley should look for the maxima of the columns, then pick the smallest of the maxima. This is called the Minimax strategy. From this we see that Rafe should pick row 2 while Carley should pick column 2. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

23
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Maxmin Strategy (R’s move) For each row (payoff matrix), find the smallest entry in that row. Choose the row for which the entry found in step 1 is as large as possible. Minmax Strategy (C’s move) For each column of the payoff matrix, find the largest entry in that column. Choose the column for which the entry found in step 1 is as small as possible. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

24
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Ex. Determine the maximin and minimax strategies for each player in a game that has the payoff matrix: Row minima –2 –4 Column maxima –2 The row player should pick row 1. The column player should pick column 3. *Note: the column player is favored under these strategies (win 2) Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

25
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Optimal Strategy The optimal strategy in a game is the strategy that is most profitable to a particular player. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

26
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Strictly Determined Game A strictly determined game is characterized by the following properties: There is an entry in the payoff matrix that is simultaneously the smallest entry in its row and the largest entry in its column. This entry is called the saddle point for the game. The optimal strategy for the row (column) player is precisely the maxmin (minmax) strategy and is the row (column) containing the saddle point. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

27
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

From a previous example: This is a strictly determined game with saddle point. The optimal strategies are for row player to pick row 1 and the column player to pick column 3. The value of the game is –2. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

28
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Mixed Strategies Making different moves during a game. A row (column) player may choose different rows (columns) during the game. Ex. The game below has no saddle point. From a minimax/maximin strategy the row player should pick row 2 and the column player should pick column 3. Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

29
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

For a mixed strategy, let the row player pick row 2, 80% of the time and row 1, 20 % of the time. Let the column player pick column 1, 2, and 3, 10%, 20%, and 70% of the time respectively. Column player: Row player: To find the expected value of the game we compute: payoff Expected value = 1.1 Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

30
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Expected Value of a Game Let P and Q be the mixed strategies for the row player R and the column player C respectively. The expected value, E, of the game is given by: Payoff matrix Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

31
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Optimal Strategies for Nonstrictly Determined Games Payoff matrix Row player: Column player: Where Value of the game: Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

32
**Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc**

Ex. Given the payoff matrix, find the optimal strategies and then the value of the game. Row player should pick each row 50% of the time. Column player should pick column 1, 90% of the time. Value: Copyright (c) 2003 Brooks/Cole, a division of Thomson Learning, Inc

Similar presentations

Presentation is loading. Please wait....

OK

Markov Games TCM Conference 2016 Chris Gann

Markov Games TCM Conference 2016 Chris Gann

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on state of india Ppt on ancient 7 wonders of the world Ppt on viruses and anti viruses free downloads Ppt on post abortion care Ppt on shell scripting expect Ppt on product portfolio management Ppt on milk products for students Ppt on data collection methods in healthcare Ppt on law against child marriage in nepal Ppt on mars one