1. Stevenson and Ozgur First Edition Introduction to Management Science with Spreadsheets McGraw-Hill/Irwin Copyright © 2007 by The McGraw-Hill Companies, Inc. All rights reserved. Chapter 12 Markov Analysis Part 3 Probabilistic Decision Models

2. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–2 Learning Objectives 1.Give examples of systems that may lend themselves to be analyzed by a Markov model. 2.Explain the meaning of transition probabilities. 3.Describe the kinds of system behaviors that Markov analysis pertains to. 4.Use a tree diagram to analyze system behavior. 5.Use matrix multiplication to analyze system behavior. 6.Use an algebraic method to solve for steady-state probabilities. After completing this chapter, you should be able to:

3. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–3 Learning Objectives (cont’d) 7.Analyze absorbing states, namely accounts receivable, using a Markov model. 8.List the assumptions of a Markov model. 9.Use Excel to solve various problems pertaining to a Markov model. After completing this chapter, you should be able to:

4. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–4 Characteristics of a Markov System 1.It will operate or exist for a number of periods. 2.In each period, the system can assume one of a number of states or conditions. 3.The states are both mutually exclusive and collectively exhaustive. 4.System changes between states from period to period can be described by transition probabilities, which remain constant. 5.The probability of the system being in a given state in a particular period depends only on its state in the preceding period and the transition probabilities. It is independent of all earlier periods.

5. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–5 Markov Analysis: Assumptions Markov Analysis Assumptions –The probability that an item in the system either will change from one state (e.g., Airport A) to another or remain in its current state is a function of the transition probabilities only. –The transition probabilities remain constant. –The system is a closed one; there will be no arrivals to the system or exits from the system.

6. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–6 Table 12–1Examples of Systems That May Be Described as Markov

7. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–7 Table 12–2 Transition Probabilities for Car Rental Example

8. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–8 System Behavior Both the long-term behavior and the short-term behavior of a system are completely determined by the system’s transition probabilities. Short-term behavior is solely dependent on the system’s state in the current period and the transition probabilities. The long-run proportions are referred to as the steady-state proportions, or probabilities, of the system.

9. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–9 Figure 12–1Expected Proportion of Period 0 Rentals Returned to Airport A

10. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–10 Methods of System Behavior Analysis Tree Diagram –A visual portrayal of a system’s transitions composed of a series of branches, which represent the possible choices at each stage (period) and the conditional probabilities of each choice being selected. Matrix Multiplication –Assumes that “current” state proportions are equal to the product of the proportions in the preceding period multiplied by the matrix of transition probabilities. –Involves the multiplication of the “current” proportions, which is referred to as a probability vector, by the transition matrix.

11. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–11 Methods of System Behavior Analysis (cont’d) Algebraic Solution –The basis for an algebraic solution is a set of equations developed from the transition matrix. –Because the states are mutually exclusive and collectively exhaustive, the sum of the state probabilities must be 1.00, and another equation can be developedf rom this requirement. –The result is a set of equations that can be used to solve for the steady-state probabilities.

12. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–12 Figure 12–2Tree Diagrams for the Car Rental Example for One Period

13. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–13 Figure 12–3Two-Period Tree Diagrams for Car Rental Example

14. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–14 Table 12–3Period-by-Period Proportions for the Rental Example, and the Steady-State Proportions Based on Matrix Multiplications

15. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–15 Figure 12–4Development of Algebraic Equations

16. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–16 Table 12–4Transition Probabilities for the Machine Maintenance Example

17. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–17 Exhibit 12-1Worksheet for the Markov Analysis of the Machine Maintenance Problem

18. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–18 Figure 12–5Decision Tree Representation of the Machine Maintenance Problem: Initial State = Operation

19. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–19 Figure 12–6Decision Tree Representation of the Machine Maintenance Problem Initial State = Broken

20. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–20 Exhibit 12-2Solver Parameters Specification Screen of the Machine Maintenance Problem

21. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–21 Table 12–5Transition Matrix for Examples 12-5, 12-6, and 12-7

22. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–22 Figure 12–7Tree Diagram for Example 12-5, Starting from X (Initial State = X)

23. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–23 Figure 12–8Tree Diagram for Example 12-5, Starting from Y (Initial State =Y)

24. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–24 Exhibit 12–3Worksheet for the Markov Analysis of the Acorn University Problem

25. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–25 Exhibit 12–4Second Worksheet for the Markov Analysis of the Acorn University Problem

26. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–26 Exhibit 12–5Third Worksheet for the Markov Analysis and Steady-State Probabilities of the Acorn University Problem

27. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–27 Exhibit 12–6Parameters Specification Screen for the Acorn University Problem

28. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–28 Cyclical, Transient, and Absorbing Systems Cyclical system –A system that has a tendency to move from state to state in a definite pattern or cycle. Transient system –A system in which there is at least one state—the transient state—where once a system leaves it, the system will never return to it. Absorbing system –A system that gravitates to one or more states—once a member of a system enters an absorbing state, it becomes trapped and can never exit that state.

29. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–29 Table 12–6An Example of a Cyclical System Table 12–7An Example of System with a Transient State Table 12–8An Example of a System with Absorbing States

30. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–30 Figure 12–9Probability Transition Diagrams for the Transition Matrices Given in Tables 12-6, 12-7, and 12-8

31. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–31 Exhibit 12–7Excel Worksheet for Example 12-10: The Acorn Hospital Absorbing State Problem

32. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–32 Table 12–9Answers to Example 12-10, Part 1 a through f

33. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–33 Exhibit 12–8Worksheet for The Markov Analysis of Solved Problem 4

34. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–34 Exhibit 12–9Solver Parameters Specification Screen for the Steady-State Calculations for Solved Problem 4

35. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–35 Exhibit 12–10Worksheet for the Steady-State Calculations of Solved Problem 5

36. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–36 Exhibit 12–11Solver Parameters Specification Screen for the Steady-State Calculations for Solved Problem 5

37. Copyright © 2007 The McGraw-Hill Companies. All rights reserved. McGraw-Hill/Irwin 12–37 Exhibit 12–12Excel Worksheet for Solved Problem 6: Accounts Receivable—Absorbing State Problem