1. Markov Processes System Change Over Time Data Mining and Forecast Management MGMT E-5070 scene from the television series “Time Tunnel” (1970s)

2. Markov Process Models  Also known as Markov Chains.  Analyze how systems change over time.  Common applications include:  consumer brand loyalty tendencies  consumer brand-switching tendencies  reliability analysis for equipment  the aging / writeoff of accounts receivable  the spoilage tendencies of perishable items over time

3. Andrei Andreyevich Markov ( 1856 – 1922 ) Андрєй Aндрєєвич Марков  Ph.D, St. Petersburg University (1884)  Studied under Pafnuty Chebyshev.  Professor, St. Petersburg University (1886-1905), but taught informally until 1922.  Early work in number theory, algebraic continual fractions, limits of integrals, and least squares method.  Launched the theory of stochastic processes ( Markov Chains ) : an all new branch of probability theory.

4. Markov Process Models We will consider only the simplest types which are: We will consider only the simplest types which are:  discrete  finite  stationary  first-order THE SCOPE OF STUDY

5. Markov Process Models The states and transitions are “discrete” This means, for example, that market share among different stores can only change once per week or once per month, but not second - to - second. DISCRETE

6. Markov Process Models The number of states is “finite” Means, for example, that an accounts receivable can only age 3 months before it is written off as a bad debt. Once a bottle of wine spoils, there are no additional aging periods for it. FINITE

7. Markov Process Models Transitions depend only on the current state……….not on prior states The chances of an accounts receivable being paid off are higher if it is one month old as opposed to two months old. STATIONARY

8. Markov Process Models Transition probabilities remain constant over time For example, the defection rates of customers from a local supermarket to its competitors month-by-month will always stay the same. FIRST-ORDER PROCESSES

9. Markov Process Models BASIC EXAMPLE TWO BARBERS IN A SMALL TOWN EACH HAVE A 50% MARKET SHARE. THEREFORE, THE VECTOR OF STATE PROBABILITIES AT THIS TIME IS: 0.5 0.5 π ( 1 ) = ( 0.5, 0.5 ) PERIOD NUMBER ONE 50% MARKET SHARE A FOR BARBER “A” 50% MARKET SHARE B FOR BARBER “B”

10. Markov Process Models BASIC EXAMPLE P = [].90.25.10.75 LOYALTYPROBABILITY FOR BARBER “A” IN ANY GIVEN PERIOD DEFECTIONPROBABILITY FOR BARBER “B” IN ANY GIVEN PERIOD DEFECTIONPROBABILITY FOR BARBER “A” IN ANY GIVEN PERIOD LOYALTYPROBABILITY FOR BARBER “B” IN ANY GIVEN PERIOD THE MATRIX OF TRANSITION PROBABILITIES PER MONTH ARE:

11. Markov Process Models BASIC EXAMPLE THE MARKET SHARE EACH BARBER HAS IN THE 2 nd MONTH IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATE PROBABILITIES IN PERIOD ( MONTH ) 1 AND THE MONTHLY MATRIX OF TRANSITION PROBABILITIES: π ( 2 ) = π ( 1 ) x P π ( 2 ) = π ( 1 ) x P.90.25.10.75 ( 0.5, 0.5 ) = ( 0.575, 0.425 ) [].575.425 BARBER ‘A’ BARBER ‘B’.45.125.05.375 BARBER ‘A’ BARBER ‘B’

12. Markov Process Models BASIC EXAMPLE THE MARKET SHARE EACH BARBER HAS IN THE 3 rd MONTH IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATE PROBABILITIES IN PERIOD (MONTH) 2 AND THE MONTHLY MATRIX OF TRANSITION PROBABILITIES: π ( 3 ) = π ( 2 ) x P π ( 3 ) = π ( 2 ) x P.90.25.10.75 ( 0.575, 0.425 ) = ( 0.62, 0.38 ) [].62375.37625 BARBER ‘A’ BARBER ‘B’.5175.10625.0575.31875 BARBER ‘A’ BARBER ‘B’

13. Markov Process Models BASIC EXAMPLE THE MARKET SHARE EACH BARBER HAS IN THE 4th MONTH IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATE PROBABILITIES IN PERIOD (MONTH) 3 AND THE MONTHLY MATRIX OF TRANSITION PROBABILITIES: π ( 4 ) = π ( 3 ) x P π ( 4 ) = π ( 3 ) x P.90.25.10.75 ( 0.62, 0.38 ) = ( 0.653, 0.347 ) [].653.347 BARBER ‘A’ BARBER ‘B’.095.062.285.558 BARBER ‘A’ BARBER ‘B’

14. Steady – State Probabilities Reached when the before and after state probabilities stay the same forever, assuming no changes in the matrix of transition probabilities Here, the eventual market share in the “barber” problem. ALSO KNOWN AS THE EQUILIBRIUM OR STEADY-STATE SOLUTION

15. Equilibrium Condition BARBER EXAMPLE  The eventual market shares of the two barbers can be calculated directly from the matrix of transition probabilities.  Prior period vectors of state probabilities are not required.  The equations:.90 π 1 +.25 π 2 = π 1.10 π 1 +.75 π 2 = π 2.90.10.25.75 [ ] MATRIX OF TRANSITION

16. Equilibrium Condition BARBER EXAMPLE Since π 1 + π 2 = 1.0, π 1 = 1.0 – π 2 Therefore, “ 1.0 – π 2 “ may be substituted for π 1 in either of the two equations below:.90 ( 1 – π 2 ) +.25 π 2 = 1 – π 2.10 ( 1 – π 2 ) +.75 π 2 = π 2 or

17. Equilibrium Condition BARBER EXAMPLE.90 ( 1 – π 2 ) +.25 π 2 = 1 – π 2.90 -.90 π 2 +.25 π 2 = 1 – π 2 -.10 = -.35 π 2 π 2 =.2857 and π 1 = ( 1 -.2857 ) =.7143 SUBSTITUTING IN THE 1 st EQUATION, WE GET: EVENTUALLY BARBER ‘A’ WILL HAVE 71% OF THE MARKET WHILE BARBER ‘B’ WILL HAVE THE REMAINING 29%

18. Equilibrium Condition BARBER EXAMPLE.10 ( 1 – π 2 ) +.75 π 2 = π 2.10 -.10 π 2 +.75 π 2 = π 2.10 =.10 π 2 -.75 π 2 + 1.0 π 2.10 =.35 π 2 π 2 =.2857 π 1 = ( 1 -.2857 ) =.7143 SUBSTITUTING IN THE 2 nd EQUATION, WE GET:

19. Markov Processes with QM for WINDOWS

20. We scroll to “ Markov Analysis ”

21. We click on “ New ” to solve a new problem

22. We specify the number of states. Here, it is the market share for the two barbers

23. The “ initial “ market shares (loyalty rates) are 50% / 50% respectively. ( Barber “A” is “1” ) ( Barber “B” is “2”) The Matrix of Transition is inserted to the right. We desire to find the market shares over “4” periods ( months )

24. The “ End of Period 1 “ is actually the end of period “2” The “ End of Period 2 “ Is actually the end of period “3”, etc. The market share ( loyalty rates ) after 4 months: Barber A ( 1 ) - 66% Barber B ( 2 ) - 34%

25. The ‘Steady-State’ or ‘Equilibrium’ market shares ( loyalty rates ) are: Barber ‘A’ ( 71% ) Barber ‘B’ ( 29% )

26. Markov Processes Using

31. Template

32. Insert the Matrix of Transition here Matrix of Transition for the next three periods

33. To obtain the Steady-State market shares, ( loyalty rates ) go to Tools, Solver

36. Steady State or Equilibirum market shares ( loyalty rates )

38. Gas Station Example MARKOV PROCESSES A town has three gas stations: A,B,C. The only factor influencing the choice of station for the next purchase is the prior purchase. Each station is concerned about brand share. The town’s weekly gas sales are $10,000.00, and we assume each driver buys gas once per week.

39. Gas Station Example MARKOV PROCESSES THE MARKET SHARES AT THIS PARTICULAR TIME ARE AS FOLLOWS: STATION ‘A’ - 30% STATION ‘B’ - 40% STATION ‘C’ - 30% THEREFORE, THE VECTOR OF STATE PROBABILITIES IS: π (1) = ( 0.3, 0.4, 0.3 ) PERIOD NUMBER ONE

40. Gas Station Example THE MATRIX OF TRANSITION PROBABILITIES PER WEEK ARE:.90.05.05.10.80.10.20.10.70 P = LOYALTY RATE STATION ‘A’ LOYALTY RATE STATION ‘B’ LOYALTY RATE STATION ‘C’ ALL OTHERS ARE DEFECTION RATES MARKOV PROCESSES

41. Gas Station Example THE MARKET SHARE THAT EACH GAS STATION HAS IN THE NEXT WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATE PROBABILITIES IN PERIOD ( WEEK ) 1 AND THE WEEKLY MATRIX OF TRANSITION PROBABILITIES: π ( 2 ) = π ( 1 ) x P π ( 2 ) = π ( 1 ) x P P (A) = 0.30 A.90.05.05.27.015.015 P (B) = 0.40 X B.10.80.10 =.04.32.04 P (B) = 0.40 X B.10.80.10 =.04.32.04 P (C) = 0.30 C.20.10.70.06.03.21 P (C) = 0.30 C.20.10.70.06.03.21.37.365.265 A B C π ( 1 ) P

42. Gas Station Example THE MARKET SHARE THAT EACH GAS STATION HAS IN THE THE MARKET SHARE THAT EACH GAS STATION HAS IN THE 3 rd WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF 3 rd WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATE PROBABILITIES IN PERIOD ( WEEK ) 2 AND THE WEEKLY STATE PROBABILITIES IN PERIOD ( WEEK ) 2 AND THE WEEKLY MATRIX OF TRANSITION PROBABILITIES: MATRIX OF TRANSITION PROBABILITIES: π ( 3 ) = π ( 2 ) x P π ( 3 ) = π ( 2 ) x P P (A) = 0.37 A.90.05.05.333.018.018 P (B) = 0.365 X B.10.80.10 =.037.292.037 P (B) = 0.365 X B.10.80.10 =.037.292.037 P (C) = 0.265 C.20.10.70.053.027.185 P (C) = 0.265 C.20.10.70.053.027.185.423.337.24.423.337.24 A B C A B C π ( 2 ) P

43. Gas Station Example THE MARKET SHARE THAT EACH GAS STATION HAS IN THE 4 th WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATE PROBABILITIES IN PERIOD ( WEEK ) 3 AND THE WEEKLY MATRIX OF TRANSITION PROBABILITIES: π ( 4 ) = π ( 3 ) x P P (A) = 0.423 A.90.05.05.381.021.021 P (B) = 0.337 X B.10.80.10 =.034.269.034 P (B) = 0.337 X B.10.80.10 =.034.269.034 P (C) = 0.240 C.20.10.70.048.024.168 P (C) = 0.240 C.20.10.70.048.024.168.463.314.223.463.314.223 A B C A B C π ( 3 ) P

44. Steady–State Probabilities WE KNOW THAT WE HAVE ARRIVED AT THE EVENTUAL MARKET SHARES WHEN THE STARTING PROBABILITIES ARE EQUAL TO THE NEXT STATE PROBABILITIES: π ( Final State ) = π ( Starting ) x P π ( Final State ) = π ( Starting ) x P P (A) = 0.589 A.90.05.05.530.029.029 P (B) = 0.235 X B.10.80.10 =.024.188.024 P (B) = 0.235 X B.10.80.10 =.024.188.024 P (C) = 0.176 C.20.10.70.035.018.123 P (C) = 0.176 C.20.10.70.035.018.123.589.235.176.589.235.176 A B C π ( starting ) P

45. Gas Station Example CONCLUSION Given that the total weekly gas sales are $10,000.00, the average weekly sales per station are: A : ( 0.589 ) ( 10,000 ) = $5,890.00 B : ( 0.235 ) ( 10,000 ) = $2,350.00 C : ( 0.176 ) ( 10,000 ) = $1,760.00 Unless there is some change, the share of market will stay approximately 58.9% for station ‘A’, 23.5% for station ‘B’, and 17.6% for station ‘C’.

46. Calculation of Steady-State Probabilities GAS STATION EXAMPLE INITIALSTATEPROBABILITIES TRANSITIONPROBABILITIES NEW STATE PROBABILITIES X1X2X3ABC A B C A B C.90.05.05.10.80.10.20.10.70 A B C A B C.90X1.05X1.05X1.10X2.80X2.10X2.20X3.10X3.70X3 P(A) = X1 P(B) = X2 P(C) = X3 X =

47. Calculation of Steady-State Probabilities GAS STATION EXAMPLE P(A) =.90X 1 +.10X 2 +.20X 3 = 1X 1 P(A) =.90X 1 +.10X 2 +.20X 3 = 1X 1 P(B) =.05X 1 +.80X 2 +.10X 3 = 1X 2 P(B) =.05X 1 +.80X 2 +.10X 3 = 1X 2 P(C) =.05X 1 +.10X 2 +.70X 3 = 1X 3 P(C) =.05X 1 +.10X 2 +.70X 3 = 1X 3 1X 1 + 1X 2 + 1X 3 = 1 1X 1 + 1X 2 + 1X 3 = 1 DEPENDENT EQUATION INDEPENDENT EQUATION DEPENDENT EQUATION

48. Equation Conversion SET ALL 3 DEPENDENT EQUATIONS EQUAL TO ZERO: P(A) =.9X 1 +.1X 2 +.2X 3 = 1X 1.9X 1 – 1.0X 1 +.1X 2 +.2X 3 = 0 -.1X 1 +.1X 2 +.2X 3 = 0 P(B) =.05X 1 +.8X 2 +.1X 3 = 1X 2.05X 1 +.8X 2 – 1.0X 2 +.1X 3 = 0.05X 1 -.2X 2 +.1X 3 = 0 P(C) =.05X 1 +.1X 2 +.7X 3 = 1X 3.05X 1 +.1X 2 +.7X 3 – 1.0X 3 = 0.05X 1 +.1X 2 -.3X 3 = 0

49. Summary Equations -.1X 1 +.1X 2 +.2X 3 = 0.05X 1 -.2X 2 +.1X 3 = 0.05X 1 +.1X 2 -.3X 3 = 0 1X 1 + 1X 2 + 1X 3 = 1 DEPENDENT EQUATIONS INDEPENDENT EQUATION The 3 dependent equations will not solve for the values of X 1, X 2, and X 3. Therefore, we add the independent equation!

50. The Solution TO ELIMINATE X 1 AMONG THE DEPENDENT EQUATIONS:.05 X 1 -.2X 2 +.1X 3 = 0.05 X 1 +.1X 2 -.3X 3 = 0.05 X 1 +.1X 2 -.3X 3 = 0 -.3X 2 +.4X 3 = 0 -.3X 2 +.4X 3 = 0.05 X 1 -.2X 2 +.1X 3 = 0 “.05” ( 1.0 X 1 + 1.0 X 2 + 1.0 X 3 = 1 ) 05 X 1 +.05 X 2 +.05 X 3 =.05.05 X 1 +.05 X 2 +.05 X 3 =.05.25 X 2 +.05 X 3 = -.05 -.25 X 2 +.05 X 3 = -.05

51. The Solution WITH JUST X 2 AND X 3 LEFT TO SOLVE, WE WILL SOLVE FOR VARIABLE X 2 FIRST: -.3X 2 +.4X 3 = 0 “8” ( -.25X 2 +.05X 3 = -.05 ) -2.0X 2 +.4X 3 = -.40 1.7 X 2 =.40 X 2 =.2352

52. The Solution WE SOLVE FOR X 3 BY SUBSTITUTING “ X 2 =.2352 ” INTO EITHER OF THE TWO DEPENDENT EQUATIONS: -.3X 2 +.4X 3 = 0 -.3 (.2352) +.4X 3 = 0 -.07056 +.4X 3 = 0 -.07056 = -.4X 3.1764 = X 3.1764 = X 3 WITH “X 2 ” and “X 3 ” NOW KNOWN, WE EASILY SOLVE FOR “X 1 ” USING THE INDEPENDENT EQUATION: SINCE 1X 1 + 1X 2 + 1X 3 = 1 1X 1 = 1 – 1X 2 – 1X 3 1X 1 = 1 – 1X 2 – 1X 3 1X 1 = 1 – (.2352 ) – (.1764 ) 1X 1 = 1 – (.2352 ) – (.1764 ) 1X 1 = 1 – [.4116 ] 1X 1 = 1 – [.4116 ] X 1 =.5884 X 1 =.5884

53. Gas Station Market Shares STATE PROBABILITIES WeekP(A)P(B)P(C) 1.30.40.30 2.37.365.265 3.423.337.240 4.463.314.223 5.492.297.211 6.515.283.202

54. Gas Station Market Shares STATE PROBABILITIES WeekP(A)P(B)P(C) 13.577.243.180 14.579.242.179 15.581.240.179 16.583.239.178 17.584.238.178 18.585.237.177

55. Gas Station Market Shares STATE PROBABILITIES WeekP(A)P(B)P(C) 24.588.236.176 25.588.236.176 26.588.236.176 27.588.236.176 28.589.235.176 29.589.235.176

56. Markov Processes with QM for WINDOWS

57. We specify the number of initial states. Here, it is the market share of the three ( 3 ) gas stations

58. The “ Matrix of Transition “ is inserted to the right of the “ Initial States “ We arbitrarily select twelve ( 12 ) periods ( weeks ) for analysis

59. The ‘ Steady - State ‘ or ‘ Equilibrium ‘ market shares ( loyalty rates ) are: Station ‘A’ - 58.9 % Station ‘B’ - 23.5 % Station ‘C’ - 17.6 %

60. The market shares ( loyalty rates ) for weeks 2, 3, 4, and 5 respectively

61. The market shares ( loyalty rates ) for weeks 6, 7, 8, and 9 respectively

62. The market shares ( loyalty rates ) for weeks 10, 11, 12, and 13 respectively

63. Markov Processes Using

67. Template

68. Changes to the transition matrix over 3 periods Excel will not show the market shares for each gas station on a period-by-period basis !

69. By selecting Tools, Solver, the program will provide the equilibrium (steady) state market shares ( loyalty probabilities )

77. Markov Process Models FOUR BASIC ASSUMPTIONS I.The probability of an element changing from one state to another state remains constant from one period to another. II.There are a limited number of states of nature.

78. Markov Process Models FOUR BASIC ASSUMPTIONS III. If we know the present state and the matrix of transition, we can predict any future state. IV.The parameters of the system do not change. That is, none of the states of nature are eliminated, and elements are not entering or leaving the system.

79. Absorbing States  This is where the Markov model contains at least one absorbing or trapping state.  When the system reaches that absorbing state(s), it remains there forever !  Here, steady-state probabilities are no longer mean- ingful. Other measures of performance such as the mean number of transitions until absorption become important. MARKOV PROCESSES

80. Absorbing States LIQUOR STORE EXAMPLE A liquor store owns 10,000 bottles of select wines. Each December, it permits its special customers to purchase wines from its collection. Some bottles that have not aged sufficiently are not offered and are kept for sale in future years. Currently, 3,000 bottles are not available for sale. Of the bottles available for sale in any year, some are not sold until future years. In addition, a bottle not sold may turn bad during the year and will not be available the next year. REQUIREMENT: How many of the 10,000 bottles will eventually be sold?

81. Absorbing States LIQUOR STORE EXAMPLE THE FOLLOWING FOUR STATES ARE DEFINED: 1.AVAILABLE AND SOLD 2.TURNED BAD AND LOST 3.NOT SUFFICIENTLY AGED 4.AVAILABLE BUT NOT SOLD

82. Liquor Store Example THE FOLLOWING TRANSITION MATRIX APPLIES: FROM TO 4 321 1 2 3 4 1 0 0 0 0 1 0 0.1.3.4.2.5.2 0.3 1. AVAILABLE AND SOLD, 2. TURNED BAD AND LOST, 3. NOT SUFFICIENTLY AGED, 4. AVAILABLE BUT NOT SOLD

83. Absorption State Problems SUB - MATRIX SYMBOLS Transition Matrix “T” or “P” Fundamental Matrix “F” or “N” Transition Matrix for Absorption in the “R” or “A” Next Period Transition Matrix for Movement Between “Q” or “B” Non-Absorption States Identity Matrix “I”

84. The Identity Matrix State 1 Available and Sold State 2 Turned Bad and Lost State 1 Available and Sold State 2 Turned Bad and Lost and Lost 1 0 0 1 0 1 “I” ( UPPER LEFT QUADRANT OF THE TRANSITION MATRIX ) FROM TO

85. The “Q” or “B” Matrix State 3 Not Sufficiently Aged State 4 Available but not Sold State 3 Not Sufficiently Aged State 4 Available but not Sold.4.2 0.3 0.3 “Q” ( BOTTOM RIGHT QUADRANT OF THE TRANSITION MATRIX ) FROM TO

86. The “R” or “A” Matrix State 1 Available and Sold State 2 Turned Bad and Lost State 3 Not Sufficiently Aged State 4 Available but not Sold not Sold.1.3.5.2.5.2 “R” ( BOTTOM LEFT QUADRANT OF THE TRANSITION MATRIX ) FROM TO

87. Initial Calculations 1 0.4.2.6 -.2 1 0.4.2.6 -.2 0 1 0.3 0.7 0 1 0.3 0.7 I - Q a b c d 1 -.4 0 -.2 0 - 0 1 -.3 ==- LIQUOR STORE EXAMPLE Q or B Matrix Identity Matrix

88. The Fundamental Matrix F = [ I – Q ] = d/e – c/e -b/e a/e =.7/.42.2/.42 0 /.42.6/.42 Where “e” = ( a x d ) – ( b x c ) = (.6 x.7 ) – ( 0 x -.2 ) = [.42 – 0] =.42 Identity Matrix “Q” or “B” Matrix Negative Power ( matrix inversed )

89. The Fundamental Matrix F = 1.667.476 0 1.429 0 1.429 SUM2.1431.429 FOR “S 3 ” UNAVAILABLE BOTTLES, THIS IS THE MEAN NUMBER OF YEARS UNTIL DISPOSAL ( 2.14 years ) FOR “S 4 ” AVAILABLE BOTTLES, THIS IS THE MEAN NUMBER OF YEARS UNTIL DISPOSAL ( 1.429 years ) THE NUMBER OF PERIODS ( HERE YEARS ) THAT BOTTLES WILL BE IN ANY OF THE NON - ABSORBING STATES BEFORE ABSORPTION FINALLY OCCURS

90. The FR Matrix FR = 1.667.476 0 1.429 0 1.429X.1.3.5.2 ab cd “F” MATRIX “R” MATRIX ef gh =.1667 +.238 0 +.7145 0 +.7145.5001 +.0952 0 +.2858 0 +.2858 ae + bg ce + dg af + bh cf + dh

91. The FR Matrix FR =.4047.5953.7145.2858 S3S3S4S4S3S3S4S4 S 1 S 2 THE PROBABILITIES OF ABSORPTION GIVEN ANY STARTING STATE 40.47% chance that an inadequately-aged bottle will eventually be sold 71.45% chance that an available, but not sold bottle will eventually be sold. 59.53% chance that an inadequately-aged bottle will eventually turn bad and be lost 28.58% chance that an available, but not sold bottle will eventually turn bad and be lost.

92. The Final Solution LIQUOR STORE EXAMPLE 10,000 bottles of wine are currently owned by the liquor store not now 3,000 bottles of wine are not now available for sale ( given ) now available 7,000 bottles of wine are now available for sale ( inferred ) EVENTUAL ( EXPECTED ) SALES in bottles :.405 S 3 :.405 x 3,000 bottles = 1,215 bottles.714 S 4 :.714 x 7,000 bottles = 4,998 bottles 6,213 Σ = 6,213 bottles

93. There are 4 states: 1.Available & sold 2.Turned bad & lost 3.Not sufficiently aged 4.Available but not sold

94. Since we do not know the initial state probabilities, we can arbitrarily assign an equal probability to each event ( 25% ) The ‘ Matrix of Transition ‘ is inserted to the right of the initial state probabilities

95. The ‘ FR ‘ Matrix 40.47% chance that an inadequately - aged bottle will eventually be sold 71.45% chance that an available but not yet sold bottle will eventually be sold 59.53% chance that an inadequately - aged bottle will turn bad and be lost 28.58% chance that an available but not yet sold bottle will eventually turn bad and be lost

97. Eventually, 52.98% of all bottles will be sold & 47.02% of all bottles will be lost

98. The “ B “ or “ Q “ Matrix The “ F “ or “ Fundamental Matrix “ The “ FA “ or “ FR “ Matrix

99. Markov Processes Using

103. Matrix of Transition Development ( period by period ) State 1 Available + Sold State 2 Turned Bad + Lost State 3 Not Aged Enough State 4 Available, Not Sold

105. State 1 Available + Sold State 2 Turned Bad + Lost State 3 Not Aged Enough State 4 Available, Not Sold 40.5% chance bottles not aged enough will be sold eventually 59.5% chance bottles not aged enough will be lost eventually 71.4% chance bottles available will be sold eventually 28.6% chance bottles available will be lost eventually

106. Markov Processes System Change Over Time