1. Markov versus Medical Markov Modeling – Contrasts and Refinements Gordon Hazen February 2012

2. Medical Markov Modeling We think of Markov chain models as the province of operations research analysts However … The number of publications in medical journals – using Markov models – to address medical cost-effectiveness – approaches 300 per year! 2

3. Medical Markov Modeling Why the large buy-in from the medical community? – Easy-to-use software that combines decision trees and Markov models (Data, TreeAge) – Simplicity of models Discrete time Transient 3

4. Overview of this talk 1.Background on medical Markov modeling 2.Population modeling versus individual-level modeling 3.Product structure in medical Markov models

5. Overview 1.Background on medical Markov modeling 2.Population modeling versus individual-level modeling 3.Product structure in medical Markov models.

6. Medical Markov Modeling The kind of modeling that is typical 6 IHD = Ischemic heart disease MI = Myocardial infarction (heart attack) A simplification of: Palmer S, Sculpher M, Phillips Z, Robinson M, Ginnelly L, Bakhai A et al. Management of non-ST elevation acute coronary syndrome: how cost-effective are glycoprotein IIb/IIIa antagonists in the U.K. National Health Service?. International J Cardiology 100 (2005) 229-40.

7. The kind of modeling that is typical Cohort analysis 7

8. Our preference: Continuous-time Cohort analysis in continuous time 8 p MI =  t p 0 =    t p 1 =    t

9. Our preference – continuous time Discounted expected quality-adjusted life years: 9

10. Cohort analysis in continuous time Intervention: Post-MI mortality rate  1 = 0.1/yr is decreased by 75% and the MI incidence rate = 0.12/yr is decreased by 50%. 10 6.62 QALY/patient10.28 QALY/patient

11. Continuous-time version of cohort analysis Let dt  0 to obtain 11 … the Kolmogorov differential equations. The cohort analysis procedure is merely the Euler method for solving the Kolmogorov equations.

12. Overview 1.Background on medical Markov modeling 2.Population modeling versus individual-level modeling 3.Product structure in medical Markov models.

13. Question: How to incorporate population issues? An intuitive approach: Restart following death 13 Then compute steady-state probabilities in the resulting irreducible chain. Open routing process Closed routing process

14. Question: How to incorporate population issues? Balance equations for steady-state probabilities: 14 Intervention assumptions: – Post-MI mortality rate  1 = 0.1/yr is decreased by 75% – MI incidence rate = 0.12/yr is decreased by 50%. Results:  PostMI increases from 23.0% to 38.5% The population is less healthy! So what is wrong here?

15. Population issues: A more rigorous approach Observation: A population of non-interacting individuals is equivalent to a Jackson network of infinite-server queues. 15

16. Equilibrium results Jackson network balance equations 16 Theorem (e.g. Serfozo 1999): The counts n j of individuals in health state j are, at equilibrium, independent Poisson variables with means  j given by the solution to the balance equations.

17. Equilibrium results Solve balance equations with entrance rate = 1000/yr More survivors under intervention!

18. The closed routing process again Convert the open routing process to a closed one in the following way 18 OpenClosed

19. Open versus closed routing Theorem (Hazen and Huang 2011): One may obtain equilibrium means from steady state probabilities, and vice versa: 19 Equilibrium means  j Steady-state probabilities  j.

20. Open versus closed results 20 Equilibrium means  j Steady-state probabilities  j.

21. Example: Re-analysis of the preventive use of tamoxifen Original analysis: Col et al 2002 Tamoxifen – an estrogen agonist/antagonist – an effective therapy against established breast cancer Evidence that it can reduce breast cancer incidence But life-threatening side effects – endometrial cancer – vascular events. Would the benefit of its prophylactic use in healthy women be worth the associated risks? 21

22. Preventive use of tamoxifen: Our model Cure rate models for breast and endometrial cancer treatment – Mortality decreases in time survived after cancer diagnosis. – This cannot be directly modeled as a Markov model – it is semi-Markov. – Cure rate model with unobserved states Cured/ Not Cured allows implicit mortality to decrease over time survived. 22 Breast cancer incidence and treatment Endometrial cancer incidence and treatment

23. Preventive use of tamoxifen: Our model Overall model is the Cartesian product of the two factors below and a third Background Mortality factor. More on this later … 23 Breast cancer incidence and treatment Endometrial cancer incidence and treatment

24. Preventive use of tamoxifen: Our model Estimated parameters (max likelihood estimates) 24

25. Preventive use of tamoxifen: Our model Product structure for quality of life – Q jk = Q bj Q ek – more on this later 25 Model entry rate 0 = 110,000/yr – 2.3 M women reaching age 50 each year x 4.8% at high risk for breast cancer

26. Preventive use of tamoxifen: Results A more nuanced picture of the effects of this intervention than just incremental QALYs. 26 Incremental QALYs/woman Incremental equilibrium means Incremental equilibrium probabilities

27. Overview 1.Background on medical Markov modeling 2.Population modeling versus individual-level modeling 3.Product structure in medical Markov models.

28. Markov models with product structure Product structure is relatively common in medical Markov models 28 Roach PJ, Fleming C, Hagen MD, Pauker SG. Prostatic cancer in a patient with asymptomatic HIV infection: are some lives more equal than others? Med Decis Making. 1988 Apr- Jun;8(2):132-44.

29. Markov models with product structure Much simpler depiction of model structure: Independent factors 29

30. Product structure is relatively common Schousboe et al. considered 5 different types of fractures: – hip fracture – clinical vertebral (Cv) fracture – radiographic vertebral (Rv) fracture – distal forearm (Df) fracture – other fracture In principle this should allow 2 5 = 32 state combinations corresponding to 5 factors each at 2 possible levels. What their model actually did: 6 states – 5 states corresponding to a single fracture type – 1 other state corresponding to the combination of the worst two possible fracture types – Disadvantage: Such a model “forgets” past fractures when a new fracture occurs, which the 32-state model would not do. 30 Schousboe JT, Nyman JA, Kane RL, Ensrud KE. Cost-effectiveness of alendronate therapy for osteopenic postmenopausal women. Ann Intern Med. 2005 May 3;142(9):734-41.

31. Advantages of explicitly accounting for product structure Model formulation: Simpler to merely consider one factor at a time Model presentation: Simple factors easier to understand and critique. – Model is less likely to be perceived as a “black box” Computational advantages as well when factors are independent. 31

32. Computation of QALYs under product structure 32

33. Computation of QALYs under product structure 33

34. Computation of QALYs under product structure 34

35. Computation of expected cost under product structure 35

36. Advantages of factored computation Computational work in cohort analysis is proportional to the number of state transitions Suppose the number of transitions in a factor with s non- death states is roughly s also. Then assuming s states in each factor and f factors, – s f transitions in the overall model under naïve cohort analysis – s  f transitions in cohort decomposition – Big advantage for large f Caveat: s and f are not usually large. 36

37. Cohort decomposition issues How often are factors independent? – Ans: More often not probabilistically independent. – But one factor is almost always probabilistically independent: Background mortality. How reasonable is the product form for the quality coefficient v(x)? – Empirical support for product form in HUI literature – additive decomposition is not supported. – Often only one factor carries quality adjustments, in which case product form holds by default. 37

38. Summary These are just the basics – Population modeling Population model  Jackson network One can get at equilibrium population issues by solving the usual balance equations for steady-state probabilities and scaling them up appropriately. – Product structure Common feature of medical Markov models Recognizing it can assist in model formulation and presentation, as well as computation. – Drawbacks for continuous-time models Medical researchers don’t “get” the models Software not widely available There is more to do here …

39. Questions?