1. Economic Evaluation Journal Club: PINT Trial Dmitry Dukhovny, MD MPH Instructor in Pediatrics, Harvard Medical School Neonatologist, Beth Israel Deaconess Medical Center Journal Club February 19, 2013

2. Conflicts of Interest I have no conflicts of interest

3. Agenda Why Economic Evaluations? Brief overview of PINT Trial PINT EE by Kamholz et al.

4. Objectives 1. To understand the key components of an economic evaluation 2. To identify sources to assist with critical appraisal of economic evaluations 3. To be able to critically assess an economic evaluation

5. Why do an economic evaluation?

6. SPR Workshop: Dollars and Sense Framing: Type of Analysis Costing Cost-minimization Cost-effectiveness Cost-utility Cost-benefit “Incomplete” Economic Evaluations “Complete” Economic Evaluations © 2012 Dukhovny, Lorch, Profit, Kamholz, Zupancic

7. Framing: Study Design Frankenstein’s Monster? Vampire of Trials? O’Brien B. Med Care. 1996;34(12 Suppl):DS99-108 Decision AnalysisRandomized Trial Slide used with permission from J AF Zupancic

8. SPR Workshop: Dollars and Sense Cost-Effectiveness Study Cost-Effectiveness = Costs of Treatment A – Costs of Treatment B Effects of Treatment A – Effects of Treatment B © 2012 Dukhovny, Lorch, Profit, Kamholz, Zupancic

9. Health Expenditures as a Percentage of Gross Domestic Product (GDP) in Selected OECD Countries, 1960–2009. Fineberg HV. N Engl J Med 2012;366:1020-1027.

10. How can we can Neonatology more efficient? 1. 1. Decrease cost 2. 2. Improve or not change quality 3. 3. Don’t push anything off to someone else David Cutler, BIDMC Epi confernece 2/7/2013

11. Overview of the PINT Trial

12. PICOT P ELBW (BW<1,000 g), GA<31 wks, <48 hrs old at enrollment I transfusion algorithm C higher vs. lower threshold (depends on DOL, respiratory support and type of sample) O – –Primary: Death before d/c home or survival with BPD, severe ROP (3-5), Brain injury (PVL, ventriculomegally) – –Secondary: Death or CP, Cognitive Delay (MDI<70), severe visual (<20/200 in 1 eye) or hearing impairment (amplification or cochlear implant) T – –Primary: Discharge Home – –Secondary: 18 to 21 months’ corrected age

13. Figure 1 from Kirpalani et al. J Pediatr 2006;149:301-7.

14. Table 2 from Kirpalani et al. J Pediatr 2006;149:301-7.

15. Table 1 from Kirpalani et al. J Pediatr 2006;149:301-7. How were these thresholds determined?

16. Figure 2 from Kirpalani et al. J Pediatr 2006;149:301-7.

17. Table 5 from Kirpalani et al. J Pediatr 2006;149:301-7. Primary Outcome

18. Table 6 from Kirpalani et al. J Pediatr 2006;149:301-7. Secondary Outcomes

19. Table 3 from Whyte RK, et al. Pediatrics 2009;123:207-13. Follow Up Trial: n=430/451 If cut off is MDI<85, cognitive delay favors high threshold group: Adjusted OR 1.81 [1.12, 2.93] (p=0.016)

20. Conclusions Higher Hgb level resulted in more transfusions, but little evidence of benefit at: – –First discharge home – –18 to 21 months’ corrected age

21. If the outcome is equivalent, then how do you decide? PRO Less transfusions –Blood product exposure –Medical Errors –Less IVs –COST CON Trends towards slightly worse outcomes – –NEC/Bowel Perforation – –Length of Stay – –Death or Impairment at 18- 21 months’ corrected

22. Kamholz et al. Archives of disease in childhood Fetal and neonatal edition 2012;97:F93-8

23. Big Picture RCT: – –Generalizability – –In what directions do the issues in the trial bias the result? EE: – –Generalizability – –Stakeholder – –Policy

24. Framing of EE Type of Study Design Perspective Time Horizon Cost-Effectiveness Analysis Alongside RCT Third Party Payer 18-21 months’ CGA

25. Costs 2008 Canadian Dollars Discount Rate 3% (a dollar in your hand right now is worth more then the same dollar in 1 hour or 1 month) Used case report forms from RCT –Per diem costs (based on respiratory support)  Adjusted for nurse to patient ratio –Transfusion Costs –Surgery –Physician fees –Re-hospitalization post discharge home

26. Table 1 from Kamholz et al. Archives of disease in childhood Fetal and neonatal edition 2012;97:F93-8

27. SPR Workshop: Dollars and Sense Types of Resource Costs Health Care Related Costs – –Direct Medical Costs   Variable: Drugs, personnel, tests   Fixed: Land, equipment Non-Health Care Related Costs – –Direct Non-Medical Costs   Child care, parking, meals, gym membership Productivity Costs – –“Absenteesim”   Work absence of family or patient due to illness – –“Presenteeism”   Decreased productivity of family or patient due to illness – –Employment choices due to condition (eg CP) © 2012 Dukhovny, Lorch, Profit, Kamholz, Zupancic

28. Effectiveness Survivor without BPD (at 1 st discharge home) Survivor without NDI (at 18-21 months’)

29. TrialAuthorYearPerspectiveTime Horizon Measure of Effectiveness Surfactant Rescue Backhouse19943 rd party payer1 yearSurvivor w/o impairment iNO for PPHNLorch2004SocietalDischarge home Per life year gained; QALY ECMOPetrou20063 rd Party Payer7 yearsPer life year gained TIPPZupancic20063 rd party payer18 months CGA Survivor w/o impairment NO CLDZupancic20093 rd party payerD/C homeSurvivor w/o BPD SODMcBride20093 rd party payerD/C home; 1 year of age Survivor w/o BPD; Chronic Respiratory morbidity averted ET ROPKamholz20093 rd party payer9 monthsCost per eye with severe visual impairment averted PINTKamholzDraft3 rd party payer18 to 21 months Survivor w/o impairment

30. Analysis

31. SPR Workshop: Dollars and Sense Cost-Effectiveness Study Cost-Effectiveness = Costs of Treatment A – Costs of Treatment B Effects of Treatment A – Effects of Treatment B © 2012 Dukhovny, Lorch, Profit, Kamholz, Zupancic

32. Table 2 from Kamholz et al. Archives of disease in childhood Fetal and neonatal edition 2012;97:F93-8

33. Uncertainty/Sensitivity Analyses Deterministic Probabilistic

34. Supplement Table from Kamholz et al. Archives of disease in childhood Fetal and neonatal edition 2012;97:F93-8

35. iCER Plot Figure 1 from Kamholz et al. Archives of disease in childhood Fetal and neonatal edition 2012;97:F93-8

36. Cost-Effectiveness Acceptability Curve Figure 2 from Kamholz et al. Archives of disease in childhood Fetal and neonatal edition 2012;97:F93-8

37. Limitations International Trial, but only used Canadian Costs Time horizon at 2 years Quality of Life Data Societal Perspective (would taking family expenses move the estimate in one direction or another?)

38. Summary PINT trial showed similar outcomes at discharge and 18-21 months’ corrected age for different transfusion thresholds – –Trends for better in higher threshold group The cost estimate appears favorable towards the higher threshold group, but a wide confidence interval around it

39. Take Home Points If clinical equivalence between 2 treatment options, must consider a “risk-benefits” calculus Systematically look at Economic Evaluations just like you do at RCTs

40. References for Critical Appraisal of Economic Evaluations Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ 1996;313(7052):275-83. – –British Medical Journal has a checklist that is required to be filled out along with the paper submission: http://resources.bmj.com/bmj/authors/checklists-forms/health-economics Ungar WJ, Santos MT. The Pediatric Quality Appraisal Questionnaire: an instrument for evaluation of the pediatric health economics literature. Value Health 2003;6(5):584-94.