1. Studying hospital quality using mixed methods Elizabeth H. Bradley, PhD Yale University School of Medicine

2. 2 Disclosure and Acknowledgments The research is funded by the Agency for Health Care Research and Quality (#R01HS10407-01), and the Donaghue Medical Research Foundation (#02-102) Conducted in collaboration with Genentech and the National Registry of Myocardial Infarction (NRMI) Investigators

3. 3 Purpose of the presentation Provide an example of using mixed methods to identify determinants of hospital quality Highlight the benefits and special challenges in employing mixed methods

4. 4 Background ACC/AHA guidelines recommend beta- blocker prescription for patients hospitalized with acute myocardial infarction (AMI) However, many patients do not receive beta-blockers after AMI, and hospitals vary substantially in rates of beta blocker use and improvement in those rates over time

5. 5 Improvement in hospital beta- blocker rates, 1996-1999

6. 6 Objective To identify success factors in hospitals’ increased rates of beta-blocker use for patients with AMI… what works?

7. 7 Mixed methods Qualitative study Site visits in higher/lowerperformers In-depth interviews with key staff Quantitative study Closed-ended survey of random sample of hospitals

8. 8 Qualitative study Site visits with in-depth interviews (n=45) in 8 hospitals with higher versus medium/lower performance in beta-blocker rates 14 physicians 15 nurses 11 quality improvement staff 5 administrators (senior and mid-level) Constant comparative method for data analysis of qualitative data

9. 9 A taxonomy to classify QI efforts along key dimensions Goals – content, specificity, sharedness Admin support – philosophy, resources Clinical support – physician, nurse, ancillary Systems design – standing orders, pathways, reminders, care coordinators, etc. Data feedback – validity, timeliness Contextual factors – size, teaching status, system affil, financial constraints, market and regulatory pressures, etc.

10. 10 Hypotheses about “what works” In beta-blocker performance, presence of clinical champions and administrative support for quality improvement are more important than systems design interventions Data feedback, especially when it is physician- specific, can be viewed as punitive and can backfire as an improvement effort

11. 11 Quantitative study Cross-sectional study of 234 hospitals from those participating 30+ months during Apr 96-Sept 99 in the National Registry of Myocardial Infarction (reflects 54.2% response rate) Patients: 60,363 treated for AMI in these hospitals during 1998-1999, the years just after beta-blocker recommendations were widely published Telephone survey of QI directors at each hospital Hierarchical models to estimate p (high-rate hospital)

12. 12 Hospital sample (n=234) Beta-blocker rates Mean, range60%;19% - 89% Urban83% Teaching39% Annual AMI volume (median) 137 patients

13. 13 QI efforts Type of QI effortPrevalence Standing orders57% Clinical pathways58% Educational efforts76% QI teams80% Care coordinators50% Reminder forms28% Computer support34%

14. 14 QI efforts (continued) Type of QI effortPrevalence Data feedback reports 97% Quarterly reports81% Public display of data34% Data reports last 6 months39% Data are physician-specific11% Has physician champion92%

15. 15 QI efforts and performance QI effortAdj ORp-value Standing orders 2.3.066 Physician champion10.5.001 Admin support [1-5] 2.0.009 Data feedback that is physician-specific 0.1.001

16. 16 Qualitative study benefits The qualitative study augmented conceptual background for quantitative study: 1.Taxonomy with which to characterize QI efforts, a multifaceted intervention 2.Hypotheses about systems interventions, clinical champions, administrative support, and data feedback 3.Comprehensible language for survey design

17. 17 Special challenges of using mixed methods Integration the qualitative and quantitative work – benefit comes from their integration but easy to split them off The “juicy” ideas in qualitative work can be difficult to measure (organizational change, culture, etc.) and test in larger samples Qualitative work often slows down and delays the quantitative work Publishing mixed methods – a special challenge (length, reviewers’ tolerance for unknown methods)

18. 18 Why use mixed methods? “Grounds” our work, so that we ask the important questions, have realistic hypotheses, and use sensible language Increases the potential that research will be more easily translated into practice

19. 19 Conclusions Mixed methods studies are particularly advantageous for some, but certainly not all, topic areas Research inquiries that involve multifaceted interventions, interdisciplinary interactions, or innovation and organizational change are good candidates for mixed methods