1. Adoption of an Electronic Medical Record-Based Decision Support Tool for Otitis Media Alexander G. Fiks, MD, MSCE, Peixin Zhang, PhD, A. Russell Localio, PhD, Robert W. Grundmeier, MD, Charles Bailey, MD, PhD, Saira Khan, MPH, James Massey, RN, Stephanie Mayne, MHS, Evaline A. Alessandrini, MD, MSCE, Christopher B. Forrest, MD, PhD

2. DISCLOSURE STATEMENT Speaker: Alexander Fiks Dr. Fiks is a co-inventor of the “Care Assistant” that was used to provide clinical decision support in this study. He holds no patent on the software and to date has earned no money from this invention.

3. Unapproved or Off-Label Disclosures for Alexander Fiks Presenter: I have documented that my presentation will not involve discussion of unapproved or off-label, experimental or investigational use.

4. Background  Physicians commonly fail to adhere to practice guidelines  Clinical decision support (CDS) systems provide intelligently filtered, appropriately timed, and actionable information to clinicians at the point of care  Such systems help overcome barriers to guideline- based treatment  However, like guidelines themselves, CDS systems are commonly ignored

5. Background  Otitis Media (OM) is one of the most common disorders in childhood  Acute Otitis Media (AOM): Active infection  Otitis Media with Effusion (OME): Fluid in the middle ear without active infection  OM is the third most common reason for a pediatric office visit and is the principal diagnosis in up to 12% of all office visits  The AAP, CDC, and AHRQ have developed guidelines for OM; however, studies have shown that adherence to guidelines remains low

6. Study Objectives  Within the context of a randomized clinical trial of an EHR-based CDS tool designed to improve OM guidelines: 1) To assess the impact of performance feedback on adoption of CDS 2) Measure the impact of CDS use on quality of care

7. Methods  Design :  Prospective study  Part of a larger randomized trial examining the effect of CDS and clinician feedback on guideline adherence  CDS alone for 12 months then Feedback added for 10 months (8 practices)  CDS only through the study period (8 practices)  Study Population:  Physicians from 16 practices within The Children’s Hospital of Philadelphia’s Pediatric Research Consortium (PeRC)

8. Clinical Decision Support System  Developed by research team for the randomized clinical trial  Delivered using a web service  Appears seamlessly in the electronic health record for children with current ear complaints or history of OM care  Practices were trained regarding CDS use and OM guidelines in 1-hour, in-person sessions lead by pediatricians on the research team

9. Visual Display of OM Events during past 24 months

10. Facilitates documentation of the clinical encounter

11. Supports clinicians’ ordering of guideline-based care

12. Clinician Feedback  After 12 months of CDS only, practices were cluster-randomized to receive feedback or not  Feedback documented physicians’ level of CDS use and monthly adherence to OM guidelines, change in adherence over time, and compared to others in their practice and health system

13. Methods  Aim 1:  Outcome: Physician use of the decision support tool at eligible visits  Exposure: Absence or presence of feedback  Aim 2  Outcome: Adherence to OM Guidelines  Exposure: CDS Use  Covariates:  Visit, clinician, and patient-level characteristics  Study Sample:  41,391 visits at 16 practices with 108 clinicians

14. Overall CDS Use Frequency -Clinicians used the CDS at a mean of 21.3% of eligible visits (median: 8.8%, range: 0-84.8%) -Practices used the CDS at a mean of 16.8% of eligible visits (median: 15.1%, range 0-51%)

15. Characteristics Associated with CDS Use  CDS more likely to be used by clinicians who diagnose OM less frequently (25% vs. 14% of visits, p=0.05)  CDS more likely to be used at visits with less complexity  Sick visits versus preventive (17% vs. 8%, p<0.001)  Visits with 1 diagnosis compared to 6 or more (21% vs. 9%, p<.01)  Children >24 months or age compared to those <6 months of age (18% vs. 14%, p=0.02)  CDS more likely to be used when an action was needed  Visits with AOM (antibiotic ordering) versus OME (18% vs. 13%, p<0.001)

16. Impact of Feedback on CDS Use  Among clinicians with access to CDS, feedback resulted in significant increases in CDS use.  Those with no feedback had a mean decrease in CDS use of -6.8%, while those with CDS and feedback had a mean increase of 2.2%,.  Mean difference in difference of 9.0 percentage points (p=0.001)

17. Impact of CDS Use on OM Guideline Adherence  For all OM:  48% relative increase in pain treatment (p<0.001)  For AOM:  5% increase in use of amoxicillin as a first-line therapy (p=0.007)  5% increase in appropriate antibiotic for penicillin-allergic patients (p=0.04)  17% increase in high dose amoxicillin (p=0.02)  For OME:  12% increase in adequate diagnostic evaluation (p=0.01)  Comprehensive Quality Measures:  For visits at which at least 3 quality measures were relevant, there was an increase in perfect care for AOM and OME (8%, p<0.001 and 9%, p=0.01, respectively)

18. For Whom is CDS Most Effective  We divided our sample of clinicians into those with low, medium, and high guideline adherence at baseline (tertiles).  CDS improved adherence in all groups  Clinicians with low adherence at baseline improved to the greatest extent  70% increase in pain treatment for low vs. 13% improvement for high  30% increase in adequate AOM diagnostic documentation for low vs. 12% decrease for high  Clinicians with low adherence also improved for the most metrics  Improvement in 6/12 metrics for low vs. 4/12 for high

19. Limitations  This study was conducted at a single health care network in one region of the country  The study is not an evaluation of a clinical trial as randomized, but a description of CDS adoption among the study arm randomized to receive CDS  The limited time frame of the study prevents full understanding of:  how long feedback programs can influence provider behavior change  what happens when feedback is removed  how long feedback must persist to achieve optimal effect

20. Study Conclusions  Even effective CDS is often ignored  Implementing clinician feedback along with CDS increases CDS adoption  CDS, when used, improves the quality of care for OM  Benefits are greatest for clinicians with low baseline quality  Results support the overall importance of CDS adoption for improving guideline adherence and the use of feedback to increase CDS adoption

21. Acknowledgements  We thank the network of primary care physicians, their patients and families for their contribution to clinical research through the Pediatric Research Consortium (PeRC) at CHOP.  This project was supported by the Agency for Healthcare Research and Quality (R18 HS017042) and the Eunice Kennedy Shriver National Institute of Child Health & Human Development (K23 HD059919) (AGF).

22. Questions?