Presentation is loading. Please wait.

Presentation is loading. Please wait.

Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.

Similar presentations

Presentation on theme: "Using Audit and Feedback to Improve Quality of Care December 7-8, 2012."— Presentation transcript:

1 Using Audit and Feedback to Improve Quality of Care December 7-8, 2012

2 What are we talking about here? A definition of Audit and Feedback: summary of clinical performance over a specified period of time and provision to health care provider(s)/organization(s) NOT: reminders at the point of care NOT: relay of individual, patient-specific clinical data between providers

3 Why Talk about Feedback? the limiting factor to Self- Directed Learning for practicing physicians may be their limited ability to accurately self-assess Davis et al. JAMA 2006 Sep 6;296(9): You cant manage what you cant measure

4 Feedback and Behaviour Change Old behaviours New behaviours Performance Feedback


6 Objective To develop a research agenda to improve the effectiveness of audit and feedback interventions to reliably improve quality of care. Goals: 1.Establish the foundation for a program of research aimed at improving the effectiveness of AF interventions 1a) prioritize elements of AF design that should be tested 1b) identify settings that provide opportunities for testing AF design 1c) utilize methodology to efficiently test and improve upon AF interventions 2. Develop a knowledge translation plan: 2a) integrated collaboration with clinical, administrative, policy stakeholders 2b) disseminating the results of the meeting 2c) developing a wiki-based platform to support ongoing collaboration 2d) establishing plans for new operating grants that will carry out the agenda developed in the meeting INTRODUCTIONS

7 Agenda

8 Results from the Cochrane Review What do we know about the impact of AF? – Cochrane Review and Meta-regression – Cumulative Meta-analysis (not yet published)

9 Findings of 2006 Cochrane Review 88 comparisons from 72 studies dichotomous outcomes median adjusted risk difference of compliance with desired practice = 0.05 (IQR 0.03 to 0.11) Intensity of audit and feedback might also help to explain variation in the absolute effect (p = 0.04). Intensive(individual recipients) AND ((verbal format)OR (a supervisor or senior colleague as the source)) AND (moderate or prolonged feedback) Non-intensive ((group feedback) NOT (from a supervisor or senior colleague)) OR ((individual feedback) AND (writ-ten format) AND (containing information about costs or numbers of tests without personal incentives)) Moderately intensive(any other combination of characteristics than described in Intensive or Non-intensive group). Jamtvedt G et al. Cochrane Database of Systematic Reviews 2006, Issue 2. Art. No.: CD DOI: / CD pub2.

10 Unclear how to do AF an unreliable approach to quality improvement until we learn how and when it works best Foy R. et al. BMC Health Services Research, 2005;5, 50.

11 2012 Update: Changes in Review Methodology ANALYSIS OF HETEROGENEITY Meta-regression Format (verbal; written; both; unclear) Source (supervisor or senior colleague; review organization or employer; investigators; unclear) Frequency (weekly; monthly; less than monthly; one-time) Instruction for improvement (explicit goal; action plan; both; neither) Direction of change required (increase behaviour; decrease; mix/unclear) Recipient (physician; other health professional) Baseline performance (continuous) Risk of bias (high; unclear; low) Plus exploratory analyses: Targeted behaviour (prescribing; test ordering; dm/cvd)

12 CharacteristicN% N% Publication Year Classification of Intervention AF alone Multifaceted case mgmt/team change32 before clinician education (not outreach)4834 Risk of Bias educational outreach2820 Low4532 clinician reminders, cdss1712 Unclear7050 patient intervention86 High2518 continuous qi96 Number of Arms in Trial financial incentives54 Two9870 Three2216 Targeted Behaviour(s) Four2014 DM/CVD mgmt3021 Clinical Setting Laboratory testing/radiology2115 Outpatient9467 Prescribing3122 Inpatient3626 Other5041 Other/unclear107 Targeted Health Professional(s) Medical Specialty(s) Physician12186 GP8460 Nurses1611 Internists6043 Pharmacists54 Other4029 Other Update: Study Characteristics

13 Characteristics of Included Studies: Features of Feedback Design CharacteristicN% N% Format verbal only139 written only8460 both3223 Instructions for Improvement unclear118 Goal-setting118 Source Action planning4129 supervisor/colleague139 Both43 employer1511 Neither8460 investigators/unclear11280 Direction of Change Required Frequency Increase current behaviour5741 weekly118 Decrease current behaviour2921 monthly1914 Mix or unclear5539 less than monthly3626 once only6849


15 Meta-Regression CharacteristicEffectCharacteristicEffect Format of feedbackp=0.020Instructions for improvementp<0.001 Verbal3.4Target/Goal2.52 Written9.5Action plan9.57 Both Verbal and Written11.2Both Goal and Action plan11.09 Not clear5.3Neither6.2 Source of feedbackp<0.001Direction of change requiredp<0.001 A clinical supervisor or colleague16.5Increase current behaviour4.34 A PSRO' or employer2.4Decrease current behaviour10.54 The investigators5.0Chg behaviour/mix/unclear7.16 Not clear5.5Baseline performancep=0.007 Frequency of feedbackp<0.001at 25%9.11 Frequent (up to weekly)1.4at 50%7.07 Moderate (up to monthly)9.8at 75%5.03 Infrequent (less than monthly)4.8Profession of recipientp=0.561 Once only2.6Physician7.9 Unclear;18.1Non-physician6.8 Risk of biasp=0.679 Low risk of bias7.68 Unclear 7.02 High risk of biasn/a

16 Meta-Regression - Exploratory CharacteristicEffect Type of professional practiceP<0.001 Diabetes/CVD5.91 Laboratory testing/radiology referrals4.21 Prescribing11.11 Other4.71 Direction of change requiredP=0.525 Increase current behaviour6.64 Decrease current behaviour7.13 Change behaviour or mix or unclear5.7

17 Summary AF is effective o One quarter of studies find relatively large effect (>16% aRD) AF may be more effective when: o baseline performance is low, o the source is a supervisor or senior colleague, o delivered both verbally and written, o provided more than once, o and includes both explicit targets and an action plan Targeted behavior plays an important role

18 Growing Literature; Stagnant Science?

19 Cumulative Meta-Analysis

20 Feedback and Behaviour Change Old behaviours New behaviours Performance Feedback

21 Questions? Clinical: Personal:

22 Agenda

23 Role of Theory What can it offer?

24 Illustrative Examples Good, bad, and ugly

25 Moving forward Tension between theory and practice?

26 Agenda

27 Small groups: part 1 Building a complete list of potential effect modifiers What are the things to consider when thinking about doing audit and feedback interventions?

28 Small groups: part 2 Prioritizing effect modifiers for testing What are the top five things that we need to know in order to sort out how to make audit and feedback more effective?

29 Agenda




33 What is the impact of engaging the recipient in design of AF What is the impact of adding to AF

34 5 Research Questions 1.What is impact of engaging the recipient in design implementation vs. engaged in none of them 2.What is the impact of adding to AF: Incentives/penalties – financial, CME credit + licensing Tools/practise aid – eg. Clinical decision tool… Practise redesign, coaches, facilitation, mentorship 3.Skill of the person providing feedback make a significant difference 4.Take the top quintile of AF studies and replicate their interventions 5.The 5 most important aspects of AF to study are: 1.Frequency, individual vs. group, evidence based, in person vs electronic delivery, number of targets 2.Developing strategies for replicating and implementing successful intervention in other settings, looking at

35 Dinner 6.30pm Empire Grill 47 Clarence Street (in Byward Market) (About 20 min walk)

Download ppt "Using Audit and Feedback to Improve Quality of Care December 7-8, 2012."

Similar presentations

Ads by Google