Presentation on theme: "Enhanced feedback from perioperative quality indicators: Studying the impact of a complex QI intervention Jonathan Benn Centre for Patient Safety and Service."— Presentation transcript:
Enhanced feedback from perioperative quality indicators: Studying the impact of a complex QI intervention Jonathan Benn Centre for Patient Safety and Service Quality Imperial College London Glenn Arnold Imperial College Healthcare NHS Trust Research Group: Danielle DLima Joanna Moore Igor Wei Alan Poots Alex Bottle Stephen Brett
Declaration of funding and conflicts of interest Project funding: NIHR CLAHRC Northwest London NIHR HS&DR Research Programme Conference attendance funded by: NIHR & Imperial College London Conflicts of Interest: None: No payment received for presentation
QI concept: Provision of real-time feedback on quality of anaesthetic care (for anaesthetists) Anaesthetists rarely receive systematic, routine feedback on the quality of anaesthetic care delivered (and as experienced by the patient) in post-operative recovery
Review of quality indicators in anaesthesia (2009) Perioperative morbidity and mortality data lacks the sensitivity and specificity necessary for analysis of variation in quality of anaesthesia. Few validated indicators incorporating the patient's perspective on quality of anaesthetic care.
Survey of use of quality indicators in perioperative units (2012) Local data collection driven by theatre productivity and external reporting requirements Patient satisfaction with anaesthesia infrequently monitored Post-op patient temperature, pain and nausea data is not reliably monitored and utilised at local level, in the majority of perioperative units
Concept for a quality monitoring and feedback initiative A continuous control loop representing learning at the individual and micro-system levels:
Concept for a quality monitoring and feedback initiative A continuous control loop representing learning at the individual and micro-system levels: Research questions for improvement science: Can we conceptualise data feedback as the core of a quality improvement intervention? Under what conditions are data feedback initiatives effective in improving care?
Contributions from improvement science Continuous process monitoring - an industrial model: Provides a continuous signal, representing variation over time, rather than a snapshot view of standards at one point in time Emphasises reliability rather than the extent of specific deviations Supports open and objective discussion about variations in performance and learning from best practice examples Supports rapid detection and correction of problems in near real- time Effects of QI interventions are observable, iterations are systematic and guided by empirical evidence Disaggregates data onto a level that is meaningful for users Fosters local ownership of data and responsibility for improvement Data collection is integrated within routine operations Metrics are stable and reliable
Research basis for data feedback interventions Systematic reviews of the effects of feedback on professional practice typically show small to moderate positive effects (e.g. Jamdtvedt, 2005) Adding elements (such as education & quality improvement methods) to basic data feedback reports enhance their effectiveness (van der Veer, 2010; de Vos, 2009) Qualitative research suggests that effective data feedback for quality improvement has a number of characteristics (Bradley, 2004) Timeliness Specific to the local context Originates from credible/respected sources Is non-punitive Is sustained over time
IMPAQT (CLAHRC project): Anaesthetic quality monitoring & feedback at St Marys, London CLAHRC improvement model: Iterative change (PDSA) Focus upon local multidisciplinary engagement Supported by continuous measurement and evaluation (SPC) Quality monitoring in PACU: Temperature on arrival in recovery (NICE Guideline) Quality of recovery/anaesthetic: Patient reported Quality of Recovery (QoR) score (Myles, 1999) Post Operative Nausea and Vomiting (PONV) (Categorical) Pain scales (Categorical and continuous scales) Patient transfer efficiency (Ward Wait Time) Additional data is routinely compiled from the theatre and patient administration systems.
St Marys Main Theatres: Data process PACU Surgical wards Pre & Intra- operative care Database Excel templates Anaes. feedback report PACU data posting Ward feedback report Intra-operative care pathway Data validation & cleansing Quality of Recovery, PONV, Pain, Temp, Patient transfer delays Feedback anaesthetic quality indicators (personal level data) Feedback quality of recovery and transfer efficiency metrics Feedback patient transfer efficiency metrics (ward level data)
Monthly PACU & Ward Feedback Data posted in recoverySurgical ward reports
Personalised feedback for anaesthetists (Version 1: Sep 2010)
Enhanced feedback reports (Version 3: Feb-July 2012) Developed based on interviews with end-users Programme of active, trust- wide engagement and work with specialty sub-groups Enhanced monthly report features: Inclusion of multi-site data Comparative perspective: individual vs peer group Longitudinal view on variation in personal and group practice Identification and description of statistical outlying cases to support case-based learning Specialty-specific reporting of Pain scores (to better account for case mix)
Mixed-methods evaluation of anaesthetics QI initiative (NIHR HS&DR) Evaluation of effects upon perioperative process and outcome indicators Interrupted time series analysis of quality indicators dataset merged at case level with hospital administrative data Semi-structured investigation of implementation context and perceived acceptability of the initiative Theoretically-informed qualitative research interviews with consultant anaesthetists and perioperative service leads 2 rounds of interviews: 1) formative, 2) evaluative End-user evaluation Survey data collected at multiple time points Baseline (pre-feedback) Multiple post-implementation follow-ups at three hospital sites
Effects of implementation of feedback on perioperative warming Main anaesthetist cohort, all St Marys surgical cases Mar 2010 - Sep 2013 No FeedbackBasic feedbackEnhanced feedback
Effect of introduction of enhanced feedback (multi-site data) Proportion of patients with temp below 36 degrees: Stepwise decrease of 9% with introduction of enhanced feedback (p<0.01) Proportion of patients reporting no pain or mild pain (compared to moderate or severe): Stepwise increase of 8% with introduction of enhanced feedback (p<0.01) Proportion of patients free from nausea: Small improvement in rate of change over time following introduction of enhanced feedback (p<0.01) No significant effect of feedback on Surgical Site Infection rate No significant effect of feedback on 30 day mortality
Qualitative investigation: Anaesthetists views on feedback I know that Im able to immediately affect the outcome of these measures, so I can do things to make these measures different. I thought: My goodness, I do quite a lot of patients; my goodness, oh, some of them are in more pain than I thought they would be in. So I did some things to change it. For me to improve my practice I would need to first have my own data over a month or over a year.....and also how does my data compare to other anaesthetists that do exactly the same thing I think having departmental level data is important, data for the department that identifies areas where the department as a whole needs to improve or is performing adequately. I dont think were particularly adversarial here, and I think we generally, discuss things and were quite open with each other about our data and about how we do things.
Comparison of pre and post feedback implementation Longitudinal survey evaluation: Usefulness of locally available data for QI Scale: 1 Completely inadequate to 8 Excellent Item descriptions Level of analysis: Relevance of data to personal practice Timeliness: Adequate frequency for monitoring variation Communication: Effectiveness of channel and method of dissemination Data presentation: Clarity and usefulness of graphical formats Credibility: Perception of trustworthiness and freedom from bias