PHS / Department of General Practice Framework for the evaluation and implementation of clinical prediction rules Susan Smith, Emma Wallace and Tom Fahey

PHS / Department of General Practice Overview Implementation of CPRs –Impact analysis Current implementation strategies –Criteria applied to existing studies Issues arising Proposed framework

PHS / Department of General Practice Stages of development of CPR

PHS / Department of General Practice Implementation literature Reilly et al; Strategies for overcoming barriers for effective use of decision rules Lang et al –Knowledge, Attitude and Behaviour Overlap with knowledge translation and guideline implementation (Toll et al; Glasziou et al)

PHS / Department of General Practice Barriers to uptake KnowledgeAttitudesBehaviour Sources of Behaviour Change Lack of familiarity 1.Time Investment 2.Inadequate infrastructure Lack of agreement 1.Uncertain application 2.Unclear applicability External Barriers 1.Lack of time 2.Resources 3.GP Software companies 4.Poor integration into workflow Barriers to Innovation Adapted from Lang et al. 2007

PHS / Department of General Practice ©2006 by American College of Physicians Strategies to overcome barriers to implementation (Reilly et al) Three phases –Before rule introduced –During introduction of rule –After impact analysis Barriers and potential solutions addressed for each phase

PHS / Department of General Practice Combined approaches examining behaviour change and phases of CPR implementation Used in 15 identified impact analysis studies Considered strategies used before and during introduction of CPR Examined ‘after phase’

PHS / Department of General Practice A. Before introducing rule 1.Preparation of clinicians (Knowledge) 2.Simulated impact analysis (Attitudes) 3.Decision rule adopted as standard of care 4.Characteristics of rule (Attitudes) a)Face validity b)User friendly

PHS / Department of General Practice B. During use of rule 1.Eliciting clinician concerns 2.Feedback (Behaviour) a)Use of rule (extent of overruling) b)Impact on outcomes 3.Incentives (Behaviour) C. After impact analysis

PHS / Department of General Practice Criteria N = 15 YesNoNot reported Notes Before: Preparation of clinicians 924 Simulation1140Reilly et al CPR as standard of care 1311Variation+ Face validity4011Design of rule User friendly2013N=NR

PHS / Department of General Practice Criteria N = 15 YesNoNot reported Notes During: Clinician concerns 4110 Feedback on use of rule 4110 Feedback on outcomes 4?110Exact feedback not specified Incentives114Research assistance After 2132 discuss generalisability

PHS / Department of General Practice Issues arising Majority about decision support Majority part of a complex intervention –CPRs incorporated in clinical care guidelines and critical care pathways Clinician preparation and training could be improved Usually about avoiding overtreatment Underuse of computerised decision support Feedback during use of rule not standard

PHS / Department of General Practice Issues arising (2) Generalisability an issue (system barriers) –Chest pain rule Define impact –Efficacy vs effectiveness (actual vs potential impact) –Safety and efficiency

PHS / Department of General Practice Potential output: CPRs as complex interventions Framework for the evaluation and implementation of clinical prediction rules

PHS / Department of General Practice Framework for the evaluation and implementation of clinical prediction rules Phase 1: Exploratory Systematic review of validation studies Phase 1: Exploratory Systematic review of validation studies Phase 2: Prior to impact analysis Clinician preparation, simulation of impact analysis Phase 2: Prior to impact analysis Clinician preparation, simulation of impact analysis Phase 3: Impact analysis Appropriate design and feedback Phase 3: Impact analysis Appropriate design and feedback Phase 4: Dissemination Continuous quality improvement Phase 4: Dissemination Continuous quality improvement