Presentation on theme: "Opening the "black box" of PDSA cycles: Achieving a scientific and pragmatic approach to improving patient care Chris McNicholas, Professor Derek Bell,"— Presentation transcript:
1 Opening the "black box" of PDSA cycles: Achieving a scientific and pragmatic approach to improving patient careChris McNicholas, Professor Derek Bell, Dr Julie ReedNational Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) Northwest London, Imperial College LondonThe Health FoundationUnited KingdomFunded by CLAHRC NWL and Health FoundationCLAHRC NWL is…. Practical experience of improvement methods…
2 Types of evidenceKey questions about the QI methodologySources of evidence to answer those questionsTheoretical evidenceHow and why does it work? What is the underlying ‘programme theory’?Descriptions of the methodology's intended mechanism or action, setting out the programme logic or intended causal sequence and drawing on appropriate social science theoryEmpirical evidenceWhen, for whom and how well does it work? What effects does it have? What does it cost?Qualitative and quantitative evaluations of the methodology's implementation, using rigorous and robust comparative methods to quantify effects, and undertaken independentlyExperiential evidenceWhat is it like to use? What has been learned about its application in a wide variety of settings or contexts?Descriptive accounts of the methodology in use, synthesis of practitioner experience and feedback, collation of learning and interchange among networks of usersA more sceptical and scientifically rigorous approach to the development, evaluation and dissemination of QI methodologies is needed, in which a combination of theoretical, empirical and experiential evidence is used to guide and plan their uptake. Similarity of improvement methodsNeed for theoretical, empirical and experiential evidenceImprovement science research not outcomes of a single projectNeed broader overarching look at improvement methods
3 Outline Theory of PDSA (Systematic Review) Reality of PDSA (International Observational Study)Revisiting Theory PDSAEmpiricalExperientialStart with theoryPresent our preliminary resultsRevisit thinking about theory
4 PDSA Cycles – Why the interest? Why PDSA – crux of change, scientific method, popular, varied effectiveness, little overarching view of its applicationWhat systematic review was – theory – say 5 thingsFindings – brief/variationNeed to unpack Black boxThe Plan-Do-Study-Act (PDSA) cycle is a common tool to test changes in a pragmatic and scientific fashion.The effectiveness and application of the method, however, is varied with a lack of adherence to key principles (1).This research examines facilitators and challenges to achieving good quality use in improvement initiatives and at the influence of organizational context (2).
5 Research QuestionsWhat are the perceived functions and benefits of PDSA method?Are these functions and benefits applied in practice?How do social and organisational contextual factors influence the improvement work and the use of PDSA in practice?1. Read Qs
6 An International Observational Study 4 International SitesSpecific improvement initiatives, Organisational improvement support, Broader organisational contextMethods: Interviews (65), Observations (70 hours), Focus groups (6), Document analysis (PDSA cycles)Technical, Social and Contextual research lensesInternational qualitative observational studySpending prolonged period of time with frontline improvement teamsTheory – technical (systematic review), social (boundary objects/communities of practice), context – MUSIQ – super quick – only for those in the knowA qualitative observational study comparing improvement initiatives in 3 healthcare organizations based in Australia, UK and USA.Data collection included non-participant observation (n=70 hours), document analysis, semi-structured interviews (n=65) and focus groups (n=6).Participants included improvement initiative members and organization leaders.Thematic analysis included both deductive coding to build on established theoretical frameworks (1) (2) and inductive coding to support the identification of new themes and concepts.MUSIQ (Model for Understanding Success in Quality) Kaplan et al, 2013)
7 Observing the reality of PDSA FeatureOrg 1Org 2Org 3Org 4Documentation of cyclesIterative cyclesStart testing on small scaleUse of regular data over timePrediction-based test of changeSelected Themes from Preliminary AnalysisUsing quantitative data to inform progression of cyclesManaging complexity of emergent learning - scaling up and iterative cyclesSocial factors influencing PDSA useVaried compliance with technical theoryDocumentation – question value which results in low completitionOur observations reveal the intricacy of PDSA in practiceThe 3 themes we have selected to present todayPhrase as related to audience – I’m sure you have experienced the challenges of… type thingAll recognized the value of PDSA cycles to structure tests of change and empower staff to lead change in complex environments.
8 Using quantitative data to inform progression of cycles The nitty gritty of having data metrics in a database……we had a concept, we revisited it and we said we need this; by the way, that data isn’t currently being captured…that had to be designed, that had to be added, the data started being collected… and maybe three, four, five months goes by when all that is happening and now our data just started last week…And then, of course, the physicians will get frustrated, because it’s, like - ‘I thought we defined this months ago’.Data concept to data metric takes timeDisappointed physician – read quoteThis prevented effective learning and iteration as teams did not effectively pause and conduct a “study” stage.
9 Scale Up and the Disappearing PDSA DATA?Daily verbal remindersReminders in notesScale of testingFormal educationAll patients for another weekWhat we are taughtNot linear – complex/messyBlurred as scale increasesUnderstand why this isAll patients for one week5 patients3 patients1 patientTime
10 Unpacking a “single” large scale PDSA Doctor AvailabilityMacrosystemCoding ofPatient NotesJobPlansScale of testingData AvailabilityProcess 2Not Following PlanSphere of contextual influenceWard A1 example – was described to us as a single large scale PDSA – but in reality mutliple PDSATaking place different levels of influence – think how much detail to go in to – 1 or 2 examples?Positive that PDSA can help navigate this complexityData part of this challengePDSA cycles are typically taught as a smooth chain of cycles testing change. Changes are adopted, abandoned or adapted. These decisions inform future cycles. Scale of testing grows as confidence of success, whatever that may be, grows.Process 1Following PlanCompletion of Post Take NotesMicrosystemTime
11 “its all about social skills – the technical are important but you wont be successful without social skills”Using the Plan to negotiate different perspectivesPrior QI experience“That’s actually where I think the most value comes in… you have to have a conversation with people to realise most of us don’t hold with all of it, right? …that’s a two-hour conversation sometimes …just getting to that point is what takes a long time, but also where… the most valuable conversation can happen.”Engagement tacticsNo Prior QI experience"If I got my laptop out in the meeting and went through a PDSA, people wouldn't come back. It's a fine line between being useful and pushing people away”Example of social skills need to engage – different tactics employedWhen no prior experience – don’t mention jargon, use words like testing – ask people for predictionsWhen experienced – use as tool to negotiate different perspectivesAlso from an organisational perspective PDSA was method of choice as it was seen to empowers users to take responsibility and provides a freedom to actDeveloping planNegotiating the ideal and real worldsUnderstanding context – QI and frontline staff collaborationCommunicating plan to those doing testBuddy system– toilet break?Director promoting testsEnsuring someone is there to ‘observe’ test to maximise learningUnclear inclusion criteriaImpartial observersReliability of data being reportedLearning the numbers game (learning vs performance)Continual enquiryReflection on results against predictionsDoes data relate to learning and/or interventionsPredictions routinely madeDeciding implications for next stepsLimited sphere of Influence/opposing organisation initiativesSenior level buy-inAgreeing action plan for next stepsHow to prioritise/limited scope – root cause vs symptomPlans for ramping
12 Enhancing the theory of PDSA EmpiricalExperientialPDSA as complex social-technical toolPDSA as boundary object between different groupsUsing quantitative data in social and contextual dependent worldHow can we structure the management of PDSA cycles?How do we prepare people for the reality of using PDSA?What are the generic implications for change management, learning organisations and knowledge mobilisation?These practical examples are intended to support better use of PDSA cycles and further ground the method a scientific approach to improve patient care. This work outlines an important enquiry into the fidelity of use of improvement methods by comparing their application to the key principles of the method and exploring how fidelity of use is influenced by facilitators and barriers within the local context.This study can act as a research template to unpack the “black box” of PDSA cycle and other QI methods to inform education and conduct of QI. By furthering this field of research we are able to better understand how the use of improvement methods influences improvement success.12
13 Structuring Complexity Learning and Improvement UsabilityApplicabilityScalabilityCurrent processAbility to measureSustainability(self-sufficient)‘Maintaining’?Daily verbal remindersReminders in notesScale of testingFormal education“Implementing”All patients for another weekWhat we are taughtNot linear – complex/messyBlurred as scale increasesUnderstand why this isAll patients for one week“Testing”5 patients3 patients1 patientTime
14 Walshe, K. (2009). Pseudoinnovation: the development and spread of healthcare quality improvement methodologies. International journal for quality in health care, 21(3),Taylor, M. J. et al (2013) Systematic review of the application of the plan-do-study-act method.BMJ Quality & Safety. doi: /bmjqsKaplan, H. C. et al (2012) The Model for Understanding Success in Quality (MUSIQ). BMJ Quality & Safety, 21(1), 13-2Ogrinc, G., & Shojania, K. G. (2013). Building knowledge, asking questions. BMJ quality & safety, bmjqs-2013.Funders:National Institute of Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) Northwest LondonThe Health Foundation