2 Learning ObjectivesReview with Infant Mental Health Advisory Board: Part 1-- Foundational Continuous Quality ImprovementResults of the Self-Assessment Survey resultsVote on priority area to develop a CQI plan based on the self- assessment survey and Advisory Board feedback
4 Performance Management and CQI Two primary components:Performance Management: Using data to improve performanceCQI (Continuous Quality Improvement): Involving all stakeholders to examine the performance of a process and address identified gapsPerformance Management and CQI are distinct processes. While some of the components may overlap, there are marked differences as well.
5 Quality ImprovementUses data on how you are performing to drive activities which lead to true improvement.Utilizes stakeholders to address identified gaps continually improve processesAddresses the question: How do we get better?
6 CQI Process: PDSA Cycle The use of a deliberate anddefined improvement processPlan-Do-Study-Act Cyclewill be used for this project
7 CQI Process: PDSA Cycle When applying PDSA there are 3 key questions and 4 QI principles that will guide efforts.The 3 key questions to keep front and center during QI work are:What are we trying to accomplish?How will we know that a change is an improvement?What changes can we make that will results in improvement?
8 CQI Process: PDSA Cycle The 4 CQI principles are:Develop a strong client focusContinually improve all processesInvolve stakeholdersMobilize both data and team knowledge to improve decision-making
9 PDSA Cycle: Nine StepsThe PDSA cycle is used to improve existing processes, not to plan or implement new programsAlways relies on data to support its decisions
10 PDSA Cycle: Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Steps 1-5)Step 1: Identifying a problemStep 2: Assembling a Q.I. team that can address the problemStep 3: Identify what the true cause(s) of the problem may be, map the process to understand the existing activities that lead to the problematic issueStep 4: Brainstorming and conducting background research to look for possible solutionsStep 5: Working to develop a theory of improvement that can be tested
11 Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Step 6) Step 6: Test your theory of improvementimplement a small-scale test of the change to your process.Collect, chart, and display data to determine the effectiveness of the improvement.Document problems, unexpected observations, and unintended side effects, because these occurrences will aid in the learning process.
12 Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Step 7) Step 7: Study the Results of your TestUse data to determine if the test of your theory was successfulCompare the results of your test to the baseline dataSome questions to consider include:Did your test work? How do you know?Did the results match your theory/prediction? What do the data show?Are there trends in your data?Did you have unintended side effects?Is there improvement?Do you need to test the improvement under other conditions?
13 Plan-Do-Study-Act Plan-Do-Study-Act Cycle (Steps 8 and 9) Step 8: Either standardize your improvement (if test was successful) or develop a new theory for improvement (if test was unsuccessful)Step 9: Establish Future PlansSustaining change or beginning PDSA again
14 Tips for Beginning the Journey CQI can be confusing, daunting - lots of jargon, different tools, methods, approaches (Google = 2,760,000 results)Avoid starting with a large-scale project; think small and simpleChoose a QI effort related to a familiar process, program or areaBe patient – change takes timeUnanticipated results are not failures. They are learning opportunities for your next trial.We will be going through the journey together!Feelings of uncertainty about how to begin or what steps to take next
15 FOCUS First Find an opportunity to improve. Organize a team who understands the process.Clarify the current knowledge of the process. In this stage, you are gathering the "who, what, when, and where“.Understand the cause of process variation. Here, you ask yourself the "why" question.Select the piece of the process you want to improve.This is not technically part of the “model for improvement” but is part of the PDSA cycle sometimes left off or forgotten. I thought it was actually important and relevant as a way to think about how to focus your improvement efforts.What we have already done in this process:-Found an opportunity to improve: Family Enrollment & Engagement-Organzied teams (you guys) – though you may want to bring in others who can add understanding and help as you:-Clarify the current knowledge of the process – which is what we’ll do today –And you’ll work to understand process variation and select your improvement project when you go home today.
16 Self-Assessment Survey Assessed Knowledge, Attitudes and Practices (KAP) about CQI and Professional Development:Role of CQI within OrganizationUse of CQI Strategies and TechniquesInvolvement in CQIUse of CQI Techniques to examine Enrollment/EngagementDesire to improve Enrollment/EngagementInstitutional supports in place to support a focus on Enrollment/EngagementUnderstanding CapacityWhat was the basis for these categories?
17 Self-Assessment Survey Conducted in November 2013Purpose: to capture what is known and unknown about the training process and the use of Continuous Quality Improvement (CQI) principles in five component areas:professional development trainingquality of professional developmentrole of continuous quality improvement in professional development activitiesevaluation activitiesdata collection and analysis21 responses received
19 Summary of Survey Results Section 1: Professional Development TrainingMembers of the ECMHAB are involved in early childhood mental health initiatives, supportive of offering cross-disciplinary training and knowledgeable about the populations they serve:57% report having a full understanding of who needs PD76% work with other programs to enroll individuals in PDAll agreed that ECMH trainings should be offered across disciplines
20 Summary of Survey Results Section 1: Professional Development TrainingLess is known about trainings themselves1/3 of ECMHAB members don’t know if the number of trainings meet the need, if trainings are well attended, if registration is easy to understand and complete, or if cost is a barrier76% of respondents disagreed that current communication methods adequately reach all individuals needing PD
21 Summary of Survey Results Section 1: Professional Development TrainingThere may be regional or specialty-specific deficits:>half of respondents reported that the number of trainings is not sufficient to meet the need in their region or specialty.Nearly half of respondents reported that participants do not know where to register and find training opportunities.
22 Summary of Survey Results Section 2: Quality of Professional DevelopmentFeedback from survey was generally positive regarding the quality of current PD efforts.Exceptions were:Nearly ¼ of respondents disagreed(1) trainers are able to accommodate differing literacy levels and language barriers(2) early childhood programs are implementing evidence-based programs and practices
23 Summary of Survey Results Section 2: Quality of Professional DevelopmentA significant number of individuals reported “Don’t Know” for several indicators of quality centering around two themes:The demographics and values of the training participantsTheir belief in evidence-based practicesThe quality of the trainers themselvesAccommodating ethnic and cultural needs and differing literacy levels and language barriers
24 Summary of Survey Results Section 3: Role of CQI in PD ActivitiesMembers of the ECMHAB are knowledgeable about both the DSS and ECCS project, reporting familiarity (95%) and confidence (80%) in both logic models.50/50 split between those with familiarity and experience in both CQI and PDSA and those without62% report needing training on CQI
25 Summary of Survey Results Section 4: Evaluation ActivitiesMost respondents are aware of the ongoing evaluation activities included in both the DSS and ECCS projects.There is a significant percentage who report not knowing about evaluation activities:Formally measuring participant knowledge and understanding after training (38%)Evaluation plans (19%)Collect quantitative and qualitative data to measure progress (24%)Interesting to note that while only 24% didn’t know that both quantitative and qualitative data are collected, on average at least 50% of respondents reported not knowing whether or not specific outcomes were measured following each training.
26 Survey of Summary Results Section 5: Data Collection and AnalysisMembers are aware of the opportunity they have to provide input into quality improvement measures.A lack of awareness regarding what outcome data is available and collected for PD.
27 Next Steps Provide ongoing training and education: CQI PDSA cycle Specific information and updates on the data collected under the evaluation planPD qualifications for trainers2. Vote on priority area to develop a CQI plan based on the self- assessment survey and Advisory Board feedback
28 Focus for 2014 CQI Plan Professional Development Training process Qualifications of trainersDemographics and values of training attendees (Who is attending?)Types of data collected for each training
29 Focus for 2014 CQI Plan What does the Board feel needs to be the focus Suggestions follow based on the survey
30 Focus for 2014 CQI Plan 2. Quality Cultural competence Evidence based trainingsStakeholder input
31 Focus for 2014 CQI Plan 3. Evaluation Review of process and outcome measuresMonitoring of any factors that inhibit project implementation
32 Focus for 2014 CQI Plan4. Performance Measure/ Outcomes - Review and monitoring of data collected following each training
33 Focus for 2014 CQI PlanInvestigate areas potentially in need of improvement:Effectively reaching all individuals in need of PDCommunication methods and networksEnsuring participants know where to register and how to find training opportunitiesUnderstanding if, where, and/or what type of gaps exist in training