Evidence-Based Practice: Selection and Implementation A Story about Better Outcomes CMHACY Conference 2007 Todd Sosna, Ph.D.

CMHACY 2007Selecting and Implementing EBPs2 Table of Contents Chapter I “And the third little pig built his house out of bricks” Chapter II “Where’s the beef?” –Action book enclosed Chapter III “Car ownership is a serious responsibility”

CMHACY 2007Selecting and Implementing EBPs3 Evidence-Based Practices “…the integration of the best research evidence with clinical expertise and patient (consumer) values” Based on the definition used in “Crossing the Quality Chasm: A New Health System for the 21st Century” (2001), by the Institute of Medicine

CMHACY 2007Selecting and Implementing EBPs4 Evidence-Based Practices Clinician expertise and judgment based on education and experience Consumer and family beliefs, values, preferences, choices based on personal life experience, family, and culture Effectiveness research based on controlled studies

CMHACY 2007Selecting and Implementing EBPs5 Levels of Science Effective/Efficacious--achieves outcomes, controlled rigorous research (random assignment, matched between-groups comparisons) Not effective--significant evidence of a null, negative, or harmful effect Promising--some positive research evidence, quasi-experimental, of success and/or expert consensus Emerging practice--recognizable as a distinct practice with “face” validity or common sense test Unknown--not clearly articulated nor evaluated

CMHACY 2007Selecting and Implementing EBPs6 Evidence-Based Practices Specific to area of need or concern Specific to outcomes achieved Clearly articulated practices--can be replicated EBPs are not always effective Incorporation of EBPs is a developmental process of building on successive advances--it is not an end but a beginning

CMHACY 2007Selecting and Implementing EBPs7 Evidence-Based Practices Increase hope Increase choice Increase individualized care Improves achievement of outcomes Reduces adverse consequences of inappropriate care Achieves outcomes sooner Outcomes last longer Ethical Cost effective

CMHACY 2007Selecting and Implementing EBPs8 Selecting an EBP What outcomes do you want to achieve For whom? EBPs are specific to outcome and population What is the level of evidence? Need to know the research methodology Higher levels mean more confidence that if implemented in your community (with high model adherence) similar good outcomes will be achieved. Consider lower levels of science when there is no alternative at a higher level, or interested in a practice-to-science program. Be cautious of promotion in advance of research

CMHACY 2007Selecting and Implementing EBPs9 Studies Qualitative studies –Anecdotal observations & descriptions Quasi- or partially controlled experimental studies –Within-subjects or longitudinal (pre and post comparison) studies –Between groups comparisons without random assignment Controlled experimental studies –Random clinical trials (between groups) –Random clinical trial-longitudinal studies Replication

CMHACY 2007Selecting and Implementing EBPs10 Static Spontaneous recovery –Individuals are constantly seeking support and self-improvement Non-specific factors –Treatment leads to outcomes but for reasons unrelated to the theory or active components Independent factors –Employment –Friends Bias –Selection bias--more hope and motivation –Consumers bias--want to please –Providers bias--want to succeed

CMHACY 2007Selecting and Implementing EBPs11 Reviews of Effectiveness SAMHSA National Registry of Evidence Based Practices California Evidence-Based Clearinghouse for Child Welfare g/ g/ Helping America’s Youth

CMHACY 2007Selecting and Implementing EBPs12 Implementing and Sustaining Traditional Postgraduate training Medi-Cal compliance Generalist Quantity of service Evidence-Based Practice-specific training Model adherence Specialist Service effectiveness

CMHACY 2007Selecting and Implementing EBPs13 More Than Training Training and supervision is an ongoing process, not a moment in time Learning a practice for the first time typically takes about a year and includes: –Intensive initial training and booster trainings –Periodic (weekly) supervision –Fidelity monitoring (checklists, videotape reviews) Continuing training and supervision needs to be routine Need supportive managers, coordination of referrals, monitoring of model adherence and outcomes, replacement training

CMHACY 2007Selecting and Implementing EBPs14 Preventing Drift Staff are not enthusiastic about the practice Not enough training and supervision Failure to adhere to practice caseload standards and program components Select staff with interest based on an understanding of the practice Make use of all training and supervision activities Understand the practice, commit to caseload standards and program fidelity

CMHACY 2007Selecting and Implementing EBPs15 Preventing Drift Practitioners have competing duties Insufficient within and between agency coordination involving referrals and funding Mid-managers do not proactively support the practice Understand time commitments and staff accordingly Plan thoroughly in advance Involve mid-managers so they understand and support the practice

CMHACY 2007Selecting and Implementing EBPs16 Preventing Drift Attrition of trained practitioners Delays between training and using the practice Insufficient attention to fidelity Eagerness to expand and adapt the practice before it is well established Expect and prepare for replacement training Synchronize training and referrals Prioritize and monitor fidelity from the outset Implement with adherence and demonstrate positive outcomes before making adjustments

CMHACY 2007Selecting and Implementing EBPs17 Preventing Drift Unrealistically high expectations Increased scrutiny and accountability Numerous agency demands and initiatives Do not over sell the practice Be sensitive to practitioners’ feeling scrutinized Evaluate new and existing practices (all programs scrutinized) Designate a single responsible administrator--has responsibility and authority

CMHACY 2007Selecting and Implementing EBPs18 Todd Sosna –(916) Bill Carter –(916) ext. 130