Presentation is loading. Please wait.

Presentation is loading. Please wait.

Challenges to Measuring Provider Fidelity in a Statewide Dissemination/Implementation Project Rochelle Hanson, Ph.D. Benjamin Saunders, Ph.D. Presentation.

Similar presentations


Presentation on theme: "Challenges to Measuring Provider Fidelity in a Statewide Dissemination/Implementation Project Rochelle Hanson, Ph.D. Benjamin Saunders, Ph.D. Presentation."— Presentation transcript:

1 Challenges to Measuring Provider Fidelity in a Statewide Dissemination/Implementation Project Rochelle Hanson, Ph.D. Benjamin Saunders, Ph.D. Presentation at the Implementation Science Retreat SC Hospital Association, Yates Conference Center Columbia, SC March 1, 2013

2 Introduction Exposure to violence/abuse = significant risk factor for myriad psychological, social, and physical health consequences Exposure to violence/abuse = significant risk factor for myriad psychological, social, and physical health consequences # of efficacious and effective treatments identified, with Trauma- focused Cognitive Behavioral Therapy (TF-CBT) currently evidencing most empirical support # of efficacious and effective treatments identified, with Trauma- focused Cognitive Behavioral Therapy (TF-CBT) currently evidencing most empirical support Not all children/families have ready access to these trauma- informed, evidence-based services Not all children/families have ready access to these trauma- informed, evidence-based services They are not well-integrated into many communities They are not well-integrated into many communities Trauma-focused EBTs arent the standard practice of care throughout our mental health and child welfare service systems. Trauma-focused EBTs arent the standard practice of care throughout our mental health and child welfare service systems. 2 Challenge: How do we do a better job of getting these EBTs into communities??

3 Project BEST (funded by Duke Endowment) Mission: to ensure that all abused children and their families in every community in South Carolina receive appropriate, evidence supported mental health assessment and psychosocial treatment services. Spreading and building the capacity of every community to deliver Evidence Supported Treatments (ESTs) Grant No SP and 1790-SP from The Duke Endowment

4 Colleagues Benjamin E. Saunders, Ph.D., PB Co-Director Medical University of South Carolina M. Elizabeth Ralston, Ph.D., PB Co-Director Dee Norton Lowcountry Childrens Center Michael de Arellano, Ph.D. Medical University of South Carolina Angela Moreland, Ph.D. Medical University of South Carolina Key Staff Jan Koenig, Program Coordinator Rachael Garrett, DNLCC Project BEST Manager Anna Shaw, DNLCC Sarah Sweeney & Kate Measom, Program Assistants and Data Coordinators Colleagues

5 Project BEST Coverage: Phase 1: Start date: July 2007 Phase 2: Start date: Jan 2011 Pee Dee CBLC 1 Lower State LC 1 Durant Childrens Center Coastal CBLC 1 Pioneer CBLC 1 Dee Norton Lowcountry Childrens Center Dorchester Childrens Center Childrens Recovery Center Upstate CBLC 2 Midlands Northcentral 2 Edisto 2

6 Balancing the Scale 6 Clinical Providers Brokers Consumers Build Supply Build Demand

7 Socio-ecological framework; Tabak et al., 2012 Aarons, Hurlburt & Horwitz, 2011; Proctor et al., 2010

8 Community-Based Learning Collaborative (CBLC) Implementation Model Multistage - Exploration, Adoption Decision/Preparation, Active implementation, Sustainment/Feedback (Aarons et al., 2011; Green & Aarons, 2011) Multistage - Exploration, Adoption Decision/Preparation, Active implementation, Sustainment/Feedback (Aarons et al., 2011; Green & Aarons, 2011) Multilevel Multilevel (Targets: clinicians, brokers, supervisors, senior leaders across multiple agencies nested within communities) (Targets: clinicians, brokers, supervisors, senior leaders across multiple agencies nested within communities) Goal: to create supply and demand for TF-CBT; build community capacity for delivery of trauma-informed EBPs Goal: to create supply and demand for TF-CBT; build community capacity for delivery of trauma-informed EBPs Differs from traditional LC: Differs from traditional LC: Community focus Community focus Train broker + clinical professionals Train broker + clinical professionals

9 Community-based Learning Collaborative (CBLC) Community Agency Agency Clinicians and Clinical Supervisors Senior Leaders Brokers Community Change Team

10 Focus on Fidelity: Challenges to Measuring Fidelity Conceptualization of the construct: What is fidelity? Conceptualization of the construct: What is fidelity? Model adherence – (did therapy occur as intended?) Model adherence – (did therapy occur as intended?) Competence Competence Treatment differentiation Treatment differentiation Lack of reliable/valid measures Lack of reliable/valid measures Gold Standard : Expert real time observation of treatment sessions and rating of fidelity. Gold Standard : Expert real time observation of treatment sessions and rating of fidelity. In-person In-person Electronic observation (video, telephone) Electronic observation (video, telephone) Feasibility for use in routine care (i.e., low burden, inexpensive) Feasibility for use in routine care (i.e., low burden, inexpensive) 10

11 Research Questions What individual (e.g., clinical experience; attitudes towards EBPs; theoretical orientation) and organizational (e.g., availability of supervision; leader support for EBP) level factors are related to therapist fidelity to TF-CBT? What individual (e.g., clinical experience; attitudes towards EBPs; theoretical orientation) and organizational (e.g., availability of supervision; leader support for EBP) level factors are related to therapist fidelity to TF-CBT? What role do broker professionals play in therapists use/fidelity to TF-CBT? What role do broker professionals play in therapists use/fidelity to TF-CBT? What are the relationships between therapist self-report of fidelity to TF-CBT and child outcomes? What are the relationships between therapist self-report of fidelity to TF-CBT and child outcomes? 11

12 Phase 2 Completion Rates (to date) CBLCLearning Session 1# Completed% Complete PHASE 2 (Jan 2011) Brokers % Senior Leaders201995% Clinicians % Clinical Supervisors % Total % Total Clinicians = 98 Total of 307 training cases N=122 with pre/post data 24 weeks of metrics

13 Clinical Metrics Weekly brief online survey Training case seen Training case seen For each training case For each training case Caregiver involvement (at least 15 minutes)Caregiver involvement (at least 15 minutes) Component usedComponent used Sense of clinical competence for the componentSense of clinical competence for the component Barriers to adherenceBarriers to adherence

14 Weekly Clinical Metrics Supervision minutesSupervision minutes Registered cases seenRegistered cases seen Each Case Parental involvementParental involvement

15 Percent of Therapists Using TF-CBT Components by Session

16 Questions/Collaborations Ideas for low burden/low cost measures of fidelity Ideas for low burden/low cost measures of fidelity Best ways to determine what constitutes model adherence? Best ways to determine what constitutes model adherence? What % of model components should be present to be considered good fidelity? What % of model components should be present to be considered good fidelity? How to determine a fidelity score for each case?: How to determine a fidelity score for each case?: 0 or 1 for each component 0 or 1 for each component Max total = 11 Max total = 11 Score for perceived competence (1-5 for less than adequate to expert) Score for perceived competence (1-5 for less than adequate to expert) Aside from fidelity issue – others who are interested in discussing ways to develop/sustain collaborative relationships across community providers for ESI implementation? Aside from fidelity issue – others who are interested in discussing ways to develop/sustain collaborative relationships across community providers for ESI implementation? 16

17 Contact Information Rochelle F. Hanson, Ph.D. Phone: (843) Fax: (843) Address: National Crime Victims Research & Treatment Center, Medical University of SC, 67 President Street, Charleston, SC


Download ppt "Challenges to Measuring Provider Fidelity in a Statewide Dissemination/Implementation Project Rochelle Hanson, Ph.D. Benjamin Saunders, Ph.D. Presentation."

Similar presentations


Ads by Google