Presentation is loading. Please wait.

Presentation is loading. Please wait.

Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.

Similar presentations


Presentation on theme: "Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality."— Presentation transcript:

1 Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality care: Results from the Reinforcing Therapist Performance (RTP) experiment Bryan R. Garner, Susan H. Godley, Michael L. Dennis, Brooke D. Hunter, Christin M. L. Bair, & Mark D. Godley Chestnut Health Systems Bloomington-Normal, IL R01AA (PI: Garner) Contract # Opinions are those of the authors and not official positions of the government

2 Learning Objectives To learn more about… 1.Methods to improve the implementation of evidence-based practices 2.The effectiveness of pay-for-performance methods 3.The relationship between performance measures and client outcomes

3 Group of treatment organizations that are: Receiving the same level of funding. Implementing the same evidence-based treatment. Receiving the same comprehensive training model. Documenting treatment implementation using the same web-based program. Documenting client outcomes using the same evidence-based assessment instrument. AAFT AAFT as the foundation of the Reinforcing Therapist Performance (RTP) Experiment

4 AAFT A-CRA Clinician Training and Certification Process Feedback Monitor Clinician Fidelity Record Therapy Sessions Achieve Basic A-CRA Certification In 9 procedures Treatment Manual and Training Workshop Participant in Coaching Calls with Supervisors/Coaches Adapted from Godley, Garner, Smith, Meyers, & Godley (2011)

5 AAFT A-CRA Training Workshop A-CRA Treatment Manual + A-CRA Clinician Training and Certification Process

6 AAFT Feedback Monitor Clinician Fidelity Record Therapy Sessions Achieve Basic A-CRA Certification In 9 procedures Treatment Manual and Training Workshop Participant in Coaching Calls with Supervisors/Coaches A-CRA Clinician Training and Certification Process

7 AAFT Digital Session Recordings (DSRs) A-CRA Clinician Training and Certification Process

8 Upload DSR to EBTx.org website

9 AAFT Feedback Monitor Clinician Fidelity Record Therapy Sessions Achieve Basic A-CRA Certification In 9 procedures Treatment Manual and Training Workshop Participant in Coaching Calls with Supervisors/Coaches A-CRA Clinician Training and Certification Process

10 AAT 1: First Year AAFT 1: Second Year AAFT 1: Third Year 15 sites funded in October 2006 AAFT 2: First Year AAFT 2: Second Year AAFT 2: Third Year 17 sites funded in October 2007 Improving the Implementation Process within AAFT How might we improve the process?

11 Recommended Pay-for-Performance (P4P) as a promising method to improve the delivery of high quality of care. The literature evaluating the effectiveness of pay for performance consists of fewer than 20 studies, yielding mixed conclusions on overall impact. Rewarding Provider Performance: Aligning Incentives in Medicare (IOM, 2007)

12 AAFT 1: First Year AAFT 1: Second Year AAFT 1: Third Year 15 sites funded in October 2006 AAFT 2: First Year AAFT 2: Second Year AAFT 2: Third Year 17 sites funded in October 2007 RTP study funded in October 2008 How might we improve the process? Improving the Implementation Process within AAFT

13 AAFTRTP Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes. Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P) $50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification. $200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010). Targets: Monthly A-CRA Competence Target A-CRA Aims and Design of RTP Experiment

14 Target A-CRA 6-month Remission Status + + Treatment Implementation Performance Measures Treatment Effectiveness A-CRA Competence RTP hypotheses to be tested + Assignment to Experimental Pay-For-Performance Intervention

15 Sample size & Data structure Therapist 104 Site 29 Level 3: 29 Sites Level 2: 105 Therapists Level 1: 986 Clients … Therapist Therapist 1 Site 1 Therapist 3Therapist 2 … …

16 Therapist Characteristics (N=105) a p <.05; b See Donnellan, Oswald, Baird, & Lucas (2006) for more information about Mini-International Personality Item Pool (Mini-IPIP) % or Mean (SD) IAU (n = 52) P4P (n = 53) Therapists (N=105) Female67%81% White58%51% Age37.2 (11.7)36.0 (10.7) Masters degree or higher60%49% Months experience60.3 (80.2)35.9 (37.5) a In Recovery6% Mini-IPIP personality factors b Extraversion3.6 (0.73)3.5 (0.77) Agreeableness4.4 (0.47)4.4 (0.50) Conscientiousness3.9 (0.73)4.2 (0.59) a Neuroticism2.5 (0.63)2.5 (0.69) Intellect/Imagination4.0 (0.67)3.9 (0.72)

17 Client Characteristics (N=986) % or Mean (SD) IAU (n = 534) P4P (n = 452) Clients (N=986) Female22%27% a White36%28% a Age15.8 (1.34)15.9 (1.47) Criminal Justice Involved64%68% Prior substance use treatment37%28% a Percent-of-days-abstinent.58 (.37).66 (.34) a a p <.05;

18 AAFTRTP Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes. Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P) $50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification. $200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010). Targets: Monthly A-CRA Competence Target A-CRAResults

19 Results: A-CRA Competence IAUP4P 9.2% 23.7% Odds Ratio = 2.2 p =.02

20 AAFTRTP Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes. Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P) $50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification. $200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010). Targets: Monthly A-CRA Competence Target A-CRAResults

21 Results: Target A-CRA IAUP4P 3.3% 15.9% Odds Ratio = 5.2 p =.01

22 AAFTRTP Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes. Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P) $50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification. $200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010). Targets: Monthly A-CRA Competence Target A-CRAResults

23 Results: Remission Status

24 IAUP4P 50.8% 41.8% No Significant Difference Average Remission Status was 24% in Cannabis Youth Treatment (CYT) study (Dennis et al., 2004) Results: Remission Status ?

25 IAUP4P 65% 56% Poor Follow-up Rates Remission Status Unknown for 44% Remission Status Unknown for 35% Differential Client Attrition!

26 IAUP4P 65% 56% Poor Follow-up Rates Remission Status Unknown for 44% Remission Status Unknown for 35% No Significant Differences Significant Differences Follow-up group was significantly more severe at baseline

27 Target A-CRA 6-month Remission Status Treatment Implementation Performance Measures Treatment Effectiveness + A-CRA Competence + Assignment to Experimental Pay-For-Performance Intervention Not Supported Supported Post hoc analyses

28 Take home messages 1.Modest-sized incentives can have large impacts on improving treatment implementation 2.Despite the large impact of P4P on the two treatment implementation measures, there remained considerable room for improvement even in the P4P condition.

29 Next Step Examine the extent to which the improvements in A-CRA Competence and Target A-CRA were Cost-Effective.

30 Thank You.


Download ppt "Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality."

Similar presentations


Ads by Google