1. NIDA Portfolio on Performance and Outcome Research Sarah Duffy Using Performance Measures and Outcomes to Improve Treatment Sheraton Gateway Hotel, Los Angeles March 21, 2008

2. NIDA Services Research Branch Mission: Support science that improves the “quality” of the drug abuse treatment system Quality = Effective, efficient care to all who need it

3. Measures Development Addiction Severity Index Development, refinement, administration, translation Measures from Administrative Data Margaret Lee, Brandeis/Harvard NIDA Center “Adolescent Substance Abuse Performance Measures” – Washington Circle measures

4. Measures Used in Research No single standard Examples Abstinence, reduction in use, longest duration abstinent Criminal justice, employment, housing etc., outcomes Changes in ASI composite scores Washington Circle Measures Treatment completion, goal achievement, retention Transition to after care Self reports, biological tests, administrative data

5. Provider Incentives to Improve Quality (Brandeis/Harvard Center) Maine’s Performance-Based Contracting System Shen 2003 - Favorable selection of clients? Pre/post analysis of % clients rated as dysfunctional in outpatient treatment % dysfunctional among state-funded ↓ from 27%-20% % dysfunctional among Medicaid ↑ from 24% - 26% Dumping? Or…

6. Incentives to Improve Quality, Cont. Lu et al., 2003: PBC incentive for appropriate referral? Compared referral rates pre- and post- PBC implementation using data on 5 levels of care Found that, after PBC referral rates ↑ dysfunctional clients to higher intensity settings same dumping rate Quality improvement strategies warrant rigorous research

7. Current Grants: CJ DATS CJ Client Evaluation of Self and Treatment (CJ-CEST) (Knight, TCU IBR) Assesses need and performance in Tx Motivation, psych functioning, social functioning, Tx progress Criminal Thinking Scales Power orientation, personal irresponsibility, cold heartedness, entitlement, criminal rationalization and justification.

8. Current Grants: NIATx Develop a practical model for improving addiction treatment (Gustafson) RCT, 240 organizations, 5 states to evaluate 5 versions of NIATx model in improving Time to treatment No-show rates Admission Continuation Cost effectiveness of different models

9. Current Grants: Brandeis/Harvard Center Provider profiling vs incentives on quality of behavioral health care Collaborating with CT’s DMHAS RCT Programs that treat low-income general assistance clients in CT’s MBHC program Evaluating process/performance measures

10. Use of PBCs for OP SAT in Delaware (Stewart, Brandeis/Harvard Center) Did PBC meet stated goals? Did facility characteristics affect ability to meet goals Did facilities change? If so, how? Current Grants Cont

11. Current Grants, Cont International Conference on Outcomes Measurement in Substance Abuse IRT/Rasch measurement techniques Sept 11-13 2008, Bethesda, MD http://www.icom-2008.org

12. Research Interests Measurement Development Process/performance measures → outcomes? Would those who show up do better anyway? Measures tracking the entire course of care? Performance Management Unintended consequences? Can they be attenuated by better designs? Medicare Surgical DRGs How to prevent, account for gaming (reporting changes, “teaching to the test”, dumping) Performance systems for larger states

13. Research Interests, Cont. How are improvements accomplished? Effects on organizations? How to disseminate EBPs? (CT MET-CBT) What is the most cost-effective way to achieve quality improvement? Value-Based Purchasing? What kind, how much is enough to pay? Other mechanisms (technical assistance; NIATx, reporting)?

14. Relevant Program Announcements Health Services Research on the Prevention and Treatment of Drug and Alcohol Abuse (PA07-119[R01], P06- 307[R21], PA07-308[R03]): “Performance measures (e.g., those developed by the Washington Circle Group) and standards of care to improve assessment of client needs, treatment progress, and outcomes.”

15. Other Avenues Economics PA (PA 07-122) – anything having to do with $$s National Research Service Awards (F31, F32, etc.) National Institute on Alcohol Abuse and Alcoholism – Same program announcements Provider involvement critical here, too!

16. Contact duffys@nida.nih.gov http://www.nida.nih.gov/about/organization/ despr/srbHome.html http://www.nida.nih.gov/about/organization/ despr/srbHome.html Talk to a program official

17. References Shen, Y. (2003) “Selection Incentives in a Performance-Based Contracting System” Health Services Research 38:2 (April) 535- 552 Lu, M., et al. (2003) “Risk Selection and Matching in Performance-Based Contracting” Health Econ. 12: 339-354

18. Questions?