1. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Using Data to Inform Practice Pennsylvania Mental Health and Justice Center of Excellence Workshop: Expanding Specialized Police Response and Collaboration with BHS in Pennsylvania Grantville, PA November, 2010 Carol A. Schubert Edward P. Mulvey Marcel Schipper

2. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 What we’ll cover today  Introductions/Goals  Self Assessment  Case study – Day Reporting Center  Exercise /application to CIT

3. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 The Technical Assistance Role Of the Center of Excellence

4. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Why Data Matters Anecdote versus numbers Quality Improvement Funding

5. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Common Barriers Confidentiality Issues Funding Time Lack of cooperation/buy-in Knowledge/skills

6. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Self Assessment

7. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 How the Center of Excellence Can Help Assessing existing database structure and content Planning for data collection (e.g. identification of outcomes) and analysis strategies Designing data collection instruments Implementing standardized reporting components Monitoring data quality Integrating relevant information from multiple sources Analyzing and interpreting data analyses

8. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Advantages of COE Technical Assistance Individualized Experienced Responsive Flexible Free

9. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Case Study: Westmoreland County Day Reporting Center

10. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Practical Application to CIT

11. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Key Consideration Audience Purpose Data collection Interpretation

12. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Purpose

13. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Data Collection

14. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Data Collection: Some lessons learned Get data in specific form. Letting people “choose all that apply” creates uninterpretable data. People will rarely distribute forms for you to recruit people or to fill out surveys. All data points have to be collected from everyone in the same way. All fields in a form have to be filled out. Open-ended responses can be valuable, but turning them into data that is reportable across a whole sample is an involved coding task. Response options to any question have to be mutually exclusive. The name given to a scale doesn’t necessarily mean that it measures it.

15. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Analysis and Interpretation

16. Using Data to Inform Practice Carol Schubert Marcel Schipper PA Community Providers Association Conference October, 2010 Analysis and Interpretation: Some lessons learned There is nothing magical about statistical significance; think of clinical significance. Effect size does matter. Just because a variable can be found in an official record doesn’t mean it is worth coding or analyzing. Any data set can only do so much. It usually only does what it is designed to do. Patterns of missing data matter. Sensitivity analyses can be very convincing to make a point. The observed performance of any predictive instrument in a particular sample is as good as it is going to get. A “predictor” variable in a regression equation doesn’t really “predict” anything.