1. Bisma Sayed, M.S.W. University of Miami Department of Sociology John Dow, M.S.W. South Florida Behavioral Health Network

2.  Understand the value of utilizing data for decision-making  Determine what should be measured and what data elements should be used  Data analysis and interpretation  Recognize limitations  Validate findings using other data sources

3.  The recent recession coupled with health care reform has had cascading consequences on behavioral health care service delivery in Florida.  Current funding limitations and budget cuts have increased the urgency for cost-effective and efficient delivery of behavioral health care services.

4.  How can behavioral health care organizations lower cost, raise quality, and still offer accessible services to increasing numbers of consumers? ◦ Meet standards ◦ Coordinate ◦ Demonstrate outcomes ◦ Eliminate duplication ◦ Produce reportable, effective, and sustainable results

5.  Quality Improvement vs. Quality Assessment

6.  Quality Improvement Processes allow organizations to analyze current practices, identify strengths and weaknesses, set goals, and monitor progress  Quality in the behavioral health care setting may be defined as the ‘extent to which a health care service or product produces a desired outcome’

7.  Quality of care measures ◦ Effective ◦ Appropriate ◦ Safe ◦ Efficient ◦ Responsive ◦ Accessible ◦ Continuous ◦ Capable ◦ Sustainable

8.  Examine current organizational functioning  Identify target problems  Identify quality of care measures  Identify goals (short term or long term)  Measure baseline performance on quality measures

9.  Develop and conduct interventions designed to affect the targeted measures  Repeat measurement of performance based on quality indicator  Document and disseminate results.

10.  “If you do not measure it (or cannot measure it), it didn’t happen.”  How can we measure it?

11.  Data provides the foundation for quality improvement initiatives ◦ Timely ◦ Transparent ◦ Presented with humility ◦ Based on past lessons learned ◦ Accountability ◦ Presented with compassion and understanding

12.  The shift to evidence based care coupled with increased technological and statistical advances have resulted in an explosion of data...

13. ... However, this knowledge remains to be harnessed

14.  The influx of data has led organizations to report data, rather than analyze data.  Data Reporting Data Analysis

15. Data Information Knowledge Decision Action Data Information Knowledge Decision Action

16.  Quality Improvement ◦ What is happening? ◦ What factors affect delivery ◦ How can we influence them  Reactive and Proactive  We need data to guide this.  “Data helps to push improvement (by identifying problems) and pull improvement (by identifying opportunities)”

17.  Facts and statistics collected together for reference or analysis Surveys Literature Reviews Key informants Surveillance data Focus Groups Surveys Literature Reviews Key informants Surveillance data Focus Groups

18.  Develop overall goal for improvement  Identify objectives using quality of care measures  Identify target populations  Identify data to be collected

19.  Determine data sources and/or collection method.  Determine data storage, management, and analysis techniques.  Analyze and Interpret Data  Utilize data for decision-making

21.  Plan ◦ Consider scope and purpose ◦ Target Audience  Learn ( Do not reinvent the wheel) ◦ Literature Reviews ◦ Other sources of data  Test ◦ Pilot-test on a smaller scale to identify challenges  Team work ◦ Involve and Integrate

22.  Internal Data  External Data  Administrative or Clinical  Regardless of source of data or type of data, it must be reliable and valid ◦ What is reliability and validity?

23.  Process mapping: (Who? How long? Steps? Costs?)  Brainstorm  Quantitative or Qualitative ◦ Nominal ◦ Ordinal ◦ Interval ◦ Ratio

24.  Surveys and questionnaires ◦ Ethical Standards ◦ Confidentiality and Anonymity ◦ Response Rates ◦ Existing Surveys ◦ Guidance ◦ Pilot test

25.  What is your target population? ◦ Consumers? Their families? Providers? Community?

26.  Clear and Understandable ◦ Specific ◦ Not loaded or leading ◦ No double barreled question ◦ No jargon or acronyms  Allow choice of only one option  Provide reasonable ranges of variation in the response options

27.  Social Desirability Bias  Target towards population ◦ Appropriate for age, culture and literacy  Include adequate demographic information

28.  Why do we sample?  Sampling must be representative of your population  Selection bias

29.  Important step that can cause significant error if not done properly  Identify inconsistencies ◦ For example, the mean age of adolescents sampled across the nation is 23.5. The range is 13-56. ◦ Why do we have a 56 year old adolescent?

30.  Spreadsheet programs ◦ Reporting, not analysis  Database programs ◦ Database changes – Store data with reports ◦ Reporting, not analysis  Statistical Programs ◦ Analysis

32.  Understand the variables ◦ Categorical and numerical variables  Frequency Distribution  Median and Percentile  Counts and Sums  Measures of central tendency  Measures of variability

33.  Measures of Central Tendency ◦ Mean ◦ Median ◦ Mode

34.  Range  Standard Deviation  What does this tell you about your population?

35.  What is the goal of data analysis in QI?  Descriptive Analyses and Measures of Variation are useful, but...  Inferential statistics can add to the power of your conclusions. ◦ Examine Relationship/Estimate size of difference ◦ Confidence Intervals ◦ Tests of statistical significance

36.  Correlation Analysis ◦ Correlation Coefficient: Pearson Product Moment Correlation Coefficient (r)  Scatter plots ◦ Linear Relationships ◦ Non-Linear Relationships  Correlation does not equal causation

37.  Nominal Level Data: Non-Parametric Tests ◦ Chi Square ◦ Cramer’s V/ Contingency Coefficient/Others  Numerical Data: Parametric Tests ◦ T-tests (independent or dependent) ◦ ANOVA ◦ Regressions  Confidence Intervals ◦ What are they? ◦ How can they be used? ◦ Sample size matters

38.  When you combine your sample value with the margin of error, you obtain a confidence interval.  The confidence interval is the level of confidence that the sample value represents the true value as seen in the overall population.

39.  For example, the waiting time for appointments for clients referred to your clinic might be expressed as a mean of 13.5 weeks with a 95% confidence interval of 11.6 to 15.3 weeks (95% CI 11.6-15.3).  This means that you expect your population on average would wait between 11.6 and 15.3 weeks for an appointment.

40.  The p value is the probability that the difference you have observed in your study samples could be due to chance.  Smaller p value = lowered probability that results are due to chance  Statistical Significance

41.  The size of the p value depends on the size of the sample, so be aware of possible mistakes that can occur in interpreting these values.  Statistical significance does not mean clinical significance.

42.  Keep it simple  Consistent units  Decimal Points  Include raw numbers and percentages  Always include n  Identify missing data  Group data appropriately

43.  Keep it simple  Avoid complexity  Clear headings  Scale Carefully  Raw numbers and percentages  Always include n  Group data appropriately

44.  Basic population characteristics: Pie chart; bar graph  Measures of magnitude including comparisons: Bar chart or box plot

45.  Frequency: Pie chart; bar chart  Trends over time: Line graph  Distribution of Data: Histogram; Scatter plot  Relationship between two things: scatter diagram

47.  Whether you are collecting your own data or relying on external sources, there is a difference between compiling/reporting data and analyzing data ◦ Data : petabytes ◦ Reports : terabytes ◦ Excel : gigabytes ◦ PowerPoint : megabytes ◦ Insights : bytes ◦ One business decision based on actual data: Priceless 1

48.  What is the problem?  What can you improve?  How can you improve?  Have you achieved improvement?  Have we sustained improvement?

49.  State and national datasets provide important information about key health indicators and can serve as basis for comparison.  However, we must be careful in interpreting and analyzing this data. ◦ Understand limitations  Understand how data is presented ◦ Mean, Median, Mode ◦ Raw sums or percentages

50.  Level of variables ◦ Individual ◦ Community ◦ State  State level data can help guide decisions, but you must examine individual data in your community to determine if the problem exists at a local level.

52.  What does data drive? ◦ Assessment ◦ Priority setting ◦ Allocation of resources ◦ Directives to staff and community ◦ Evaluation of clinical outcomes ◦ Basis of QI for providers ◦ Feedback ◦ Sets the groundwork for comprehensive planning

53.  Assess performance and identify gaps  Understand the needs and opinion of stakeholders  Prioritize problems and improvement projects  Establish overall aims and targets for improvement

54.  Establish a clear case for the need for improvement.  Data assists in sustained improvement: feedback to reinforce change and demonstrate benefits.