1. Performance Improvement Projects: From Idea to PIP
Amy McCurry Schwartz, Esq., MHSA
EQRO Consultant
Behavioral Health Concepts, Inc.
Amie Miller, Psy.D., MFT
Quality Improvement Manager
Monterey County Behavioral Health

2. Performance Improvement Projects
The purpose of PIPs
To assess and improve processes, and thereby outcomes, of care.
42 CFR (d) defines PIPs as having a “focus on clinical and non-clinical areas.”

3. Performance Improvement Projects
Clinical PIPs Might Target
Prevention and care of acute and chronic conditions
High-volume services
High-risk procedures
Special health care needs

4. Performance Improvement Projects
Non-Clinical PIPs Might Target
Coordination of Care
Appeals, Grievances Process
Access or Authorization
Member Services

5. Step 1: Select the Study Topic
Bilingual Treatment Plans
Why is this an important topic?

6. Impact Affects a significant portion of consumers
Study Topic Impact
Affects a significant portion of consumers
Has a potentially significant impact on consumer health, functional status or satisfaction
Desire to provide client centered care

7. Data Sources of data used to select this topic
Average person remembers 15 – 20% of things discussed after a visit
Need to employ a variety of tools to communicate what treatment looks like
78% of population is Latino in Monterey County, data showed MCBH that only 11% of their clients that spoke Spanish had treatment plans in Spanish.

8. Data

9. Data

10. Step 2: Define the Study Question
Do interventions increase the number of bilingual(Spanish/English) treatment plans completed for individuals receiving Outpatient County services whose primary language is identified as Spanish?
Is this client focused? Does this go to client outcomes?

11. Step 2: Define the Study Question
The outcomes are framed in a way that sets MCBH up to measure only staff performance, not client outcomes.
This is a measure of the effectiveness of an intervention…it is an indicator of success.
Measuring how many more treatment plans are providing in Spanish, is not focused on the client outcomes that having a treatment in your language of choice may bring.

12. Step 2: Define the Study Question
Study Questions should be stated in a way that:
Supports the ability to determine whether the intervention has a measurable impact for a clearly defined population

13. Step 2: Define the Study Question
This study question should focus on the outcomes for the consumer:
Increased client participation in treatment
Client satisfaction with treatment received
Ability to accomplish next step in treatment plan

14. Step 2: Define the Study Question
Better Study Question?
Will clients who receive a treatment plan in their language of choice report they are more satisfied with their treatment?
Will clients who receive a treatment plan in their language of choice be more engaged in treatment?
Will providing treatment plans in the client’s language of choice…

15. Step 2: Define the Study Question
How to measure satisfaction?
How to measure engagement?

16. Step 3: Use a Representative and Generalizable Study Population
Two options
Review all EHR data for Treatment Plans in language of choice
Review a sample of EHR data for Treatment Plans in language of choice

17. Step 4: Select the Study Indicator(s)
A study indicator is:
A quantitative or qualitative characteristic reflecting a discrete event or continuous status to be measured
Used to track performance and improvement over time

18. Step 4: Select the Study Indicator(s)
If goal is: More participation/engagement in treatment
Indicator – “More treatment plans are provided to clients in Spanish.
Indicator – “On follow up visit, clinician reported the consumer was engaged in treatment and willing to work towards treatment goals.”
If goal is: Consumer satisfaction
Indicator – “Pre and post satisfaction testing”

19. Step 5: Sampling
If sampling applies here – is sampling technique reliable and valid?
Probability sampling
Non-probability sampling

20. Step 6: Reliably Collect Data
Data must be valid and reliable.
Valid – indication of accuracy
The data measures what is intended to be measured
Reliable – indication of repeatability
Produces consistent results

21. Step 6: Reliably Collect Data
Baseline data showing that 11% of treatment plans were provided to consumers in Spanish.
This collects number of treatment plans in Spanish…may be an indicator of satisfaction…but, is not an outcome.
How was this data collected?
EHR
Improving Quality Report – IQ Report…very relevant…by each MHSA age groups…looks at which age ranges are struggling to engage non-English speaking people
Who collected this data?
UR Review
Annual Satisfaction Tool
Plan to collect data for this and other indicators in the future?

22. Reliably Collect Data
UR tool – is there evidence that a client was substantively involved in their treatment
Increasing engagement of Latino clients – client frequency

23. Reliably Collect Data

24. Reliably Collect Data
This chart displays the count of clients served grouped by clients
who received 3 or less services and clients who received 4+ services

25. Step 6: Reliably Collect Data
Develop a data collection plan:
Clearly identify data to be collected
For satisfaction, participation
Identify data sources and how/when the baseline and repeat indicator data will be collected
Surveys, treatment notes
Specify who will collect the data and that they are qualified to collect the data
Clinicians
Identify the data collection instruments to be used

26. Step 6: Reliably Collect Data
Will the data be collected on the entire population or a sample?
Entire population vs. children
Time needed to replicate data
Will collected data be compared to the results of previous studies? If so, will the data gathered be comparable to the earlier study?
Satisfaction surveys
Previous data regarding treatment participation
Numbers who graduate?

27. Step 6: Reliably Collect Data

28. Step 7: Analyze Data and Interpret Study Results
Valid interpretations occur when:
Analysis is conducted according to the data analysis plan
Results present numerical data that is accurate, clear and easily understood
Analysis addresses the comparability of initial and repeat measurements, including factors that threaten validity
A conclusion is drawn as to the success of the PIP and planned follow-up activities are described

29. Step 8: Implement Intervention and Improvement Strategies
Interventions should:
Relate to causes/barriers identified through data analysis
Be culturally and linguistically appropriate
Be “implementable” system-wide
Not be “one time” efforts

30. Step 8: Implement Intervention and Improvement Strategies
Interventions proposed by MCBH:
1. Use script (when Spanish is primary language) to include a window prompt when a treatment plan form is opened “friendly reminder to complete treatment plan in English followed by Spanish language”.
2.  Staff training (bilingual staff and supervisors only)  to educate on expectations for bilingual Tx plans and use of Google Translate.
3.   Immediate QIAR when conducting UR and Tx plan not in Spanish language.
4.   Include a radio button that is activated for the treatment plan participation consent  and/or Treatment plan (when Spanish is identified language).
5.    Make it a supervision issue/ performance issue; supervisors must review Tx plan for bilingual plans for the next 6 months.

32. Step 8: Implement Intervention and Improvement Strategies
Goal (as stated by MCBH):
Increase the number of bilingual treatment plans from 10% to 50% by June 2015, to 80% by December 2015, and to 90% by Feb in order to improve cultural sensitivity services in the individual’s primary language.

33. Step 9: Plan for “Real” Improvement
When a change in performance occurs, determine whether the change is real, attributable to an event unrelated to the intervention, or random chance
Results of the intervention must be “statistically significant”

34. Step 10: Achieve Sustained Improvement
If real change has occurred, the project should be able to achieve sustained improvement.
Sustained improvement is demonstrated through repeated measurements over time.