1. Performance Improvement Projects (PIPs): A Year in Review (FY16 -17)

2. Performance Improvement Projects (PIPs) Submissions
Total
Clinical
Non-clinical
PIPs Submitted
Themes:
Access to Care 20 7 13
Timeliness to Care 4 16
Quality of Care 22 15
Outcomes to Care 30 25 5
Status:
Active and Ongoing 52 24 28
Completed 8
Concept Only 9

3. Access to Care PIPs Active or Completed PIPs 16 PIPs
Concept Only, Not Yet Active
4 PIPs
Nearly twice as many non- clinical PIPs
13 non-clinical PIPs
7 clinical PIPs

4. Access to Care PIPs
Most clinical PIPs dealt with post hospitalization follow-up
Most non-clinical PIPs addressed ACCESS Call Lines or 24/7 ACCESS

5. Timeliness of Care PIPs
Active or Completed PIPs
17 PIPs
Concept Only, Not Yet Active
3 PIPs
Four times as many non- clinical PIPs
16 non-clinical PIPs
4 clinical PIPs

6. Timeliness of Care PIPs
Most clinical PIPs dealt with improving psychiatric wait times
Most non-clinical PIPs addressed improving timeliness to an appointment
Timeliness to Appointment
4 clinical PIPs
8 non-clinical PIPs

7. Quality of Care PIPs Active or Completed PIPs 16 PIPs
Concept Only, Not Yet Active
6 PIPs
Twice as many clinical PIPs
15 clinical PIPs
7 non-clinical PIPs

8. Quality of Care PIPs
Most clinical PIPs dealt with initial assessment or engagement
Most non-clinical PIPs addressed integration of care

9. Outcomes of Care PIPs Large number of Concept Only, Not Yet Active
12 of the 25 PIPs rated as Concept Only were from this category
Overwhelmingly clinical
25 clinical PIP submissions
5 non-clinical PIP submissions

10. Outcomes of Care PIPs Most clinical PIPs dealt with Recovery services
Most non-clinical PIPS addressed implementation of a new service or team

11. Poll Results

12. Strengths of FY PIPs
Success of Clinical PIPs
Defining the study population
Identifying Interventions

13. Success of Clinical PIP Submissions
More non-clinical PIPs (51) were submitted for review than clinical PIPs (41)
91% submission rate Non-Clinical PIPs
73% submission rate Clinical PIPs
78% of Clinical submissions were Active and Ongoing or Completed
69% of Non-Clinical submissions were Active and Ongoing or Completed

14. Active and Ongoing PIP strengths
Defining the study population
Identifying Interventions

15. CMS Protocol: Validating Performance Improvement Projects
OMB Approval No.
EQR PROTOCOL 3:
VALIDATING PERFORMANCE IMPROVEMENT PROJECTS (PIPs)
A Mandatory Protocol for External Quality Reviews (EQR)

16. CMS Protocol: Validating Performance Improvement Projects
10 Protocol Steps:
Select the study topic(s)
Define the study question(s)
Use a Representative and Generalizable study population
Select the study variable(s)
Use sound sampling techniques

17. Step 3: Use a Representative and Generalizable Study Population
Each PIP must represent the entire Specialty Medi-Cal or DMC-ODS population to which the study indicators apply
The Plan must decide whether to review data for the entire population or a sample of that population

18. Defining the Study Population – FY 16-17 PIPs
No-Shows are higher for one segment of the MHP/DMC population
In a Q4 FY’16-’17 PIP, the health plan analyzed data
Determined that the Youth and Young Adult (YAYA) Division experienced high no-show rates for psychiatry services, as high as 36% in FY13-14, and 32% in FY
The health plan also reviewed literature which reflected no-shows as a sustained negative impact on treatment effectiveness.
The health plan analyzed demographic variables of the YAYA targeted population as well.

19. Examples of Study Populations – FY 16-17 PIPs
All clients with an inpatient hospitalization
Medi-Cal eligible children and youth under the age of 18 after discharge from the Crisis Stabilization Unit.
All adult clients
“The target population included all adult mental health consumers regardless of age, race, gender, or language”

21. CMS Protocol: Validating Performance Improvement Projects
10 Protocol Steps:
Select the study topic(s)
Define the study question(s)
Use a Representative and Generalizable study population
Select the study variable(s)
Use sound sampling techniques

22. Identifying Interventions – FY 16-17 PIPs
Revise the referral and aftercare forms, streamline the referral process, and have regular meetings between Child Stabilization Unit (CSU) and Children and Family Services (CFS) staff.
Support staff performing reminder calls, with two in advance and one the day of an appointment.

23. Identifying Interventions – FY 16-17 PIPs
Assuring that the clinics were in possession of current contact/phone information, which is fairly dynamic.
An appointment within three days given at discharge.
This appointment is followed up by a telephone call to begin the engagement process.

24. Opportunities – FY 16-17 PIPs
Topic Development
Connecting Interventions to Indicators
Project Planning and Implementation Strategies

25. Topic Development
Defining a relevant issue that would benefit consumers, or that would improve consumer services. The purpose of the PIP is to assess and improve processes, and thereby outcomes, of care.
Study Topic – this is the foundation of the PIP.
Development of a PIP is creating a tool (study) that informs the MHP about program effectiveness. The focus of the PIP is to create an environment that encourages improvement – with the goal of improving services to consumers.

26. Topic Development
Development of the study topic should be based on data collection and an analysis of the comprehensive aspects of consumer need, care and services (topics chosen should have as large a scope as possible and impact as many consumers as possible. Limiting the scope of a PIP limits its impact on improving the MHP’s services and consumer outcomes.)
This should include research or a literature review that assists in supporting the topic choice. If national level information is used, the study topic discussion should also explain how the issue is pertinent to the local population.

27. Topic Development – Do We Have A PROBLEM???
Use DATA to identify a problem
Determine possible reasons for the problem – providing more DATA
Use MHP data and demographics to assist in topic development. Identify consumer barriers and develop PIPs w/ strategies to address these barriers.
Establishing a baseline for the problem – more DATA

28. Assess Ask: Do we have a way to measure that?
What data do we have that would show us if there is in fact an issue?
Do we know if our issue is “out of whack” with other
similarly sized counties?

29. Topic Development – Use Data to Identify a Problem – FY 16-17 PIPs
The County was subject to a state audit, which found that psychotropic prescribing practices for foster care youth exceed state standards which had the potential to negatively impact the clinical outcomes of youth consumers.
The baseline data of 17% for 30 day readmissions, is derived from admission records. Of a total of 566 admissions, 95 participants were readmitted. The readmission data is reported quarterly at CQI meetings. The current goal set by CQI for readmissions is 14%. Seven day readmission rates are also tracked and reported.

30. Assess Ask: Do we know if this is problem nationally?
What are the national averages for
this problem/issue/condition?
Is there a norm we are not meeting?
Is this a problem for our county?

31. Topic Development – Use Data to Identify a Problem – FY 16-17 PIPs
A review of timeliness of access for Level 1 consumers (those who are discharging from acute care services) showed the MHP did not meet standards.
MHP contractual requirements and industry practice standard calls for follow-up to be within five business or seven calendar days.

32. Assess
If the MHP or DMC-ODS feels there is an issue, a baseline measurement must be established.

33. Topic Development
MHPs should consider areas in which to develop PIPs on a continuous basis. If an issue that requires improvement has a potential impact on consumer outcomes, the MHP should consider how a PIP could be developed to improve the issue.

34. Continuous Quality Improvement
How not to Assess:
“I think we have an issue with X”.
“Oh, ok, let’s make that a PIP.”
How to begin to Assess:
“I think we have an issue with Y.”
“Why do you think there is an issue?”
“Well, Dr. Jones says people are never here for their scheduled appointment.” OR
“I keep seeing the same patients back in 10 days after discharge.” OR
“Patients’ ANSA scores don’t seem to be improving”.
Not yet - A PIP doesn’t start here.

35. Plan
The MHP or DMC- ODS should gather input from stakeholders to explore solutions to the problem.
What barriers exist?
Can the MHP effect those barriers?
How will the MHP/ DMC-ODS measure their effect?
Not yet - A PIP hasn’t started yet.

36. Improve
Identify interventions (Do not start a PIP with the intervention and then try to make it work)
New treatment we want to try
New training we want to try
The MHP or DMC- ODS should decide what interventions they want to use to try to improve the issue after reviewing the causes and barriers.
Then put those interventions in to place.
A PIP has started.

38. Connecting Interventions to Indicators
Develop measurable interventions –
Appropriate interventions are key to a PIP’s ability to bring about improved mental health care outcomes.
Any intervention should relate to causes/barriers identified through data analysis; should be implementable throughout the MHP system; and not be a “one-time” effort.

39. Connecting Interventions to Indicators
Think about what next steps are necessary to impact the identified problem
Identify interventions that will create the desired change.
Develop at least one new intervention each year.
Create interventions that can be shown to demonstrate a correlation between the issue identified and the interventions put in place.

40. Connecting Interventions to Indicators
Interventions should not focus on procedures, but on consumers.

41. Interventions – FY16-17 PIPs
Adjustments and new interventions as a result of identifying barriers were implemented July 1, The MHP has decided to continue the PIP for another year to assess the results of these Interventions.
Interventions added or changed:
1) Intervention adjusted time for staff to call consumer within 6 days of access opening episode.
2) Intervention changed protocol that instead of consumer receiving a letter confirming an appointment and asking them to call, they will receive a letter confirming appointment and staff will call consumer.

42. Interventions – FY16-17 PIPs
Strengths model training;
Policy and procedure (making sure that all clients have an active MORS); and
Improvement in the EHR.
These interventions are both client focused and administrative and will most likely have clinical impact.

43. Interventions – FY16-17 PIPs
Interventions included:
Benzodiazepine contract between medication only clients and the MHP;
Medication reconciliation with CURES (Controlled Substance Utilization Review and Evaluation System) database;
Drug testing and coordination between primary care and emergency room physicians.

45. Project Planning and Implementation Strategies
Understand and implement a sound method of data collection and analysis.
The data collection plan should:
Clearly identify all data to be collected.
Identify data sources, and how/when the baseline and repeat indicator data will be collected.
Identify who will collect data.
Identify instruments to be used.
Control for variability in clinician expertise and their ability to effectively measure the variable in question.

46. Project Planning and Implementation Strategies
Data Analysis Plan
Determine if qualitative or quantitative data (or both) are to be collected.
Determine how often data will be collected (at least quarterly is recommended).
Determine how to ensure that collected data are comparable to previously collected data.

47. Data Analysis Plan – FY 16-17 PIPs
Clients hospitalized are tracked in an Excel spreadsheet on a SharePoint site so that all County staff involved in this project can view the population intended for intervention.
The hospitalization data is pulled on a monthly basis from the Report 109 – Inpatient Tracking in Avatar. This is essentially the same information as held on the spreadsheet, but pulling from Avatar promotes consistency in the data retrieval. It also ensures that post discharge information is pulled for the proper clients.
Based upon the names/dates in the Report 109, the presence or non‐presence of post‐discharge service delivery is checked manually in the Progress Notes of the medical record. This is labor intensive.

48. Data Analysis Plan – FY 16-17 PIPs
Hospital Discharge data is collected on a daily basis with each admission and is summed monthly. This data has historically been collected by the SCMH Contracts Project Manager who is responsible for processing claims from providers. The Contracts Project Manager collects data from the EHR “Report 109 ‐ Inpatient Tracking.” The data in this particular report is pulled from the “Inpatient Tracking Non‐Episodic” screen in Avatar.
Referrals are tracked ongoing. Clients who are already engaged in a case management program are not referred. The current program is expected to conduct a visit with their client while in the hospital and provide the 7‐day post‐discharge service.
Results are summarized monthly for review with the PIP committee. Quarterly the results are presented in the form of charts. The committee reviews the information and identifies improvements needed.

49. Project Planning and Implementation Strategies
Data Analysis Plan
If manual data collection is the only choice, the PIP design must include:
Appropriate staff who have appropriate clinical knowledge and skills, including conceptual, organizational, and documentation skills.
Data collection instruments that provide for reliable and accurate data collection over the time of the study.

50. Project Planning and Implementation Strategies
When periodic data is collected and analyzed, new or enhanced interventions should be considered, particularly when the data provides evidence that an intervention is not effective.
Be sure that indicators are producing enough data to analyze.

51. Project Planning and Implementation Strategies
Ensure that adequate data analysis occurs for all interventions, and that all variables involved in an initiative are analyzed.
If a PIP implements multiple interventions, ensure that each are measured and all data from each intervention is analyzed.

52. Interventions – FY 16-17 PIPs

53. Project Planning and Implementation Strategies
PIPs should involve consumer feedback as much as possible; consumers’ input can be valuable in determining the direction and effectiveness of interventions. This information should be included in data analysis.

54. Project Planning and Implementation Strategies
Remember that effective analysis is not just presenting data, but providing narrative analysis of the data, with conclusions drawn as to the success of the PIP and planned follow-up activities are described.

55. Project Planning and Implementation Strategies
When a change in performance occurs, determine whether the change is real, attributable to the interventions employed; or to an unrelated event, or random chance.
Determine if the outcomes achieved are statistically significant.
Real improvement results from a continuous cycle of measuring performance and ensuring that effective changes become part of the MHP’s regular organization operations.
If real improvement has occurred, it should be able to achieve sustained improvement.
Sustained improvement is demonstrated through repeated measurements over time.

57. Technical Assistance (TA)
BHC offers TA through several venues, and throughout the year – we want you to be successful!
Consult the PIP library on our website:
Watch a video
See a PowerPoint slideshow
Read through other county PIPs
Use the PIP Validation Tool to self-score your own PIPs
Consult with your lead reviewer early and often
Begin (or continue) working on your PIPs directly after your annual review
Collect and analyze data AT LEAST quarterly, but monthly is better
Submit a draft for feedback
Set up a call to discuss ideas and/or findings

58. Thank you & Happy Holidays!