1. Performance Improvement Project (PIP) Clinic: Topic Selection, Indicators, Interventions
June 21, 2018
Amy McCurry Schwartz, Esq., MHSA
EQRO Consultant
Behavioral Health Concepts, Inc.

2. VALIDATING PERFORMANCE IMPROVEMENT PROJECTS (PIPs)
OMB Approval No.
EQR PROTOCOL 3:
VALIDATING PERFORMANCE IMPROVEMENT PROJECTS (PIPs)
A Mandatory Protocol for External Quality Reviews (EQR)

3. CMS Protocol: Validating Performance Improvement Projects
10 Protocol Steps:
Select the study topic(s)
Define the study question(s)
Use a Representative and Generalizable study population
Select the study variable(s)
Use sound sampling techniques

4. CMS Protocol: Validating Performance Improvement Projects
10 Protocol Steps Continued:
Reliably collect data
Analyze data and interpret study results
Implement intervention/improvement strategies
Plan for “real” improvement
Achieve sustained improvement

5. Step 1: Select the study topic(s)
Look for topics that:
Affect a significant portion of consumers
Have a potentially significant impact on consumer health, functional status or satisfaction
Reflect high-volume or high-risk conditions of the population served

6. Step 1: Select the study topic(s)
Do a thorough data review before beginning the PIP
County specific data
Review local data linked to your challenge/problem
Review system procedures and process linked to your challenge/problem
National data
Helpful background, benchmarks and possible interventions
SAMHSA, NIDA, NIMH, NIATX, AHRQ, NCQA, ASAM, NSDUH, etc

7. Assess Ask: Do we know if this is problem nationally?
What are the national averages for
this problem/issue/condition?
Is there a norm we are not meeting?
Is this a problem for our county?

8. Step 1: Select the study topic(s)
Data sources that may help:
County eligibility database
County encounter/claims data
County performance on standardized measures
External data – Medicaid FFS, other health plans, local or national public health reports (be sure to show this data has a direct correlation to the County’s population)

9. Step 1: Select the study topic(s)- Facts
Identify a problem area or opportunity for improvement in treatment or process, but do not take it “to the bank” without first reviewing your facts and data
Include information from clients, families, staff and community partners
Client, families, community partner and staff input should be obtained on problem
What do they see as the barriers to better care?
Why do they think the problem is happening?
What ideas do they have to fix it?

10. Assess Ask: Do we have a way to measure that?
What data do we have that would show us if there is in fact an issue?
Do we know if our issue is “out of whack” with other
similarly sized counties?

11. Assess
If the MHP or DMC-ODS feels there is an issue, a baseline measurement must be established.

12. Plan
The MHP or DMC- ODS should gather input from stakeholders to explore solutions to the problem.
What barriers exist?
Can the County effect those barriers?
How will the MHP/ DMC-ODS measure their effect?

13. Questions

14. Step 2: Define the Study Question(s)
Make the Question:
Obvious
Simple
Answerable

15. Step 2: Define the Study Question(s)
State the Question in a way that:
Supports the ability to determine whether the intervention has a measurable impact for a clearly defined population

16. Step 2: Define the Study Question(s)
Vague Study Question Example:
An “implied” study question, or
“Is a mail intervention adequate to improve medication adherence over time?”

17. Step 2: Define the Study Question(s):
Clearer Study Question Example:
“Will providing educational materials to providers and members increase the number and percentage of consumers who participate in a medication clinic, and decrease the number and percentage of consumers who use crisis services?”

18. Step 2: Define the Study Question(s):
Clearer Study Question Example:
“Does creating a Care Coordination Team with strategies to engage and provide timely access to outpatient services increase engagement and mental health treatment and increase overall client satisfaction?”

19. Step 2: Define the Study Question(s):
This step is critical because it sets the framework for data collection and analysis of results
So take the time to develop a strong study question!

20. The BEST PIPs have….
Knowledge of what might have changed in your environment to contribute to or improve the problem (examples, opening or closing problems due to budget changes or new medications to help improve recovery for specific SUDs or diagnoses)
A clear study question based on thorough data collection and analysis of the problem and potential causes

21. Questions

22. Step 4: Select the Study Indicator(s)
A study indicator is:
A measurable characteristic, quality, trait, or attribute of a particular individual, object, or situation being studied.

23. Step 4: Select the Study Indicator(s)
Criteria
Each PIP should have one or more measured indicators to track performance and improvement over a specific period of time. All measured indicators should be:
Objective; and
Clearly defined; and
Based on current clinical knowledge or health services research; and
Enrollee outcomes (e.g. health or functional status, enrollee satisfaction); or a valid indicator of these outcomes.

24. Step 4: Select the Study Indicator(s)
The number and complexity of indicators (measures) may vary depending on:
The Study Question(s);
The complexity of existing practice guidelines for a clinical condition; and
Availability of data and resources to gather data

25. Step 4: Select the Study Indicator(s)
Indicators may be quantitative or qualitative and continuous or discrete.
Discrete or categorical indicators have a limited number of possible categories
An individual has/has not attended medication support in the last 6 months.

26. Step 4: Select the Study Indicator(s)
Continuous indicators have unlimited possible values within the limits of the indicator range
ANSA or CANSA score
# of appointments attended
Data collected on a continuous indicator such as # of appointments attended can be used for a discrete indicator (e.g. a client attended at least one appointment in the past 3 months)

27. Step 4: Select the Study Indicator(s)
Study Indicator “tips”:
The study indicator should be objective and clearly defined
The indicator must be “measurable” – do you have the necessary data (or can you get it)?
Use an existing, relevant health care guideline to develop the study indicator
Use an existing indicator generally accepted in the health care community (ex. HEDIS; Clinical Practice Guidelines…)

28. Step 4: Select Study Indicator(s)
Potential Sources of Supporting Information
Clinical and non-clinical practice guidelines
Administrative Data
Claims data
Enrollment files
Medical Records

29. Step 4: Select Study Indicator(s)
Examples of measures currently existing within the public health community or managed care industry include:
NCQA Healthcare Effectiveness Data Information Set (HEDIS)
Measures developed by CMS and AHRQ (Pediatric or Adult Core Measures)

30. Example of Study Indicators
If goal is engagement/participation in medication clinic: Indicators- “Consumers enrolled in medication clinic with <2 crisis visits/calls in a 6 month period.” “Consumers enrolled in medication clinic with attendance at clinic >4 times in a 6 month period.” “Consumers enrolled in medication clinic with attendance at clinic 6 times in a 6 month period and 0 crisis visits."

31. Example of Study Indicators
If goal is: More participation in treatment
Indicator – “On follow up visit, clinician reported the consumer was engaged in treatment and willing to work towards treatment goals.”
If goal is: More ability to recall what client was to accomplish before the next visit
Indicator – “Consumer was able to accomplish 1 or more next steps in their treatment plan by their return visit”.
If goal is: Consumer satisfaction
Indicator – “Pre and post satisfaction testing”

32. Questions

33. Connecting Interventions to Indicators
Interventions should not focus on procedures, but on consumers.

34. Step 8: Implement Intervention and Improvement Strategies
Interventions should:
Relate to causes/barriers identified through data analysis
Be culturally and linguistically appropriate
Be “implementable” system-wide
Not be “one time” efforts

35. Step 8: Implement Intervention and Improvement Strategies
Topic: Decrease no-show rate
Interventions:
Reminder calls (standardized)
Create a more welcoming environment
Incentivize attendance

36. Step 8: Implement Intervention and Improvement Strategies
Topic: Decrease no-show rate
Interventions:
Scheduling to accommodate public transportation times for clients using public transportation
Help transport clients to their medication support appointments who are severely disabled.

37. Step 8: Implement Intervention and Improvement Strategies
Topic: Improve access and retention
Interventions:
Implemented an integrated walk-in screening tool at one adult behavioral health clinic
To improve the accurate identification of co-occurring diagnoses and to provide “right size” services to consumers from the start.
One provider contacts each client

38. Step 8: Implement Intervention and Improvement Strategies
An improvement strategy is an intervention designed to change behavior at an institutional, practitioner, or beneficiary level.
Real, sustained improvement results from a continuous cycle of measuring performance and implementing system-wide improvements.
(Plan, Do, Check, Act)

39. Questions

40. What makes a PIP successful?
Meeting the Requirements of EQRO
Submit a completed submission tool
Use data to identify an issue and measure success
Use of stakeholder input
Measure early and often

41. PIP Ratings Active and Ongoing
Baseline established on at least some of the indicators and at least some interventions have started. Any combination of these will be acceptable. This WILL be rated.
Completed
In the past 12 months or since the prior EQR. This WILL be rated.
Baseline established on at least some of the indicators and at least some interventions have started. Any combination of these will be acceptable. This WILL be rated.

42. PIP Ratings Concept Only, Not Yet Active
This is an opportunity to provide TA step-by-step and help the MHP/DMC-ODS move this towards an active PIP. Baseline may have been done, but interventions have not been started. This will NOT be rated.
Inactive, Developed in a Prior Year
Rated in a prior year and not rated this year. For example: If a new intervention was not introduced in the current year, a PIP would be considered inactive. These will NOT be rated.
Submission Determined Not to be a PIP
Use this category when the write-up contains a lack of plan, data, or indication where the County will get their data, and the County has not implemented anything.

43. What makes a PIP successful?
Potential to produce good results
Implemented as planned
Positive change occurs
Positive change is sustained over time
Initiative becomes an ongoing part of continuous quality approach to care

44. PIP Scoring Met Partially Met Not Met Each MET element = 2 points
Each PARTIALLY MET element = 1 point
The MET and Partially MET elements are divided by the Total Applicable Elements to.

46. PIP Scoring
The MET and Partially MET elements are divided by the Total Applicable Elements to arrive at a score.
(# MET * 2) + (# Partially MET)
(# of Applicable Elements *2)

48. PIP Scoring
The score is a reflection of the EQRO’s confidence in the PIP’s ability to show improvement (reach the goals).
The score is NOT a rate of completion alone.

49. Amy McCurry Schwartz, Esq., MHSA
Contact Information
PIP Technical Assistance is available from your assigned Quality Reviewer
Amy McCurry Schwartz, Esq., MHSA
(855) x 103

50. Questions