1. Leadership Academy Conference September 29, 2010
Pennsylvania’s Practice Model and Quality Service Review (QSR) Protocol
Leadership Academy Conference
September 29, 2010
HANDOUTS: 1. Power point
2. PA Practice Model
3. Summary of QSR Indicators
4. Ratings Sheet
5. Case Scenario
6. Indicator ex’s with ratings
Purpose of session:
value of the practice model
understand that the qsr measures practice model performance
understand the connection between the standards and standards with the practice model
cqi rollout plan 1

2. Activity
What do you want children, youth, and families to achieve as result of your intervention?
What do you want children, youth, and families to experience as you work with them?
Child, youth, family outcomes – push to define, spell out what they mean by safety, permanence, wellbeing
Practice model - your approach to working with each other and children, youth, and families to achieve these goals? 2

3. Practice Model Defined (NRCOI, a working document series, July 2008)
It is the agency’s guide to the daily interactions among employees, children, families, stakeholders, and community partners working together to achieve defined outcomes.
It should be the explicit link connecting the agency’s policy, practice, training, supervision, and quality assurance with its mission, values, and strategic plan.
Purpose of a practice model:
Help staff know what their jobs are and how to do them correctly
Staff, stakeholders, and families will understand the agency’s purpose
Systems of service will be aligned under the same philosophical vision
Builds on standards by connecting to overarching values and principles. In development. 3 3

4. What is the Quality Service Review?
Strategies for Success: Tools to Support Supervision and Quality Practice
5/6/2010
What is the Quality Service Review?
Measures child, youth, and family outcomes
Reveals the practice model being used in actual cases.
QSR is an organizational learning process offering helpful ways of knowing what’s working and not working in practice -- for which children, youth and families and why.
QSR connects results to local frontline conditions.
QSR supports teaching & action learning processes that clarify expectations, provide useful feedback, and affirm good work.
QSR stimulates actions taken to improve practice and results at all levels of the organization.
Acknowledge QSR is copyrighted by Human Systems & Outcomes
QSR is an organizational learning process offering helpful ways of knowing what’s working and not working in practice -- for which children and families and why.
We can use it to find out how children and families are benefiting from services received and how well locally coordinated services are working for children and families.
QSR reveals the practice model being used in actual cases.
Connects the values and principles to the case practice – which is different from last PA QSR which focused solely on CFSR outcomes of safety, permanency, well-being and 7 systemic factors – but did not connect to practice model or standards
QSR connects results to local frontline conditions.
QSR Identifies LOCAL conditions under which practice is being done; provides timely results to specific cases as well as systemic/practice strengths and concerns
QSR supports teaching & action learning processes that clarify expectations, provide useful feedback, and affirm good work.
Strength-based approach. Unlike previous PA QSR this process provides immediate feedback to the CW and Supervisor for that child/family, as well as practice performance areas that are needed to improve outcomes
QSR stimulates actions taken to improve practice and results at all levels of the organization.
Recommendations and sharing strengths, sharing areas needing improvement, sharing perspective learned after talking with everyone on the TEAM.
2010 Spring Supervisor Training Event 4 4

5. Pennsylvania background
Practice Standards
A non-regulatory method of establishing benchmarks for defining optimal, but achievable, results for child welfare services in Pennsylvania.
Evolution of standards
1999 – stakeholder group
2008 – stakeholde)r group , revisions to standards, addition of values and principles (handout
PRACTICE MODEL GUIDE WILL BE DEVELOPED TO ASSIST COUNTIES IN MOVING THEIR PM FORWARD…BASED ON QSR FINDINGS. 5 5

6. Pennsylvania background (continued)
Quality Service Review history
CFSR round 1
CFSR round 2
What’s different from round 1 to 2
Cfsr 1 – between 2004 – 2007 – every county – over 600 cases reviewed, smaller counties in clusters, 15 per review
Cfsr 2 – many similar findings, focus on implementing change at local level, lead to creation of CQI, which qsr is a component
WHAT IS QSR and how is THIS time different from 1st round of PA QSRs
Who in the room has participated in the QSR process before?
What worked about QSRs?
What didn’t work about QSRs?
This time the QSR is connected to CQI – not something to be done in isolation or out of context with where each individual county is…..QSR is NOT to exist by itself – it is but ONE component of the CQI process –
DAPIM process – flywheel next slide
qsr pilot history 6

7. PA’s Model of CQI1
DAPIMTM - (Define, Assess, Plan, Implement, Monitor)
Organizational Effectiveness (OE) is a “systematic and systemic approach to continuously improving an organization’s performance, performance capacity and client outcomes”. DAPIMTM is APHSA’s model for systematic continuous improvement. APHSA has found that to improve something, you have to do the following:
DAPIMTM MODEL
DEFINE
ASSESS
PLAN
IMPLEMENT
MONITOR
Step One: Define what the thing is in operational terms,
Step Two: Assess the current and desired state,
Step Three: Plan both rapid and longer-term improvements,
Step Four: Implement those plans in full, and
Step Five: Monitor implementation progress and impact for accountability and on-going adjustments.
DAPIMTM is both sequential and cyclical. It operates as a flywheel in real time. When beginning a DAPIMTM process definitions are crafted regarding what is the problem to be worked on. Following the work of defining, assessments are completed regarding an organization’s current state in relation to the defined problem. This assessment generates strengths and gaps which lead to findings of actionable root causes for the defined problem. While in the DAPIMTM process, an organization’s finding are continuously refined by lessons learned and new data generated while planning for continuous improvement, implementing those plans, or monitoring plans. Each of these areas of the flywheel are explained in more detail in the coming pages. 7 7 7

8. How Does QSR Work? State and local review team to review one case
Strategies for Success: Tools to Support Supervision and Quality Practice
5/6/2010
How Does QSR Work?
State and local review team to review one case
Review of the record
Focused interviewers with every member of the child/family TEAM
Rates the status of the child/family and the system performance on a 1-6 scale
Narrative that “tells the story” of what was learned from the TEAM
Immediate feed back to CW and Supervisor with strengths/needs/recommendations
Case specific
Agency specific
Includes focus groups with stakeholders
Debrief of preliminary findings
Final findings report
QSRs contain a very small sample but…. Every individual case that is examined has something to teach us about what is working now and finding ways to do things better tomorrow (not a large scientifically valid process and not focused on “who screwed up”)
One case: In-depth case reviews to measure current status, recent progress, and adequacy of current practices in getting results for children and families being served in local sites. [Drill Downs] Case review provides a MICRO VIEW of a case – Each family served is a unique “test” of the service system. The QSR views the unfolding of practice in the lives of children and families as highly interactive, and dynamically complex in nature (not linear) which means that practice performance is highly dependent on the local conditions under which practice is occurring. The QSR process relates present case practice and results to local conditions and goodness-of-fit between the practice model and the children and families who are receiving the services. IT FOCUSES ON “FUNCTIONAL PRACTICE PERFORMANCE RATHER THAN SIMPLE COMPLIANCE with Policies and procedures.” The case review is a PROFESSIONAL (honest and informed) APPRAISAL of: (1) the status of a focus child and parent/caregiver on key indicators; (2) recent progress made on applicable change indicators; and (3) adequacy of performance of essential service functions for that child and parent/caregiver. Each focus child served is a UNIQUE and VALID point-in-time “test” of frontline practice performance in a local system.
Case file review – includes having the reviewers gain some case knowledge to prepare for interviews
Focused interviews – incorporates in person/phone interviews of all integral members of the child/family team; focus is on gathering information, not on sharing information
Rating – uses a scale of 1-6 to determine the status of the child/family and the performance of the team in working with that child/family; Uses aggregate quantitative patterns of qualitative indicators to reveal and describe the quality and consistency of local practice.
Narrative - Uses the power of the story to reveal what is happening and working for families at the practice points. [Stories Teach]
Power of story – we can’t understand what is behind the #’s (the data) without learning from what the stories can teach us – usefulness of case stories during individual supervision and unit meetings, etc, can be used with stakeholders
Feedback: Two purposes – 1) Understand current status and facts of the case, as well as sharing new facts of the case that come to light during the interview process; 2) Offer helpful successful options/recommendations for the child/family situation and for the organization (team) as a whole on how to improve practice with the child/family; Focus is on learning and improvement, review of ratings, and substantiates scores.
Focus Groups: Local focus group and key stakeholder interviews along with case stories, data patterns, and local working conditions to find and affirm what’s working now and to surface areas where even better results might be achieved in the future.
Focus groups – to provide a macro-view of local site and local conditions
Provides information about local practice/resources/collaboration/coordination/working conditions-helps contextualize/explain findings
Debrief on preliminary findings: Second level QA on the reviews of the cases; providing an overview of the preliminary rating scores; discussion about validity of findings based on agency’s perspective – does this surprise you, are there “a-ha” moments?
Final findings report: Review of demographic data; Discussion about findings (trends); Sharing of case story information – recommendations; Identification of how the county will utilize the findings; Feedback loop for the state on how they can support the county in addressing their findings and applying trends/findings to larger statewide outcomes - turning the flywheel of DAPIM – applying the findings/data to drive continuous quality improvement. 3 follow up ?’s to use with Counties following the QSR:1) What did you learn? (what is working right now, and what can we do better?); 2) What are you going to do? (Act locally and what steps make sense to you? – the change needs to be locally driven); 3) How can we help? (partnerships with State, providers, TA….etc.)
2010 Spring Supervisor Training Event 8 8

9. Indicators in the PA QSR Protocol
Strategies for Success: Tools to Support Supervision and Quality Practice
5/6/2010
Indicators in the PA QSR Protocol
Child, Youth and Family Status Indicators
1a. Safety: Exposure to Threats of Harm
1b. Safety: Risk to Self/Others
2. Stability
3. Living Arrangement
4. Permanency
5. Physical Health
6. Emotional Well-being
7a. Learning & Development: Early Learning and Development
7b. Learning & Development: Academic Status
8. Pathway to Independence
9. Parent and Caregiver Functioning
PA QSR protocol still in DRAFT form – see handout of summary of all indicators.
Child/Youth Safety, Permanence and Well-Being Indicators * most use 30 day timeframe to rate
1. Safety of the Child/Youth from:
a. Exposure to Threats of Harm
b. Risk to Self/Others
2. Stability (looks at 12 months back and 6 months ahead)
3. Living Arrangement
4. Permanency
5. Physical Health
6. Emotional Well-Being
7. Learning and Development:
a. Early Learning and Development (0-4 years)
b. Academic Status (5-15+ years)
8. Pathway to Independence (16+ years)
Caregiving Indicators
9. Parent and Caregiver Functioning
2010 Spring Supervisor Training Event 9 9

10. Indicators in the PA QSR Protocol Practice Model Functions
Strategies for Success: Tools to Support Supervision and Quality Practice
5/6/2010
Indicators in the PA QSR Protocol
Practice Model Functions
1a. Engagement: Engagement Efforts
1b. Engagement: Role & Voice
2. Teaming
3. Cultural Awareness & Responsiveness
4. Assessment & Understanding
5. Long-Term View
6. Child/Youth and Family Planning Process
7. Planning for Transitions and Life Adjustments
8. Timely Permanence
9. Intervention Adequacy & Resource Availability
10. Maintaining Family Relationships
11. Tracking & Adjusting
Practice Performance Indicators
1. Engagement:
a. Engagement Efforts
b. Role and Voice
2. Teaming - looks at FORMATION & FUNCTIONING
3. Cultural Awareness and Responsiveness
Assessment and Understanding
Long-Term View
Child/Youth and Family Planning Process
Planning for Transitions and Life Adjustments – looks retrospectively over the past 90 days and prospectively over the next 90 days
8. Timely Permanence
Intervention Adequacy and Resource Availability
10. Maintaining Family Relationships
11. Tracking and Adjustment
2010 Spring Supervisor Training Event 10 10

11. (see handout)
Each indicator is rated on a scale of 1 through 6. And there are a few ways that you can break down and look at the data. VERY high standards
1-3 – unacceptable range means something needs to change here and some type of concerted action should be taken 11 11 11

12. Activity Rate a scenario based on assigned indicator with your partner
Discuss ratings 12

13. Next steps: connections to CQI
Finalization of QSR protocol and process (including manual and training
Roll out of CQI in phased-in approach across the state
Finalization Of Practice Model & Resource To Support Implementation (GUIDE BOOK) 13

14. How can I get involved?
If you are not a phase one county, are you interested in phases two or three?
Can volunteer staff to be reviewers in phase one county QSRs 14

15. Who to contact for more information:
Mike Byers
(717)
Stephanie Maldonado
(717) 783 – 7376
Jeanne Schott