1. Implications of CFSR 3 for IVE Programs
Richard P. Barth & Terry V. Shaw
October 23, 2014
CSWE APM
Tampa, Florida

2. Purpose of the Presentation
To stimulate thinking and discussion about:
What is the meaning of CFSR 3?
Does it create new opportunities for SSW University-Agency Partnerships?
Does it suggest changes in SW Education?

3. CFSR 3 Guiding Principles
States and the Federal Government are unified in a commitment to safety, permanency, and well-being for the children served by the public child welfare system.
The CFSRs have led to the identification of opportunities for:
1. program improvement
2. improvement in outcomes for children and families.
Further improvements in the CFSR and in the related CQI processes are critical to the continued CWs advances
Scientifically valid
Feasible
Understandable
Meet All Federal Requirements

4. National Standards (Issues)
Points of Agreement
Areas for Discussion
Proposal
Input
Aggregate administrative data on safety and permanency
Recommendation to endorse CB proposal; or
Compute State-specific rates and calculate a national average rate
Applied Methodology
Multi-level Model
Risk Adjusted
Recommendation to endorse CB proposal; or
Endorse with additional factors; or; Do not risk adjust
Output
Set National Standard at national average
for all measures
Proposal is reasonable 4

5. National Standards
Safety Outcome 1: Children are, first and foremost, protected from abuse and neglect.
Safety Performance Measure 1: Maltreatment in Foster Care
CFSR Round 2
CFSR Round 3
Differences
Of all children in foster care during the reporting period, what percent were not victims of substantiated or indicated maltreatment by a foster parent or facility staff member?
Of all children in foster care during a 12-month period, what is the rate of victimization per day of foster care? (most likely reported in a rate per 100,000 days)
Child maltreatment while in care is calculated by directly linking data from NCANDS (numerator) with the AFCARS file (denominator) using CHILDID.
All instances of maltreatment will be included regardless of type of perpetrator (including parents).

6. Re-Report of Maltreatment (Issues)
Points of Agreement
Areas for Discussion
Proposal
Of all the children who were victims of substantiated or indicated abuse or neglect during the first 6 months of the reporting period, what percent experienced another screened-in report with a 12-month period?
It is important to include a measure of repeat maltreatment.
No support for the measure as proposed
“…..getting the measure right should trump concerns over current data capacities.”
Recommendations for appropriate measure(s)?
Recommendations around data availability, quality and testing proposed measures? 9

7. National Standards
Safety Outcome 1: Children are, first and foremost, protected from abuse and neglect.
Safety Performance Measure 2: Re-report of Maltreatment
CFSR Round 2
CFSR Round 3
Differences
Of all children who were victims of substantiated or indicated maltreatment allegation during the first 6 months of the reporting period, what percent were not victims of another substantiated or indicated maltreatment allegation within a 6-month period?
Of all children with a screened-in report of alleged maltreatment in a 12-month period, what percent had another screened-in report within 12 months of their initial report?
Lengthens the timeframes being considered (from 6 months to 12 months)
increases the scope to include all screened-in reports and not just Substantiated or Indicated.

8. National Standards
Permanency Outcome 1: Children have permanency and stability in their living situations.
Permanency Performance Measure 1: Permanency in 12 months for children entering foster care
CFSR Round 2
CFSR Round 3
Differences
Composite 1.3: Of all children entering foster care for the first time in a 6-month period, what percent discharged to reunification (or live with relative) within 12 months of entering foster care or by the time they reached 18?
Of all children who enter foster care in a 12 month period, what percent discharged to permanency within 12 months of entering foster care?
Expanded the exits from reunification alone, to permanency (reunification, live with relative, adoption or guardianship).
Expands base time from 6 months to 12 months and includes all entries, not solely first entries.

9. National Standards
Permanency Outcome 1: Children have permanency and stability in their living situations.
Permanency Performance Measure 2: Permanency in 12 months for children in foster care for 2 years or more
CFSR Round 2
CFSR Round 3
Differences
Composite 3.1: Of all children in foster care on the first day of a 12-month period who had been in foster care (in that episode) for 2 or more years, what percent discharged to permanency within 12 months of the first day or by the time they reached 18?
Of all children in foster care on the first day of a 12-month period who had been in foster care (in that episode) for 2 or more years, what percent discharged to permanency within 12 months of the first day?
No differences.

10. National Standards
Permanency Outcome 1: Children have permanency and stability in their living situations.
Permanency Performance Measure 3: Re-entry in 12 months
CFSR Round 2
CFSR Round 3
Differences
Composite 1.4: Of all children discharged from foster care to reunification or live with a relative in a 12- month period, what percent re-entered foster care in less than 12 months from the date of discharge?
Of all children who enter foster care in a 12- month period and discharged within 12 months to reunification, live with relative, or guardianship, what percent re-entered foster care within 12 months of their date of discharge?
The measure is now based on an entry cohort instead of an exit cohort.
The denominator has been expanded and now includes Guardianship as well as reunification, and live with relative.

11. National Standards
Permanency Outcome 1: Children have permanency and stability in their living situations.
Permanency Performance Measure 4: Placement stability
CFSR Round 2
CFSR Round 3
Differences
Composite 4.1: Of all children served in foster care during the 12- month period, what percent had two or fewer placement settings?
Of all children who enter foster care in a 12- month period, what is the rate of placement moves per day of foster care? (most likely expressed as a rate per 1,000 days)
The measure is now based on an entry cohort instead of all children served and in care for a period.
All placement moves will be counted during a period rather than just identification of stable vs. not stable.

12. Risk Adjusted Analyses
And, yet, we might include child sex despite the absence of any data to suggest it should be used.

13. Performance Goals and Thresholds (issues)
Points of Agreement
Areas for Discussion
Proposal
Input
State specific data on the most recent 3 years of performance
Bootstrapping is reasonable
Recommendation to incorporate trend consideration
Applied Method
Min. & Max. Goals & Thresholds
Min. = Top of bottom 5th
Max. = Bottom of top 5thth
Use Companion Measures
Applied Method
Bootstrapping to set baseline
Estimate grand mean and standard deviation
Recommendation to adjust calculations to 1 SD
Recommendation to set at 5th and 45th State performance level
Out Put
Improvement Factor Grand Mean/Grand Mean + 2SD
Goal Calculation
Baseline * Improvement Factor
Threshold Calculation
Baseline – 2SD
Companion Measures are reasonable
Output
Measurement at 3 & 5 years for some measures and 4 & 5 years for one measurement 7
Measurement timeframes are reasonable

14. Case Review Standards (issues)
Points of Agreement
Areas for Discussion
Proposal
Case Review determines substantial conformity for 24 items that assess case-specific practices in both in-home and foster care cases.
A single standard is applied to all Case Review items.
Case Review is an important component of a CQI system.
A single standard should be applied.
The Panel suggested that Case Review should not be used to determine Substantial Conformity.
How would Case Review results be used to meet legislative objectives to measure State performance and improvement?
What should the standard be?

15. Case Review Standards
In prior rounds onsite case reviews were conducted during a one-week visit by external reviewers.
In round 3, qualifying states may conduct their own reviews using a revised CFSR onsite review instrument. States have to meet three criteria in order to use their own case review process and be able to meet the following time lines.
Must use a sample period of 4/1 to 9/30 of year prior to review year.
Conduct reviews from 4/1 to 9/30 of the year of review and provide information to the Children’s Bureau
Report results no later than 11/15 of the year of review to the Children’s Bureau
If a state receives approval to use its own case review process in conjunction with the federal CFSR onsite review instrument they must provide to CB its case review schedules and other relevant information. This will allow the federal team to participate in the state’s case review process. At a minimum, CB participation will include observing the state’s case review process while present in the state and reviewing completed instruments. As noted above, stakeholder interviews will also still be conducted jointly by the federal and state team during the period when case reviews are conducted. Federal oversight by CB may be necessary to ensure the state conforms to our case review criteria and produces reliable results for the CFSRs.
If a state does not meet the necessary criteria to use their case review process or chooses not to do their own case reviews, CB will work with the state to prepare for a more traditional week-long case review conducted jointly by the state and CB.

16. Case Review Standards
Criterion 1 - The states operates an internal case review process at least annually that assesses statewide practice performance for the key child welfare areas using a uniform sampling process and methodology
Criterion 2 - The state has a process in place for ensuring accurate and consistent case review ratings.
Criterion 3 - The state uses the federal onsite review instrument and its instructions using the sample and method established above to collect data to be used for the initial determination of conformity.
Criterion 1 - The states operates an internal case review process at least annually that assesses statewide practice performance for the key child welfare areas using a uniform sampling process and methodology.
This means that the sampling methodology must address the following specific criteria:
• A statewide schedule that selects cases randomly from the entire state universe; or a stratified schedule of counties or jurisdictions, which consists of a cross-section of state child welfare practice and includes the largest metropolitan area and significant tribal or other populations that is representative of state demographics. That same stratification will then be replicated for ongoing performance measurement.
• The state uses a simple random sample design but may include additional stratification to achieve an adequate representation of key program areas.
• The sample consists of a minimum of 65 cases served during the sample period with a minimum of 40 foster care cases and 25 in-home cases, inclusive of alternative response cases. Samples larger than 65 should reflect the state ratio of foster care and in-home cases as long as the minimums are met for both case types.
• The sampling frame for the state foster care population consists of the listing of children served statewide or by jurisdiction strata according to the states’ AFCARS defined reportable cases for the CB defined sample period. To allow for ongoing review to occur timely, a state may use its AFCARS defined reportable cases for each day in a quarter.
Criterion 2 - The state has a process in place for ensuring accurate and consistent case review ratings.
This means:
• The state provides consistent training of all reviewers on the case review process.
• The state adheres to instructions contained in the case review instrument for rating cases.
• The state has a process in place to ensure consistency of ratings across multiple sites and reviewers, and includes third party (i.e., someone who has not reviewed the case) quality assurance of cases reviewed for accuracy of ratings in accordance with the instrument and instructions.
• The state ensures that individuals who had direct contact, supervision, oversight or consultation for the case being reviewed do not complete the case review or quality assurance.
Criterion 3 - The state uses the federal onsite review instrument and its instructions using the sample and method established above to collect data to be used for the initial determination of conformity.
• The state uses CB’s onsite review instrument and its instructions to collect information on all necessary items and implements and adheres to guidance CB issues to accompany the instrument.
• The state includes case specific interviews of key informants on every case to inform the ratings, including all of the following individuals: child (if age and developmentally appropriate), parents, caregiver/foster care provider, and caseworker or supervisor, and follows a written protocol for acceptable case specific exceptions to an interview.

17. Sampling (issues) Areas for Discussion Points of Agreement Proposal
Sample to allow for statistical inferences
Combined foster care and in-home case sample
Simple random sampling design
Statewide with options for geographic representation
Option to reduce sample size for ongoing program improvement
No support for a sampling approach that could eliminate children in rural areas
No support for reducing sample size for ongoing program improvement
Should sample size be increased to allow for statistical inferences for foster care and in-home cases?
Is the proposed sample size feasible?
Is the proposed random sampling design feasible?
What are alternative sampling strategies?
Panel suggestion that targeted reviews could have a role in ongoing improvement monitoring.

18. Sampling
Random selection; or a stratified selection representative of the states demographics and includes the largest metropolitan area.
Minimum of 65 cases served during the sample period (40 foster care cases; 25 in-home cases, including Alternative Response);
Out of home cases must be consistent with the listings reported to AFCARS during the period.
In home cases (including AR) must have been opened for at least 45 consecutive days during the sampling period .
The state must be able to consistently address document case elimination.

19. Standards for Systemic Measures (issues)
Points of Agreement
Areas for Discussion
Proposal
The State’s CQI system would identify sources for quantifiable data for items TR1 – TR4 and FAP 2.
A single standard is applied to all systemic capacity items.
The prior method of assessing systemic measures was subjective.
The evidence connecting the systemic factors to family outcomes is limited.
What is the recommended standard and why?
90%
75%
Should the standard be different for systemic capacity items assessed through case review?

20. Implications
The measures are simpler which will help in explaining what they mean to all the various audiences - this should make it clearer to develop strategies to influence outcomes.
Opportunities for SSWs to increase their role regarding development and implementation of CQI processes
Opportunities to work with state partners on the importance of understanding the trajectory of children into and through the service systems
Greater focus on CQI in social work education
Critiquing CFSR 3 or Designing CFSR 4 could become a useful activity for a research or CWS course