1. Considerations for Establishing Baseline and Setting Targets for Indicators C3 and B7
Kathy Hebbeler,
Lynne Kahn, Christina Kasprzak ECO/NECTAC
June 16, 2009 1

2. Two related calls
Last Tuesday, June 9, 2009 on “Review of Summary Statements”
Powerpoints on the ECO website
Today, “Considerations for Setting Targets”

3. What we’ll cover today Two strategies for examining data
Data quality
Potential for program improvement
Parameters, guidance for target settings from OSEP
Potential for improvement – refers to program improvement, not data improvement. I think I would insert word program

4. Can you trust the data? Begin by identifying outliers
Examples: look at the percentages reported for category a and category e across local programs

5. Percentages reported in category “a” across 30 local programs

6. Remove the outliers State percentage for “a” with all data= 3.9%
Revised percentage for “a” with outliers removed= 2.4%

7. Percentages reported in category “e” across 30 local programs

8. Remove the outliers State percentage for “e” with all data= 32.1%
Revised percentage for “e” with outliers removed= 27.7%

9. Example of data with outliers removed
Progress Category
Original %
Clean % a 4 2 b 15 17 c 27 30 d 31 e 24 20
Sum St 1 75 76
Sum St 2 54 51
Clean data (without the outliers) may be a more accurate picture of where you are starting

10. Suggested strategy
Analyze your data with your local LEA/program outliers included and excluded so you can gauge the impact they are having on your state level data.

11. Note Note Note
Consider clean data when deciding about reasonable targets, BUT
Turn in the original data to OSEP in the SPP report!
You can discuss the clean data in the rationale for your targets.

12. Which local programs can be targeted for program improvement?
Compare the summary statement data by local program to identify which programs have the most potential for improvement.

13. Summary Statement Percentages by Local Program
I think we are recommending doing this with the outliers removed because the implication is all of this is at least somewhat trustworthy data….we should probably say that explictly --- the outliers are out of here (or not)…

14. Considerations
What do you know about the programs/LEAs with the least and the most progress in the summary statements? i.e. the programs w/
the lowest and highest percentages of children at age expectation at exit
the lowest and highest percentage of children making greater than expected gains

15. Examples of Key Questions
Are the children similar at entry?
Are the higher performing programs/LEAs participating in special projects? e.g. a state initiative, TACSEI or CELL?
Are there systems issues in lower performing programs/LEAs that would explain differences in outcomes? e.g. personnel shortages

16. Bottom-line Question
Could either system or practice focused improvement activities targeted toward the lowest performing programs/LEAs improve the child outcomes?

17. The Math of Target Setting
How much would the data change if the lowest local programs moved toward the mean?
Improvements in the lowest programs will result in improvement in your statewide data
Experiment with your data to determine what targets are reasonable in your state

18. Timelines In Feb, 2010, in SPP format:
Baseline
Targets for 2 reporting years
Improvement activities for 2 reporting years
In Feb, 2011 and 2012, in APR format
Actual data, progress and slippage, etc.
Local reporting of [summary statement %s]

19. Questions and comments?

20. Outcomes Conference
June 22 and 23, Bethesda, MD Resources at The-ECO-Center.org