1. Prepared for the ECO Advisers Meeting February 2, 2011 National Data on Child Outcomes: Are We There Yet?

2. Computing national numbers 2 Early Childhood Outcomes Center Department of Education needs national numbers for GPRA for Part C and Part B preschool Ideally, we would add up the numbers from each of the states and get the national numbers but….

3. States are in various stages with regard to building their child outcomes measurement systems Some phrased in Some changed approaches 3Early Childhood Outcomes Center Child Outcomes: What we know

4. Producing National Numbers: The Issues Missing data? –Which states are reporting data on all the children who stayed 6 months and exited in 08-09?* Data quality –Which states have invested sufficient resources to be producing credible data? –Which states have been producing outcomes data long enough to have credible data? * A very few states are sampling. 4 Early Childhood Outcomes Center

5. Missing/Incomplete Data: Part C 5 Early Childhood Outcomes Center Aver=35.7%; 31 Sts. > 33%

6. Missing/Incomplete Data: Part B Preschool 6 Early Childhood Outcomes Center Aver= 21.1%; 22 Sts. > 25%

7. Data Quality: Outliers 7 Early Childhood Outcomes Center Some data quality issues are obvious States whose data look VERY different from other states or from what would be expected.

8. What would we expect? 8 Early Childhood Outcomes Center

9. What would we expect? 9 Early Childhood Outcomes Center

10. What would we expect? 10 Early Childhood Outcomes Center

11. What would we expect? 11 Early Childhood Outcomes Center

12. Producing National Estimates 1.Divide the states into those with large, medium, and small child counts. 2.Identify the states that appear to have the best data. 3.Weight by child count and calculate the overall OSEP percentages using the “best data” states in each category. 4.Use the OSEP percents and total child count for each category to get the national estimates. 12 Early Childhood Outcomes Center

13. 1. Divide the states into those with large, medium, and small child counts. Size of Child CountStates LargeCA, NY, PA….. MediumTN, CO, AZ… SmallMT, ND, DE….. 13 Early Childhood Outcomes Center

14. How Weighting Works STATES w/ USEABLE DATA a%a% b %c %d %e %Child CountNew # for e A302000600 B252400600 C201500300 D3030001000 E332400800 Total11,3003,300 Re-calculated a-e for this set of states29.2% 14 Early Childhood Outcomes Center

15. And now the results…. 15 Early Childhood Outcomes Center

16. 16 Note: Analysis based on data from 19 states. Data were weighted by child count to represent the nation.

17. Early Childhood Outcomes Center17 Note: Analysis based on data from 19 states. Data were weighted by child count to represent the nation.

18. 18 Early Childhood Outcomes Center Note: Analysis based on data from 15 states. Data were weighted by child count to represent the nation.

19. 19 Early Childhood Outcomes Center Note: Analysis based on data from 15 states. Data were weighted by child count to represent the nation.

20. How trustworthy are these numbers? One possible test: What do we get when we compare this method with other methods for calculating the national numbers? 20 Early Childhood Outcomes Center

21. 3 Approaches to National Numbers 1.Use data from states with the best data (least missing, not an outlier on “a” or “e”). Weight their data to represent states of similar size.* 2.Use data from all states. Average across the percentages. Each state weighted as one. 3.Use data from all the states. Weight each state according to their child count. * We did this twice with a smaller set of states and a slightly larger set. 21 Early Childhood Outcomes Center

22. Comparing the three methods See handout 22 Early Childhood Outcomes Center

23. Our Questions for You What do these data say to you about the outcomes of Part C and Part B Preschool? What method makes the most sense for generating the calculations? What would you use as indicators that the national data are valid and meaningful? When would you trust the data? –When all (most?) states can demonstrate valid data? –When the alternate methods converge more closely? 23 Early Childhood Outcomes Center