1. 1 Trends in Child Outcomes (C-3 / B-7) and Family Outcomes (C-4) Analysis and Summary Report of All States’ 2008-2009 Annual Performance Reports Christina Kasprzak, Robin Rooney, Siobhan Colgan Lynne Kahn, Kathy Hebbeler (NECTAC / ECO) November 30, 2010 4:00 PM EST

2. 2 1. National data on child and family outcomes (C3/B7 and C4) 2. Challenges related to collecting and reporting on this indicator 3. Improvement activities Webinar Focus

3. Quick Poll 1 Who is joining us on the call today?

4. 4 Child Outcomes Data Summary of 2010 APR Data

5. 5 Part C State Approaches (n=56) to Measuring Child Outcomes Child Outcomes Summary Form (COSF) –41 (73%) states Single assessment statewide –7 (13%) states Publishers’ online assessment systems –3 (5%) states Other approaches –5 (9%) states

6. 6 HI GU AS MP Legend: COSF Publishers’ on-line systems One tool statewide Other State Approaches to Child Outcomes Measurement Part C Program Early Childhood Outcomes Center –August 2010

7. 7 7 619 State Approaches (n=59) to Measuring Child Outcomes Child Outcomes Summary Form (COSF) –36 (61%) states Single assessment statewide –9 (15%) states Publishers’ online assessment systems –6 (10%) states Other approaches –7 (12%) states

8. 8 MH HI GU PW FM AS MP Legend: COSF Publishers’ on-line systems One tool statewide Other State Approaches to Child Outcomes Measurement Section 619 Programs Early Childhood Outcomes Center –August 2010

9. 9 The number of Part C children with outcome data is increasing! Part C Total Number of Children States Included in Progress Data # of children reported # of States FFY07 # of States FFY08 99 or less136 100-4992516 500-999613 1000-1999911 2000+310 Range = 5-6452Range = 11-7998

10. 10 The number of 619 children with outcome data is increasing! 619 Total Number of Children States Included in Progress Data # of children reported # of States FFY07 # of States FFY08 99 or less128 100-499146 500-9991012 1000-199987 2000-2999510 3000+915 Range= 3-10,157Range= 3-9,967

11. Quick Poll 2 Number of children in the data

12. 12 Missing Data ECO additional analysis State efforts to identify missing data State efforts to reduce missing data

13. 13 National Conference Call on Data Quality – Coming Soon What do you know the quality of your state’s outcomes data? Do you know how much missing data you have? How much is reasonable? Missing data is still a major problem for many states. Join us to learn about how much progress has been made and how your state compares to the national numbers.

14. 14

15. 15

16. 16 Part C Progress data trends FFY07 FFY08

17. 17 619 Progress data trends FFY07 FFY08

18. Quick Poll 3 Progress Data Trends

19. 19 Part C - Category ‘e’ by % served

20. 20 FFY2008 Summary Statements Baseline FFY08 Part C619 SS1SS2SS1SS2 Outcome 163% 76%61% Outcome 268%54%76%53% Outcome 369%62%75%65%

21. 21 Part C - Summary Statement 2 by % Served

22. 22 What States are Doing for Improvement Continuing training and TA on data collection system Enhancing data systems Developing data analysis Identifying and addressing data quality issues Identifying areas for program improvement

23. Open Discussion Questions? Comments?

24. 24 Family Outcomes Data Summary of 2010 APR Data

25. 25 State Approaches to Family Outcomes Measurement Part C Program Legend: ECO Family Outcomes Survey State-developed survey NCSEAM survey HI GU AS MP Early Childhood Outcomes Center – August 2010

26. 26 Survey timing and family population Timing Family Population Total All families Families with ≥6 months* Point in time (or time period)251237 Based on child participation89 17 -at exit(3)(6) -at IFSP meeting(s)(3)(2) -IFSP and/or exit(2)(1) Total332154 *One State used ≥9 months, and one State used ≥12 months

27. Quick Poll 4 Survey distribution

28. 28 Response rate variables

29. 29 Response rate by state size

30. 30 Representativeness: Comparison data Thirty-nine states (70%) reported the source of data used: –Part C population/ 618 data: 31 states –Program population data: 3 states –Target population: 3 states –State data (not specified): 2 states Remaining 17 states did not specify

31. Quick Poll 5 Addressing representativeness

32. 32 Criteria used for evaluating representativeness Forty-six states (89%) reported the criteria they used for determining representativeness –Race/ ethnicity: 73% (41 states) –Geography (district, county, region): 50% (28 states) –Sex: 21% (12 states) –Child’s age: 20% (11 states) –Disability/ eligibility category: 9% (5 states) –Length of time in services: 9% (5 states) –Program size : 9% (5 states) Previous years: 2009: 44/56 (78.6%) and 2008: 37/56 (66.1%) reported criteria used Mean number of criteria used this year: 2.7 Previous years: 2009: 2.6 criteria and 2008: 2 criteria.

33. 33 Performance and trends Early intervention has helped… A.Families know their rights: 84% +3% from last year B.Families effectively communicate children’s needs: 85% +2% from last year C.Families help their children develop and learn: 90% +2% from last year

34. 34 Performance by Survey Type

35. 35 Performance by scoring criteria

36. 36 Performance by state size

37. 37 Performance by percent served

38. 38 Were data representative? Forty-four states reported whether their data were representative (79%) –Yes, some data provided: 36% (20 states) –Yes, no data provided: 14% (8 states) –No: 11% (6 states) –Varied results: 18% (10 states) No conclusions re: representativeness reported among the remaining 12 states (21%)

39. 39 State Highlights Analyzing and reporting outcomes among subgroups Improvement activities based on detailed analysis (e.g. by subgroups of families) Data collection improvement activities Partnering w/parent organizations

40. 40 Suggested Formats for February 2011 APR Reporting http://www.fpg.unc.edu/~eco/pages/ fed_req.cfm#ECOSuggestedFormats (this link is also available from the webinar series page)

41. Open Discussion Questions? Comments?

42. 42 Full APR analysis Reports are Available Online Part C: http://www.nectac.org/~pdfs/calls/2010/ partcapr/2010_spp_partc_report.pdfhttp://www.nectac.org/~pdfs/calls/2010/ partcapr/2010_spp_partc_report.pdf Part B: http://www.nectac.org/~pdfs/calls/2010/ partcapr/2010_spp_partc_report.pdfhttp://www.nectac.org/~pdfs/calls/2010/ partcapr/2010_spp_partc_report.pdf (these links are also available from the webinar series page)

43. 43 Thank you for participating. Presentations from this series and their related resources are made available on the NECTAC website at: http://www.nectac.org/~calls/2010/partcapr/partcapr.asp Trends in Child Outcomes (C-3 / B-7) and Family Outcomes (C-4)