1. Early Childhood Outcomes Center Orientation to Measuring Child and Family Outcomes for New People Kathy Hebbeler, ECO at SRI Lynne Kahn, ECO at FPG/UNC Early Childhood Outcomes Meeting Baltimore, Maryland August 2007

2. Early Childhood Outcomes Center 2 We will cover Why the federal government is interested in outcomes How these outcomes were selected Review the SPP/APR indicators and what they mean Questions encouraged throughout!!

3. Early Childhood Outcomes Center 3 How Looking at Child Indicators for EI/ECSE Became Important

4. Early Childhood Outcomes Center 4 Focus on Accountability Followed an era of focusing on and measuring actions/activities Call for interest in the “ultimate result”: child and family outcomes Cut across both the public and private sectors Funders want data to determine whether a program is doing what it is supposed to do

5. Early Childhood Outcomes Center 5 Critical Events in EI/ECSE Accountability 1992 –Osborne and Graebler, Reinventing Government 1993 – GPRA (Government Performance and Results Act) passed Intervening years…. 2002 – PART (Program Assessment Rating Tool) finds there are no data on outcomes for Part C or Part B Preschool 2003 – OSEP begins to ask states for child outcome data (and funds the ECO Center!) 2005, 2006 – OSEP releases revisions to the reporting requirements

6. Early Childhood Outcomes Center 6 PART Review for Part C and Part B Preschool Results Not Demonstrated Part C “While the program has met its goal relating to the number of children served, it has not collected information on how well the program is doing to improve the educational and developmental outcomes of infants and toddlers served.” Part B Preschool “The Department has no performance information on preschool children with disabilities served by this program.” Read more at Expectmore.gov

7. Early Childhood Outcomes Center 7 What was happening in the intervening years? Special Education – National study found poor outcomes for special education adolescents Results Push to include students with disabilities in statewide assessment systems Early Childhood Debate about whether child outcomes should be measured at all Discussion of the many problems in trying to measure outcomes for young children with disabilities

8. Early Childhood Outcomes Center 8 Measuring Child Indicators for EI/ECSE The PART findings put an end to the debate about whether or not to do it Unfortunately, almost no progress had been made in the intervening years as to HOW to do it

9. Early Childhood Outcomes Center 9 Why These Outcomes?

10. Early Childhood Outcomes Center 10 Early Childhood Outcomes Center Funded in October 2003 Promote the development and implementation of child and family outcome measures for infants, toddlers and preschoolers with disabilities that can be used in national and state accountability systems

11. Early Childhood Outcomes Center 11 Origin of the Child Outcomes ECO Center convened stakeholder meetings followed by public comment period (2004) First, collected themes and ideas Then, drafted and re-drafted wording based on input from stakeholders Made recommendation to OSEP (2005)

12. Early Childhood Outcomes Center 12 Themes from the Stakeholders: Child Outcomes Consistent with IDEA and legislative intent Reflect what EI and ECSE are trying to do One set for birth to 5 Reflect what is known about development and learning

13. Early Childhood Outcomes Center 13 Themes from the Stakeholders Be compatible with best practice (esp. transdisciplinary service models, functional behaviors) Do not base them on domains Have potential to influence practice in a positive way Incorporate universal design Be readily understood

14. Early Childhood Outcomes Center 14 Make Outcomes/Indicators Functional Functional refers to things that are meaningful to the child in the context of everyday living Refers to an integrated series of behaviors or skills that allow the child to achieve the outcomes Not discrete skills; not domain-based

15. Early Childhood Outcomes Center 15 Recommendations Decision: One set of indicators birth to 5 Decision: Functional outcomes Decision: Global, not specific

16. Early Childhood Outcomes Center 16 The need for an introductory statement Family and child outcomes are linked There are overarching goals for children and family that cut across the outcomes Helpful to frame the outcomes with these overarching goals

17. Early Childhood Outcomes Center 17 Concepts in the introduction: Goal for children The ultimate goal is for young children to be active and successful participants now and in the future in a variety of settings– in their homes, in their child care, preschool or school programs, and in the community http://www.fpg.unc.edu/~eco/pdfs/eco_outcomes_4-13-05.pdf

18. Early Childhood Outcomes Center 18 Concepts in the introduction: Goal for children Active and successful participants Now and in the future In variety of settings

19. Early Childhood Outcomes Center 19 Concepts in the introduction: Goal for families Enable families to provide appropriate care for their child Have resources they need to participate in community activities

20. Early Childhood Outcomes Center 20 Concepts in the preface: Outcomes and accountability Outcomes determined by variety of factors Not all families and children will achieve all outcomes BUT, system should still strive to achieve them

21. Early Childhood Outcomes Center 21 ECO Child Outcomes Children have positive social relationships Children acquire and use knowledge and skills Children take appropriate action to meet their needs

22. Early Childhood Outcomes Center 22 ECO Family Outcomes Understand their child’s strengths, abilities, and special needs Know their rights and advocate effectively for their children Help their children develop and learn Have support systems Access desired services, programs, activities in their community

23. Early Childhood Outcomes Center 23 What kind of data should be reported on child outcomes? Had extensive discussions with stakeholders about status vs. progress data Status – where child is Progress – how child has changed Progress data alone is weak evidence How much progress should be considered “good progress” or “enough progress?”

24. Early Childhood Outcomes Center 24 ECO Recommendations to OSEP ECO submitted recommendations to OSEP in May 2005 on the kind of percentages that should be requested from states. http://www.fpg.unc.edu/~eco/pdfs/ECO_respon se_to_OSEP_5-9-05.pdf

25. Early Childhood Outcomes Center 25 The APR/SPP Requirements Related to Outcomes

26. Early Childhood Outcomes Center 26 OSEP Reporting Requirements: Child Outcomes Positive social emotional skills (including positive social relationships) Acquisition and use of knowledge and skills (including early language/ communication [and early literacy]) Use of appropriate behaviors to meet their needs Part C #3; Part B #4

27. Early Childhood Outcomes Center 27 OSEP Reporting Categories Percentage of children who: a.Did not improve functioning b.Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c.Improved functioning to a level nearer to same-aged peers but did not reach it d.Improved functioning to reach a level comparable to same-aged peers e.Maintained functioning at a level comparable to same-aged peers 3 outcomes x 5 “measures” = 15 numbers

28. Early Childhood Outcomes Center 28 Additional information about child indicators Child must be in program at least 6 months Data must be collected at (near) entry and (near) exit Entry data for year July 1, 2005 to June 30, 2006 to be reported in February 2007 Progress data to be reported in February 2008 and therafter

29. Early Childhood Outcomes Center 29 Implications for Measurement Compares entry to exit data for each child- 2 measurement points are needed for each child for the data analysis. Individual children’s entry data will need to be maintained and matched to exit data Information about children’s functioning in each area needs to be compared to age level expectations

30. Early Childhood Outcomes Center 30 Implications for Measurement (continued) Outcomes are broad Functional outcomes summarize each child’s current functioning across settings and situations Not asking for domains Best practice for assessing young children recommends the use of multiple measures

33. Early Childhood Outcomes Center 33 Comparing to Same Aged Peers? Deficit model? Not individualized? Progress alone is weak evidence Goal of EI/ECSE: Active and successful participation now and in the future Kindergarten, school readiness, having friends, community participation Setting high expectations

34. Early Childhood Outcomes Center 34 Comparison of OSEP requirements to ECO recommendations: Family outcomes (Part C only) Understand their child’s strengths, abilities, and special needs Know their rights and advocate effectively for their children Help their children develop and learn Have support systems Access desired services, programs, activities in their community Percent of families participating in Part C who report that EI services have helped the family Know their rights Effectively communicate their children’s needs Help their children develop and learn ECOOSEP (#4)

35. Early Childhood Outcomes Center 35 OSEP APR Requirements for Families for Part B Preschool (#8) Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities ECO recommends that states collect additional information about families of children 3-5

36. Early Childhood Outcomes Center 36 Additional information about family indicators Timing is not specified (e.g., every May, at exit, etc.) States have options with regard to how they collect the information Survey, interview Which tool

37. Early Childhood Outcomes Center 37 Family Outcome Issue OSEP reporting requirements do not call for data on family outcomes for C or B

38. Early Childhood Outcomes Center 38 Family Outcomes Issues All states are doing some kind of data collection on family issues OSEP requirements + satisfaction +/or parent involvement +/or outcomes, etc. Some states are opting to collect data on family outcomes

39. Early Childhood Outcomes Center 39 Tools for Indicator Measurement Child Outcomes Summary Form (COSF) Family Outcomes Survey (FOS) NCSEAM Survey

40. Early Childhood Outcomes Center 40 Finding out more Additional information about OSEP requirement and state activities ECO and other resources including training materials for the COSF, Family Outcomes Survey, resources on assessment and typical child development, and materials for parentswww.the-eco-center.org