Orientation to Outcomes Lynne Kahn Kathy Hebbeler The Early Childhood Outcomes (ECO) Center October 16, 2012 Early Childhood Outcomes Center

Early Childhood Outcomes Center What We Will Cover Background on child and family outcomes Family Indicator C4 Indicators C3 and B7 The three child outcomes Progress categories & summary statements Approaches to measuring child outcomes Emerging trends Conference overview Early Childhood Outcomes Center

Background on Measuring Outcomes Why are state early intervention and preschool special education agencies collecting data on child and family outcomes? Early Childhood Outcomes Center

Early Childhood Outcomes Center Need for Outcomes Data Accountability Federal government (Office of Special Education Programs, U.S. Department of Education) requires that states submit data on outcomes In some states, policy-makers are asking for outcome data Program Improvement State agencies (and local programs) want to use data on outcomes to improve services for children and families Early Childhood Outcomes Center

Early Childhood Outcomes Center Background Part C and Part B, Section 619 are line items programs in IDEA funding OSEP had no data on these programs. In 2003, funded the ECO Center to convene stakeholders to make recommendations on the outcomes and what to report. Early Childhood Outcomes Center

Why do we care about outcomes for families? Pt. C of IDEA is based on expected benefit to families Achieving child outcomes requires healthy families Families can be affected by having a child with a disability- early intervention/preschool can reduce negative impacts and promote positive adaptation Family outcomes may be especially important for families whose children, despite intensive early intervention/preschool, make relatively little progress and suffer serious health or developmental challenges Early Childhood Outcomes Center

Ultimate Goal for EI and ECSE: Families “to enable families to provide care for their child and have the resources they need to participate in their own desired family and community activities. .” Based on the ECO stakeholder process when identifying outcomes Early Childhood Outcomes Center

Considerations in Developing Family Outcomes for Part C and 619 How can a national system to assess family outcomes accommodate the wide range of individualized outcomes? What measurement strategies should be used that are objective yet sensitive to the fact that many outcomes are perceptions? Should a similar set of family outcomes be expected of early intervention and preschool programs? Early Childhood Outcomes Center

Input Process and Recommendations Technical Workgroup on Family Outcomes Developed priority outcome statements based on commonalities across the literature ECO Constituent Workgroups Reacted to and tweaked the resulting list of outcome statements Recommended that the same set of family outcomes be used birth through five Recommended family report of perceptions as the measurement strategy Early Childhood Outcomes Center

Stakeholder Recommendation: Family Outcomes Families understand their child’s strengths, abilities, and special needs Families know their rights and advocate effectively for their child Families help their child develop and learn Families have support systems Families are able to gain access to desired services and activities in their community Note: 1 through 3 were recommended for EI and ECSE. There was not consensus on whether 4 and 5 applied to ECSE. Early Childhood Outcomes Center

Family Outcomes: Additional Resources Family and Child Outcomes for Early Intervention and Early Childhood Special Education. The Early Childhood Outcomes Center (April 2005) Recommended Outcomes for Families of Young Children with Disabilities. Bailey, D. B. Jr., Bruder, M.B., Hebbeler, K., Carta, J., Defosset, M., Greenwood, C., Kahn, L., Mallik, S., Markowitz, J., Spiker, D., Walker, D., & Barton, L. (2006). What Is the Future of Family Outcomes and Family-Centered Services? Bailey, D., Raspa, M., & Fox, L. (2012). Early Childhood Outcomes Center

Early Childhood Outcomes Center Family Indicator: C4 Early Childhood Outcomes Center

APR Requirements for Part C Percent of families participating in Part C who report that EI services have helped the family: Know their rights Effectively communicate their children’s needs Help their children develop and learn Early Childhood Outcomes Center

APR Requirements for Part B, Section 619 Preschool Programs Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities no specific indicator for preschool, nor the expectation for examining preschool family involvement separately from Part B Early Childhood Outcomes Center

State Approaches for Family Indicator: 2012 Original ECO Family Outcomes Survey, Items 16-18 (17 states) Revised ECO Family Outcomes Survey (6 states)* NCSEAM Impact on Family Scale (25 states) State Survey (7 states) *One additional state used both ECO Surveys Early Childhood Outcomes Center

Family Survey: Additional Resources ECO Family Surveys http://projects.fpg.unc.edu/~eco/pages/tools.cfm#SurveyVersions NCSEAM* Survey http://www.accountabilitydata.org/FamilyInvolvmentNCSEAMMeasures.htm *National Center for Special Education Accountability Monitoring Early Childhood Outcomes Center

Early Childhood Outcomes Center

Challenges Related to Family Surveys Need to increase the response rates Determining and increasing the representativeness of the data Interpreting the data to improve outcomes for families Early Childhood Outcomes Center

Early Childhood Outcomes Center Child Outcomes: C3 and B7 Early Childhood Outcomes Center

Themes from the Stakeholders: Child Outcomes Consistent with IDEA and legislative intent Reflect what EI and ECSE are trying to do Develop one set of outcomes for all disabilities Develop one set of outcomes for birth to five Make the outcomes functional Have the potential to positively influence practice; do not drive the field backwards Early Childhood Outcomes Center

Ultimate Goal for EI and ECSE “To enable young children to be active and successful participants during the early childhood years and in the future in a variety of settings – in their homes with their families, in child care, preschool or school programs, and in the community.” Based on the ECO stakeholder process when identifying 3 functional outcomes Outcomes aren’t just about accountability… This is where we want our children (EI and ECSE children) to accomplish and so we want to be accountable for helping children/families achieve this goal… Early Childhood Outcomes Center

Early Childhood Outcomes Center Understanding the Three Child Outcomes Early Childhood Outcomes Center

Early Childhood Outcomes Center Three Child Outcomes Children have positive social-emotional skills (including social relationships) Children acquire and use knowledge and skills (including early language/ communication [and early literacy]) Children use appropriate behaviors to meet their needs Early Childhood Outcomes Center

Outcomes Are Functional Functional outcomes: Refer to using skills to accomplish things that are meaningful to the child in the context of everyday life Refer to an integrated series of behaviors or skills that allow the child to achieve the important everyday goals Early Childhood Outcomes Center

Children Have Positive Social Relationships Involves: Relating with adults Relating with other children For older children, following rules related to groups or interacting with others Includes areas like: Attachment/separation/autonomy Expressing emotions and feelings Learning social rules and expectations Social interactions and play Early Childhood Outcomes Center

Children Acquire and Use Knowledge and Skills Involves: Thinking Reasoning Remembering Problem solving Using symbols and language Understanding physical and social worlds Includes: Early concepts—symbols, pictures, numbers, classification, spatial relationships Imitation Object permanence Expressive and receptive language and communication Early literacy Early Childhood Outcomes Center

Children Take Appropriate Action to Meet Their Needs Involves: Taking care of basic needs Getting from place to place Using tools (e.g., fork, toothbrush, crayon) In older children, contributing to their own health and safety Includes: Integrating motor skills to complete tasks Self-help skills (e.g., dressing, feeding, grooming, toileting, household responsibility) Acting on the world to get what one wants Early Childhood Outcomes Center

Taking Action to Meet Needs Includes Integrating various skills (gross motor, fine motor, communication skills) to complete tasks Self help skills (feeding, dressing, toileting, household task) Acting on the world to get what he or she wants Not JUST acting on the world: takes APPROPRIATE action to meet needs Early Childhood Outcomes Center

Outcomes Reflect Global Functioning Each outcome is a snapshot of: The whole child Status of the child’s current functioning Functioning across settings and situations Rather than: Skill by skill In one standardized way Split by domains Early Childhood Outcomes Center

Child Outcomes: Additional Resources Family and Child Outcomes for Early Intervention and Early Childhood Special Education. The Early Childhood Outcomes Center (April 2005) Child Outcomes Video: Step by Step Early Childhood Outcomes Center

OSEP Reporting Categories Percentage of children who: a. Did not improve functioning b. Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged peers but did not reach it d. Improved functioning to reach a level comparable to same-aged peers e. Maintained functioning at a level comparable to same-aged peers 3 outcomes x 5 “measures” = 15 numbers Early Childhood Outcomes Center

Who is included in the reporting In February 2013, states will report progress on all children who Exited the program between July 1, 2011 and June 30, 2012, and Were in the program for at least 6 months. Early Childhood Outcomes Center

Helping Children Move Toward Age-Expected Functioning Assumption: Children can be described with regard to how close they are to age expected functioning for each of the 3 outcomes By definition, most children in the general population demonstrate the outcome in an age-expected way By providing services and supports, EI and ECSE are trying to move children closer to age expected behavior Early Childhood Outcomes Center

Functioning

Entry

Entry Exit Early Childhood Outcomes Center

Entry Exit Early Childhood Outcomes Center

Early Childhood Outcomes Center Key Points The OSEP reporting categories describe types of progress children can make between entry and exit Data from two time points (entry and exit) are needed to calculate what OSEP category describes a child’s progress Early Childhood Outcomes Center

Understanding the reporting categories a - e e. % of children who maintain functioning at a level comparable to same-aged peers Early Childhood Outcomes Center

Entry Exit Both 6 and 7 indicate age appropriate functioning. Early Childhood Outcomes Center

Entry Exit Entry Exit Early Childhood Outcomes Center

d. % of children who improve functioning to reach a level comparable to same-aged peers Early Childhood Outcomes Center

Entry Exit Entry Exit Early Childhood Outcomes Center

An exit rating of 6 or 7 indicates the child is now showing age expected functioning. Entry Exit Early Childhood Outcomes Center

c. % of children who improved functioning to a level nearer to same aged peers, but did not reach it Early Childhood Outcomes Center

Child showed sufficient improvement in functioning to move to a higher developmental trajectory (i.e., higher point on scale) but had not yet achieved age appropriate functioning. Entry Exit Early Childhood Outcomes Center

Entry Exit Early Childhood Outcomes Center

b. % of children who improved functioning, but not sufficient to move nearer to same aged peers Early Childhood Outcomes Center

Note the slope - the line is not flat; the child is showing improved functioning Entry Exit Early Childhood Outcomes Center

Note the slope - the line is not flat; the child is showing improved functioning Entry Exit

Entry Exit Early Childhood Outcomes Center

a. % of children who did not improve functioning Early Childhood Outcomes Center

Entry Exit Early Childhood Outcomes Center

Entry Exit Early Childhood Outcomes Center

The Summary Statements Of those children who entered the program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turned 3 [6] years of age or exited the program. c + d/ (a+b+c+d) The statements on which states set targets this year. Early Childhood Outcomes Center

The Summary Statements 2. The percent of children who were functioning within age expectations in each Outcome by the time they turned 3 [6] years of age or exited the program. d + e/(a+b+c+d+e) Targets set for summary statements this year. Early Childhood Outcomes Center

The concepts are easier than the words or the formulas Summary statement 1: How many children changed growth trajectories during their time in the program? Summary statement 2: How many children were functioning like same aged peers when they left the program? Other ways to think of SS1 – how many made more than expected growth. Sometimes called ‘significant.’ SS2 – for preschoolers – ‘ready for kindergarten.’ At age level. Early Childhood Outcomes Center

State Approaches to Measuring Child Outcomes – 2010-11 Part C (N=56) Preschool (N=59) COS* 7 pt. scale 43/56 (77%) 36/59 (61%) One tool statewide 7/56 (13%) 9/59 (15%) Publishers’ online analysis 3/56 (5%) 6/59 (10%) Other 5/56 (9%) 8/59 (14%) Early Childhood Outcomes Center

Early Childhood Outcomes Center The National Data Child Outcomes Two-page summary Some recent presentations APR analysis Part C Preschool Family Indicator for Part C Early Childhood Outcomes Center

ECO Child and Family Measurement Frameworks and Self-Assessment Identify components of a high-quality measurement system for child and family outcomes Assist states in making informed decisions as to which option is most effective and best suited to the state's context. Child and Family Outcomes Measurement System Tools Early Childhood Outcomes Center

Integrating Outcomes into IFSP/IEP Process Using the child and family outcomes as a framework to guide practice as well as outcomes measurement Framework for assessment, planning, service delivery Resources on integrating outcomes Learning community Contact kathi.gillaspy@unc.edu Early Childhood Outcomes Center

Themes of Agenda Sessions Measurement System Framework Areas Purpose: These sessions address aspects of articulating and communicating the purpose of your state’s measurement system. Data Collection and Transmission: Sessions address data collection procedures (e.g. entering, transmitting, and storing data) and professional development related to data collection. Analysis: Sessions may focus on processes of data analysis, accountability, and identification of program improvement questions. Reporting: Topics address how state and local programs interpret, report, and communicate information related to child and family outcomes. Using Data: These sessions address how state and local programs can use data to improve programs. Evaluation: These sessions focus on evaluation of measurement systems. Cross-system Coordination: These sessions address longitudinal data, coordination across Part C and Section 619 and early childhood programs. Program Improvement Improving Practices: These sessions specifically highlight practices aimed at improving outcomes. Early Childhood Outcomes Center

Contact ECO for assistance (please) How we can help General: staff@the-eco-center.org Specific staff person: http://projects.fpg.unc.edu/~eco/pages/people.cfm for email and phone numbers for ECO staff Early Childhood Outcomes Center

Early Childhood Outcomes Center Questions or comments? www.the-eco-center.org Early Childhood Outcomes Center