1. Early Childhood Outcomes: Using Data for Program Improvement
Kathy Hebbeler
ECO at SRI International
Robin Rooney and Christina Kasprzak
ECO at Frank Porter Graham Institute
Illinois
October, 2008

2. Objectives
Understand the purposes of the child outcomes data collection
Be familiar with key considerations related to accurately completing the Child Outcomes Summary Form
Be able to evaluate child outcomes data for accuracy as part of supervision
Understand the use of child outcomes data for program improvement, including for instruction, resource allocation, and professional development
Early Childhood Outcomes Center

3. Why are we doing this?
Early Childhood Outcomes Center

4. Keeping our eye on the prize:
High quality services for children and families that will lead to good outcomes.
Early Childhood Outcomes Center

5. Goal of early childhood special education
“…To enable young children to be active and successful participants during the early childhood years and in the future in a variety of settings – in their homes with their families, in child care, in preschool or school programs, and in the community.”
(from Early Childhood Outcomes Center,
Early Childhood Outcomes Center

6. High Quality Data on Outcomes
Data are a piece of a system that helps to achieve overarching goals for children and families
Data yield
Evidence that allows you to make an inference that should lead to specific actions to improve the system.
Early Childhood Outcomes Center

7. System for Producing Good Child and Family Outcomes
Adequate funding
Good outcomes for children and families
Good Federal policies and programs
High quality services and supports for children 0-5 and their families
Good State policies and programs
Good Local policies and programs
Strong Leadership
Prof’l Development
Preservice
Inservice

8. The Vision: Using Data as a Tool for Program Improvement
Illinois will have quality data available on an ongoing basis about multiple components of the system
Outcomes for children and families
Programs and services provided
Personnel (types, qualifications, etc.)
Etc.
Early Childhood Outcomes Center

9. Driving Force for Data on Child Goals Comes from the Federal Level
Government Performance and Results Act (GPRA)
Program Assessment Rating Tool (PART)
Individuals with Disabilities Education Act (IDEA)
Early Childhood Outcomes Center

10. Government Performance and Results Act (GPRA) passed in 1993
Requires goals and indicators be established for IDEA
Indicators and data collection further along for school age population than for EC
Previously, for early childhood data had been collected on:
Number of children served
Settings
Notes from OSEP’s slides: Lou
GPRA:
Passed in 1993
Applies to all federal agencies
Requires that a set of indicators be established in order to judge the effectiveness of programs meeting their goals
OSEP has established GPRA indicators for all IDEA programs, including Parts B, C, and D that are reported to Congress annually.
For the Part C indicators, OSEP has collected data on the total number of children served and the number of children served under the age of one, and on settings where services are provided to eligible children
For 619, OSEP has collected data on settings where children receive special education and related services
Thus far OSEP has not yet collected and reported data on the functional abilities of children and family outcomes
Early Childhood Outcomes Center

11. PART evaluation results (2002)
130 programs examined in 2002; 50% programs had no performance data
Programs looking at inputs, not results
Part C and Section 619
No long-term child outcome goals or data
Need to develop a strategy to collect annual performance data in a timely manner
During the 2003 budget period, the Administration rated approximately 130 federal programs on their effectiveness using part
50% of the these programs had no performance data
Programs tend to use inputs or process data, rather than results or outcome data
Specifically for Part C and Section 619: did not do well in the area of accountability/results because there were no long-term child outcome goals; no child outcome data. The recommendation from the PART is that OSEP needs to develop a strategy to collect annual performance data (child and family outcomes) in a timely manner
Early Childhood Outcomes Center

14. Federal Funding for Preschool Special Education
Total U.S.
Illinois
2004
$387,699,000
$17,943,956
2005
$381,385,691
$17,812,936
2006
$380,751,030
$17,650,453
Early Childhood Outcomes Center

15. Early Childhood Outcomes Center

16. Individuals with Disabilities Education Act
SEC <<NOTE: 20 USC 1416.>> MONITORING, TECHNICAL ASSISTANCE, AND ENFORCEMENT. ``(a) Federal and State Monitoring.-…..
…..
``(2) Focused monitoring.--The primary focus of Federal and State monitoring activities described in paragraph (1) shall be on-- ``(A) improving educational results and functional outcomes for all children with disabilities;
Early Childhood Outcomes Center

17. How Office of Special Education (OSEP) responded
Required states to submit outcome data in their Annual Performance Report (APR)
Funded the Early Childhood Outcomes (ECO) Center to do research, make recommendations, and assist states
Early Childhood Outcomes Center

18. Federal reporting requirements
Where are we now:
Federal reporting requirements
Early Childhood Outcomes Center

19. OSEP Reporting Requirements: the Outcomes
Positive social emotional skills (including positive social relationships)
Acquisition and use of knowledge and skills (including early language/ communication [and early literacy])
Use of appropriate behaviors to meet their needs
Early Childhood Outcomes Center

20. OSEP Reporting Categories
Percentage of children who:
a. Did not improve functioning
b. Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers
c. Improved functioning to a level nearer to same-aged peers but did not reach it
d. Improved functioning to reach a level comparable to same-aged peers
e. Maintained functioning at a level comparable to same-aged peers
3 outcomes x 5 “measures” = 15 numbers
Early Childhood Outcomes Center

21. Reporting Schedule
Due February 2008
Data in reporting categories at exit for all children who have been in the program for at least 6 months
Must be reported for the year beginning July 1, 2006
Repeat with next year’s data in 2009, etc.
2010
States must set targets
Summary statements for targets recommended; not yet official*
States must report data to public by school districts for these target numbers
*See for more information on recommended targets
Early Childhood Outcomes Center

22. State decisions and activities
Where are we now:
State decisions and activities
We are focusing the remaining discussion on states activities – because this work is unfolding in states.
Early Childhood Outcomes Center

23. WHY? Purpose (State Version) To have data for program improvement and
to respond to federal reporting requirements
To respond to federal reporting requirements
Early Childhood Outcomes Center

24. WHY? Purpose (Local Version) To have data for program improvement and
to provide data to the state
To provide data to the state
Early Childhood Outcomes Center

25. State approaches
Most states have embraced outcomes measurement and are collecting outcomes data for their own purposes.
Many states are building bigger systems than needed to produce the federal data.
Go to for more information about what other states are doing
Early Childhood Outcomes Center

26. How are states collecting child outcomes data?
Possible state approaches to collection of child data
Child Outcomes Summary Form (COSF)
Publisher’s online assessment system
Single assessment statewide
Other approaches
Early Childhood Outcomes Center

27. State Approaches to Measuring Child Outcomes
Part C
(56 states)
Preschool
(59 states)
One tool statewide
8/56 (14%)
13/59 (22%)
Multiple Publishers’ online tools
2/56 (4%)
3/59 (5%)
COSF 7 pt. scale
40/56 (71%)
36/59 (61%)
Other
6/56 (11%)
7/59 (12%)

28. Checking in – How is it going?
Implementing the Child Outcomes Summary Form Process
What is going well?
What is not going well?
Are there other issues or concerns that need to be addressed about the process?
Early Childhood Outcomes Center

29. Child Outcome Summary Form
Let’s Review:
Child Outcome Summary Form
Early Childhood Outcomes Center

30. Learning from Each Other
Did you attend a training on the COSF?
Have you participated in a COSF team rating process?
For more than 10 children? 20? 30?
Have you reviewed COSF forms completed by others?
Have you provided training on the COSF?
How confident are you in your knowledge of the COSF process?
Early Childhood Outcomes Center

31. Essential Knowledge for Completing the Child Outcomes Summary Form
Between them, team members must:
Know about the child’s functioning across settings and situations
Understand age-expected child development
Understand the content of the three child outcomes
Know how to use the rating scale
Understand age expectations for child functioning within the child’s culture
Early Childhood Outcomes Center

32. Important point
It is not necessary that all team members be knowledgeable in all 5 areas
Especially, no expectation that parents understand the rating scale or typical child development
But the professionals have to!
Early Childhood Outcomes Center

33. Essential Knowledge for Completing the Child Outcomes Summary Form
Between them, team members must:
Know about the child’s functioning across settings and situations
Understand age-expected child development
Understand the content of the three child outcomes
Know how to use the rating scale
Understand age expectations for child functioning within the child’s culture
Early Childhood Outcomes Center

34. 1. Know about the child’s functioning across settings and situations
How we learn about the child’s functioning across settings and situations:
Good
assessment
Early Childhood Outcomes Center

35. DEC* recommended practices for assessment
Involve multiple sources
Examples: family members, professional team members, service providers, caregivers
Involve multiple measures
Examples: observations, criterion- or curriculum-based instruments, interviews, norm-referenced scales, informed clinical opinion, work samples
*Division for Early Childhood
Early Childhood Outcomes Center

36. Assessment practices appropriate for outcomes measurement: ASHA*
ASHA recommended practices:
Gather information from families, teachers, other service providers
Collect child-centered, contextualized, descriptive, functional information
(*American Speech-Language-Hearing Association)
A balanced assessment, according to ASHA, includes information from families, teachers, other service providers, and information that is child-centered, contextualized, descriptive and functional.
Early Childhood Outcomes Center

37. Assessment instruments
Assessment the tool vs. assessment the process
Assessment tools can inform us about children’s functioning in each of the three outcome areas
Challenge:
There is no assessment tool that assesses the three outcomes directly
Early Childhood Outcomes Center

38. Essential Knowledge for Completing the Child Outcomes Summary Form
Between them, team members must:
Know about the child’s functioning across settings and situations
Understand age-expected child development
Understand the content of the three child outcomes
Know how to use the rating scale
Understand age expectations for child functioning within the child’s culture
Early Childhood Outcomes Center

39. Resources for understanding age-expected child development
ECO link
(under “ECO Tools”)
New course coming soon
Watch ECO web site
Early Childhood Outcomes Center

40. Essential Knowledge for Completing the Child Outcomes Summary Form
Between them, team members must:
Know about the child’s functioning across settings and situations
Understand age-expected child development
Understand the content of the three child outcomes
Know how to use the rating scale
Understand age expectations for child functioning within the child’s culture
Early Childhood Outcomes Center

41. Outcomes Jeopardy Biting $100 $100 $100 $200 $200 $200 $300 $300 $300
Pointing to the cabinet for cereal
Reading the letter “S” on the Stop sign
Washes hands before lunch
Biting
Plays by himself in the classroom
Plays with rhyming words
Building a castle from blocks with a friend
Problems sleeping
Sharing a cookie at lunchtime
$100
$100
$100
$200
$200
$200
$300
$300
$300
Early Childhood Outcomes Center

42. Children have positive social relationships
Involves:
Relating with adults
Relating with other children
For older children, following rules related to groups or interacting with others
Includes areas like:
Attachment/separation/autonomy
Expressing emotions and feelings
Learning rules and expectations
Social interactions and play
Early Childhood Outcomes Center

43. Children acquire and use knowledge and skills
Involves
Thinking
Reasoning
Remembering
Problem solving
Using symbols and language
Understanding physical and social worlds
Includes:
Early concepts—symbols, pictures, numbers
Imitation
Object permanence
Expressive language and communication
Early literacy
Early Childhood Outcomes Center

44. Children take appropriate action to meet their needs
Involves:
Taking care of basic needs
Getting from place to place
Using tools (e.g., fork, toothbrush, crayon)
In older children, contributing to their own health and safety
Includes:
Integrating motor skills to complete tasks
Self-help skills (e.g., dressing, feeding, grooming, toileting, household responsibility)
Acting on the world to get what one wants
Early Childhood Outcomes Center

45. Essential Knowledge for Completing the Child Outcomes Summary Form
Between them, team members must:
Know about the child’s functioning across settings and situations
Understand age-expected child development
Understand the content of the three child outcomes
Know how to use the rating scale
Understand age expectations for child functioning within the child’s culture
Early Childhood Outcomes Center

46. The two COSF questions
a. To what extent does this child show age-appropriate functioning, across a variety of settings and situations, on this outcome? (Rating: 1-7)
b. Has the child shown any new skills or behaviors related to [this outcome] since the last outcomes summary? (Yes-No)
Early Childhood Outcomes Center

47. 7 – Completely
The child shows behaviors and skills expected in all or almost all everyday situations that are part of the child’s life
Home, store, park, child care, with strangers, etc.
The child’s functioning is considered appropriate for his/her age
No one has significant concerns about the child’s functioning in this outcome area
Early Childhood Outcomes Center

48. 6 – Between completely and somewhat
The child’s functioning generally is considered appropriate for his or her age
but there are some significant concerns about the child’s functioning in this outcome area
Early Childhood Outcomes Center

49. 5 – Somewhat
The child shows functioning expected for his/her age some of the time and/or in some situations
The child’s functioning is a mix of age-appropriate and not appropriate functioning
The child’s functioning might be described as like that of a slightly younger child
We use the terminology that behaviors MIGHT BE more like those of younger children intentionally. For example, a child with autism does not display typical behaviors, but the behaviors observed would not be described as those of a younger child either.
Early Childhood Outcomes Center

50. 4 – Between a 5 and a 3
Child shows some age appropriate functioning some of the time or in some situations or settings but most of the child’s functioning would be described as not yet age appropriate.
The child’s functioning might be described as like that of a younger child
Early Childhood Outcomes Center

51. 3 – Emerging
The child does not yet show functioning expected of a child his/her age in any situation
The child’s behaviors and skills include immediate foundational skills on which to build age-appropriate functioning
The child’s functioning might be described as like that of a younger child
Early Childhood Outcomes Center

52. 2 – Between 3 and 1
The child does not yet show functioning expected of a child his/her age in any situation
The child’s behaviors and skills does have some the immediate foundational skills on which to build age-appropriate functioning but these are not displayed very often
The child’s functioning might be described as like that of a younger or even much younger child
Early Childhood Outcomes Center

53. 1 – Not yet
The child does not yet show functioning expected of a child his/her age in any situation
The child’s skills and behaviors also do not yet include any immediate foundational skills on which to build age-appropriate functioning
The child’s functioning might be described as like that of a much younger child
Children with 1 ratings still have skills, just not yet at an immediate foundational level
Early Childhood Outcomes Center

54. Rating Scale Jeopardy $100 $100 $100 $200 $200 $200 $300 $300 $300
Age appropriate functioning – no concerns
Mix of age appropriate and not age appropriate functioning
No age appropriate functioning – not yet showing immediate foundational skills
Some age appropriate functioning but very little
No age appropriate functioning – lots of immediate foundational skills
Age appropriate functioning – some concerns
Rarely shows age appropriate functioning
No age appropriate functioning – some immediate foundational skills
Age appropriate functioning
$100
$100
$100
$200
$200
$200
$300
$300
$300
Early Childhood Outcomes Center

55. Early Childhood Outcomes Center

56. Essential Knowledge for Completing the Child Goals Outcome Form
Between them, team members must:
Know about the child’s functioning across settings and situations
Understand age-expected child development
Understand the content of the three child outcomes
Know how to use the rating scale
Understand age expectations for child functioning within the child’s culture
Early Childhood Outcomes Center

57. Point of clarification
Process is NOT about comparing groups of children – it IS about asking how close children are to being able to do what is expected at their age
Early learning guidelines
Kindergarten and access to the general curriculum
Early Childhood Outcomes Center

58. What we are learning nationally
The process of training for child outcomes data collection has uncovered other areas of significant need related to professional development.
Early Childhood Outcomes Center

59. Providers need to know more about:
Assessment
How to gather assessment data to reflect functioning across settings and situations, especially how to gather child functioning information from families
Understanding the results of the assessment
Sharing assessment results sensitively and honestly with families
Typical child development is especially important for the COSF process but is important overall for working with young children.
Early Childhood Outcomes Center

60. Providers need to know more about:
Functional outcomes
What are they?
How do they differ from outcomes organized around domains?
What do they mean for how professionals from different disciplines operate as a team?
Typical child development
What are the functional expectations for children at different ages with regard to each of the 3 goal statements?
Typical child development is especially important for the COSF process but is important overall for working with young children.
Early Childhood Outcomes Center

61. Let’s meet Ava
Early Childhood Outcomes Center