1. OSEP Initiatives on Early Childhood Outcomes
Kathy Hebbeler
Early Childhood Outcomes Center
SRI International
Presented at
ECEA-SCASS Meeting
Savannah, Georgia
October, 2010 1

2. Early Childhood Outcomes Center
Topics
What state are required to report
State approaches
Most recent data
Child Outcomes Measurement Framework
Early Childhood Outcomes Center

3. Early Childhood Outcomes Center
Reporting Requirement for Early Intervention and Preschool Special Education
Early Childhood Outcomes Center

4. Why does the federal government want data on child outcomes?
Government Performance and Results Act (GPRA)
Program Assessment Rating Tool (PART)
Individuals with Disabilities Education Act (IDEA)
Early Childhood Outcomes Center

5. Government Performance and Results Act (GPRA) passed in 1993
Requires goals and indicators be established for IDEA
Indicators and data collection for school age population included data on outcomes
Previously, for early childhood data had been collected on:
Number of children served (Part C)
Settings (both Part C and 619

6. OSEP: PART evaluation results (2002)
130 programs examined in 2002; 50% programs had no performance data
Programs looking at inputs, not results
Part C and Section 619
No long-term child outcome goals or data
Department of Education needs to develop a strategy to collect annual performance data in a timely manner
Early Childhood Outcomes Center

7. IDEA 2004
SEC <<NOTE: 20 USC 1416.>> MONITORING, TECHNICAL ASSISTANCE, AND ENFORCEMENT. ``(a) Federal and State Monitoring.…..
(2) Focused monitoring.--The primary focus of Federal and State monitoring activities described in paragraph (1) shall be on-- (A) improving educational results and functional outcomes for all children with disabilities;

8. OSEP Reporting Requirements: Child Outcomes
Positive social emotional skills (including positive social relationships)
Acquisition and use of knowledge and skills (including early language/ communication and early literacy)
Use of appropriate behaviors to meet their needs

9. What States Report: OSEP Reporting Categories
Percentage of children who:
a. Did not improve functioning
b. Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers
c. Improved functioning to a level nearer to same-aged peers but did not reach it
d. Improved functioning to reach a level comparable to same-aged peers
e. Maintained functioning at a level comparable to same-aged peers
3 outcomes x 5 “measures” = 15 numbers

10. Early Childhood Outcomes Center

11. Early Childhood Outcomes Center
Reporting details
Progress for all children who exited between July 1, 2009 and June 30, 2010
Stayed in the program at least 6 months
Data will be reported to OSEP in February 2011
Data reported for the first time for children who exited in year.
Early Childhood Outcomes Center

12. The Summary Statements
Of those children who entered the program below age expectations in each outcome, the percent who substantially increased their rate of growth by the time they turned 3 [6] years of age or exited the program.
The percent of children who were functioning within age expectations in each outcome by the time they turned 3 [6] years of age or exited the program.

13. Formula for SS 1
(c+d)/(a+b+c+d) 13

14. Formula for SS 2
(d+e)/(a+b+c+d+e)

15. Early Childhood Outcomes Center
State approaches
Early Childhood Outcomes Center

16. Early Childhood Outcomes Center

17. Early Childhood Outcomes Center

18. State Approaches to Measuring Child Outcomes
Part C
(56 states/jur)
Preschool
(59 states/jur)
One tool statewide
7/56 (13%)
9/59 (15%)
Publishers’ online analysis
3/56 (5%)
6/59 (10%)
COSF 7 pt. scale
41/56 (73%)
38/59 (64%)
Other
5/56 (9%)
7/59 (10%)
Percents changed this year for several reasons: (1) we recoded the online analysis with one tool as online analysis due to more in common, (2) at least one more state provided data so the N is different, and (3) a couple states changed categories
Changes from preliminary data due to ‘week of clarification’ documents and also taking out ‘at risk’ from those states, as appropriate.

19. Child Outcomes Rating Form (COSF)
7-point rating scale with defined criteria for each point
Criteria describe child’s functioning relative to same aged peers
Child’s team uses multiple sources of information to assign rating
Rating assigned at program entry and program exit
Early Childhood Outcomes Center

20. Early Childhood Outcomes Center
The State Data for ‘08-’09
Early Childhood Outcomes Center

21. N=66,000

24. N=113,700

27. Early Childhood Outcomes Center
A Framework and Self Assessment for Building a Child Outcomes Measurement System
Early Childhood Outcomes Center

28. Purpose of the Framework
Provide a common language for ECO and other TA providers to use in discussing COMSs with states.
Provide a organizing structure of categorizing resources and state examples related to implementation of a COMS.
Serve as the organizing structure for the self assessment
Early Childhood Outcomes Center

29. Framework and Self-Assessment
Set of components and quality indicators
Provides the structure for the self-assessment
Self-assessment
Scale that provides criteria for levels of implementation within each quality indicator
Rating assigned based on level of implementation within each indicator
Early Childhood Outcomes Center

30. Process for Framework Development
Built off what we had learned from ECO work with states and previous ECO conceptual framework
Literature review
Repeated discussion and review internally and with 7 Partner States

31. Framework Partner States
Part C
619
California X
Colorado
Delaware
Maine
Minnesota
New York
Ohio

32. COMS Framework Components
Purpose
Analysis
Data Collection and Trans-mission
Reporting
Using Data
The difference between a data system and an outcomes measurement system.
Cross-system Coordination
Evaluation
Early Childhood Outcomes Center

33. COMS Framework Components
Purpose
The state has effective procedures for collecting, storing, and transmitting data to the state.
Analysis
Data Collection and Trans-mission
Reporting
Using Data
The difference between a data system and an outcomes measurement system.
Cross-system Coordination
Evaluation
Early Childhood Outcomes Center

34. COMS Framework Components
Purpose
State coordinates child outcomes measurement and data use across EC systems.
Analysis
Data Collection and Trans-mission
Reporting
Using Data
Cross-system Coordination
Evaluation
Early Childhood Outcomes Center

35. Early Childhood Outcomes Center
Quality Indicator
Provides additional detail as to what constitutes quality implementation of the component.
18 quality indicators across the 7 components
Early Childhood Outcomes Center

36. Quality Indicators for Data Collection and Transmission
2. Data collection procedures are carried out efficiently and effectively.
3. Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment.
4. State's method for entering, transmitting, and storing data is effective and efficient.
Early Childhood Outcomes Center

37. Structure of Self Assessment
Components (7) = Major areas of framework
Quality Indicators (18 total) = Statements of basic requirements of a quality COMS
Elements (number varies with each indicator) = Define what constitutes high quality on the Quality Indicator.
Early Childhood Outcomes Center

38. Early Childhood Outcomes Center

39. Early Childhood Outcomes Center

41. Early Childhood Outcomes Center

43. The Scale for the Quality Indicators
Implementation of Elements
Quality Indicator Score
All elements are fully implemented 7
Nearly all elements are fully implemented and the rest are in process 6
Most of the elements are fully implemented and the rest are in process. 5
At least one element is fully implemented and the rest are in process 4
All of the elements are in process 3
Some of the elements are in process 2
None of the elements are yet in process 1
Early Childhood Outcomes Center

44. Early Childhood Outcomes Center

45. Recommended State Approach for Using the Self Assessment
Complete the entire self assessment.
Identify the component(s) and quality indicators to address first.
Develop action plan to improve the related elements.
Implement improvement activities.
Re-assess status and identify “next step” priorities at regular intervals
Early Childhood Outcomes Center

46. Early Childhood Outcomes Center
Is and Isn’t
Is a comprehensive resource to alert states to all the pieces that need to be in place to have a well functioning COMS
Is not a cookbook or roadmap with each step in the process spelled out.
Way too many decisions!
Early Childhood Outcomes Center

47. Where would a state start?
Questions
(Analysis)
Purpose
Early Childhood Outcomes Center

48. Early Childhood Outcomes Center
Packaging
Entire tool will exist online
Live link from each element to a “back up” section
Profile will be filled automatically based on the QI pages
Will develop a version with live links to the back up and profile that will operate off line.
Version that can be printed off as a manual.
Early Childhood Outcomes Center

49. Early Childhood Outcomes Center
Next steps for ECO
Populate the COMS framework with resources related to the components
Revise the self assessment based on feedback
Develop a framework for a Family Outcomes Measurement System
Early Childhood Outcomes Center

50. System for Producing Good Child and Family Outcomes
Prof’l Development
Preservice
Inservice
Evidence Based Practice
Good outcomes for children and families
High quality services and supports for children 0-5 and their families
Good Federal policies and programs
Good State policies and programs
Good Local policies and programs
Part of the intentionality we need to building our systems is the availability and use of good information about the direct service component and the other components in the systems. Without good information, we are operating in a “pay and pray” mode. Without good data, we don’t know what we are buying with our money. And we don’t have the tools to identify program weaknesses and address them.
Strong Leadership
Adequate funding
Information infrastructure

51. Early Childhood Outcomes Center
Early Childhood Information Infrastructure: Data Needed for Program Improvement
WHO
SERVICES
OUTCOMES
PERSONNEL
COST
Who –
Ages
Disabilities
Race/ethnicity
Services – what, how much, how long – part c and 619 – many different services; not so much with state operated preschool
Early Childhood Outcomes Center

52. Early Childhood Outcomes Center
For more information
For updates to the framework and the self-assessment and resources to support the quality indicators:
Early Childhood Outcomes Center