1. Building a Quality Measurement System
Kathy Hebbeler
Early Childhood Outcomes Center
SRI International
Presented at
Division for Early Childhood
International Conference on Young Children with Special Needs and Their Families
October, 2010 1

2. Early Childhood Outcomes Center
"Success should be judged by results, and data is a powerful tool to determine results. We can't ignore facts. We can't ignore data." President Barack Obama July 24, 2009
Early Childhood Outcomes Center

3. System for Producing Good Child and Family Outcomes
Prof’l Development
Preservice
Inservice
Evidence Based Practice
Good outcomes for children and families
High quality services and supports for children 0-5 and their families
Good Federal policies and programs
Good State policies and programs
Good Local policies and programs
Part of the intentionality we need to building our systems is the availability and use of good information about the direct service component and the other components in the systems. Without good information, we are operating in a “pay and pray” mode. Without good data, we don’t know what we are buying with our money. And we don’t have the tools to identify program weaknesses and address them.
Strong Leadership
Adequate funding
Information infrastructure

4. Early Childhood Outcomes Center
A good information infrastructure is an essential tool to providing high quality services and supports.
(You cannot have data based-decision making without good data.)
This really is about children and families.
Early Childhood Outcomes Center

5. Early Childhood Outcomes Center
Early Childhood Information Infrastructure: Data Needed for Program Improvement
WHO
SERVICES
OUTCOMES
PERSONNEL
COST
Who –
Ages
Disabilities
Race/ethnicity
Services – what, how much, how long – part c and 619 – many different services; not so much with state operated preschool
Early Childhood Outcomes Center

6. Early Childhood Outcomes Center
A Framework and Self Assessment for Building a Child Outcomes Measurement System
Early Childhood Outcomes Center

7. Purpose of the Framework
Provide a common language for ECO and other TA providers to use in discussing COMSs with states.
Provide a organizing structure of categorizing resources and state examples related to implementation of a COMS.
Serve as the organizing structure for the self assessment
Early Childhood Outcomes Center

8. Framework and Self-Assessment
Set of components and quality indicators
Provides the structure for the self-assessment
Self-assessment
Scale that provides criteria for levels of implementation within each quality indicator
Rating assigned based on level of implementation within each indicator
Early Childhood Outcomes Center

9. Process for Framework Development
Built off what we had learned from ECO work with states and previous ECO conceptual framework
Literature review
Repeated discussion and review internally and with 7 Partner States

10. Framework Partner States
Part C
619
California X
Colorado
Delaware
Maine
Minnesota
New York
Ohio

11. COMS Framework Components
Purpose
Analysis
Data Collection and Trans-mission
Reporting
Using Data
The difference between a data system and an outcomes measurement system.
Cross-system Coordination
Evaluation
Early Childhood Outcomes Center

12. COMS Framework Components
Purpose
The state has effective procedures for collecting, storing, and transmitting data to the state.
Analysis
Data Collection and Trans-mission
Reporting
Using Data
The difference between a data system and an outcomes measurement system.
Cross-system Coordination
Evaluation
Early Childhood Outcomes Center

13. COMS Framework Components
Purpose
State coordinates child outcomes measurement and data use across EC systems.
Analysis
Data Collection and Trans-mission
Reporting
Using Data
Cross-system Coordination
Evaluation
Early Childhood Outcomes Center

14. Early Childhood Outcomes Center
Quality Indicator
Provides additional detail as to what constitutes quality implementation of the component.
18 quality indicators across the 7 components
Early Childhood Outcomes Center

15. Quality Indicators for Data Collection and Transmission
2. Data collection procedures are carried out efficiently and effectively.
3. Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment.
4. State's method for entering, transmitting, and storing data is effective and efficient.
Early Childhood Outcomes Center

16. Structure of Self Assessment
Components (7) = Major areas of framework
Quality Indicators (18 total) = Statements of basic requirements of a quality COMS
Elements (number varies with each indicator) = Define what constitutes high quality on the Quality Indicator.
Early Childhood Outcomes Center

17. Early Childhood Outcomes Center

18. Early Childhood Outcomes Center

20. Early Childhood Outcomes Center

21. The Scale for the Quality Indicators
Implementation of Elements
Quality Indicator Score
All elements are fully implemented 7
Nearly all elements are fully implemented and the rest are in process 6
Most of the elements are fully implemented and the rest are in process. 5
At least one element is fully implemented and the rest are in process 4
All of the elements are in process 3
Some of the elements are in process 2
None of the elements are yet in process 1
Early Childhood Outcomes Center

22. Early Childhood Outcomes Center

23. Recommended State Approach for Using the Self Assessment
Complete the entire self assessment.
Identify the component(s) and quality indicators to address first.
Develop action plan to improve the related elements.
Implement improvement activities.
Re-assess status and identify “next step” priorities at regular intervals
Early Childhood Outcomes Center

24. Next steps for ECO in developing the self assessment
Populate the COMS framework with resources related to the components
Revise the self assessment based on feedback
Develop a framework for a Family Outcomes Measurement System
Early Childhood Outcomes Center

25. Early Childhood Outcomes Center
For more information
For updates to the framework and the self-assessment and resources to support the quality indicators:
Early Childhood Outcomes Center