1. Layers of Evaluation & Meaningfulness of Results
3CSN and ELI Collaborate: Data Collection Workshop
September 6, 2013
Mehmet Dali Öztürk, Ph.D.

2. Meaningfulness of Results
Layers of Evaluation
Meaningfulness of Results
Impact Assessment
Most Meaningful
Outcome Analysis
More Meaningful
Least Meaningful
Output Analysis

3. Layers of Evaluation
Impact assessment focuses on determination of changes in outcomes that can be attributed to polices, programs, initiatives and projects for schools, individuals, families, or communities. For example, the change in school graduation rates due to a specific program in comparison to schools who did not receive the same program.
Outcomes focus on intended and expected results of polices programs, initiatives and projects. For example, the percent increase in schools meeting state standards.
Outputs are the types and quantities of engagements and efforts. For example, the number of mentoring hours produced by program personnel.

4. Output Outcome Impact Program X
Number of students in program
Number of students completing program
Percent gain in standard test scores in math
Significant changes in percent gain in standard math test scores at the focus schools where program is implemented
Percent gain in standard test scores in reading
Significant changes in percent gain in standard test scores in reading at the focus schools where program is implemented
Percent of students demonstrating a grade increase in math on report cards
Significant changes in percent of students demonstrating a grade increase in math on report cards at the focus schools where program is implemented
Percent of students demonstrating a grade increase in reading on report cards
Significant changes in percent of students demonstrating a grade increase in reading on report cards at the focus schools where program is implemented

5. Providing a Context for the Evaluation Process
Are there differences in outcome variables?
Are the differences significant?
Are the differences meaningful?
What do these differences mean with regard to the overall community?

6. Program Effectiveness Scale/Rating System
Program Effectiveness/Scale Rating System, which was developed to be a tool to help policy-makers better understand the impact of programs, strategies, and innovations.
By placing evaluation results into a format that is easy to understand, the Program Effectiveness Scale can help policy-makers make objective judgments on the worth of a program or strategy.