1. From Program Evaluation to Program Improvement
21st CCLC Training Meeting
January 14, 2011
Presented by:
Carrie Furrer & Dunya Minoo,
Portland Public Schools
Greg Belisle, Impact NW

2. Evaluation  Improvement
Improvement: Figure out where to make changes
Formative: Plan & think about your work
Summative: Find out if program is making a difference
Implementation: Understand service delivery & participation

3. Demystifying the ‘black box’ of program improvement
Program Inputs
Resources
Planning
Staff
Partners
21st CCLC
Programming
Long-term Student Outcomes
Program Outputs  Short-term Student Outcomes

4. What’s in the Black Box? Theory of Change
What is your theory about how your program works to improve children’s lives?
PROGRAM OUTPUTS
Quantity: number of students served; hours served
Quality: positive staff-student interactions; prosocial climate; social support
Communication/collaboration: relationships with partners; collaboration with school staff
SHORT-TERM OUTCOMES: Student engagement in program, motivation to participate in activities, sense of belonging

5. Logic Model = Theory of Change
Program Inputs
Program Outputs
Short-term Outcomes
Long-term Outcomes
Homework club
Students served 4 hours/week
Collaboration with school day teachers
Qualified staff
Students motivated to complete homework
Students turn in homework
Better understanding of class material
Improved grades

6. Program Improvement: A Systematic Review of….
Program Inputs: Program Planning & Design
Program Outputs: Quality, Quantity, Communication/Collaboration, Other
Short-term Outcomes: Students’ “baby steps” toward the long-term outcomes

7. Resources
Bagby, J. H. (Ed.). (2008). A resource guide for planning and operating afterschool programs (3rd ed.). Austin, TX: SEDL.
Chapin Hall at the University of Chicago.
Eccles, J., & Gootman, J. A. (Eds.). (2002). Community programs to promote youth development. Washington, DC: National Academies Press.
Education Northwest. educationnorthwest.org/resource/focus/Evaluation.
Harvard Family Research Project.
Little, P. M. D. (Ed.). (2003, Spring). Evaluating out-of-school time. The Evaluation Exchange, 9(1).
SEDL.
U.S. Department of Education. (2003, February 3). Evaluation shows improvements needed in after school program. Washington, DC: Author.