Program Evaluation “Most often defined as a process used to determine whether the design and delivery of a program were effective and whether the proposed outcomes were met.” “Most often defined as a process used to determine whether the design and delivery of a program were effective and whether the proposed outcomes were met.”

Program Evaluation Can be formal or informal Can be formal or informal Can be formative or summative Can be formative or summative Centers around the issue of accountability Centers around the issue of accountability Is accountability always a good thing or are there examples of it creating more harm than good? Is accountability always a good thing or are there examples of it creating more harm than good?

Program Evaluation Judgment process not an easy task… Judgment process not an easy task… Can be difficult to demonstrate direct correlation between program content and specific outcomes Can be difficult to demonstrate direct correlation between program content and specific outcomes Clear criteria, particularly when qualitative, can be difficult to finalize Clear criteria, particularly when qualitative, can be difficult to finalize Some people may not want to have such judgments made about their programs Some people may not want to have such judgments made about their programs

Program Evaluation How does evaluation connect with the other elements of Caffarella’s model? How does evaluation connect with the other elements of Caffarella’s model?

Program Evaluation Systematic Program Evaluation Systematic Program Evaluation Caffarella’s 11 step model Caffarella’s 11 step model The CGSC model…known as the Accountable Instruction System. The CGSC model…known as the Accountable Instruction System.

Program Evaluation Approaches to Program Evaluation Approaches to Program Evaluation Levels of Evaluation Approach Levels of Evaluation Approach Objectives-based Approach Objectives-based Approach The basics of what CGSC uses The basics of what CGSC uses Mixed with the levels of approach method for a blended scenario at CGSC Mixed with the levels of approach method for a blended scenario at CGSC

Program Evaluation Collecting Data Collecting Data Many techniques Many techniques Observations Observations Interviews Interviews Questionnaires Questionnaires Tests Tests Portfolios Portfolios Etc, etc, etc Etc, etc, etc Based on your program objectives and context Based on your program objectives and context

Program Evaluation Data Analysis Data Analysis Depends on Focus and context Depends on Focus and context Participants’ learning Participants’ learning Program Operations Program Operations Organizational Issues Organizational Issues Societal Issues Societal Issues Normally divided by: Normally divided by: Quantitative Quantitative Qualitative Qualitative

Program Evaluation Making Judgments Making Judgments Based on criteria related to program processes or outcomes Based on criteria related to program processes or outcomes Reached by comparing results of data analysis with criteria that were set Reached by comparing results of data analysis with criteria that were set Can be difficult to prove outcome is the result of the program versus some outside factor…have to try and engineer that out of the equation Can be difficult to prove outcome is the result of the program versus some outside factor…have to try and engineer that out of the equation