The purpose of evaluation is not to prove, but to improve.
Why Evaluate? To see if the program works What do we mean by “works?” Was the program/course successful? Define “success” measurements
Underlying Premises for Evaluation It is an important process Need to know if course objectives are being met and how to improve them Every evaluation has a political context Who will use the findings? Need to have clearly defined purpose prior to conducting evaluation
Evaluation helps in decision making Improve the way a program/course operates Decide fund & resource allocation Add or drop program/course components Initiate similar program/course elsewhere Continue or discontinue a program/course Accept or reject specific program/course approach or theory
Some Evaluation Types Formative Summative Impact Outcome Process
Formative Evaluation Usually carried out during the formation or development of the program or course May include pre-testing (e.g. component pre-testing) and pilot studies of components Usually answering questions like: Has the appropriate/qualified teacher been hired? Do we have adequate educational materials and resources for the students? How does it fit in with the rest of the curriculum?
Process Evaluation Process evaluation is a type of formative evaluation in that it usually takes place at the beginning and during the program/course; ongoing Quality assurance Used to determine the appropriateness of: Content Teaching methods, materials, media, etc. Methods: Teacher – quizzes, observe audience/students, unbiased outside observer, surveys & questionnaires during course Students – ask questions; get feedback; logs; journals, focus groups
Summative Evaluation Permits conclusions to be drawn about impact, outcome, or benefits of a program, course or methodology Was there a change/increase in knowledge, attitudes, skills, or behaviors? Which approach worked best? Did the health changes/improvements continue after the course ended?
Impact Assessment of immediate impact of a program/course on: Short-term goals Knowledge Interview & Discussion Discussion & Observation Written test Attitudes Observation – what they say Questionnaire with rating scale Behavior Observe – demonstration – keep record of participants
Outcome Evaluation Health outcomes – long term changes Assess changes in or improvements to health status indicators, such as: Morbidity and mortality rates Incidence or prevalence rates Other “statistical” data that may demonstrate improvement
Illinois Learning Improvement Plan A broad strategy for program evaluation Based on Illinois Learning Standards Illinois Goal Assessment Programs (IGAPS) LAP’s – Learning Assessment Programs. School make up objectives based on learning standards for specific grades. Then assess whether these objectives are being met through written exams and other methods. School funding ($$$$!) from State and Federal (No child left behind) sources depends in large part how students demonstrate what they know and are able to do.