Program Evaluation DR. MAJED WADI

Objectives  Design necessary parameters used for program evaluation  Accept different views of program evaluation  Realize the effect of program evaluation parameters on his/her peers

Program evaluation  A ‘‘program’’, in medical education, it could be class session, a course, or a clerkship rotation in medical school or the whole of an educational program.  Evaluation is defined in the Collins English Dictionary (1994) as ‘‘the act of judgment of the worth of...’’.  Program evaluation is the ‘‘systematic collection and analysis of information related to the design, implementation, and outcomes of a program, for the purpose of monitoring and improving the quality and effectiveness of the program.’’

Reasons of program evaluation  To determine the effectiveness of programmes for participants  To document that programme objectives have been met  To provide information about service delivery that will be useful to programme staff and other audiences  To enable programme staff to make changes that improve programme effectiveness

Models of program evaluation  Kirkpatrick’s model  Logic model  CIPP Model

Kirkpatrik’s model  It focuses on learner outcomes 1. learner satisfaction or reaction to the program; 2. measures of learning attributed to the program (e.g. knowledge gained, skills improved, attitudes changed); 3. changes in learner behavior in the context for which they are being trained; and 4. the program’s final results in its larger context.

Logic model

CIPP Model

Designing program evaluation  To design and conduct an evaluation plan one should consider the  Purpose of evaluation  Audiences  Negotiation and client involvement  Key methods(standards)  Ethical consideration  Data management  Time line and resources implications  Reporting

Methods of data collection  Questionnaires / checklists  Interviews  Documentation review  Observation  Focus groups  Case studies

Kirkpatrik’s model  Level 1 : learner satisfaction or reaction to the program  Methods: questionnaires, interview  Level 2 : measures of learning attributed to the program  Methods : analysis of exams  Level 3 : changes in learner behavior in the context for which they are being trained  Methods : work-based observation  Level 4 : the program’s final results in its larger context  Methods : evaluation of patients outcomes and improved health care