Presentation on theme: "SNDT Women's University Introduction to Evaluation and Assessments Presented by Kathleen (Kat) Miller, Senior Consultant with Booz Allen Hamilton 4."— Presentation transcript:
SNDT Women's University Introduction to Evaluation and Assessments Presented by Kathleen (Kat) Miller, Senior Consultant with Booz Allen Hamilton 4 August, 2007
2 SNDT Women's University Evaluation Model – Donald Kirkpatrick In 1959, Donald Kirkpatrick developed what has become one of the most popular models for evaluating education programs. Kirkpatrick's system has four levels of evaluation.
4 SNDT Women's University Main Reasons for Evaluation Determine how to improve future programs Determine whether a program should be continued or dropped. Justify the existence of the training department
5 SNDT Women's University Kirkpatrick’s Level 1 - Reaction Measure of “Customer Satisfaction” Measures the emotional reaction of participants after a learning experience. –It attempts to answer questions regarding the participants' perceptions: –Did they like it? –Was the material relevant to their work? –Did it meet their expectations? –How should the learning experience be improved?
6 SNDT Women's University Why Measure Reaction? It provides valuable feedback and suggestions It makes the trainees feel that their input is important It is easy to do effectively A positive reaction promotes motivation!
7 SNDT Women's University Guidelines for Evaluating “Reaction” 1.Determine exactly what you want to find out 2.Design a form the will quantify reaction 3.Encourage written comments and suggestions 4.Get a 100 percent immediate response 5.Get honest responses 6.Develop acceptable standards 7.Measure reactions against standards and that action 8.Communicate (as appropriate)
8 SNDT Women's University Kirkpatrick’s Level 2 - Learning Measures whether or not a student achieved the learning objectives during the learning event or during a series of such events. For example: we might test for written communication skills after instruction to determine if the learner is now qualified for the job. Before Training After Training
9 SNDT Women's University Why Measure “Learning?” Kirkpatrick defines learning as, “the extent to which participants change attitudes, increase knowledge and/or skill as a result of attending a program.” Was knowledge increased? Were skills developed? Were attitudes changed?
10 SNDT Women's University Guidelines for Evaluating “Learning” 1.Use a control group if possible 2.Use pre/post tests –Written for knowledge measurement –Performance for skills 3.Get 100 percent response 4.Use results to take appropriate action
11 SNDT Women's University Kirkpatrick’s Level 3 - Behavior Measures whether learners are able to apply their new knowledge and skills to their job. –Are they using those skills on the job? –Are there other issues that are stopping them from being successful on the job? –Has their behavior changed? Information for Level 3 evaluations is generally gathered via surveys and personal interviews.
12 SNDT Women's University Four Conditions for Behavior to Change The person must have the desire to change The person must know what to do and how to do it The person must work in the right climate The person must be rewarded for changing
13 SNDT Women's University Guidelines for Evaluating “Behavior” Use a control group if possible Allow time for the change in behavior to take place Evaluate before and after the program Survey or interview the trainee, as well as someone who observes their behavior Get 100 percent response Repeat the evaluation at appropriate times Consider cost versus benefits
15 SNDT Women's University Challenges to Evaluating “Results” Some are intangible Results may not occur for a long time Many other factors impact results The trainer cannot control all factors Such is hard to control all those wild cats!
16 SNDT Women's University Guidelines for Evaluating Results Use a control group if possible Allow time for results to be achieved Measure both before and after Repeat the measurement at appropriate times Consider cost versus behavior Be satisfied with evidence, if proof is not possible