This project is financed by the European Union 1 The project is implemented by a European Profiles S.A. led consortium Evaluation of Training By Senior Sort Term International Expert: Kyriakos DIMITRIOU Place: UCCI Tashkent Training Centre Ташкент - July 9, 2014 Training of Trainers

This project is financed by the European Union 2 The project is implemented by a European Profiles S.A. led consortium Our objectives To define the need of evaluation when dealing with training activities To get familiar with the evaluation procedures To determine models of evaluating and how

This project is financed by the European Union 3 The project is implemented by a European Profiles S.A. led consortium Evaluation Generally speaking, evaluation is used to determine the degree to which program objectives are met through the procedures used. If we want to define “evaluation in training” as a term, we could say that is a systematic procedure to give a “value” qualitative and quantitative in a training session, using concrete criteria and methodology.

This project is financed by the European Union 4 The project is implemented by a European Profiles S.A. led consortium The procedure What When (by) Whom 3 W How H

This project is financed by the European Union 5 The project is implemented by a European Profiles S.A. led consortium Evaluation; “What” I know that you know so I am listening … …… Curriculum (content) Way (methods, techniques) Training facilitation General satisfaction Impact Trainer Ourselves External factors (facilities etc)

This project is financed by the European Union 6 The project is implemented by a European Profiles S.A. led consortium Evaluation; “When” When the evaluation is taking place? The answer in this question is: in all phases of the training program  Before (ex ante)  During (on going) or sometime interim  After (ex post)

This project is financed by the European Union 7 The project is implemented by a European Profiles S.A. led consortium Ex ante Ex-ante evaluation enables analysis of the anticipated impacts of the planned programme. It’s a quite demanding process in comparison with on going and ex post evaluation. o Structure of the activity o Sequence of priorities o Internal coherence and o External coherence

This project is financed by the European Union 8 The project is implemented by a European Profiles S.A. led consortium Ex-ante assesses Ex-ante evaluation assesses whether objectives are clear, coherent and adjusted to the situation, helps the realistic numeric quantification of these and defines indicators enabling the monitoring of implementation, as well as reporting on results achieved. Indicators relate to financial resources, physical attributes, results and impact.

This project is financed by the European Union 9 The project is implemented by a European Profiles S.A. led consortium Evaluation; “By whom” Who is responsible to evaluate? Everyone has the right to evaluate any actor (participant). Usually: Learners Trainer and External evaluator (usually on behalf of beneficiaries, stakeholders) INTERNAL

This project is financed by the European Union 10 The project is implemented by a European Profiles S.A. led consortium Evaluation; “How” Quite a lot of ways is the answer and by various means (techniques):  Each “actor” evaluates others  Self evaluation Means: Questionnaires Interviews Round tables etc

This project is financed by the European Union 11 The project is implemented by a European Profiles S.A. led consortium Ex post evaluation a) determining what the participants have learned b) giving the learners time to reflect on their learning c) getting useful feedback in an organized manner, to help with future training planning d) ensuring learners follow-up their training with relevant actions to apply, improve, develop and reinforce learning attained.

This project is financed by the European Union 12 The project is implemented by a European Profiles S.A. led consortium Bloom's Taxonomy in evaluation Benjamin S Bloom ( ) -USA Taxonomy means “a set of classification principles”, or “a structure” Bloom's Taxonomy model is in three parts, or “overlapping domains”. Bloom used rather academic language, but the meanings are simple to understand:

This project is financed by the European Union 13 The project is implemented by a European Profiles S.A. led consortium The Taxonomy Cognitive domain (intellectual capability, ie., knowledge, or 'think') Affective domain (feelings, emotions and behavior, ie., attitude, or 'feel') Psychomotor domain (manual and physical skills, ie., skills, or 'do')

This project is financed by the European Union 14 The project is implemented by a European Profiles S.A. led consortium Simplification /1 Cognitive (knowledge) 1.Recall data 2.Understand 3.Apply (use) 4.Analyse (structure/elements) 5.Synthesize (create/build) 6.Evaluate (assess, judge in relational terms)

This project is financed by the European Union 15 The project is implemented by a European Profiles S.A. led consortium Simplification /2 Affective (attitude) 1.Receive (awareness) 2.Respond (react) 3.Value (understand and act) 4.Organise personal value system 5.Internalize value system (adopt behavior)

This project is financed by the European Union 16 The project is implemented by a European Profiles S.A. led consortium Simplification /3 Psychomotor (skills) 1.Imitation (copy) 2.Manipulation (follow instructions) 3.Develop Precision 4.Articulation (combine, integrate related skills) 5.Naturalization (automate, become expert)

This project is financed by the European Union 17 The project is implemented by a European Profiles S.A. led consortium Evaluating Training Programs: Kirkpatrick's 4 Levels The most widely used and popular model for the evaluation of training programs is known as "The Four Levels of Learning Evaluation" The model was created by Donald L. Kirkpatrick in a series of articles that appeared in the US Training and Development Journal. Kirkpatrick redefined the evaluation model with his 1998 book "Evaluating Training Programs: The Four Levels."

This project is financed by the European Union 18 The project is implemented by a European Profiles S.A. led consortium Kirkpatrick's model The four levels of the model are: Level 1: Reaction Level 2: Learning Level 3: Behavior Level 4: Results

This project is financed by the European Union 19 The project is implemented by a European Profiles S.A. led consortium Level 1: Reaction Kirkpatrick refers to Level one as a measure of customer (learner) satisfaction. Most of the forms that people fill out at the end of a training or a similar (coaching, mentoring) activity are instruments for measuring Level 1.

This project is financed by the European Union 20 The project is implemented by a European Profiles S.A. led consortium Level 2: Learning Kirkpatrick defines learning as the extent to which participants change attitudes, increase knowledge, and/or increase skill as a result of attending a program. Level 3: Behaviour Level three can be defined as the extent to which a change in behaviour has occurred because someone attended a training program

This project is financed by the European Union 21 The project is implemented by a European Profiles S.A. led consortium Level 4: Results This involves measuring the final results that occurred because a person attended a training session. This can include increased production, improved work quality, reduced turnover, etc. Level four can be difficult because you must determine what final results occurred because of attendance and participation in a training program

This project is financed by the European Union 22 The project is implemented by a European Profiles S.A. led consortium Evaluation has a meaning when  Is a procedure to collect valid, reliable and useful information and data  Extend in explaining of the results  Is combined with feedback  Is not only quantitative  Its results are used by appropriate persons in terms of improvement

This project is financed by the European Union 23 The project is implemented by a European Profiles S.A. led consortium Example The general structure of the training The Content of the training The quality of the delivery The quality of the material provided The opportunity given for interaction Sufficiency of time Improvement of knowledge

This project is financed by the European Union 24 The project is implemented by a European Profiles S.A. led consortium Our own example Are we able to create our own example in evaluation? Lets try…

This project is financed by the European Union 25 The project is implemented by a European Profiles S.A. led consortium Summary Are there additional questions? Will you help me to summarize our training action of the day?

This project is financed by the European Union 26 The project is implemented by a European Profiles S.A. led consortium Спасибо за ваше Внимание Вопросы, Комментарии и Предложения (до 17.30)