2. © Dariusz Danilewicz2 Competency development as the process From strategy to evaluation Human resources development programs evaluation Donald Kirkpatricks Model Tools Balanced Scorecard Approach

3. Competency development needs assessment Designing and developing of the program Delivering of the program Appraisal of the program Organizational mission The strategy of the organization The strategic plan of HR development HR development principles © Dariusz Danilewicz3

4. Dariusz Danilewicz4 Training needs assessment Designing and developing of the program and manuals Delivering the program Knowledge transfer (knoeledge difussion) Evaluating the program

5. Personnel strategy Business strategy Communicating systems Organizational culture Dariusz Danilewicz5

6. t© Dariusz Danilewicz Business strategy Personnel strategy (and substrategies) Business strategy Personnel strategy (and substrategies) Business strategy Personnel strategy (and subsrategies) Reactive model Interactive model Active model 6

7. 20022007 Yes66,4%55,7% No33,6%44,3% 20022007 Yes26,4%48,3% No73,6%51,7% Long-term personnel strategy (perspective: 3 years&more) Short-term personnel strategy (perspective: <3 years) © Dariusz Danilewicz7

8. The role of the HR Departament is to build the STRONG culture. The strong culture aim is to favour the HR development. © Dariusz Danilewicz8

9. No connection between training programs and strategy/aims/ mission Training programs treated as cost (not investition) Training program as formal decission Non-active methods & theoretical training Training program as a formal responsibility of HR Departament Top managers and line managers not included in the process Focusing on TRAINING, not on DEVELOPMENT (continual process) © Dariusz Danilewicz9

10. Permanent learning as the condition of company development in the turbulent environment Conections between business strategy and individual needs of employees Impact on development on- the-job (continual process) Different methods of learning (to acquire knowledge&skills, to form an attitude) New forms of learning: distance learning,e-learning, blended learning, self-improvement) The role of training evaluation quintet: senior management, the trainer, line management, the training manager, the trainee. © Dariusz Danilewicz10

11. Criteria of cost Criteria of implementation Criteria of risk Criteria of side effect Criteria of resistance © Dariusz Danilewicz11

12. 20022007 Yes66,4%79,5% No33,6%20,5% 20022007 Yes9,4%36,9% No90,6%61,1% Long-term training strategy (perspective: 3 years&more) Short-term training strategy (perspective: 3 years&more) © Dariusz Danilewicz12

13. a. Pression from the markete. CEO/high level management/ owners decission b. Day-to-day needsf. Motivating of key workers c. Short-term plansg. Developing of key competencies d.Long-term strategyh. Others © Dariusz Danilewicz13

14. © Dariusz Danilewicz14

15. Training needs Non-training needs © Dariusz Danilewicz15

16. Structure of company Management HR management Job description Procedures, systems Information system Environment Private problems © Dariusz Danilewicz16

17. © Dariusz Danilewicz17

18. Documentation analysis Strategic HR Others Questionnaires From-the-shelf Tailor-made © Dariusz Danilewicz18

19. Observations Self-observation Expert Mystery shopping Simulations Panel of experts © Dariusz Danilewicz19

20. Interview Standard Non-structurized IDI FGI Critical incidents © Dariusz Danilewicz20

21. © Dariusz Danilewicz21 a. Planning competency development during the proces of selection b. Development Centre c. Coaching/mentoring d. Employees satifaction anaysis e. (In-)formal group focused on efficiency growth f. Meetings/chats with management

22. © Dariusz Danilewicz22 a. Interwievs with clients b. Questionnaries c. Other metods of collecting clients opinions

23. © Dariusz Danilewicz23 a. Questionnaries assessing training needs b. Tests, exams c. Workers appraisal system

24. © Dariusz Danilewicz24 a. Balanced Scorecard b. Companys market value c. Labour turnover (fluctuation) d. Labour absenteeism e. Clients number and structure f. Sales g. Quality of products/services

25. © Dariusz Danilewicz25

26. © Dariusz Danilewicz26

27. © Dariusz Danilewicz27

28. Business strategy, personnel strategy, mission, vison, aims, values Training needs assessment (key cometencies) Recruitment and selection Building of organizational culture Appraisal system HR development © Dariusz Danilewicz28

29. senior management the training manager the trainee line management the trainer © Dariusz Danilewicz29

30. Awareness of the need and value of training to the organization. The necessity of involving the Training Manager (or equivalent) in senior management meetings where decisions are made about future changes when training will be essential. Knowledge of and support of training plans. Active participation in events. Requirement for evaluation to be performed and require regular summary report. Policy and strategic decisions based on results and ROI data. © Dariusz Danilewicz30

31. Work-needs and people identification. Involvement in training program and evaluation development. Support of pre-event preparation and holding briefing meetings with the learner. Giving ongoing, and practical, support to the training program. Holding a debriefing meeting with the learner on their return to work to discuss, agree or help to modify and agree action for their action plan. Reviewing the progress of learning implementation. Final review of implementation success and assessment, where possible, of the ROI. © Dariusz Danilewicz31

32. Management of the training department and agreeing the training needs and the program application Maintenance of interest and support in the planning and implementation of the program, including a practical involvement where required The introduction and maintenance of evaluation systems, and production of regular reports for senior management Frequent, relevant contact with senior management Liaison with the learners' line managers and arrangement of learning implementation responsibility learning program for the managers Liaison with line managers, where necessary, in the assessment of the training ROI. © Dariusz Danilewicz32

33. Involvement in the planning and design of the training program where possible Involvement in the planning and design of the evaluation process where possible Obviously, to take interest and an active part in the training program or activity. To complete a personal action plan during and at the end of the training for implementation on return to work, and to put this into practice, with support from the line manager. Take interest and support the evaluation processes. © Dariusz Danilewicz33

34. 1. Reaction & satisfaction 2. Learning 3. Application & implementation (behaviour) 4. Results © Dariusz Danilewicz34

35. © Dariusz Danilewicz35 Reaction evaluation is how the trainees feel, and their personal reactions to the training or learning experience Do the trainees enjoy the training? Do they consider the training relevant? Is it a good use of their time? Do they like the venue, the style, timing, domestics, etc? Level of participation. Ease and comfort of experience. Level of effort required to make the most of the learning. Perceived practicability and potential for applying the learning.

36. © Dariusz Danilewicz36 Typically 'happy sheets'. Feedback forms based on subjective personal reaction to the training experience. Verbal reaction which can be noted and analysed. Post-training surveys or questionnaires. Online evaluation or grading by delegates. Subsequent verbal or written reports given by delegates to managers back at their jobs.

37. © Dariusz Danilewicz37 Can be done immediately the training ends. Very easy to obtain reaction feedback Feedback is not expensive to gather or to analyse for groups. Important to know that people were not upset or disappointed. Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same.

38. © Dariusz Danilewicz38 Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before to after the learning experience Do the trainees learn what intended to be taught? Do the trainee experience what was intended for them to experience? What is the extent of advancement or change in the trainees after the training, in the direction or area that was intended?

39. © Dariusz Danilewicz39 Typically assessments or tests before and after the training. Interview or observation can be used before and after although this is time-consuming and can be inconsistent. Methods of assessment need to be closely related to the aims of the learning. Measurement and analysis is possible and easy on a group scale. Reliable, clear scoring and measurements need to be established, so as to limit the risk of inconsistent assessment. Hard-copy, electronic, online or interview style assessments are all possible.

40. © Dariusz Danilewicz40 Can be done immediately the training ends. Very easy to obtain reaction feedback Feedback is not expensive to gather or to analyse for groups. Important to know that people were not upset or disappointed. Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same.

41. © Dariusz Danilewicz41 Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour, and this can be immediately and several months after the training, depending on the situation: Do the trainees put their learning into effect when back on the job? Are the relevant skills and knowledge used? Is there noticeable and measurable change in the activity and performance of the trainees when back in their roles? Is the change in behaviour and new level of knowledge sustained? Would the trainee be able to transfer their learning to another person? Is the trainee aware of their change in behaviour, knowledge, skill level?

42. © Dariusz Danilewicz42 Observation and interview over time are required to assess change, relevance of change, and sustainability of change. Arbitrary snapshot assessments are not reliable because people change in different ways at different times. Assessments need to be subtle and ongoing, and then transferred to a suitable analysis tool. Assessments need to be designed to reduce subjective judgement of the observer or interviewer, which is a variable factor that can affect reliability and consistency of measurements. The opinion of the trainee, which is a relevant indicator, is also subjective and unreliable, and so needs to be measured in a consistent defined way.

43. © Dariusz Danilewicz43 360-degree feedback is useful method and need not be used before training, because respondents can make a judgement as to change after training, and this can be analysed for groups of respondents and trainees. Assessments can be designed around relevant performance scenarios, and specific key performance indicators or criteria. Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols. Self-assessment can be useful, using carefully designed criteria and measurements.

44. © Dariusz Danilewicz44 Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation. Simple quick response systems unlikely to be adequate. Cooperation and skill of observers, typically line- managers, are important factors, and difficult to control. Management and analysis of ongoing subtle assessments are difficult, and virtually impossible without a well-designed system from the beginning.

45. © Dariusz Danilewicz45 Evaluation of implementation and application is an extremely important assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job, therefore evaluation in this area is vital, albeit challenging. Behaviour change evaluation is possible given good support and involvement from line managers or trainees, so it is helpful to involve them from the start, and to identify benefits for them, which links to the level 4 evaluation below.

46. © Dariusz Danilewicz46 Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test. Measures would typically be business or organisational key performance indicators, such as: Volumes, values, percentages, timescales, return on investment, and other quantifiable aspects of organisational performance, for instance; numbers of complaints, staff turnover, attrition, failures, wastage, non-compliance, quality ratings, achievement of standards and accreditations, growth, retention, etc.

47. © Dariusz Danilewicz47 It is possible that many of these measures are already in place via normal management systems and reporting. The challenge is to identify which and how relate to the trainee's input and influence. Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training, so they understand what is to be measured. This process overlays normal good management practice - it simply needs linking to the training input. For senior people particularly, annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training.

48. © Dariusz Danilewicz48 Individually, results evaluation is not particularly difficult; across an entire organisation it becomes very much more challenging, not least because of the reliance on line-management, and the frequency and scale of changing structures, responsibilities and roles, which complicates the process of attributing clear accountability. Also, external factors greatly affect organisational and business performance, which cloud the true cause of good or poor results.

49. © Dariusz Danilewicz49

50. © Dariusz Danilewicz50

51. Dariusz Danilewicz51 Training needs assessment Designing and developing of the program and manuals Delivering the program Knowledge transfer Evaluating the program

52. Personnel strategy Business strategy Communicating systems Organizational culture Dariusz Danilewicz52

53. Personnel strategy Business strategy Communicating systems Organizational culture Training needs assessment Designing and developing of the program and manuals Delivering the program Knowledge transfer Evaluating the program Knowledge organization Dariusz Danilewicz53 Knowledge worker