Conceptual Developments in Evaluation of Training Effectiveness & Feedback Dr. Shulagna Sarkar Assistant Professor , Jt. Coordinator – PGDM – HRM IPE, Hyderabad

EVALUATION OF TRAINING EFFECTIVENESS – THE SOLUTION

Why Should A Training Program Be Evaluated? Identifying the program’s strengths and weaknesses Assessing whether content, organization, and administration of the program contribute to learning and the use of training content on the job Distinguishing trainees who benefited most or least from the program Determining the financial benefits and costs of the programs Comparing costs and benefits of training versus non- training investments Comparing costs and benefits of different training programs to choose the best program

Challenges Setting the right objectives and meeting those those objectives Communicating outcome expectations to trainers as well as trainees. Cooperation from each participants and people involved. Change in Attitude: Shift from evaluation of the person to evaluation of the correctness of the program Removing TRAINING WASTE - training that isn't needed is conducted, and the required training or other solution that would be truly beneficial is not even identified.

Introduction Training effectiveness refers to the benefits that the company and the trainees receive from training Training outcomes or criteria refer to measures that the trainer and the company use to evaluate training programs Training evaluation refers to the process of collecting the outcomes needed to determine if training is effective Evaluation design refers to from whom, what, when, and how information needed for determining the effectiveness of the training program will be collected

Evaluation Formative evaluation – evaluation conducted to improve the training process Helps to ensure that: the training program is well organized and runs smoothly trainees learn and are satisfied with the program Provides information about how to make the program better Summative evaluation – evaluation conducted to determine the extent to which trainees have changed as a result of participating in the training program May also measure the return on investment (ROI) that the company receives from the training program

The Product – The training What can be evaluated Remember 3 Ps The Plan The Process The Product – The training S.M.Israr

How to evaluate the Plan Course Objectives Appropriate selection of participants Timeframe Teaching Methods

How to Evaluate the Process Planning Vs. Implementation Appropriate participants Appropriate time Effective use of time Imparting training according to set objectives

How to Evaluate the Product Is only evaluation of the product sufficient? Time Ultimately all stages require evaluation in any case Changes in effectiveness Impact Analysis Achieving Targets Attracting Resources Satisfying Interested Parties

Kirkpatrick’s Four-Level Framework of Evaluation Criteria Focus 1 Reactions Trainee satisfaction 2 Learning Acquisition of knowledge, skills, attitudes, behavior 3 Behavior Improvement of behavior on the job 4 Results Business results achieved by trainees

Only by assessing each level can we yield actionable results Level 4 - Results Was it worth it? Level 3 - Behavior KSA being used on the job? Check Requirements, Systems and Processes Level 2 - Knowledge Did they learn anything Check Performance Environment Level 1 – Reaction Was the environment suitable for learning? Improve Knowledge/Skill transfer Go from top down What if we did nothing? Applications of percentage – level 1, level 2, level 3 etc. Improve Learning Environment

Outcomes Used in Evaluating Training Programs: Return on Investment (ROI) Comparing the training’s monetary benefits with the cost of the training direct costs indirect costs Benefits

Return on Expectations: (ROE) Showing the value of training begins before the program even starts. The end is the beginning. Value must be created before it can be demonstrated. "Trainers must begin with desired results and then determine what behavior is needed to accomplish them. Then trainers must determine the attitudes, knowledge, and skills that are necessary to bring about the desired behavior(s). The final challenge is to present the training program in a way that enables the participants not only to learn what they need to know but also to react favorably to the program."

Achieving Targets Level of variation in product Productivity Processing Time Profit Operating Cost Rates of meeting deadlines Cost/Income ratio % of tasks incorrectly done Level of variation in product Ability to cope with circumstances Time to reach job competency levels of supervision required Frequency and costs of accidents

CIRO Model Context Input Reaction Outcomes – Immediate, Intermediate and Ultimate Outcome Cognitive outcome Skill based outcome Affective outcome

CIPP Model Content Input Process Product

Hamblin’s Model Reaction Level: It measures the reactions of the trainees to the content and methods of the training, not the trainer, and to any other factors perceived as relevant. It determines what the trainee thought about the training. Learning Level: It measures the learning attitude of the trainees during the learning period. It collects information that did the trainees learn what was intended.

Job Behavior Level: The job behavior of the trainees in the work environment at the end of the training period i.e. did the training got transferred to the job? Effect on the Department: Has the training helped the trainees in improving the department’s performance? The Ultimate Level: It measures that has the training affected the ultimate well being of the organization in terms of the business objectives.

Kaufman’s Five level Level Evaluation area 1a Enabling Quality input availability like, human and physical resources 1b Reaction Methods, means and process acceptability and efficiency 2 Acquisition Individual and small group mastery and competency 3 Application Individual and small group utilization within the organization 4 Organizational Output Payoff to the organization 5 Societal Outcomes Payoff to the society

Outcomes Used in Evaluating Training Programs: Return on Investment (ROI) Comparing the training’s monetary benefits with the cost of the training direct costs indirect costs Benefits Simple ROI = Gains – Investment costs Investment costs ROI = Learning Experience + Work Environment = Business Results

Facilities rental (taking into account the opportunity cost) Savings Cost of training Fewer errors Reduced customer turnover Less equipment downtime Increased revenue collection Faster equipment start – up time Reduced employee turnover Higher workplace morale Reduced recruitment cost Less time loss to grievance hearings Facilitators fees Training design Course Material Videos – Workbooks Facilities rental (taking into account the opportunity cost) Equipment rentals Production downtime

Factors That Influence the Type of Evaluation Design How Factor Influences Type of Evaluation Design Change potential Can program be modified? Importance Does ineffective training affect customer service, product development, or relationships between employees? Scale How many trainees are involved? Purpose of training Is training conducted for learning, results, or both? Organization culture Is demonstrating results part of company norms and expectations? Expertise Can a complex study be analyzed? Cost Is evaluation too expensive? Time frame When do we need the information?

The key lies in developing a Training Plan Training plan sets the platform for evaluating training programs.

Thank you…