Presentation is loading. Please wait.

Presentation is loading. Please wait.

Kirkpatrick Four Level Evaluation Certification Program Presented Jim Kirkpatrick, PhD Hosted by Booz Allen Hamilton Herndon, VA November 23-24, 2009.

Similar presentations


Presentation on theme: "Kirkpatrick Four Level Evaluation Certification Program Presented Jim Kirkpatrick, PhD Hosted by Booz Allen Hamilton Herndon, VA November 23-24, 2009."— Presentation transcript:

1 Kirkpatrick Four Level Evaluation Certification Program Presented Jim Kirkpatrick, PhD Hosted by Booz Allen Hamilton Herndon, VA November 23-24, 2009

2 These slides are provided as a courtesy to those who have attended one of our classes or presentations. These are for internal use within your organization only; all rights reserved. We thank you in advance for respecting our intellectual property and refraining from duplicating, distributing, selling, or otherwise using this presentation in whole or part for your own profit or promotion. The following marks are the property of Kirkpatrick Partners, LLC : The Kirkpatrick Business Partnership Model SM, KBPM SM, Return on Expectations SM, ROE SM, and Chain of Evidence SM

3 Kirkpatrick Foundational Principles 1.The end is the beginning. 2.ROE SM is the ultimate indicator of value. 3.Business partnership is necessary to bring about positive ROE. 4.Value must be created before it can be demonstrated. 5.A compelling Chain of Evidence SM demonstrates your bottom line value. ©2009, all rights reserved.

4

5

6

7 Kirkpatrick Four Levels Level 4: RESULTS The final outcomes that occur as a result of the training and reinforcement ©2009, all rights reserved.

8 Kirkpatrick Four Levels Level 3: BEHAVIOR The extent to which participants apply what they learn on the job ©2009, all rights reserved.

9 Kirkpatrick Four Levels Level 2: LEARNING The extent to which participants acquire intended knowledge, skills, and attitudes ©2009, all rights reserved.

10 Kirkpatrick Four Levels Level 1: REACTION How training participants react to the training ©2009, all rights reserved.

11 Why Evaluate? How can the program be improved? How can we maximize training effectiveness by reinforcing knowledge, skills, and attitudes? How can we demonstrate the value of training? ©2009, all rights reserved.

12 “What is your job here at the hotel?” “I am a window washer.” ©2009, all rights reserved.

13 “What is your job here at the resort?” “I am part of a team that creates great experiences for our guests!” ©2009, all rights reserved.

14 LearningResultsBehaviorReaction The Kirkpatrick Business Partnership Model SM ROE SM Business need identified Measure L1 Reaction Measure L2 Learning Measure L3 Behavior Present L1 Reaction findings Present L2 Learning findings Present L3 Behavior findings Present L4 Results findings Prepare Chain of Evidence SM to demonstrate ROE TARGET critical behaviors and required drivers Determine required KSAs, Learning Objectives REFINE expectations to define outcomes Analyze findings, adjust, repeat steps as necessary Consider necessary learning environment Design and build learning program and evaluation tools Initiate ongoing reinforcement and monitoring Measure L4 Results Deliver learning program ADDRESS jury issues Identify NECESSITIES For Success PLEDGE to work together © All rights reserved.

15 Critical Behaviors Business Results Recognition Coaching Refreshers Mentoring Accountability Level 3 Evaluation Executive Modeling Drivers – encouragers or discouragers ©2009, all rights reserved.

16 DOT Example 1.Jury – safety managers, district engineers, traffic control, risk managers, maintenance supervisors. 2.Expectations – smooth traffic flow; increased safety for workers and motorists in highway work zones; better planning. 3.Success Outcomes – less than 15 minute delays; reduction in injuries and fatalities; work zone projects completed on time and within budget. 4a. Key New Behaviors – written work zone plans; plan communicated to public; proper work zone set-up; flaggers following procedures. 4b. Success Drivers – observation, feedback and coaching by supervisors; ongoing compliance tracking; ongoing execution of formal and informal recognition programs. ©2009, all rights reserved.

17 Methods, Tools, and Tips Evaluation Methods – the process that is used to gather evaluation data Evaluation Tools – the actual document that is utilized to gather the evaluation data and/or information Evaluation Tips – what you will do to gain elicit the desired candor and response rates ©2009, all rights reserved.

18 Guidelines for Evaluating Reaction Determine first what you want to find out Consider a blend of objective and subjective questions Consider your rating scale If indicated, get delayed reactions Develop with the learner in mind Set the table for your learners ©2009, all rights reserved.

19 WHO HOW? WHEN? WHO? WHAT? WHY? Level 1 Reaction Sheets Interviews Focus Groups ©2009, all rights reserved.

20 Guidelines for Evaluating Learning Start with the issues of usefulness and credibility Measure before and after attitudes, knowledge, and/or skills if indicated Beware of an overemphasis on assessing knowledge Evaluate during the course as well as after Use a performance test for skills ©2009, all rights reserved.

21 WHO HOW? WHEN? WHO? WHAT? WHY? Level 2 Knowledge Checks Knowledge Tests Skills Observation Refreshers Teach Backs Developing Action Plans Case Studies ©2009, all rights reserved.

22 Guidelines for Evaluating Behavior Start with need and importance Be sure to set the table for this Don’t overwhelm – more is not better Measure on a before/after basis if indicated Consider immediate implementation Repeat if indicated Consider more than one source Answer the ?s, “If so, why?” and “If not, why not?” ©2009, all rights reserved.

23 WHO HOW? WHEN? WHO? WHAT? WHY? Level 3 Behavior Survey Behavior Checklist Work Review Focus Groups Monitor Action Plans Action Learning ©2009, all rights reserved.

24 Guidelines for Evaluating Results Determine based on responses from stakeholders – ROE SM Borrow metrics when possible Gather data and information when indicated Consider a tiered approach (short and long term measures) Consider evidence versus proof Don’t make it harder than necessary ©2009, all rights reserved.

25 WHO HOW? WHEN? WHO? WHAT? WHY? Level 4 Borrowing Metrics –Business –HR Surveys Focus Groups ©2009, all rights reserved.

26 Evaluation Methods Evaluation Levels Methods 1 Reaction 2 Learning 3 Behavior 4 Results Survey●●●● Questionnaire / Interview●●●● Focus Group●●●● Knowledge Test / Check● Case Studies● Work Review●● Skills / Behavior Observation●● Presentation / Teach Back● Action Planning●●● Action Learning● Key Business HR Metrics● ©2009, all rights reserved.

27 1 Post Session L1 Level 1 Level 2 Level 3 Level 4 Was session delivered effectively? Post Session Reinfcmt, Coaching & L3 Expected results will occur. Build your chain of evidence & showcase Post Session L2 Did targeted learning occur? Was there successful transfer of learning to behavior? Improve Reinforcement and/or Coaching NO YES NO YES Improve Session Delivery of Learning Event L1 & L2 Formative Evaluation Was failure due to session? Kirkpatrick Four Level Evaluation Feedback Loop to Eliminate ‘Snags’ ©2009, all rights reserved.

28 Percentage of learning transfer 1975: Percentage of formal learning that is actually applied to the job: 15% 2005: Percentage of formal learning that is actually applied to the job 15% Dana Robinson, ASTD ICE, 2008 ©2009, all rights reserved.

29 Brinkerhoff Study Josh Bersin and Associates, 2008 ©2009, all rights reserved.

30 Causes of “Training Failure” 2006 ASTD Study ©2009, all rights reserved.

31 Typical Learning Investment Dr. Brent Peterson, University of Phoenix, 2004 ©2009, all rights reserved.

32 The increase in “informal learning” Josh Bersin and Associates, 2008 ©2009, all rights reserved.

33 Transferring Learning to Behavior Balancing two major forces: ACCOUNTABILITY SUPPORT ©2009, all rights reserved.

34 Data Collection Plan LevelMethodsToolsTiming 1 Surveying Focus Group #1 Reaction Sheet F.G. Questionnaire Immediately after course 1 week after course 2 Knowledge test Skills Observation Teach backs Focus Group #1 (see above) Objective test Checklist Instructions F.G. Questionnaire Pre and post course During course 1 week after course 3 On-the-job Observation Surveying – 180 Focus Group #2 Action learning Checklist Survey (partic / mrg.) F.G. Questionnaire Instructions 1 week + monthly 4 months after course 6 months after course 4 Survey – Customer Focus Group #2 (see above) Review Bus. & HR Metrics Survey F.G. Questionnaire Business & HR Metrics 4 & 8 months after course 4 months after course 6 &12 months after course ©2009, all rights reserved.

35 Chain of Evidence SM Gather data at all four levels and show that your training delivers true value to your organization Level 1 Reaction Level 2 Learning Level 4 Results Level 3 Behavior ©2009, all rights reserved.

36 Four Practical Ways to Make a Good Business Case 1.Show them the data 2.Conduct an impact study and showcase it 3.Review your training curriculum and redeploy your resources 4.The Brunei Window Washer ©2009, all rights reserved.

37 More Information kirkpatrickpartners.com Feb. 2010


Download ppt "Kirkpatrick Four Level Evaluation Certification Program Presented Jim Kirkpatrick, PhD Hosted by Booz Allen Hamilton Herndon, VA November 23-24, 2009."

Similar presentations


Ads by Google