1. Strategic Planning for Training Evaluation
Presented by:
Nita C. Lescher, M.S.W., L.I.C.S.W.
American Humane Association
Briefly discuss AHA’s experience with training evaluation. Go around the room,
ask participants to introduce themselves and explain their experience with training
evaluation.

2. Before, During, and After Training
When to Evaluate?
Before, During, and After
Training
Introduce topic, explain that evaluation is important, not just as an overall topic,
but also as concrete activities to be conducted before, during, and after training.
Explain that we will discuss the purposes for each phase of evaluation next.

3. Why Evaluate? Before Training Identify needs or problems.
Identify specific job competencies to close gaps.
Identify the type of evaluation needed.
Be sure to ask trainees, What can evaluation before training do for your
organization? Try to generate these responses, if not given go over.

4. During Development of Training
Why Evaluate?
During Development of Training
Determine which staff should be the targets of training.
Provide feedback to curriculum designers on content, methods, and materials.
Determine if course delivery is consistent and follows curriculum specifications.
Be sure to ask trainees, What can evaluation during the development of training
do for your organization? Try to generate these responses, if not given
go over.

5. Why Evaluate? After Training
Note changes in participants’ attitudes and reactions to the course.
Assess trainees’ mastery of knowledge.
Assess trainees’ mastery of skills.
Assess transfer of new knowledge and skills to the job.
Measure whether or not performance gaps have begun to close.
Measure relationship of training to agency goals and client outcomes.
Be sure to ask trainees, What can evaluation after training do for your
organization? Try to generate these responses, if not given go over. Explain that
this category is typically what organizations think about when evaluation
is even thought about in the first place.

6. Strategic Plan for Training Evaluation
Why do it?
Specify clear relationships among agency goals, resources, purposes of training evaluation and decisions about what to evaluate.
Ask, what is the purpose of strategic planning for training evaluation. Provide this definition.

7. Factors to Consider Purpose of training Cost of evaluation
Centrality of subject matter to competent job performance
Consequences of poorly developed/delivered training
Number of staff to be reached
Length of course
Legal mandates, public and political pressures
Agency goals
Go over these factors. Explain that they will help decide on the components
of a strategic plan. Ask participants, anything else?

8. AHA Training Evaluation Model
The famous “Apple Model” for a strategic plan for training evaluation.

9. Apple Peel Levels Level 1 = Course Evaluation of the training: Content
Structure
Methods
Materials
Delivery
Go over each level. Explain the purpose of each level.
Ask participants, What experience have you had with this level?

10. Level 2 = Satisfaction (Happiness Sheets)
Apple Peel Levels
Level 2 = Satisfaction (Happiness Sheets)
Assesses trainees’ feelings about:
Trainer
Quality of material presented
Methods of presentation
Environment (e.g., room temperature)
This is the most common method for training evaluation.
Ask participants, What experience have you had with this level?

11. Apple Peel Levels Level 3 = Opinion
Assesses trainees’ attitudes/perceptions about:
Utilization of training
Own learning
Training’s value
Most higher level evaluation is done at this level.
Ask participants, What experience have you had with this level?

12. Apple Peel Levels Level 4 = Knowledge Acquisition
Assesses learning and recalling terms, definitions, and facts
Level 5 = Knowledge Comprehension
Assesses understanding concepts and relationships, recognizing examples in practice, and problem solving
Ask participants, What experience have you had with this level?
Ask for examples.

13. Level 6 = Skill Demonstration
Apple Peel Levels
Level 6 = Skill Demonstration
Assesses application of learned material to new and concrete situations within the training environment
Level 7 = Skill Transfer
Assesses application of new knowledge and skills outside the classroom and on the job
Most organizations may strive for this level, but it is expensive and the effects
of training may be confounded by other factors. However,
evaluation at this level will tell you whether training made a difference.
Ask participants, What experience have you had with this level?
Ask for examples.

14. Level 9 = Client Outcomes Level 10 = Community Impacts
Apple Peel Levels
Level 8 = Agency Impact
Level 9 = Client Outcomes
Level 10 = Community Impacts
Assesses impact of training at agency, client, and community levels
Ask participants, What experience have you had with this level?
Ask for examples.
Ask participants:
Why is it difficult to evaluate at this level? Possible answers:
Cost
Difficulty of isolating effects
Lack of support from agency

15. Model for Successful Training Course Evaluation
Step 1: Identify the purposes of the evaluation.
Step 2: Determine the levels of evaluation needed.
Step 3: Develop an evaluation design.
Step 4: Select or develop instruments.
Step 5: Develop a data collection plan.
Step 6: Pilot and revise instruments and procedures.
Step 7: Collect evaluation data.
Step 8: Analyze evaluation data.
Step 9: Report results.
Step 10: Provide feedback/disseminate results.
Go over these steps. Explain that it is a linear process that requires a commitment
to the process.
When discussing Step 3, refer participants to Handout
“Sample Designs for Common Types of Evaluation”. (Need to find electronic
version or have this handout scanned.)