1. Barry Williams1 Designing & Conducting Formative Evaluation Dick & Carey Chp. 10

2. Barry Williams2 Formative Evaluation - Why? To determine Weakness(es) in the Instruction To determine Weakness(es) in the Instruction

3. Barry Williams3 Weaknesses Teachers/Trainers are not good predictors of the effectiveness of instructional content (Rosen, 1986, Rothkipf, 1963) Instructional Theory is not a perfect science Few instructional products are ever evaluated with learners Designer can’t see the forest for the trees

4. Barry Williams4 Formative Evaluation Has Six Stages Design Review Expert Review One-To-One Small Group Field Trials Ongoing Evaluation

5. Barry Williams5 Design Reviews (Ask these questions) Does the instructional goal match the problem identified in the needs assessment? Does the learner & environmental analysis match the audience? Does the task analysis include all the prerequisite skills? Are the test items reliable and valid, and do they match the objectives?

6. Barry Williams6 Expert Review Is the content accurate & up-to-date? Does it present a consistent perspective? Are examples, practice exercises, & feedback realistic & accurate? Is the pedagogy consistent with current instructional theory? Is the instruction appropriate to the audience?

7. Barry Williams7 One-On-One Review Is the message clear? What is the impact on: –learner attitudes –achievement of objectives & goals Feasibility of training

8. Barry Williams8 Small Group Review Look for the effects caused by the changes made in the one-to-one review Identify any remaining learning problems

9. Barry Williams9 Field Trials (Review) Look for effects in changes made in small group Can the instruction be used in the context in which it was intended

10. Barry Williams10 Ongoing Evaluation Viewed through the Lens of: Project Size Life span of content Audiences change One-To-One Small Group Tryouts Field Trials

11. Barry Williams11 Learner Evaluation “Testing” Do learners understand the instruction? Do they know what to do during the practice & the tests? Can they interpret graphics in the text? Can they read all the material? How much time does it take?

12. Barry Williams12 Chpt. Summary Formative Evaluation is critical Complete as many stages as practical Don’t depend on SME’s opinion Document your evaluation