1. 1 DLXXI Courseware Guidelines Workshop The University of Southern California Center for Cognitive Technology & Institute for Creative Technology Dr. Richard E. Clark, Rich DiNinni, Kenneth Yates Lesson 1

2. 2 Checklist Foundation Background research on DL learning and motivation funded by Office of Naval Research Results put in checklist form

3. 3 Workshop Goals You will learn to: Apply the checklist for DL course design review with DL contractors Catch and remedy design problems before expensive development begins Speed the design process Increase learning and transfer for soldiers and others

4. 4 Reasons for Workshop Clear and easy-to-use checklist format for your supervision of DL contractors Makes design and development phases more efficient and less costly by identifying problems at the blueprint stage Finished course will help trainees become more proficient more quickly

5. 5 Problems we hope to solve Need for rapid design of DL courses Confusion between information and training (telling is not training)  Current focus is on information but need evidence- based standards for training methods  Information must be translated into knowledge types and taught effectively  Currently, tests are often irrelevant – focused on memory not performance DL media techniques often overload trainees and prevent or slow learning

6. 6 The Foundation of this Workshop: GEL* Design 1.Goals (what trainees will be able to do) 2.Reasons (Benefits and Risks) 3.Overview (What you need to know) 4.Demonstration (of the procedure) 5.Practice and Feedback. * Guided Experiential Learning

7. 7 Instructional Strategies Show each checklist item in sequence with an example we found in the Scout Leaders Course design document You decide whether the example fits the checklist requirement(s) We show you our decision and explain We discuss different perceptions to sharpen our understanding You practice evaluating the Armor Captains Career course in teams Final “whole task” exercise with Armor Artillery Surveyor design document for USC Certification

8. 8 1. Evaluating Course and Lesson Introductions  Checklist Questions 1.1 through 1.4.3 2. Evaluating Training Design  Questions 2.1 through 2.3.6 3. Practice, Feedback, & Assessment Design  Questions 3.1 through 3.3.2 4. Evaluating Multimedia Design  Questions 4.1 through 4.4.5 Workshop Overview

9. 9 1. Course introductions and objectives ELO’s & TLO’s clearly stated in terms that describe what trainees will be able to do after training The benefit of learning and risk of avoiding Overview that relates content to prior knowledge, points out new learning Lessons sequenced as job performed or from easy to difficult Training methods are clearly described

10. 10 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.1 Are the learning objectives for the course: GONO-GO 1.1.1. Stated in words all trainees will understand about: 1.1.1.1. What they will be able to do following the training?  Objective from Scout Leader Course, page 9: “This volume will train the scout leader to understand the Opposing Force, the Intelligence Preparation of the Battlefield (IPB), and to evaluate routes and obstructions, and other critical reconnaissance skills.”

11. 11 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.1.1. Stated in words all trainees will understand about: GONO-GO 1.1.1.1. What they will be able to do following the training?  X Objective from Scout Leader Course, page 9: “This volume will train the scout leader to understand the Opposing Force, the Intelligence Preparation of the Battlefield (IPB), and to evaluate routes and obstructions, and other critical reconnaissance skills.”

12. 12 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.1.1. Stated in words all trainees will understand about: GONO-GO 1.1.1.2. The transfer setting in which they will be able to do it?  Page 9 in design document : “This volume will certify the scout leader through hands-on equipment specific training on heavy or light cavalry equipment and weapons”.

13. 13 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.1.1. Stated in words all trainees will understand about: GONO-GO 1.1.1.2. The transfer setting in which they will be able to do it?  X Page 9 in design document: “This volume will certify the scout leader through hands-on equipment specific training on heavy or light cavalry equipment and weapons”.

14. 14 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.1.1. Stated in words all trainees will understand about: GONO-GO 1.1.1.3. The speed and accuracy with which they will be required to perform it after the training?  End of ELO tests or quizzes page 33 : “Questions will be performance/scenario based. Passing score for these tests is 70%.”

15. 15 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.1.1. Stated in words all trainees will understand about : GONO-GO 1.1.1.3. The speed and accuracy with which they will be required to perform it after the training?  X End of ELO tests or quizzes – Questions will be performance/scenario based. Passing score for these tests is 70%. Feedback Check to see if speed or accuracy of skill deployment is a key issue for this training

16. 16 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.2. Are the reasons for the course and lessons stated in words that all trainees will understand about: GONO-GO 1.2.1. The benefits of the course to trainees?  Taken from online course, lesson introduction. There are many examples in history of scout platoons that have distinguished themselves in time of war. Patton’s historic change in direction to relieve the 101 st Airborne Division at Bastogne was led by rapidly deployed reconnaissance elements..

17. 17 Evaluating Course & Learning Objectives “There are many examples in history of scout platoons that have distinguished themselves in time of war. Patton’s historic change in direction to relieve the 101st Airborne Division at Bastogne was led by rapidly deployed reconnaissance elements.” GuidelineMeets Guideline? 1.2. Are the reasons for the course and lessons stated in words that all trainees will understand about: GONO-GO 1.2.1. The benefits of the course to trainees? X 

18. 18 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.2. Are the reasons for the course and lessons stated in words that all trainees will understand about: GONO-GO 1.2.2. The risks that trainees take if they do not complete the course?  Taken from online course, lesson introduction : Many platoons are not prepared to perform their mission effectively because of inadequate planning. To prevent this from happening, the reconnaissance platoon must be proficient in troop- leading procedures.

19. 19 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.2. Are the reasons for the course and lessons stated in words that all trainees will understand about: GONO-GO 1.2.2. The risks that trainees take if they do not complete the course? X  Taken from online course, lesson introduction: Many platoons are not prepared to perform their mission effectively because of inadequate planning. To prevent this from happening, the reconnaissance platoon must be proficient in troop- leading procedures.

20. 20 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.3. Do the course and lesson overviews:GONO-GO 1.3.1 Relate the content to the trainees’ course- relevant prior knowledge?  Taken from Page 16 in the design document References to prior learningConnects new knowledge with previously gained learning.

21. 21 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.3. Do the course and lesson overviews:GONO-GO 1.3.1 Relate the content to the trainees’ course- relevant prior knowledge? X  Taken from Page 16 in the design document References to prior learningConnects new knowledge with previously gained learning.

22. 22 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.3. Do the course and lesson overviews: GONO-GO 1.3.2. Point out new or unusual elements of what trainees are learning?  Design document, page 47: “Identify operations under unusual conditions.”.

23. 23 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.3. Do the course and lesson overviews: GONO-GO 1.3.2. Point out new or unusual elements of what trainees are learning?  X Design document, page 47: “Identify operations under unusual conditions”.

24. 24 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.4. Overview and sequence of lessons – does the course begin with an outline of the lessons in which the sequence of the lessons is either GONO-GO 1.4.1. In the order in which the job of mission relevant tasks are accomplished?  From Online Course: The troop-leading process consists of eight steps, which are discussed in this lesson in the traditional order. The process, however, is not rigid, and the steps are not necessarily sequential. The tasks involved in some steps (such as initiate movement, issue a warning order, and conduct reconnaissance) may recur several times during the process. Although listed as the last step, activities associated with supervising and refining the plan and other preparations occur throughout the troop-leading process.

25. 25 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.4. Overview and sequence of lessons – does the course begin with an outline of the lessons in which the sequence of the lessons is either GONO-GO 1.4.1. In the order in which the job of mission relevant tasks are accomplished?  X From Online Course: The troop-leading process consists of eight steps, which are discussed in this lesson in the traditional order. The process, however, is not rigid, and the steps are not necessarily sequential. The tasks involved in some steps (such as initiate movement, issue a warning order, and conduct reconnaissance) may recur several times during the process. Although listed as the last step, activities associated with supervising and refining the plan and other preparations occur throughout the troop-leading process.

26. 26 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.4. Overview and sequence of lessons – does the course begin with an outline of the lessons in which the sequence of the lessons is either GONO-GO 1.4.2 If there is no necessary sequence, in an order beginning with less difficult tasks and progressing toward more difficult tasks? 

27. 27 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.4. Overview and sequence of lessons – does the course begin with an outline of the lessons in which the sequence of the lessons is either GONO-GO 1.4.2 If there is no necessary sequence, in an order beginning with less difficult tasks and progressing toward more difficult tasks?  X

28. 28 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.4. Overview and sequence of lessons – does the course begin with an outline of the lessons in which the sequence of the lessons is either GONO-GO 1.4.3. Does each lesson provide an overview of the position and importance of the lesson in the overall training environment, and then describe the instructional strategies that will be used in the course?  See next slide for overview and instructional strategy description from design document

29. 29 Evaluating Course & Learning Objectives Scout Leaders Course Map - Phase 1 (DL) Course Introduction Troop-Leading Procedures Opposing Forces in a Contemporary Operational Environment Intelligence Preparation of the Battlefield Evaluate Routes and Obstructions Calculate, Designate, and Employ Demolitions Reconnaissance and Security Urban Operations StrategyRationale Text/readingNarrative explanations, examples, and doctrinal references. Practical exercise/scenariosAllows for application of knowledge-based instruction. Drill (to be called ‘Obstacles’ in this course)Reinforces material. Examples/analogies/ anecdotes/stories Clarifies and amplifies new material within a familiar context. References to prior learningConnects new knowledge with previously gained learning.

30. 30 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.4. Overview and sequence of lessons – does the course begin with an outline of the lessons in which the sequence of the lessons is either GONO-GO 1.4.3. Does each lesson provide an overview of the position and importance of the lesson in the overall training environment, and then describe the instructional strategies that will be used in the course? X 

31. 31 Team Exercise #1 Evaluating Introductions Each team selects:  A Team Leader (coordinator, timekeeper)  A Reporter (summarizes and presents teamwork to rest of us) Assign responsibility to find examples of each introduction item (1.1 to 1.4.3) in the design for the Armor Captains Career Course Report those examples to the rest of the workshop when it is your turn

32. 32 1.1 Are the learning objectives for the course & lessons: 1.1.1. Stated in words that all trainees trainees will understand about: 1.1.1.1. What they will be able to do following the training? 1.1.1.2. The transfer setting in which they will be able to do it? 1.1.1.3. The speed and accuracy with which they will be required to perform it after the training? 1.2. Are the reasons for the course and lessons stated in words that all trainees will understand about: 1.2.1. The benefits of the course 1.2.2. The risks they take if they do not complete the training 1.3. Do the course and lesson overviews: 1.3.1. Relate the content to trainees’ course-relevant prior knowledge? 1.3.2. Point out new or unusual elements of what trainees are learning? Find Examples for All Numbered Items

33. 33 1.4. Overview and sequence of lessons – does the course begin with an outline of the lessons where the sequence is either: 1.4.1. In the order that job or mission-relevant tasks are accomplished, or 1.4.2. If there is no necessary sequence, in an order beginning with less difficult tasks and progressing toward more difficult tasks? 1.4.3. Does each lesson provide an overview of the position and importance of the lesson in the program, and then describe the instructional strategies that will be used in the course Evaluating Course and Lesson Introductions

34. 34 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.1 Are the learning objectives for the course: GONO-GO 1.1.1. Stated in words all trainees will understand about: 1.1.1.1. What they will be able to do following the training? 

35. 35 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.1.1. Stated in words all trainees will understand about: GONO-GO 1.1.1.2. The transfer setting in which they will be able to do it? 

36. 36 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.1.1. Stated in words all trainees will understand about: GONO-GO 1.1.1.3. The speed and accuracy with which they will be required to perform it after training? 

37. 37 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.2. Are the reasons for the course and lessons stated in words that all trainees will understand about: GONO-GO 1.2.1. The benefits of the course to trainees? 

38. 38 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.2. Are the reasons for the course and lessons stated in words that all trainees will understand about: GONO-GO 1.2.2. The risks that trainees take if they do not complete the training? 

39. 39 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.3. Do the course and lesson overviews:GONO-GO 1.3.1 Relate the content to the trainees’ course- relevant prior knowledge? 

40. 40 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.3. Do the course and lesson overviews: GONO-GO 1.3.2. Point out new or unusual elements of what trainees are learning? 

41. 41 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.4. Overview and sequence of lessons – does the course begin with an outline of the lessons in which the sequence of the lessons is either GONO-GO 1.4.1. In the order in which the job of mission relevant tasks are accomplished? 

42. 42 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.4. Overview and sequence of lessons – does the course begin with an outline of the lessons in which the sequence of the lessons is either GONO-GO 1.4.2 If there is no necessary sequence, in an order beginning with less difficult tasks and progressing toward more difficult tasks? 

43. 43 Evaluating Course & Learning Objectives GuidelineMeets Guideline? 1.4. Overview and sequence of lessons – does the course begin with an outline of the lessons in which the sequence of the lessons is either GONO-GO 1.4.3. Does each lesson provide an overview of the position and importance of the lesson in the overall training environment, and then describe the instructional strategies that will be used in the course? 