1. THE POWER OF QUIZZING Mark A. McDaniel Washington University NCAT Meeting, March 24, 2009

2. Kaplan Test Prep and Admissions Survey Eat Alaskan salmon for breakfast Watch back-to-back episodes of ER Wear Superman underwear

3. Laboratory research: Taking a test can serve as a learning opportunity, and enhance memory retention even to a greater extent than additional studying… the “testing effect” Central implication for course redesigns: In college classes, low or no stakes quizzes (tests) can be used to enhance student learning (quizzes not just a tool to reveal student mastery)

4. Wildman, McDaniel & Anderson (2009) In WEB-based Brain and Behavior university course, each week students covered one chapter (40 pages) 14 central facts from chapter presented to students in one of three ways (as an assignment): Read the facts to review Multiple choice quiz on the facts Short answer (fill in blank) quiz on the facts 14 other central facts not presented Summative Assessments (course exams): 84 Item Multiple Choice Exams over Three Chapters

5. Experiment 1 1.Summative tests manipulated match between quiz stem and test stem: Quiz Stem: Touch receptors (for nocioception and hapsis) on the skin have their cell bodies in ____________ and connect in the spinal cord to ascending spinothalamic cells. Exam Stem: Same Touch receptors (for nocioception and hapsis) on the skin have their cell bodies in ____________ and connect in the spinal cord to ascending spinothalamic cells. A. dorsal root ganglia, B…., C…, D….. Different Touch receptors (for nocioception and hapsis) on the skin have their cell bodies in the dorsal root ganglia and connect in the spinal cord to ___________. A……, B……,C……, D. ascending spinothalamic cells

6. 2. Quizzes presented on web Ungraded Correct answer feedback provided 3. Repeated quizzing: students required to take quizzes four times. (Maximum credit for quiz taking was earned only by taking quizzes four times)

7. Unit Test Performance SA = Short answer quiz, MC = Multiple choice quiz, QR = reading, NX = no-exposure

8. SA = Short answer quiz, MC = Multiple choice quiz, QR = reading, NX = no-exposure

9. Experiment 2 Can These Robust Test-Enhanced Learning Effects Be Replicated and Extended to condensed course format? Condensed summer-school course (7 weeks) Individual students could start late and further condense the course (essentially complete course in 3-4 weeks) Similar to Experiment 1 Summative tests manipulated match between quiz stem and test stem: Different or Same Students required to take quizzes four times (Maximum credit for quiz taking was earned only by taking quizzes four times)

10. Unit Test Performance SA = Short answer quiz, MC = Multiple choice quiz, QR = reading, NX = no-exposure

11. SA = Short answer quiz, MC = Multiple choice quiz, QR = reading, NX = no-exposure

12. Summary Repeated quizzing produces robust learning effects on classroom summative assessments Multiple choice quizzes are as effective as short-answer quizzes, when summative assessment is multiple choice and quizzes repeated With same stems on quizzes and final test, quizzing produces better performance than presenting target content for additional study

13. Can quizzing enhance richer learning of conceptual information (application of content)? Will application quiz items promote learning of related factual content?

14. Short-answer quizzes (web-based course) Factual What is the most complex human behavior? Applied If an otherwise normal child does not start talking until after his peers, what do we expect about how long it might take this child to learn to speak fluently?

17. Summary Quizzing (short-answer) can promote transfer—it enhances performance on related questions

18. Does Test-Enhanced Learning Extend to Typical Lecture Courses and Very Able Learners? Washington University Introductory Psych Class (summer) Intervening quizzing: short answer quizzed not quizzed Exam question types: Multiple Choice, Short Answer McDaniel and Sun (2008)

19. 12 Short answer items (12 Non-quiz items) QUIZ SENT BY E-Mail (2/week) UNIT EXAM Mult. choice (16): 8 same wording and 8 different wording from quiz Short answer (8): 4 same wording and 4 different wording from quiz End of Week Class lecture; reading assignment N=21 Daily

21. IS TEST-ENHANCED LEARNING PERSISTENT OVER LONG DELAYS? Columbia Middle School project 6 th, 7 th, and 8 th grade science classes Some facts on each lesson are quizzed 3 times; comparable facts are not quizzed Consider recall on tests at ends of chapters, end of the semester and end of the year

22. COLUMBIA MIDDLE SCHOOL PROJECT 8 TH GRADE SCIENCE

23. Can learner initiated quizzes (retrieval) enhance learning (when course constraints preclude any quizzing)?

24. Learner controlled testing Read (study) Recite (recall material to oneself) Review (reread the material)

25. McDaniel, Howard, & Einstein (2009, Exp. 2) Three study conditions: –Read, recite, review –Note-taking –Read-twice control Each participant read two complex, ~900 word articles –Brakes & Pumps, as in Mayer & Gallini (1990)

26. Students were tested on one of the articles immediately After a one-week delay, students were tested on both articles Testing consisted of: –Free recall –Multiple choice (4 alternative) –Short-answer inference questions

27. **

28. *

29. ** *

30. Summary Participants who used the 3R strategy performed significantly better on a range of classroom type tests relative to reading twice or note-taking (excluding immediate inference performance) 3R more efficient (takes less time) than note-taking for enhancing problem solving/inference

31. Summary of major findings Basic test-enhanced learning demonstrated in college courses and classrooms Found even when learning at high levels. Similar learning benefits were not always found from additional exposure per se Self-testing is effective for learning

32. Many reasons to expect tests to confer learning benefits in educational contexts Retrieval itself produces retrieval practice Test itself requires student to engage in the material (Sikorsky et al., 2001) Signals information to target for study. (improves metamemory—both accuracy of what you know and control of study) Leads to more (and spaced) studying—keep up with the material better (Johnson & Kiviniemi, 2009)

33. Acknowledgements Funding Institute of Education Sciences Grant #R305H030339 James S McDonnell Collaborative Activity Grant