1. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. How does testing affect learning and retention – for good or ill? HEA Self-assessment seminar 11 June 2012 John Kleeman (@johnkleeman)@johnkleeman Questionmark Chairman www.questionmark.com

2. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. A. Revise by looking at notes B. Take a self-assessment quiz C. Both will help the same D. Neither will help To prepare best for an exam in a hours time, should you? Slide 2

3. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. A. Revise by looking at notes B. Take a self-assessment quiz C. Both will help the same D. Neither will help To prepare best for an exam in a weeks time, should you? Slide 3

4. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Testing has good and ill effects, how do we measure the overall impact?

5. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Confirm when topic learned Identify weak areas that need study Feedback corrects misconceptions Engage and encourage learners Give instructors feedback Gives retention benefits from spacing out learning 6 key benefits of testing Slide 5 Learn

6. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Focus learners to cram for test not retain for long term Learners to learn for test not what is useful Some say testing hurts creativity Poor scores and poor tests can demotivate learners We dont have time for testing (or self-assessment) 5 indirect concerns for testing Slide 6

7. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Some psychology experiments that give evidence not just opinion Group AGroup B Slide 7 Learn in another way Wait a period Measure learning Learn in one way Wait a period Measure learning Is learning improved?

8. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Francis Bacon, 1620 If you read anything over twenty times, you will not learn it by heart so easily as if you were to read it only ten, trying to repeat it between whiles, and when memory failed, looking at the book. Retrieval helps you learn Slide 8

9. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. 120 students divided into groups randomly Given reading comprehension passages used in TOEFL Some students Study for 14 mins Some students Study for 7 mins Recall material for 7 mins Compared results after 5 mins and 1 week Roediger & Karpicke experiment 2006 Slide 9

10. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. How much learners retained Slide 10

11. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. 80 students in 4 groups, studying a science text Study Repeated Study Concept Mapping (mind maps) Retrieval Practice Surveyed learners on expectation of retention Tested a week later to see how much retained Karpicke & Blunt experiment 2011 Slide 11

12. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. How much learners retained Slide 12

13. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Larson experiment 2009 Quizzes reduce forgetting curve In real-life, not in the laboratory Medical residents (doctors in training) 20 doctors in each group Quiz groupRestudy group Day 1Study Day 1QuizRe-study 2 weeks laterQuizRe-study 2 weeks laterQuizRe-study 6 months laterFinal exam

14. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Part of the results Slide 14

15. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Retrieval practice helps retention of learning Applies to facts And also concepts Retrieval practice includes Recall Formative quizzes/tests Self-assessment Timing of retrieval Works best when spaced out Strongest effect when retrieval is difficult Feedback very helpful Repetition of retrieval Research suggests 5-7 spaced repetitions helpful Retrieval practice key findings Slide 15

16. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Question types: Recall (e.g. short answer) best Multiple choice work too Learners do not realize they will forget Quizzes encourage/force retrieval practice Negatives Very minor effect: retrieval can cause forgetting of unretrieved material Choosing the wrong answer can retain wrong learning: putting in feedback removes this negative Taking a test directly aids retention Slide 16

17. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. How does it balance up? Slide 17 Testing & self-assessment used right aids learning and retention Concerns Other benefits Direct retrieval benefit

18. Copyright © 1995-2012 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged. Ledyard Tucker, inventor of the Angoff method (1953): the impact of electronic computing on testing is a matter for the future. It will depend on the fertility of our imaginations. What do you think? Slide 18