2. Introducing Desirable Difficulties for Educational Applications in Science Robert A. Bjork University of California, Los Angeles Marcia C. Linn University of California, Berkeley www.psych.ucla.edu/iddeas

3. “Desirable Difficulties” Spacing rather than massing study Interleaving rather than blocking practice on separate topics or tasks Varying contextual cues Reducing feedback to the learner Testing rather than re-presenting

4. Learning versus performance What we can observe is performance, what we must infer is learning… –and the former is an unreliable index of the latter Instructors are, therefore, susceptible to choosing less-effective conditions of learning over more-effective conditions And as learners, we, too, are susceptible to confusing performance with learning

5. Generation Interleaving Spacing

6. Goals of the IDDEAS project Do such findings extend to to-be-learned materials and retention intervals that are realistic from an educational standpoint? And, more broadly, what design principles are fundamental in optimizing educational materials and practices?

7. WISE (web-based inquiry science environment): http://wise.berkeley.eduhttp://wise.berkeley.edu Advantages as a tool for teachers –Supports authoring and customization –Contains a library of tested projects –Enables collaborative learning, visible thinking, autonomous investigation –Transportable Advantages as an IDDEAS research tool

8. Current Projects WISE Platform 2 laboratory studies, UCLA 2 classroom studies, UCB Design Principles 2 studies, UCLA

10. Interleaving Motor tasks: patterns, force production, bank machine transactions (Lee & Magill, 1983, Simon & Bjork, 1990 Charles et. al, 1990, Jamieson & Robers, 2000) Sports: badminton, volleyball, baseball (Bortoli et al, 1992, Goode & Magill, 1986, Hall et al, 1994) Abstract learning tasks: mazes, tracking (Carleson et.al, 1989, Jelsma & Van Merrienboer, 1989, Jelsma & Pieters, 1989) Logic rules, boolean operators (Schneider et al, 1995, Carleson & Yaure, 1990)

12. Example of generation effects

13. Study 1: Overview Uses an existing WISE astronomy module: –How scientists determine the habitability of planets Design:4 groups –Mass and distance slides either blocked or interleaved; –Some evidence re-studied via generation or reading Forty-eight hour retention interval Posttest

14. Posttest questions Simple sentence-completion: Information presented and re-studied via generation or reading, or only presented –E. g. “The amount of heat and light emitted by the sun in our solar system has increased by ____% since the beginning of earth’s history.” Integration questions across mass slides or across distance slides –E. g., “Would an object weigh more on the planets in our solar system made mostly of gas or made mostly of rock? Why? ” Integration questions across mass and distance slides –E. g., “Imagine a planet that is smaller than Earth and that was located 1.5 AUs from its sun, which is the same strength as the Earth’s sun. How would this planet’s potential for life compare to Earth’s?”

15. Figure 1. UCLA Study 1 - Interleaving by type of Posttest Integration Question.

16. Figure 2: UCLA Study 1 -Generation effects on single fact materials

17. Generation manipulations Study 1 (UCLA): Sentence Completion –Generate: ____-type planets are mostly made up of gases. –Read: Jovian-type planets are mostly made up of gases. Study 2, (UCLA; and UCB Classroom Studies 1 and 2): – Sentence level generation E.g., “Describe in a sentence how the size of one planet's mass can affect another planet.” –Knowledge required for successful generation: Mass or distance Only Both mass and distance

18. Projects in Progress using WISE Interactions between Generation & Interleaving (UCLA and Classroom) Integration of M + D Integration of M or D Increase Contextual Interference (UCLA and Classroom) Habitability and Detectability Visual Support for Generation (Classroom) Static versus Animated

19. Design Principle: Interleaving Interleaving as spacing: Practical advantages But are effects of interleaving more than the effects of spacing? Contextual interference ideas (Battig, 1972, 1979) –Benefits dues to: Reloading/ Reconstruction (e.g. Lee & Magill, 1983, 1985) And/or Development of a higher order representation to differentiate interleaved materials (Shea & Zimny, 1983, 1988)

20. Disentangling Interleaving and Spacing Initial experiment with second-language materials (courtesy of Hal Pashler): Designed to: –Maximize contextual interference –Co-vary (i.e., unconfound) interleaving and spacing

21. Learning Materials English word EstonianSwahili “bird” “dege” “lind” -8 English words learned in both foreign languages (16 word pairs total) -6 repetitions (anticipation trials) of each word pair

22. What is tree in Swahili? ?: The correct word is: mfufumaji mufuma Sample anticipation-method learning trial

23. What is tree in Swahili? ?: in Estonian? ?: Sample Test Trial

27. Implications and necessary next steps Interleaving can cause intrusions and errors during learning that then foster long-term retention--and, possibly, transfer The effects of interleaving and spacing –May be independent and additive, –But contextual interference (competition) maybe be necessary to demonstrate benefits of interleaving that go beyond the benefits of spacing And, from an educational standpoint, it is essential to see whether the same pattern obtains with more complex and cumulative materials

28. Our thanks to  The Institute for Education Sciences and the Cognition and Student Learning program  Other members of the IDDEAS research team: –Lindsey E. Richland, Ph.D, (UCLA) –Britte H. Cheng (UC Berkeley) –Jason R. Finley (UCLA) –And a number of undergraduate students, especially Jeff Beyers, Fernando Cervantes, and Alexandra Hessenius Relevant Links –IDDEAS : http://www.psych.ucla.edu/iddeas –WISE : http://wise.berkeley.edu –SCALE: (Synergy Communities: Aggregating Learning about Education) - http://scale.soe.berkeley.edu –TELS: (Technology Enhanced Learning in Science) - http://www.telscenter.org