1. Self-regulated learning: Heuristics and illusions Robert A
Self-regulated learning: Heuristics and illusions Robert A. Bjork University of California, Los Angeles
Panel on Does Knowing What You Know Improve Study Habits and Learning?
Fifth Annual Research Conference
Institute of Education Sciences
National Harbor, Maryland
June 28-30, 2010

2. Components of becoming metacognitively sophisticated as a learner
Managing (optimally) the conditions of one’s own learning
Spacing, variation, generation, retrieval practice, …
Organizing one’s knowledge, using technology, engaging in cooperative learning, …
Judging (accurately) whether learning/comprehension that will support later recall/transfer has been achieved
Interpreting the meaning and predictive value of objective and subjective indices of current performance
Understanding that changes from the study context to the test context will impact access to what has been learned
Avoiding “foresight bias” (Koriat & Bjork, 2005)
Giving appropriate weight to the impact of retention interval and subsequent study opportunities
Avoiding “stability bias” (Kornell & Bjork, 2009)

3. Interpreting (and misinterpreting) objective indices of performance
Learning versus performance
What we can observe is performance;
What we must infer is learning;
…and the former is an unreliable guide to the latter.
Conditions of instruction that make performance improve rapidly often fail to support long-term retention and transfer, whereas
Conditions of instruction that appear to create difficulties for the learner, slowing the rate of apparent learning, often optimize long-term retention and transfer
Teachers and learners alike can be fooled
Teachers become susceptible to choosing poorer conditions of instruction over better conditions;
… and learners to preferring those poorer conditions
Examples
And, now, what I will be doing for the rest of this talk, is to describe some of these latter types of condition--conditions that have come to be called--as first coined by my collaborator and spouse, Robert Bjork--”desirable difficulties.”
I’m going to try to accomplish two main goals during this talk. First, to give you a feeling for a few types of desirable difficulties, at least one of which I will illustrate with some experimental findings. And, second in this context, I want to point out the potential for teachers and trainers--as well as students and trainees--to be misled as to what are and are not good educational practices or good conditions of learning.
As instructors, we can often be misled in this determination because what is readily available to us is the performance of our students during instruction, which can be a poor indicator of how much durable learning is actually occurring. If, for example, all we consider is the rapidity and apparent ease of learning during training and instruction, we can easily be led into preferring poorer conditions of learning to better conditions of learning. Additionally, as learners, it seems that we do not develop--through the everyday trials of living and learning—an accurate mental model, so to speak, of those operations that result in learning and those that do not. And, furthermore, we are fooled by certain indices—such as how fluently we process information during the re-reading of to-be-learned material—into illusions of learning and/or competence that then leads us to prefer poorer conditions of learning to better conditions of learning.
So, now, as promised let me turn to describing a few desirable difficulties.

4. Examples of learners being fooled
Generation Interleaving Spacing

5. Interpreting (and misinterpreting) subjective indices of performance
Perceptual fluency or familiarity
The sense of ease in processing visual or auditory information
Retrieval fluency
How readily information “comes to mind”
Fluency of induction
The sense of ease in noticing the commonalities across exemplars of a category or concept

6. Perceptual fluency or familiarity
Heuristic value
Misattributions and illusions
Misinterpreting the cause of perceptual fluency
Misinterpreting the meaning of perceptual fluency
Illusions of competence
Example: Reder (1987, 1988)

7. “What is the term in golf for scoring one under par?”
(Reder, 1987, 1988)
“What is the term in golf for scoring one under par?”

8. Retrieval fluency Heuristic value Misattributions and illusions
Interpreting performance as learning; illusions of competence
Egocentrism in instruction and social communication
Incomplete/faulty models of ourselves as learners/remembers
Example: Benjamin, Bjork, & Schwartz (1998)

9. Benjamin, Bjork, & Schwartz (1998)
Phase 1: 20 (easy) general-knowledge questions
E. g., “Who was the first president of the United States?”
Participants asked to:
(a) hit ENTER as soon as the answer “came to mind” (latency recorded);
(b) say the answer;
(c) predict the likelihood they would be able--at the end of the experiment--to free-recall having given that answer.
Phase 2: Distracting activity (spatial/map task)
Phase 3: Final test
Free recall: Write down as many of the 20 answers you gave earlier as you can;
(Original questions were not shown again)

12. Interpreting fluency of induction (Kornell & Bjork, 2008)
The ability to generalize concepts and categories through exposure to multiple exemplars.

13. “Spacing is the friend of recall but the enemy of induction.”
-Ernst Rothkopf

14. M S S M M S S M M S S M Lewis Pessani Stratulat Schlorff Wexler Juras
Mylrea
Pessani
Stratulat
Schlorff
Wexler
Juras
Mylrea
Hawkins

15. Test Feedback How negative is massing for induction?
How big will the massing effect be?

17. Results Actual Responses
Before asking this, we explained the terms massed and spaced.
Massing seemed easy.

18. A qualification
Subjective experience is not always misleading, it is sometimes even the best basis for judgments
Example: Jacoby and Kelley (1987)

19. Jacoby & Kelley (1987)
FSCAR ?????
vs.
FSCAR SCARF

20. Jacoby, Bjork, & Kelley (1994)
“Subjective experience, like the public media, is unavoidable, serves useful functions, and is not to be fully trusted”

21. Why is performance a poor guide to learning?
Performance is heavily influenced by local conditions—cues, predictability, recency—which can serve as crutches that prop up performance, but will not be there later at the time of test
Predictions of future recall are, for example, subject to a foresight bias (Koriat & Bjork, 2005, 2006)

22. Foresight bias: an illustration
Likelihood the second word will be given as a free associate to the first?
Lamp: Light
Find: Seek
Sell: Buy
Cheese: Cheddar
Citizen: Tax

23. Foresight bias: an illustration
Likelihood the second word will be given as a free associate to the first?
Lamp: Light (.71)
Find: Seek (.03)
Sell: Buy (.56)
Cheese: Cheddar (.03)
Citizen: Tax (.00)

24. Foresight bias: an illustration
Likelihood the second word will be given as a free associate to the first?
Lamp: Light (.71) Forward pair
Find: Seek (.03) Backward pair
Sell: Buy (.56) Forward pair
Cheese: Cheddar (.03) Backward pair
Citizen: Tax (.00) “Purely a-posteriori” pair

26. Foresight bias: Dynamics
Judgments of learning are made in the presence of information that will be absent, but solicited, on a subsequent test
The targets in cue-target pairs (e.g., Cheese: cheddar); or the answers to questions (e.g, the Capital of Australia is Canberra)
We are unable to anticipate the test situation, when the cue/question alone will trigger other associations
“Cheese ___?____” will trigger other strong associates, such as “mouse,” “bread,” “wine,” etc., which will compete with “cheddar.”
“Capital of Australia?”—will trigger Sydney, Melbourne, …

27. Finally, the impact of intervening events: Predicting one’s own forgetting and learning
We are subject to what Kornell and Bjork (2009) have labeled a stability bias:
The tendency to think that a memory representation, once formed, will remain stable
This bias leads to both
Overestimating remembering (i.e., underestimating forgetting); and
Underestimating learning.

28. Predicting one’s own forgetting (Koriat, Bjork, Sheffer, & Bar, 2004)
Experiment 1 (of 9):
60 paired associates
30 related;
30 unrelated
Participants judge, pair by pair, the likelihood they will remember that pair on a later cued-recall test
Retention interval to the final test (between-subjects):
Immediately after the study phase;
One day;
One week

29. Koriat, Bjork, Sheffer, & Bar (2004)
--PEOPLE DON’T KNOW THEY FORGET STUFF?--
Exp 1 : Between Item
Exp 2 : Within aggregate
Exp 3a: Between Aggregate
Exp 3b: Within Item

30. Experiment 7: Predicting one’s own learning
24 paired associates
12 related (Hill-Valley)
12 unrelated (Clemency-Idiom)
Number of study/test cycles (within-subjects): ST
STST
STSTST
STSTSTST
During the first study trial, participants judged, pair by pair, the likelihood they would remember that pair on either the first, second, third, or fourth cued-recall test cycle
Within-subjects
Response panel insured that participants predicted for the designated test

33. Actual Easy
Predicted Easy
Actual Hard
Predicted Hard

35. Concluding comment: People believe, in general,
that forgetting happens over time and that studying fosters learning,
That is, they have a theory of forgetting and a theory of learning
but they do not appear to believe that access to a given item in memory will be lost over a retention interval or increased by further study.

36. The end, probably

37. Final comment, if there is time, on our subjective experience as teachers
Egocentrism in social communication
Newton (1990) as a parable of teaching;
Piaget (1962) quote

40. Piaget (1962)
“Every beginning instructor discovers sooner or later that his first lectures were incomprehensible because he was talking to himself, so to say, mindful only of his point of view. He realizes only gradually and with difficulty that it is not easy to place one’s self in the shoes of students who do not yet know about the subject matter of the course.”

41. The end (for sure)

44. Koriat et al. (2004) conclusions
“… participants can access their knowledge about forgetting, but only when theory-based predictions are made, and then only when the notion of forgetting is accentuated —either by manipulating retention interval within individuals, or by framing recall predictions in terms of forgetting rather than remembering.”
“Once the notion of forgetting is activated, people can then take into account what they know about the specified retention interval in making their recall predictions. It is indeed quite instructive that they do not do that spontaneously—even when the specified retention interval is a year (Experiment 4c)!”

45. Koriat, Bjork, Sheffer, & Bar (2004) Predicting One's Own Forgetting: The Role of Experience-Based and Theory-Based Processes.
“We examined the hypothesis that judgments of learning (JOL), if governed by processing fluency during encoding, should be insensitive to anticipated retention interval.”
“The initial impetus … was the dual-basis view of metacognitive judgments (Brown & Siegler, 1993; Jacoby & Kelley, 1987; Kelley & Jacoby, 1996; Koriat & Levy-Sadot, 1999; Strack, 1992).”
Subjective experience: “Various mnemonic cues contribute directly to produce an immediate feeling of knowing that can serve as the basis of judgments. Thus, for example, encoding and retrieval fluency may foster a feeling of competence that can serve as [an experience-based basis for judgments of learning].”
Domain-specific knowledge retrieved from memory. “Theory-based judgments, in contrast, rely on the deliberate use of specific beliefs and information to form an educated guess about one's own knowledge. Thus, JOLs may utilize such rules as “memory performance should be better on recognition than on recall memory test.”

46. Experiment 8: Predicting one’s own learning
24 difficult (unrelated) paired associates (e.g., Clemency-Idiom)
Removing item differences should decrease participants’ tendency to base their judgments on intrinsic between-item differences
Number of study/test cycles (within-subjects): ST
STST
STSTST
STSTSTST
Participants judge, pair by pair, the likelihood they will remember that pair on the first, second, third, or fourth cued-recall test cycle
Within-subjects
Response panel insured that participants predicted for the designated test