1. Cultivating Growth Mindset
Marsha Lovett, Ph.D.
Thank you… Special topic for me
Abstract: As faculty, besides helping our students acquire key disciplinary knowledge and technical skill, we can also promote students’ development of metacognition and thereby help them become effective, life-long learners. The challenge is that metacognition all too often falls between the cracks in a curriculum. Many faculty naturally wonder: What can I do to help my students learn to learn? Isn’t this somebody else’s job? How can I find time in my already jam-packed course? This session will present research on these and related issues and will explore various strategies that are practical and powerful.

2. 2 Learners Approach Upcoming Test
Emily: slightly worried
test right after playoffs
essay tests a challenge
sets a plan, starts early
outline key ideas
notes cause-effect
stops to self-assess
Monica: also anxious
essay tests hard for me
so study harder
read/re-read text
memorize vocabulary
no explicit plan
starts night before
As instructors, we know it’s important not only that students gain the content knowledge that we can offer and use the resources we provide them, but also gain the ability to help themselves along the learning process. If we help them gain skills to direct their own learning, they can take those skills with them beyond college (i.e., lifelong learning).
ACTIVITY
Discuss with a partner:
What are key differences underlying Emily’s/Monica’s approach
(Ertmer & Newby, 1996)

3. Self-Regulated Learning (SRL)
1:00-2:00
This is the IDEAL cycle in which students regulate their own learning through reflective processes:
ASSESS the task, taking into consideration the task’s goals and constraints
EVALUATE their own knowledge, skills, strengths, weakness in relation to the task
PLAN their approach based on their self-evaluation
APPLY strategies to enact the plan; MONITOR their own progress along the way
REFLECT on whether their approach worked and adjust to restart the cycle
Student’s beliefs about intelligence and learning influences how they move through the cycle.
Ambrose, et al., (2010). How Learning Works:
7 Research-Based Principles for Smart Teaching.

4. Self-Regulated Learning (SRL)
Emily:
test right after playoffs
essay tests a challenge
sets a plan, starts early
outline key ideas
notes cause-effect
stops to self-assess
1:00-2:00
This is the IDEAL cycle in which students regulate their own learning through reflective processes:
ASSESS the task, taking into consideration the task’s goals and constraints
EVALUATE their own knowledge, skills, strengths, weakness in relation to the task
PLAN their approach based on their self-evaluation
APPLY strategies to enact the plan; MONITOR their own progress along the way
REFLECT on whether their approach worked and adjust to restart the cycle
Student’s beliefs about intelligence and learning influences how they move through the cycle.
Ambrose, et al., (2010). How Learning Works:
7 Research-Based Principles for Smart Teaching.

5. Self-Regulated Learning (SRL)
Your Students:
What do they do?
Have they been taught these skills?
Personal connection: Story about N, going to college “Will he make good choices?” Myself as a college student, not always making good choices…
Ambrose, et al., (2010). How Learning Works:
7 Research-Based Principles for Smart Teaching.

6. Students often don’t assess the task
Half of the college students who were observed as they tackled a writing assignment, ignored the instructions and instead used a generic strategy (writing-as-knowledge-telling).
(Carey and Flowers, 1989)

7. Self-assessing strengths is hard
“Ignorance more frequently begets confidence than does knowledge.”
– Darwin
Admitting you have a problem is the first step to change
"ignorance more frequently begets confidence than does knowledge” (Darwin, 1871, p. 3)
(Kruger & Dunning, 1999)

8. Many students don’t plan or plan poorly
Comparing less vs. less effective writers: Less effective writers plan less appropriately.
Comparing novices and experts in physics: Novices take more time to solve problems, but they spend almost no time planning. Experts spend more time planning than novices do.
Ss may not plan at all
Ss may know to plan but put inapprop tasks in the plan (potentially because you didn’t assess task or self accurately);
Ss may put appropr tasks in the plan but not allow enough time in the plan or execute those tasks poorly
(Carey et al., 1989; Chi et al., 1989)

9. People have trouble monitoring and adjusting their approach
Poor problem solvers continue to use a strategy even after it has failed
People often continue to use a familiar strategy that works moderately well rather than switch to a new strategy that would work better
be sure to refer back to diagram because we’re merging with the larger scale reflection process.
Programming project example (ss didn’t plan modules; they got themselves into a strategy); outlining chapters just because
Students get a negative answer for the elevation of a building; architecture example; engineering common;
Insensitive art projects
Group projects: students don’t monitor
(NRC, 2001; Fu & Gray, 2004)

10. Self-Regulated Learning
Which component gives your students the most trouble?
Can you think of an example?
1:00-2:00
This is the IDEAL cycle in which students regulate their own learning through reflective processes:
ASSESS the task, taking into consideration the task’s goals and constraints
EVALUATE their own knowledge, skills, strengths, weakness in relation to the task
PLAN their approach based on their self-evaluation
APPLY strategies to enact the plan; MONITOR their own progress along the way
REFLECT on whether their approach worked and adjust to restart the cycle
Student’s beliefs about intelligence and learning influences how they move through the cycle.
Ambrose, et al., (2010). How Learning Works:
7 Research-Based Principles for Smart Teaching.

11. Strategies Brainstorm
What strategies can we use to help students:
Assess the task
Evaluate strengths/weaknesses
Plan
Monitor and adjust their approach
(Would scaffolding or modeling help?)

12. Some possible strategies
Planning
Have students implement a plan that you provide, and eventually allow them to make their own plan
Create an assignment that focuses on (ends on) planning
Applying and Monitoring
Ask students to annotate their own work during the process
Have students self-evaluate using a designated rubric or checklist
Facilitate peer review sessions for students to evaluate and monitor each others’ work, using a rubric
Reflecting and Adjusting
Provide activities for students to reflect on their performance
Present students with multiple approaches to a problem, and ask them to pursue the one that works best for them
1:19 (EDW)

13. (Self-)Testing Enhances Learning
When students test themselves or “self-explain” when studying, they retain more and solve subsequent problems better.
Students tend not to self-assess/test themselves
Study Strategy
Re-read notes or textbook
Re-write notes
Do practice problems
Use flashcards
Fix/move
(Karpicke, Butler, & Roediger 2009)

14. (Self-)Testing Enhances Learning
When students test themselves or “self-explain” when studying, they retain more and solve subsequent problems better.
Students tend not to self-assess/test themselves
Study Strategy
% rank #1
Re-read notes or textbook 55
Re-write notes 12
Do practice problems
Use flashcards 6
Fix/move
(Karpicke, Butler, & Roediger 2009)

15. Self-Regulated Learning
1:00-2:00
This is the IDEAL cycle in which students regulate their own learning through reflective processes:
ASSESS the task, taking into consideration the task’s goals and constraints
EVALUATE their own knowledge, skills, strengths, weakness in relation to the task
PLAN their approach based on their self-evaluation
APPLY strategies to enact the plan; MONITOR their own progress along the way
REFLECT on whether their approach worked and adjust to restart the cycle
Student’s beliefs about intelligence and learning influences how they move through the cycle.
Ambrose, et al., (2010). How Learning Works:
7 Research-Based Principles for Smart Teaching.

16. Beliefs about learning
and intelligence
Growth Mindset
0:00-2:41

17. Research on Changing Beliefs
Students in an 8-week workshop on learning received either 2 lessons on “brain as muscle” (experimental) or “memory strategies” (control)
Students’ beliefs assessed before & after
Teachers’ (blind) effort ratings collected
Students’ math grades collected
(Blackwell et al, 2007)

18. Key Results
Experimental group endorsed “incremental” beliefs more after intervention
Experimental group showed more increases in effort, according to teacher ratings
Experimental group showed upturn in their math grade trajectories
(Blackwell et al, 2007)

19. Changing beliefs: Results (cont’d)
(Blackwell et al, 2007)

20. More Research on Changing Beliefs
Stanford University students recruited for pen pals program.
Program promoted view of intelligence as either “malleable” or “fixed”
Pen pals met three times to write letters
Race (African American, White) used as a blocking variable
(Aronson et al, 2002)

21. Key Results Short term effects on beliefs, as predicted
Long term effects – end of school year:
Belief changes maintained
Enjoyment of academics*
Spring quarter GPA higher for “malleable”, controlling for prior SAT
*Condition differences larger for African American students

22. The Mentor’s Dilemma Cohen, Steele, & Ross (1999)
Study participants: Black and white college students
Task: write a letter of commendation for your most memorable and effective teacher, coach, or mentor – for potential publication in an educational journal.
Experiment: Students randomly assigned to get feedback on the letter (in the form of detailed criticisms and suggestions for improvement) in one of three ways:
Unbuffered, Positive Buffer, High Standard + Assurance
Outcome measures: Students’ motivation to revise, etc.
, e.g., motivation to continue at a task, belief that effort -> improvement
Positive buffer comments prefacing feedback: Overall, nice job. Your enthusiasm for your teacher really shows through, and it’s clear that you must have valued her [him] a great deal. You have some interesting ideas in your letter and make some good points. In the pages that follow, I’ve provided some more specific feed- back and suggested several areas that could be improved.
High Standards + Assurance: t’s obvious to me that you’ve taken your task seriously and I’m going to do likewise by giving you some straight- forward, honest feedback. The letter itself is okay as far as it goes—you’ve followed the instructions, listed your teacher’s merits, given evidence in support of them, and importantly, produced an articulate letter. On the other hand, judged by a higher standard, the one that really counts, that is, whether your letter will be publishable in our journal, I have serious reservations. The comments I provide in the following pages are quite critical but I hope helpful. Remember, I wouldn’t go to the trouble of giving you this feedback if I didn’t think, based on what I’ve read in your letter, that you are capable of meeting the higher standard I mentioned.
Cohen, G.L., Steele, C.M., & Ross, L.D. (1999). The mentor’s dilemma: Providing critical feedback across the racial divide. PSPB, 25(10),

23. Unbuffered Criticism: Your letter needs work in several areas… In addition to some routine editorial suggestions, most of my comments center on how you could make the description… As it stands, your letter is vague and rambling...
Positive Buffer: Overall, nice job. Your enthusiasm for your teacher really shows through, and it’s clear that you must have valued her [him] a great deal. You have some interesting ideas in your letter and make some good points. In the pages that follow, I’ve provided some more specific feedback and suggested several areas that could be improved…
High Standards + Assurance: It’s obvious to me that you’ve taken your task seriously and I’m going to do likewise by giving you some straight-forward, honest feedback. The letter itself is okay as far as it goes—you’ve followed the instructions, listed your teacher’s merits, given evidence in support of them, and importantly, produced an articulate letter. On the other hand, judged by a higher standard, the one that really counts – whether your letter will be publishable in our journal – I have serious reservations. The comments I provide [below] are quite critical but I hope helpful. Remember, I wouldn’t go to the trouble of giving you this feedback if I didn’t think, based on what I’ve read in your letter, that you are capable of meeting the higher standard I mentioned.

24. Results & Questions for Discussion
Students’ Motivation to Revise
Students’ Identification w/ Writing
What do these results say to you about the role and impact of different approaches to feedback?
How might you leverage these results in your teaching?
What more might you want to know about this research before taking action?

25. Changing Mindset Through Feedback
What about positive feedback?
Research suggests: To enhance motivation, give praise for effort and strategies, not for ability (Mueller & Dweck, 1998)
Constructive feedback appears most effective when it communicates high standards for performance but also assurances that the student is capable of meeting those high standards (Cohen, Steele, & Ross, 1999) Constructive feedback (esp to under-represented students) is most effective when it communicates high standards AND assurances that the student is capable of meeting those high standards (Cohen, Steele, & Ross, 1999)

26. Changing beliefs: Summary
By working to change students’ beliefs about learning, intelligence, and responses to challenge, we can see:
Sustained changes in belief (for months)
Increased motivation/effort
More positive attitudes
Improved performance (even after a delay)

27. Apply it to your own course…
What is one new strategy you might you try?
Model/scaffold a specific SRL skill
Incorporate testing as a better way to study
Promote growth mindset with “high expectations + assurances” feedback

28. Conclusions
Most students do not naturally engage metacognitive skills or self-directed learning strategies.
As instructors, we can support students in developing these skills and making them habits of mind.
Doing so can pay big dividends for students’ life-long learning.

29. Transforming the meaning of effort & difficulty
Feedback to reduce stereotype threat
0:00-2:41

30. Self-Regulated Learning
One of the most common course components is also one of the biggest missed opportunities for
Self-Regulated Learning

31. EXAMS
Think about what students typically do when they receive back their graded exams
Consider what you typically do when you return graded exams

32. Exam wrappers promote self-assessment
An exam wrapper is a short, reflective exercise that students complete upon receiving back their graded exam.
Goal: Encourage students to learn from the exam results rather than ignore them
Sublimate, repress
Students complete exam wrapper upon receiving back graded exam
Completed sheets are collected by instructional staff (optionally reviewed/analyzed)
Completed sheets returned to students a week or so before next exam
Lovett, M. C. (2013). Make Exams Worth More Than The Grade, in Using Reflection and Metacognition to Improve Student Learning. Stylus Publishing.

33. Exam Wrappers ask students to reflect
1. How did I prepare for the exam?
2. What errors did I make on the exam?
3. How should I study for the next exam?

34. 1. How did I prepare for the exam?
Exam Wrappers
1. How did I prepare for the exam?
Completed a variety of practice problems
Prepared a study guide
Crammed for the test the night before.
Reviewed general concepts only
What did they do right, and how can they improve for next time?

35. 2. What errors did I make on the exam?
Exam Wrappers
2. What errors did I make on the exam?
Careless mistake
Mis-read the question
Didn’t know how to start
What did they do right, and how can they improve for next time?

36. 3. How should I study for the next exam?
Exam Wrappers
3. How should I study for the next exam?
What behaviors or habits should I change or maintain?
Student is writing a letter to his/her future self
What did they do right, and how can they improve for next time?

37. Exam Wrapper Procedure
Distribute wrappers when graded exams are returned
Students complete wrapper in class or as homework
Collect wrappers from students, so you can:
Spot check for patterns
Keep wrappers from getting lost
Return wrappers to students before next exam to review (and optionally discuss)
“How closely have you been following your designated study plan?”
“Based on your responses, what must you still do to prepare for the upcoming exam?” 3. 4.

38. Evaluating Exam Wrappers
Subjects - Students in several introductory math and science courses at CMU
Treatment – Exam wrappers were tailored for each course, but always included the three core components:
How did the student prepare for the exam?
What kinds of errors did the student make?
How should the student study for the next exam?
Questionnaire – Students were given pre- and post-course questionnaire (5-point scale) about their strategies for studying and (post-course only) asked about their experience.
All students took a pre-test about the circulatory system, which comprised of 1) matching words to corresponding definitions, 2) labeling pictures of a heart, 3) an outline of the human body in which students are supposed to draw the path of blood circulation, and 4) a prompt asking students to write everything they can about the circulatory system (i.e., essay prompt).
All students had a brief training on how to use a hypermedia learning environment (Microsoft Encarta)
Half the students received 30-min Self-Regulated Learning (SRL) training: 10 min explanation of the phases/areas of regulation, 10 min explanation/illustration of a model of SRL, and 10 min explanation of each SRL variable (e.g., prior knowledge activation, monitoring progress, summarizing, rereading, etc.). The other half received no training (control).
All students used hypermedia learning environment to learn about the circulatory system. All students were prompted to “think aloud” as used it, i.e., they were supposed to say everything they are thinking. Treatment group was specifically instructed to use the SRL variables they had learned about.
All students given the same post-test after using hypermedia. Pre- and post-tests were scored.
Lovett (2013)

39. Exam Wrapper Results
Students self-identified multiple approaches for exam preparation.

40. Overall Impact of Intervention
Did self-monitoring lead to change?
Majority of students used new strategies

41. New Strategies Highlight Monitoring
“I didn’t really know how well or not well I knew the material until I put myself to the test: really doing the problems on practice tests and re-solving homework problems without looking at the answer.”
“I went over previous tests and practice exams. Completing the practice exams in college help[ed] me to gauge what I still needed to focus in on in my study.”

42. New Strategies Address Overconfidence
“There is a big difference between actually doing problems and trying to memorize a particular solution while looking at examples. I have to say my advice is: do a lot of practice problems.”
“I began solving problems much more often, going to extra help sessions and, while I was reading or listening to lecture, looking for what I'm supposed to be getting out of it… actively reading/listening, instead of just reading/listening.”
CRG: see notes last slide

43. Pre- and post-course questionnaire
Change in rating for metacognitive strategies, as a function of exam wrapper “dose”
Results for specific metacognitive strategies showed a positive “dosage” effect on students’ self-reported skills in:
Connecting ideas
Reviewing solutions
Self-explaining
Memorizing*
*Memorization is actually considered to be a “less effective” strategy (as opposed to the other three listed), and the study predicted a negative dose-response, however it is acknowledged that memorization may actually be more helpful in the intro science and math courses than originally thought.
Lovett, M. C. (2013). Make Exams Worth More Than The Grade, in Using Reflection and Metacognition to Improve Student Learning. Stylus Publishing.

44. Key takeaways
Instructors can learn a lot about student study habits and perceptions of the course from exam wrapper exercises
Exam wrappers lead students to self-identify new (better) strategies
Repeated exposure to exam wrappers has a positive impact on study strategy ratings and metacognitive behaviors
No quantitative data to show that exam wrappers improve student scores/grades

45. Final thoughts on Exam Wrappers
Not just for exams! – Homework wrappers, lab wrappers, etc.
For more examples and ideas, please visit:
14:30-15:00