1. Metacognition: A Practical Overview Ed Nuhfer – Director of Faculty Development, Director of Educational Assessment and Professor of Geology (University of Wisconsin, University of Colorado, Idaho State University, California State University — retired) enuhfer@earthlink.net Phone 208-241-5029 enuhfer@earthlink.net Technology, Innovations, & Pedagogy Conference CSU Fresno August 18, 2014

2. Metacognition… is a way of reflecting on: “What am I really trying to do here?”

3. What quality do you MOST wish your students will gain from their undergraduate experience? Reflection Show of hands: How many picked or heard something to do with elevated thinking ability? How many picked or heard additional content knowledge?

4. What We Mostly Want To Do Worth being familiar with Important to know and do Improved Thinking & Awareness of Learning Wiggins and McTighe (1998)

5. What We Mostly Do Worth being familiar with Important to know and do Disciplinary Knowledge & Skills Wiggins and McTighe (1998)

6. Exercise to try with your students “I know I am achieving a truly good education at this University when the following occurs: …” – On opening day, ask students to complete that sentence in class or as an assignment. – Collect the responses to learn how students perceive their goals for becoming educated in comparison with how you perceive providing education in their best interests. – Save these as baseline data. During your course, strive to have students make progress in understanding what is truly important. – Re-run this exercise again near the end of your course. See if pre-post- measures reveal change.

7. A thought on Motivation Are we maybe coming at this in the wrong way, through expecting most students to be motivated to learn the subject matter of our course or discipline … while perhaps wondering where the responsibility lies: that the students motivate themselves or that WE motivate the students? Suppose we focused together as a whole… on motivating students to understand how to learn anything? Thereafter, might all subject matter become fair game for a challenging practice?

8. Metacognition Metacognition is a “self-imposed internal conversation.” Shown to improve transfer (Bransford et al. 2000) We easily assume that students are doing it, or can develop doing it on their own; both assumptions are wrong. Our challenge is to keep students in constant contact with their metacognition. Instruction must be explicit. (Pintrich, 2002)

9. Metacognition “Metacognition is thinking about thinking” "Metacognition refers to one's knowledge concerning one's own cognitive processes or anything related to them…” (Flavell, 1976)

10. Metacognition “Metacognition is thinking about thinking” "Metacognition refers to one's knowledge concerning one's own cognitive processes or anything related to them…” (Flavell, 1976)

11. Metacognition So, if “metacognition is thinking about thinking…” …Just how does one do this? What are students supposed to be “thinking about”? Let’s consider some things that are candidates for “thinking about.”

12. The brain learns by building and stabilizing neural connections. (see Leamnson, 1999). “Thinking about WHAT we are trying to “wire in” Thinking about the biological basis for learning

13. We are trying to wire in… Knowledge Skills Reasoning Making distinctions for ourselves between these different kinds of learning challenges takes some thought. We should guide students to do the same. As a start for “thinking about,” students should think how to distinguish the three and how to engage with all three effectively.

14. Our usage in this presentation Knowledge Skills Reasoning Information, mostly disciplinary content, obtained through experience, observation, and study. Abilities and basic competencies that develop and improve with intentional practice and training Thinking process that employs knowledge for the purpose of gaining understanding or taking informed action. With practice, stages of development bring increased intellectual, affective, and ethical capacities.

15. “What am I really trying to do here?” What kind of challenge is this? What can I use from my past experiences to address this kind of problem? What is the best strategy for solving it? What kind of reasoning is most appropriate? How will I know if I solved it correctly? What additional information do I need? How can I use my new understanding to solve other kinds of problems?

16. What kind of learning challenge is this? Knowledge Skills Reasoning At what scales might I best achieve these? Lessons Courses Curricula (major, GE) Degrees (major + GE)

17. When students take a general education (GE) course, what do they “think about” as being the main objective? Do they think about: “If I can take an anthropology, biology, chemistry, environmental science, geology, or a physics course interchangeably for GE science credit, why can I do that?” For that matter, how good an answer might we give to that question? “We easily assume that students are doing it, or can develop doing it on their own; both assumptions are wrong.”

18. “Metadisciplinarity” 1.Identifying groups of disciplines that hold in common an overarching framework of reasoning/way of knowing that unites them. 2.Articulating the major unifying concepts and restating these as assessable student learning outcomes (SLOs)

19. Major Academic Metadisciplines Arts Humanities Mathematics/quantitative reasoning Physical/Life/Natural Science (or “Science”) Social Science Technology

20. Metadiscipline Example - Art Performing arts (music, dance, theatre ), visual arts (drawing, painting, sculpting, jewelry making), media arts (photography, filming) and literature (creative writing and poetry) These hold in common the overarching framework of reasoning/way of knowing in the arts that students can be directed to “think about” when taking a GE art course

21. DRAFT: Metadisciplinary Outcomes for the Arts Students should be able to… 1.Explain the significance of creative expression and art to the human experience. 2.Discern objective vs. subjective scholarship, criticism and analysis of the arts. 3.Articulate in his/her own words a definition for what constitutes the arts. 4.Communicate ideas and emotions through the practice and study of the arts. 5.Recognize and value creative expression from various cultural and historical perspectives. 6.Explain in his/her own words reasons why critical thinking and problem solving have value in the arts. 7.Describe, using at least two specific examples, how art literacy is important in everyday life. Instruction must be explicit (Pintrich, 2002)

22. Lets look at three versions of the same general education science course. What students “think about” can arise from what we emphasize with our course designs and enacted teaching philosophies.

23. Knowledge General Education: – Strives to impart content knowledge that citizens should know. – This accords with the type of science literacy tested on certain science literacy tests: All radioactivity is man-made. Radioactive milk can be made safe by boiling it. The earliest humans lived at the same time as the dinosaurs. Respond by agree-disagree. (Miller, 1998)

24. Skills General Education: – Strives to impart an excitement and enthusiasm for science by engaging students in doing science…ideally with other students. – This accords with involvement in applied research experiences such as Field studies Laboratory studies ….active development of knowledge and skills in authentic experiences – And it is a successful approach to recruiting science majors. – But what about the majority…who are going to major in something else?

25. Reasoning General/Liberal Education for Citizen Literacy – Develops through "… the collaboration and integration of general education and the major.” Content and specialty skills alone do not enable easy integration across majors. But understanding a framework of reasoning and way of knowing may allow such transfer. We hope that we are promoting reasoning and an understanding of science as a way of knowing…but are we? The answer to that last query requires the dreaded “A-word”

26. G iven each of the three introductory course experiences, how might each influence students in valuing what is most relevant to becoming educated? Knowledge Skills Reasoning

27. Knowledge Skill Reasoning On a piece of paper, draw your own circles to the size scales that show the emphases you might wish to give each in your own “Ideal Course”

28. Knowledge Skill Reasoning We wanted something like this

29. Reasonin g Knowledge Skill We discovered that we had built something else. In our second workshop today, we will address how we discovered this and what we did about it.

30. Metacurriculum for Metacognition InstrumentsKnowledge or Skill Learning-across-curriculum modules Strategies and developing a learning philosophy Reading ReflectionsReflection & Monitoring Exam Wrappers Reflection, Monitoring & Evaluation Knowledge Surveys Goal-setting, Monitoring & Evaluation Developing self- assessment & self-efficacy Learning Journals Evaluation & Goal-setting Developing self-assessment & self-efficacy Metadisciplinary Concept Inventories (Assessment) Reasoning, respect for diverse ways of knowing, overarching philosophy for becoming educated

32. When students hear“higher order reasoning,” “critical thinking” or “higher level thinking,” what are they “thinking about?” Example exercise to try with your students to find out. – “Can we distinguish those who can do critical thinking from those who cannot? If so, how?” – Answer to best of your present ability.

33. Module 12 – “Events a Learner Can Expect to Experience” We will investigate two metacognitive tools in one encounter. Read Module 12 down to the exercises, and complete a reading reflection by answering the following three questions.

34. Reading Reflection What is the main point of this reading? What did you find surprising? Why? What did you find confusing? Why?

35. Reading and Reflecting Reading Reflections: Completed after each reading assignment Short responses to a few questions Submitted online before class Credit awarded for “reflective” submissions What is the main point of this reading? What did you find surprising? Why? What did you find confusing? Why?

36. Metacognitive Goals of Reflections What is the main point of this reading? Summarizing (Anderson & Thiede, 2008) Keywords (Thiede et al., 2005) What did you find surprising? Why? Misconceptions (Bransford et al., 2000) Affect (Winne & Hadwin, 1998; Pintrich and Zusho, 2007) What did you find confusing? Why? Monitoring, Evaluation, and Reflection (Ertmer and Newby, 1996; Zimmerman, 2002)

37. Reading Reflection Employ the three queries and a rubric Download the template from http://profcamp.tripod.com/rrwithrubric.pdf http://profcamp.tripod.com/rrwithrubric.pdf

38. Some Results Reading Reflections vs. Course Grades MACALESTER GEOLOGY Pearson = 0.842 p-value = <0.001 HAMLINE ECONOMICS Pearson = 0.779 p-value = <0.001

39. Classroom Activities Effect Size = 1.35 (Large) Clicker Questions Effect Size = 1.08 (Large)

41. Perry in a Nutshell Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a nuisance that obstructs getting at the right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers to all things. They respond by concluding that all opinions are equally valid and that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. William J. Perry Jr. (1968) Forms of Intellectual and Ethical Development in the College Years Our challenge is to keep students in constant contact with their metacognition

42. SCALE - Change in Levels of Thinking by Design

43. Perry in a Nutshell Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers to all things. They respond by concluding that all opinions are equally valid and that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. You are frustrated with responding to a written take-home essay assignment and literally screaming “What does the professor WANT?” What developmental stage does this typify?

44. Perry in a Nutshell Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers to all things. They respond by concluding that all opinions are equally valid and that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. A professor who teaches college juniors reads the responses for suggested improvements on her student ratings form. Five students wrote: “Please, just give us the facts.” What does this reveal?

45. Perry in a Nutshell Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers to all things. They respond by concluding that all opinions are equally valid and that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. “I saw the evidence, but it did not change my mind.” Probable Stage?

46. Perry in a Nutshell Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers to all things. They respond by concluding that all opinions are equally valid and that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. “I believe this because someone I trust told me so.” Probable Stage?

47. Perry in a Nutshell Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers to all things. They respond by concluding that all opinions are equally valid and that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. “I carefully considered the arguments, but I favor this one because it has the most support from the evidence.” Probable Stage?

48. Perry in a Nutshell Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers to all things. They respond by concluding that all opinions are equally valid and that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. “For a long time I really opposed this argument. Now, I realize just how compelling the evidence for it is. I realize that I opposed it largely because I did not want to believe it.” Probable Stage?

49. Perry in a Nutshell Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers to all things. They respond by concluding that all opinions are equally valid and that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. “I observed that people in California readily accepted the evidence, but there are reasons that Midwesterners will not be inclined to do so.” Probable Stage?

50. Perry in a Nutshell Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers to all things. They respond by concluding that all opinions are equally valid and that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. “Yes, there is evidence, but everyone has a right to their opinions. In order to respect others, we must accept that all opinions are equally valid.” Probable Stage?

51. Perry in a Nutshell Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. Level 3 thinkers realize that authority is fallible and doesn't have all answers to all things. They respond by concluding that all opinions are equally valid and that arguments are just about proponents thinking differently. Evidence doesn't change this. Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. Level 5 thinkers accept that evaluations that lead to best solutions can be relative to the context of the situation within which a problem occurs. Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence to explore alternatives. They recognize that the most reasonable answers often depend upon context and value systems. Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. “There are good arguments on all sides, so I am going to just do what I feel is best.” Probable Stage?

52. Available Learning-Across-the-Curriculum Modules From CSU Faculty Developers

53. Metacognition, like the affective domain, was dismissed as a serious topic for decades. Good Resources on Metacognition

54. We reach our aspirations by consistently developing the desired qualities from the start.