1. Materials on Specific Aspects of Peer Instruction Dr. Stephanie V. Chasteen Physics Department & Science Ed. Initiative University of Colorado – Boulder http://STEMclickers.colorado.edu Stephanie.Chasteen@colorado.edu Co-presenters have included Steven Pollock, Jenny Knight, Trish Loeblein, and Kathy Perkins. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Scince Education Initiative/ CU-Boulder This Powerpoint provides our workshop slides organized by topic

2. This presentation is copyrighted under the Creative Commons License Attribution Non-Commercial Share-Alike That means: Please watch it, share it, and use it in your presentations. Just give us credit, don’t make money from it, and use the same kind of license on the works that you create from it. More information about Creative Commons licenses here: http://creativecommons.org/licenses/ Credit should be given to: Stephanie Chasteen and the Science Education Initiative at the University of Colorado, http://colorado.edu/sei

3. About these slides We have created a variety of workshops on clickers and Peer Instruction for faculty and K12 teachers. These slides represent the presentations and activities that we have produced through this work. You are free to use this material with proper attribution (see previous slide). Not all slides or activities were used in every workshop. Activities are designated with a peach background to the slide You can find the full handouts and activity descriptions under Workshop Materials at http://STEMclickers.colora do.edu

4. To make your own workshop We suggest including… Introduction About Peer Instruction Facilitation OR Question Writing It is difficult to cover both Facilitation and Question Writing. In a general workshop, you might focus your activities on Facilitation and do some short activity on questions. 1. Choose your activities (typically one or two per section) 2. Choose your slides 3. Make an outline/timeline of your workshop to make sure you’re not trying to do too much. All activities have accompanying handouts which you can find at http://STEMclickers.colorad o.edu http://STEMclickers.colorad o.edu

5. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder These meta-slides provide a little bit of information for you about our presenter and what we are trying to do with our professional development workshops. Overview

6. Introducing Me 6 Applying scientific principles to improve science education – What are students learning, and which instructional approaches improve learning? Science Education Initiative Physics Education Research Group One of largest PER groups in nation, studying technology, attitudes, classroom practice, & institutional change. http://colorado.edu/SEI http://PER.colorado.edu Blogger & Consultant http://sciencegeekgirl.com Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder

7. U. Colorado clicker resources… 7 Videos of effective use of clickers http://STEMclickers.colorado.edu Clicker resource page http://STEMvideos.colorado.edu 2-5 mins long Instructor’s Guide Question banks Workshops Literature / Articles Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder

8. “Clickers” are really just a focal point We aim to help instructors: Use student-centered, interactive teaching techniques By the use of a tool (clickers) which makes a transition to that pedagogy easier Our talks are “how people learn” talks in disguise. Bransford, Brown, Cocking (1999), How People Learn Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder 8

9. The typical pattern of professional development for faculty… (we) Tell them how to do it (they) Try it (they) Fail or fade (we) Repeat (louder!) In physics, half of faculty only use Peer Instruction for a single semester What’s missing? We need to help faculty anticipate challenges and difficulties with implementing peer instruction. Lose the rose-colored glasses! We also need to provide less prescriptive “do this, don’t do that” recommendations, which are hard to remember, and instead provide a pedagogical strategy which will naturally lead to those “best practices” These workshop materials are intended to help overcome some of the challenges to sustainable improvements in teaching, as based on the research on instructional change. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder 9

10. How we try to accomplish goals: Give a clear introduction to peer instruction. What does it really look like? Give experience in peer instruction. How does it feel as a student? As an instructor? Provide disciplinary experience. Give examples from multiple disciplines; have instructors sit next to others who teach in their subject area Why does it work? The research. Respect their experience. Answer their questions/challenges, rather than being gung-ho salesman. Provide opportunity for practice and feedback. Especially in writing questions and facilitation. Practice what we preach. Do all this in a student-centered, interactive environment. Don’t lecture about how not to lecture. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder 10

11. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder These slides frame the message of the workshop: that we are talking about effective questioning techniques, and how clickers help to facilitate that questioning. We emphasize that the technology is not the same as the pedagogy. Introduction: Questioning If you start with technology, they focus on technology. Frame it as a workshop about questioning. Don’t equate the technology and the pedagogy

12. What do you teach? A. Science B. Engineering or Math C. Social sciences D. Humanities E. Other Show of hands

13. Have you used response systems (clickers) in your teaching? A. Not at all, and I haven’t seen them used B. Not at all, but I’ve observed their use somewhat C. I’ve used them a little D. I’ve used them a lot E. I could be (should be?) giving this workshop Take a clicker & turn it on If the green light flashes, your vote has been counted

14. How familiar are you with “Peer Instruction” A. Fairly familiar, and I like it B. Fairly familiar, but I’m not sure that I like it C. I’ve heard of it but only have a vague idea what it is D. Not familiar at all E. Not sure Colored cards

15. Warm-Up Activities The following slides outline several possible warm- up activities to focus the participants on the purposes of questioning in the classroom See the handouts for full descriptions and for participant worksheets

16. Warm Up #1: Why question? Why do we ask questions? When might we use questions? What is the purpose of clicker questions? Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder Warm up activity Discuss in small groups, making notes in handout. Then share-out. Warm up activity Discuss in small groups, making notes in handout. Then share-out. 16

17. Warm Up #2: Why clickers? What goals might clickers be used to achieve? Or, put another way, what might you use clicker questions to accomplish in your class? 17 Warm up activity Brainstorm on your own, then discuss in small groups, making notes in handout. Then share-out. Warm up activity Brainstorm on your own, then discuss in small groups, making notes in handout. Then share-out. What does this tool help us to do?

18. The toughest thing about asking questions in class is… A. Writing good questions B. Getting students to really think about them C. Getting students to answer the questions / Nobody responds D. The same students always respond / Not everybody responds E. It takes too long / I have a lot of content to cover Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder Warm Up #3: Clicker question about questions 18

19. Warm Up #4: Goals of Questions 19 Warm up activity Watch a short mini lecture and write up one or two questions you could ask students to assess learning or facilitate understanding (not multiple choice). Choose your favorite to share. Warm up activity Watch a short mini lecture and write up one or two questions you could ask students to assess learning or facilitate understanding (not multiple choice). Choose your favorite to share. What is the goal of your question? A.Setting up instruction B.Developing knowledge C.Assess Learning D.Something else

20. Question Cycle: Before / During / After Credit: Rosie Piller and Ian Beatty. 20 BEFORE Setting up instruction E.g.: Motivate Assess prior knowledge … (handout!) DURING Developing knowledge Application Elicit misconception … AFTER Assessing learning Relate to big picture Demonstrate success … 20

21. Question Cycle: Before/During/After Credit: Rosie Piller and Ian Beatty. 21 BEFORE Setting up instruction Motivate Discover Predict outcome Provoke thinking Assess prior knowledge DURING Developing knowledge Check knowledge Application Analysis Evaluation Synthesis Exercise skill Elicit misconception AFTER Assessing learning Relate to big picture Demonstrate success Review or recap Exit poll

23. Two way conversations with students are vital... 23...because students can misunderstand what we say

24. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder These slides outline how a clicker works, what the benefits are of using clickers, and explain peer instruction (the pedagogical technique using clickers for in-class questioning). It seems important to show example questions early in the workshop, though this can also be done later when discussing features of good questions. Seeing lots of examples seems to broaden participants’ thinking about clicker questions, and gives them some concrete examples to refer to as you discuss the abstract pedagogy. We also have them participate in a model peer instruction question. About clickers and peer instruction

25. What is special about clicker questions? Similar goals as other types of questioning techniques Multiple choice Anonymous (to peers) Every student has a voice – the loud ones and the shy ones Forced wait time You can withhold the answer until everyone has had time to think (choose when to show the histogram) 25 What does this tool help us to do?

26. 26 But not a magic bullet! Clickers are a tool for questioning Don’t equate the pedagogy with the technology. So what IS the pedagogy?

27. Browsing Questions Activity Three different activities follow which allow the participants to browse a variety of questions This serves to expose participants to a wide variety of types of clicker questions, and to open participants’ eyes to the wide variety of possible uses of clicker/peer instruction questions, so that they may be able to gain a broad vision of how this tool might be used in their classroom. These activities also help ground the abstract pedagogy in concrete examples.

28. Browsing Questions 1: Gallery Walk With a partner, look at the “example questions” trios on the wall. What do you think an instructor would be trying to accomplish with such questions? Jot down any ideas next to the question 5 minutes Aihofanz2010 on Wikimedia

29. Browsing Questions #2: Treasure Hunt Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder Question Rating Sheet Browse the questions on the handout. In your groups, rate them on a 4-point scale, with “1” being “terrible” and “4” being “terrific.” Question Rating Sheet Browse the questions on the handout. In your groups, rate them on a 4-point scale, with “1” being “terrible” and “4” being “terrific.” 29

30. Browsing Questions 3: Powerpoint Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder Powerpoint Show a series of questions via Powerpoint and discuss. Powerpoint Show a series of questions via Powerpoint and discuss. 30

31. Why use peer instruction? 31 Here we show one of two videos from http://STEMvideos.colorado.edu  Anatomy of a Clicker Question (for audiences that are already bought-in to using clickers)  Students and Teachers Speak (for audiences that could benefit from some persuasion), first few minutes Each shows the process of teaching using peer instruction, from which we can extract the “anatomy of peer instruction” on the next slide

32. Anatomy of a clicker question 32 Ask Question Peer Discussion Vote Debrief …Lecture… (May vote individually) * See also: Peer Instruction, A User’s Manual. E. Mazur. 32

33. 33 1. Asking Question Best practices Ask several times during lecture Ask challenging, meaningful questions Questioning is integral to lecture Best practices Ask several times during lecture Ask challenging, meaningful questions Questioning is integral to lecture Why do it? Students can learn by considering a question Breaks up lecture Learning is in the application of knowledge Why do it? Students can learn by considering a question Breaks up lecture Learning is in the application of knowledge

34. Example question: Biology 34 A small acorn over time can grow into a huge oak tree. The tree can weigh many tons. Where does most of the mass come from as the tree grows? A) Minerals in the soil B) Organic matter in the soil C) Gases in the air D) Sunlight Common misconception leads to answers (A) and (B). Correct answer: C Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder 34

35. 2. Peer Discussion 35 Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder Why do it? Students learn more deeply by teaching each other Makes them articulate answer Lets you see inside their heads Why do it? Students learn more deeply by teaching each other Makes them articulate answer Lets you see inside their heads Best Practices: Make it clear why you’re doing this Circulate and ask questions / model Use questions they want to discuss Allow enough time (2-5 mins) Focus on reasoning in wrap-up Show students you value their ideas Best Practices: Make it clear why you’re doing this Circulate and ask questions / model Use questions they want to discuss Allow enough time (2-5 mins) Focus on reasoning in wrap-up Show students you value their ideas

36. 3. Wrap-Up Discussion. 36 Best practices: Establish culture of respect Don’t always show the histogram immediately Ask multiple students to defend their answers Emphasize reasoning: Why are wrong answers wrong and why right answer is right Best practices: Establish culture of respect Don’t always show the histogram immediately Ask multiple students to defend their answers Emphasize reasoning: Why are wrong answers wrong and why right answer is right Why do it? Student ideas are important Instructor feedback is important So students know answer and reasoning by the end Why do it? Student ideas are important Instructor feedback is important So students know answer and reasoning by the end

37. 37 Ask Question Peer Discussion Vote Debrief …Lecture… (May vote individually Question break Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder

38. The Practice Question Following are a series of practice questions that you can use to demonstrate Peer Instruction. Choose one. A practice question provides the experience of engaging in PI as a participant, which serves to outline a model PI cycle and highlight the value of the technique Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder 38

39. 39 Practice question #1: Superpowers Which superpower would you rather have? The ability to… A. Change the mass of things B. Change the charge of things C. Change the magnetization of things D. Change the boiling point of things Question: Ian Beatty, UNC GreensboroImage: Thibault fr on Wikimedia 39

40. Practice Question #2: Twins 40 Your sister in law calls to say that she’s having twins. Which of the following is the most likely? (Assume she’s having fraternal, not identical, twins) A) Twin boys B) Twin girls C) One girl and one boy D) All are equally likely Courtesy Derek Bruff, Vanderbilt 40

41. Practice Question #3: Tennis Anyone? A tennis racket and can of balls together costs $110. The tennis racket alone costs $100 more than the can of balls. How much does the can of balls alone cost? A. $5 B. $10 C. $11 D. $100 E. None of these Courtesy Steven Pollock, CU-Boulder 41

42. Practice Question #4: Clicker Challenges A. Writing good questions B. Getting students to really think about the questions C. Getting students to share their reasoning with the whole class D. Getting students to discuss the questions with each other (peer discussion) E. It takes too long / I have a lot of content to cover I think the toughest thing about using clickers and peer instruction in class is / will be: 42

43. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder Following are a variety of slides highlighting key findings regarding the effectiveness of clickers. We like to highlight the research to show that we’re not “selling snake oil,” but at the same time we find that too much time on the research can be dull. Previous studies have found that instructors aren’t *convinced* to use a new technique based on research: rather, they are convinced because a technique seems like good teaching, or a colleague persuades them, and then they use the research to *justify* what they have chosen to do. Thus, it is good to show that there is research to back up the technique but not spend too much time on it. Links to key studies are at http://stemclickers.Colorado.Edu. Research about peer instruction

45. Peer instruction helps students learn Research shows that: Students can better answer a similar question after talking to their peers Peer discussion + instructor explanation of question works better than either one alone Students like peer instruction Peer instruction classes outperform traditional lectures on a common test 45 See http://STEMclickers.colorado.edu for various references

46. Clicker Question Honestly, I think that I’m most likely to modify this technique of peer instruction to suit me and my students. I know that there are at least ___ parts of the technique that I’ll be changing: A. None B. One C. Two-three D. Four or more

47. Is there a problem with modifications? I won’t tell you how to teach. You’re smart & you care about instruction. Be strategic about modifications. Know the research.

48. Some research on modifications 63.5% of faculty (in physics) say they are familiar with Peer Instruction 30% report that they use Peer Instruction 50% of those use Peer Instruction in the way described by developers Often dropped are:  Student discussion  Use of conceptual questions  Whole-class voting Dancy & Henderson, Pedagogical practices and instructional change of faculty, Am. J. Phys., 78(10), Oct 2010.Web survey of 722 physics faculty at various institutions, initial sample of 2000. Is this a problem? Probably. 48

49. Talking brings convergence 49 Eric Mazur - Harvard U. Before discussion B CA After discussion BCA Mazur, 1997 Why do you think this happens? (A) Students are getting answers from the ‘smart’ kids (B) They’re learning from their discussions (C) They just needed more time to think about it Why do you think this happens? (A) Students are getting answers from the ‘smart’ kids (B) They’re learning from their discussions (C) They just needed more time to think about it

50. The hypothesis: If students learn from peer discussion, they should show better performance on a similar question. Ask a second, similar question without any instructor input: Q2 Undergrad introductory genetics course. 16 Q1/Q2 pairs. Smith et al., Science. 2009, 323(5910):122. Research by Michelle Smith, Bill Wood, Wendy Adams, Carl Wieman, Jenny Knight, Nancy Guild, Tin Tin Su, MCDB.

51. n= 350 students Smith et al., Science. 2009, 323(5910):122. Are they learning from peers? Q2 Individual Q1 Individual Q1 AD After Discussion 20 40 60 80 100 0 Percent Then explain answers to Q1 and Q2 Students answer Q1 individually. 1) Students talk to neighbors and answer Q1 again (Q1 AD = Q1“After Discussion”). 2) Students answer Q2 individually. Q2 tests same concept as Q1. 3)

52. Easy (5 questions) Medium (7 questions) Difficult (4 questions) Percent correct 10 20 30 40 50 60 70 80 90 100 0 Can students answer difficult questions correctly after discussion? Q1 Q1 after discussion Q2 Very few students knew correct answer to Q1, but after discussion, many more answer correctly: students are constructing their own knowledge Smith et al., Science. 2009, 323(5910):122.

53. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder In this section of the workshop, we address best-practices in facilitation of Peer Instruction. In a workshop focused on facilitation, this forms about half the workshop. In a workshop focused on question-writing or an introduction to Peer Instruction, we must choose a small subset of these slides and activities to give an overview. We highlight best practices by explicitly addressing common challenges associated with the technique. There are several possible activities we have used. The Pedagogical Philosophies activity leads into best-practices through discussion of the underlying beliefs that would be required to facilitate Peer Instruction. The other “Challenges” activities get participants brainstorming about common hurdles and solutions. Common Challenges / Facilitation

54. Activity: Pedagogical Philosophy Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder Core Philosophies exercise What are the underlying principles that make this work? Why might this be an effective teaching strategy? What must the instructor believe? What must the students believe? Discuss in groups, and then share- out. Core Philosophies exercise What are the underlying principles that make this work? Why might this be an effective teaching strategy? What must the instructor believe? What must the students believe? Discuss in groups, and then share- out. 54

55. Some core philosophies of mine Students learn by … teaching each other … articulating their ideas It’s important for me to …. hear student ideas … know what my students understand I value and respect student ideas I want students to … know that I value student ideas … feel safe sharing their ideas Clicker questions are an integral part of my lecture

56. Activity: Challenges in the Classroom You ask students a question, and ask them to discuss. You then ask them to share their answers and reasoning in a whole-class discussion What could possibly go wrong? 56 In groups of 3-5 brainstorm some of the challenges you imagine in using this. Brainstorm some solutions that are in line with your core philosophies Write on your handout and then scribe on board In groups of 3-5 brainstorm some of the challenges you imagine in using this. Brainstorm some solutions that are in line with your core philosophies Write on your handout and then scribe on board 10 mins

57. What are the challenges? What do you think is the toughest thing about using Peer Instruction? A. Writing good questions B. Technical issues C. Tough to get students to discuss questions D. I have too much content to cover / takes too much time E. Something else Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder 57

58. 58 1. Ask Question What are some challenges/ philosophies / solutions related to asking the question? Best practices Ask several times during lecture Ask challenging, meaningful questions Don’t post until ready & give time to read Best practices Ask several times during lecture Ask challenging, meaningful questions Don’t post until ready & give time to read Philosophies Questions are integral to lecture Students can learn by considering a question Philosophies Questions are integral to lecture Students can learn by considering a question

59. Question Cycle: Before / During / After Credit: Rosie Piller and Ian Beatty. 59 BEFORE Setting up instruction E.g.: Motivate Assess prior knowledge … (handout!) DURING Developing knowledge Application Elicit misconception … AFTER Assessing learning Relate to big picture Demonstrate success …

60. 2. Peer Discussion 60 What are core philosophies in peer discussion? Philosophies: Students learn through discussion Students need to know that you value their ideas & that it’s safe to share Philosophies: Students learn through discussion Students need to know that you value their ideas & that it’s safe to share Solutions: Make it clear why you’re doing this Circulate and ask questions / model Use questions they want to discuss Allow enough time (2-5 mins) Focus on reasoning in wrap-up Solutions: Make it clear why you’re doing this Circulate and ask questions / model Use questions they want to discuss Allow enough time (2-5 mins) Focus on reasoning in wrap-up What are challenges / how can you help make it work?

61. Student buy-in is key! Here we show a portion of the video, “Tell Students Why” at http://STEMvideos.colorado.edu. This highlights the importance of student buy-in and gives an example of an instructor speech to a class on why he is using Peer Instruction

62. 3. Wrap-Up Discussion 62 Philosophies? Challenges? What might you do to facilitate an effective wrap-up discussion? Solutions: Establish culture of respect Consider whether to show the histogram immediately Ask multiple students to defend their answers Emphasize reasoning: Why are wrong answers wrong and why right answer is right Solutions: Establish culture of respect Consider whether to show the histogram immediately Ask multiple students to defend their answers Emphasize reasoning: Why are wrong answers wrong and why right answer is right Philosophies: Student ideas are important Students need to feel safe Philosophies: Student ideas are important Students need to feel safe

63. 63 Giving the answer stops student thinking!

64. Effects of increased wait time Changes in student behavior:  More students respond  More students respond without being asked (unsolicited)  Student responses are longer  More alternative explanations are offered  Student confidence increases  There are more speculative responses  Students ask more questions Other changes (on teacher!)  Quantity of questions decreased  Quality of questions increased  Expectations of slower students were revised  Teacher reactions to answers were more appropriate 64 Rowe, Mary Budd (1974) All from a few more seconds!

65. Other things we haven’t talked about? Other challenges / solutions / philosophies? 65

66. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder These slides are used in workshops where we are focusing on the skills and process of writing good clicker questions. In a workshop that is trying to give an overview, we might only do a brief “best practices in writing questions” slide. In a longer workshop focused on writing questions Writing Questions

67. Writing Questions Activity Following are two possible activities where participants write a draft question They have a chance to revisit and revise the question after additional workshop material is presented, later

68. Writing Questions #1: Pedagogical Goal Choose one of the pedagogical goals from the “Question Cycle” Write a draft clicker question that aims to achieve this goal. 3 minutes

69. Question Cycle: Before/During/After Credit: Rosie Piller and Ian Beatty. 69 BEFORE Setting up instruction Motivate Discover Predict outcome Provoke thinking Assess prior knowledge DURING Developing knowledge Check knowledge Application Analysis Evaluation Synthesis Exercise skill Elicit misconception AFTER Assessing learning Relate to big picture Demonstrate success Review or recap Exit poll

70. Writing Questions #2: Content Goal Choose a content learning goal that relates to your discipline Write a draft clicker question that aims to help students achieve this learning goal 3 minutes

71. Learning Goals Biology: Recognize the components of a cell and describe why each is necessary for the function of a cell Physics: Identify the different ways that light can interact with an object (i.e., transmitted, absorbed, reflected). Chemistry: Explain trends in boiling points in terms of intermolecular interactions Earth science: Understand the formation of the three major types of rocks (igneous, sedimentary, and metamorphic) and the processes by which they form, relating them by the rock cycle. Math: Solve a system of linear equations in two variables using algebra or graphing.

72. Writing Questions #3: Revise Existing Question A. The change in the earth’s distance from the sun during the year B. The tilt of the earths axis C. Changes in the sun’s brightness D. Changes in clouds E. None of the above What causes the seasons? Consider the following question. How might you improve upon this question, or write it differently? What is the pedagogical goal of this question?

73. A. The change in the earth’s distance from the sun during the year B. The tilt of the earths axis C. Changes in the sun’s brightness D. Changes in clouds E. None of the above Bad question. Students can answer by memorizing a word (“tilt”) Can we make a better question on the SAME topic? Yes… What causes the seasons? Writing Questions #3: Revise Existing Question

74. What would happen to the seasons if the earth’s orbit around the sun was made a perfect circle (but nothing else changed) ? A. There would be no seasons B. The seasons would remain pretty much as they are today C. Winter to spring would differ much less than now D. Winter to spring would differ much more than now Much better question. Requires reasoning! Better seasons example

75. Question-writing tips Move away from simple quizzes Use questions that prompt discussion Use questions that emphasize reasoning or process Use clear wording Use tempting distracters Use questions for a variety of instructional goals Use questions at a mixture of cognitive depth Ask challenging questions – don’t just test memorized facts See handout

76. Effective multiple-choice questions have believable “distracters.” 76 1)Talking with other instructors that have taught the course in the past. 2)Talking with your students one-on-one before class, after class, during office hours. 3)Using student responses to open- ended questions that you include in HW and exams. 4)Asking your students to come up with answers that will be used as the choices. 5)Use researched and documented student misconceptions. D. Duncan, Univ. of Colorado

77. Remember the Question Cycle (pedagogical goals) Credit: Rosie Piller and Ian Beatty. 77 BEFORE Setting up instruction E.g.: Motivate Assess prior knowledge … (handout!) DURING Developing knowledge Application Elicit misconception … AFTER Assessing learning Relate to big picture Demonstrate success …

78. Use questions at a variety of cognitive depth 78 Do the questions you use intellectually challenge your students or simply assess their factual knowledge? Higher order ---------------- Lower order handout Bloom’s Taxonomy of the Cognitive Domain

79. Writing Questions (follow-up): Revise your question Use what we’ve just talked about, and the “tips” in your handouts, to revise your question If you wish, swap with your neighbor and discuss. 5 minutes

80. Writing Questions#4: Rate and swap Use the Bloom’s Taxonomy worksheet to rate the Bloom’s level of your question Swap your question with a neighbor. Do you agree on the Bloom’s level of your question? Use the verbs on the detailed Bloom’s handout to “Bloomify up” the level of your question. 5 minutes

81. Writing Questions #4 (variation): Rate and Bloom it up Use the Bloom’s Taxonomy worksheet to rate the Bloom’s level of this question Use the verbs on the detailed Bloom’s handout to “Bloomify up” the level of this question 5 minutes A. The change in the earth’s distance from the sun during the year B. The tilt of the earths axis C. Changes in the sun’s brightness D. Changes in clouds E. None of the above What causes the seasons?

82. Share out What did you learn in this process? What worked well, what was challenging? How might you go about writing questions in your class?

83. But… The perfect question doesn’t solve all problems!

84. Action Plan Take a few minutes to write down your action plan to implement ideas you heard about in the workshop 84