1. Slides on Writing Clicker Questions Dr. Stephanie V. Chasteen Physics Department & Science Ed. Initiative University of Colorado – Boulder http://STEMclickers.colorado.edu Stephanie.Chasteen@colorado.edu Co-presenters have included Steven Pollock, Jenny Knight, Trish Loeblein, and Kathy Perkins. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Scince Education Initiative/ CU-Boulder

2. This presentation is copyrighted under the Creative Commons License Attribution Non-Commercial Share-Alike That means: Please watch it, share it, and use it in your presentations. Just give us credit, don’t make money from it, and use the same kind of license on the works that you create from it. More information about Creative Commons licenses here: http://creativecommons.org/licenses/ Credit should be given to: Stephanie Chasteen and the Science Education Initiative at the University of Colorado, http://colorado.edu/sei

3. About these slides We have created a variety of workshops on clickers and Peer Instruction for faculty and K12 teachers. These slides represent the presentations and activities that we have produced through this work. You are free to use this material with proper attribution (see previous slide). Not all slides or activities were used in every workshop. Activities are designated with a peach background to the slide You can find the full handouts and activity descriptions under Workshop Materials at http://STEMclickers.colora do.edu

4. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder These meta-slides provide a little bit of information for you about our presenter and what we are trying to do with our professional development workshops. Overview

5. Introducing Me 5 Applying scientific principles to improve science education – What are students learning, and which instructional approaches improve learning? Science Education Initiative Physics Education Research Group One of largest PER groups in nation, studying technology, attitudes, classroom practice, & institutional change. http://colorado.edu/SEI http://PER.colorado.edu Blogger & Consultant http://sciencegeekgirl.com Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder

6. U. Colorado clicker resources… 6 Videos of effective use of clickers http://STEMclickers.colorado.edu Clicker resource page http://STEMvideos.colorado.edu 2-5 mins long Instructor’s Guide Question banks Workshops Literature / Articles Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder

7. “Clickers” are really just a focal point We aim to help instructors: Use student-centered, interactive teaching techniques By the use of a tool (clickers) which makes a transition to that pedagogy easier Our talks are “how people learn” talks in disguise. Bransford, Brown, Cocking (1999), How People Learn Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder 7

8. The typical pattern of professional development for faculty… (we) Tell them how to do it (they) Try it (they) Fail or fade (we) Repeat (louder!) In physics, half of faculty only use Peer Instruction for a single semester What’s missing? We need to help faculty anticipate challenges and difficulties with implementing peer instruction. Lose the rose-colored glasses! We also need to provide less prescriptive “do this, don’t do that” recommendations, which are hard to remember, and instead provide a pedagogical strategy which will naturally lead to those “best practices” These workshop materials are intended to help overcome some of the challenges to sustainable improvements in teaching, as based on the research on instructional change. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder 8

9. How we try to accomplish goals: Give a clear introduction to peer instruction. What does it really look like? Give experience in peer instruction. How does it feel as a student? As an instructor? Provide disciplinary experience. Give examples from multiple disciplines; have instructors sit next to others who teach in their subject area Why does it work? The research. Respect their experience. Answer their questions/challenges, rather than being gung-ho salesman. Provide opportunity for practice and feedback. Especially in writing questions and facilitation. Practice what we preach. Do all this in a student-centered, interactive environment. Don’t lecture about how not to lecture. Creative Commons – Attribution. Please attribute Stephanie Chasteen / Science Education Initiative/ CU-Boulder 9

10. These slides are used in workshops where we are focusing on the skills and process of writing good clicker questions. In a workshop that is trying to give an overview, we might only do a brief “best practices in writing questions” slide. In a longer workshop focused on writing questions Writing Questions

11. Writing Questions Activity Following are two possible activities where participants write a draft question They have a chance to revisit and revise the question after additional workshop material is presented, later

12. Writing Questions #1: Pedagogical Goal Choose one of the pedagogical goals from the “Question Cycle” Write a draft clicker question that aims to achieve this goal. 3 minutes

13. Question Cycle: Before/During/After Credit: Rosie Piller and Ian Beatty. 13 BEFORE Setting up instruction Motivate Discover Predict outcome Provoke thinking Assess prior knowledge DURING Developing knowledge Check knowledge Application Analysis Evaluation Synthesis Exercise skill Elicit misconception AFTER Assessing learning Relate to big picture Demonstrate success Review or recap Exit poll

14. Writing Questions #2: Content Goal Choose a content learning goal that relates to your discipline Write a draft clicker question that aims to help students achieve this learning goal 3 minutes

15. Learning Goals Biology: Recognize the components of a cell and describe why each is necessary for the function of a cell Physics: Identify the different ways that light can interact with an object (i.e., transmitted, absorbed, reflected). Chemistry: Explain trends in boiling points in terms of intermolecular interactions Earth science: Understand the formation of the three major types of rocks (igneous, sedimentary, and metamorphic) and the processes by which they form, relating them by the rock cycle. Math: Solve a system of linear equations in two variables using algebra or graphing.

16. Writing Questions #3: Revise Existing Question A. The change in the earth’s distance from the sun during the year B. The tilt of the earths axis C. Changes in the sun’s brightness D. Changes in clouds E. None of the above What causes the seasons? Consider the following question. How might you improve upon this question, or write it differently? What is the pedagogical goal of this question?

17. A. The change in the earth’s distance from the sun during the year B. The tilt of the earths axis C. Changes in the sun’s brightness D. Changes in clouds E. None of the above Bad question. Students can answer by memorizing a word (“tilt”) Can we make a better question on the SAME topic? Yes… What causes the seasons? Writing Questions #3: Revise Existing Question

18. What would happen to the seasons if the earth’s orbit around the sun was made a perfect circle (but nothing else changed) ? A. There would be no seasons B. The seasons would remain pretty much as they are today C. Winter to spring would differ much less than now D. Winter to spring would differ much more than now Much better question. Requires reasoning! Better seasons example

19. Question-writing tips Move away from simple quizzes Use questions that prompt discussion Use questions that emphasize reasoning or process Use clear wording Use tempting distracters Use questions for a variety of instructional goals Use questions at a mixture of cognitive depth Ask challenging questions – don’t just test memorized facts See handout

20. Effective multiple-choice questions have believable “distracters.” 20 1)Talking with other instructors that have taught the course in the past. 2)Talking with your students one-on-one before class, after class, during office hours. 3)Using student responses to open- ended questions that you include in HW and exams. 4)Asking your students to come up with answers that will be used as the choices. 5)Use researched and documented student misconceptions. D. Duncan, Univ. of Colorado

21. Remember the Question Cycle (pedagogical goals) Credit: Rosie Piller and Ian Beatty. 21 BEFORE Setting up instruction E.g.: Motivate Assess prior knowledge … (handout!) DURING Developing knowledge Application Elicit misconception … AFTER Assessing learning Relate to big picture Demonstrate success …

22. Use questions at a variety of cognitive depth 22 Do the questions you use intellectually challenge your students or simply assess their factual knowledge? Higher order ---------------- Lower order handout Bloom’s Taxonomy of the Cognitive Domain

23. Writing Questions (follow-up): Revise your question Use what we’ve just talked about, and the “tips” in your handouts, to revise your question If you wish, swap with your neighbor and discuss. 5 minutes

24. Writing Questions#4: Rate and swap Use the Bloom’s Taxonomy worksheet to rate the Bloom’s level of your question Swap your question with a neighbor. Do you agree on the Bloom’s level of your question? Use the verbs on the detailed Bloom’s handout to “Bloomify up” the level of your question. 5 minutes

25. Writing Questions #4 (variation): Rate and Bloom it up Use the Bloom’s Taxonomy worksheet to rate the Bloom’s level of this question Use the verbs on the detailed Bloom’s handout to “Bloomify up” the level of this question 5 minutes A. The change in the earth’s distance from the sun during the year B. The tilt of the earths axis C. Changes in the sun’s brightness D. Changes in clouds E. None of the above What causes the seasons?

26. Share out What did you learn in this process? What worked well, what was challenging? How might you go about writing questions in your class?

27. But… The perfect question doesn’t solve all problems!

28. Action Plan Take a few minutes to write down your action plan to implement ideas you heard about in the workshop 28