1. Advanced Clickers: Practical Classroom Issues & Question Design David Vakil April 18, 2009 On Course National Conference Professor Vakil teaches astronomy & physics at El Camino College in Torrance, California. Contact info: dvakil@elcamino.edudvakil@elcamino.edu – 310-660-3593 x3134 For more clicker information or a copy of this presentation, please visit: http://www.elcamino.edu/faculty/dvakil/clickers.htm

2. Presentation outline Uses for clickers (3 min) Uses for clickers (3 min) Suggested reading (2 min) Suggested reading (2 min) What are your questions and concerns? (up to 15 min) What are your questions and concerns? (up to 15 min) Grading suggestions, pre-empting cheating (5 min) Grading suggestions, pre-empting cheating (5 min) Types of questions (3 min) Types of questions (3 min) Write your own clicker questions (10+ min) Write your own clicker questions (10+ min) Simulate class with your questions (20+ min) Simulate class with your questions (20+ min) Final tips (2 min) Final tips (2 min)

3. What can you do with clickers? Pre- and post- instruction questions. (Did they learn?) Pre- and post- instruction questions. (Did they learn?) Taking attendance (automatically calculated if any question is asked) Taking attendance (automatically calculated if any question is asked) Classroom demographics Classroom demographics Who are your students? Who are your students? How does performance relate to various characteristics? How does performance relate to various characteristics? Gathering feedback about class or getting opinions Gathering feedback about class or getting opinions like/dislike activity like/dislike activity useful/useless useful/useless Quick assessment of recently covered material Quick assessment of recently covered material Ask about facts or conceptual understanding. Ask about facts or conceptual understanding. No question is too easy to ask students. You’ll be amazed. No question is too easy to ask students. You’ll be amazed. Build their confidence or shake yours! Build their confidence or shake yours! Ice-breakers Ice-breakers Discussion starters Discussion starters

4. Uses, continued Data gathering – very useful for accountability reports Data gathering – very useful for accountability reports Cooperative learning: Think-pair-share Cooperative learning: Think-pair-share Quizzes Quizzes End of class summary End of class summary Beginning of class review Beginning of class review Determine if students did reading/homework Determine if students did reading/homework Full-length tests Full-length tests Ranking choices, putting lists in order Ranking choices, putting lists in order On-the-fly questions to suit whatever comes to mind during class. On-the-fly questions to suit whatever comes to mind during class. Some of my best & worst questions were done on the fly. Some of my best & worst questions were done on the fly. Sometimes students ask a question. DON’T ANSWER! Click! Sometimes students ask a question. DON’T ANSWER! Click! Fastest responses Fastest responses Team questions/answers Team questions/answers

5. Suggested Reading - Books I VERY STRONGLY RECOMMEND: I VERY STRONGLY RECOMMEND: Derek Bruff’s Teaching with Classroom Response Systems, Creating Active Learning Environments. Derek Bruff’s Teaching with Classroom Response Systems, Creating Active Learning Environments. 200 pages of everything you need to know. Useful Table of Contents. Well organized. Has everything EXCEPT for info about clicker manufacturers and software help. 200 pages of everything you need to know. Useful Table of Contents. Well organized. Has everything EXCEPT for info about clicker manufacturers and software help. Written for most disciplines about generic clickers. Written for most disciplines about generic clickers. QUICK READ: QUICK READ: Doug Duncan’s Clickers in the Classroom. Doug Duncan’s Clickers in the Classroom. Centered mostly on science, especially astronomy/physics. Centered mostly on science, especially astronomy/physics. THE BOOK THAT STARTED IT ALL: THE BOOK THAT STARTED IT ALL: Eric Mazur’s Peer Instruction: A User’s Manual. Eric Mazur’s Peer Instruction: A User’s Manual. Centered mostly on physics Centered mostly on physics

6. Suggested Reading - Journal Articles Physics Teacher, April 2008, “Clickers or Flashcards” by Nathaniel Lasry Physics Teacher, April 2008, “Clickers or Flashcards” by Nathaniel Lasry Clickers & flashcards result in same amount of learning Clickers & flashcards result in same amount of learning Clickers are better because clickers are both a learning AND teaching tool. Flashcards are just a learning tool. Clickers are better because clickers are both a learning AND teaching tool. Flashcards are just a learning tool. Life Sciences-CBE, Spring 2007, “Clickers in the Large Classroom” by Jane E. Caldwell Life Sciences-CBE, Spring 2007, “Clickers in the Large Classroom” by Jane E. Caldwell Doug Duncan’s 2 page clicker tips handout: Doug Duncan’s 2 page clicker tips handout: http://casa.colorado.edu/~dduncan/clickers/Tips.pdf http://casa.colorado.edu/~dduncan/clickers/Tips.pdf http://casa.colorado.edu/~dduncan/clickers/Tips.pdf Check YOUR discipline’s journals. More articles appear every day. Check YOUR discipline’s journals. More articles appear every day.

7. Your questions & concerns Topics: Topics: What are your biggest concerns about using clickers? What are your biggest concerns about using clickers? What are your unanswered questions? What are your unanswered questions? YOUR TASK: YOUR TASK: Work in small groups Work in small groups Write 3-5 questions Write 3-5 questions In 5 minutes we will share them with the room In 5 minutes we will share them with the room

8. Clicker grading Doug Duncan’s book: Clickers in the Classroom Doug Duncan’s book: Clickers in the Classroom Align grading with course goal (e.g. participate vs. correct) Align grading with course goal (e.g. participate vs. correct) Use clickers consistently if part of grade Use clickers consistently if part of grade Drop some scores – tech problem, forgot clicker, etc. Drop some scores – tech problem, forgot clicker, etc. Make worth 10% (maybe less) Make worth 10% (maybe less) small effect on grade, big effect on attendance small effect on grade, big effect on attendance Patrick Len’s article: Astronomy Education Review: Patrick Len’s article: Astronomy Education Review: Offer bonus points if class gets it 80+% correct Offer bonus points if class gets it 80+% correct Increases participation and discussion Increases participation and discussion High stakes grading DISCOURAGES talking with neighbors. High stakes grading DISCOURAGES talking with neighbors.

9. Cheating Doug Duncan’s book: Clickers in the Classroom Doug Duncan’s book: Clickers in the Classroom Paraphrased: Paraphrased: Discuss cheating at start of the semester Discuss cheating at start of the semester Using another clicker = taking their exam for them Using another clicker = taking their exam for them Little to gain, lot to lose Little to gain, lot to lose Small part of grade, big hindrance to learning Small part of grade, big hindrance to learning Most students LIKE the feedback Most students LIKE the feedback Students say feedback improves grade Students say feedback improves grade

10. Types of Questions Fact check Fact check Conceptual Conceptual Easy: one concept Easy: one concept Medium: assemble two concepts Medium: assemble two concepts Hard: assemble 3+ concepts Hard: assemble 3+ concepts Think-Pair-Share (next slide) Think-Pair-Share (next slide) Discussion starter / opinion gathering Discussion starter / opinion gathering Pre & Post test (identical question) Pre & Post test (identical question)

11. Think-Pair-Share (My personal favorite, the kind I use most often) (My personal favorite, the kind I use most often) Ask the question, students respond individually Ask the question, students respond individually If results are between 50-75% correct, consult with your neighbor and vote again. If results are between 50-75% correct, consult with your neighbor and vote again. If 40% correct, coach them a little first. If 40% correct, coach them a little first. If less than 40% correct, coach a lot or review. If less than 40% correct, coach a lot or review. Usually results improve noticeably, often 80+%. Usually results improve noticeably, often 80+%. Students understand and remember concept better Students understand and remember concept better Takes only 2-3 minutes, for rich conceptual questions. Takes only 2-3 minutes, for rich conceptual questions.

12. Write your own clicker question Write question(s) on your own for YOUR class. Write question(s) on your own for YOUR class. Then share them with group Then share them with group Estimate your questions’ difficulty level Estimate your questions’ difficulty level We’ll try your questions in here We’ll try your questions in here We’ll assess your difficulty level We’ll assess your difficulty level

13. Your questions Insert participant questions in the slides below Insert participant questions in the slides below

14. Asking the question in class Two thoughts: Two thoughts: 1. Don’t read the question out loud. The students reread it to themselves anyway. The students reread it to themselves anyway. 2. Don’t show the vote results if you may do a Think-Pair-Share Seeing the majority influences decision/discussion Seeing the majority influences decision/discussion Try not to reveal if you’re happy or sad about results Try not to reveal if you’re happy or sad about results Guide students if <50% correct on first question Guide students if <50% correct on first question

15. Challenger accident: what was the biggest problem/breakdown? 1. A 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

16. How difficult was that question? 1. Easy (1 concept) 2. Medium (2 concepts) 3. Hard (3+ concepts) 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

17. Feedback What did the question author(s) do well? What did the question author(s) do well? What suggestions do you have for this question? What suggestions do you have for this question?

18. Q 1. A 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

19. How difficult was that question? 1. Easy (1 concept) 2. Medium (2 concepts) 3. Hard (3+ concepts) 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

20. Feedback What did the question author(s) do well? What did the question author(s) do well? What suggestions do you have for this question? What suggestions do you have for this question?

21. Q 1. A 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

22. How difficult was that question? 1. Easy (1 concept) 2. Medium (2 concepts) 3. Hard (3+ concepts) 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

23. Feedback What did the question author(s) do well? What did the question author(s) do well? What suggestions do you have for this question? What suggestions do you have for this question?

24. Q 1. A 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

25. How difficult was that question? 1. Easy (1 concept) 2. Medium (2 concepts) 3. Hard (3+ concepts) 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

26. Feedback What did the question author(s) do well? What did the question author(s) do well? What suggestions do you have for this question? What suggestions do you have for this question?

27. Q 1. A 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

28. How difficult was that question? 1. Easy (1 concept) 2. Medium (2 concepts) 3. Hard (3+ concepts) 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

29. Feedback What did the question author(s) do well? What did the question author(s) do well? What suggestions do you have for this question? What suggestions do you have for this question?

30. Q 1. A 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

31. How difficult was that question? 1. Easy (1 concept) 2. Medium (2 concepts) 3. Hard (3+ concepts) 12345678910111213141516171819202122232425262728293031323334353637383940 4142434445464748495051525354555657585960 6162636465666768697071727374757677787980

32. Feedback What did the question author(s) do well? What did the question author(s) do well? What suggestions do you have for this question? What suggestions do you have for this question?

33. CONCLUSION I hope you wrote the useful tips down! That was the entire point of today’s session! I hope you wrote the useful tips down! That was the entire point of today’s session! Some final advice: Some final advice: Pay attention to difficulty level & Bloom’s taxonomy Pay attention to difficulty level & Bloom’s taxonomy Discussion questions are most memorable and lead to longer term learning Discussion questions are most memorable and lead to longer term learning Don’t write questions all by yourself or at the last minute! Don’t write questions all by yourself or at the last minute! We all write bad questions. Learn from mistakes. Try to use the “bad” question as a teachable moment. “How should this question be worded if we were trying to XXXXX?” We all write bad questions. Learn from mistakes. Try to use the “bad” question as a teachable moment. “How should this question be worded if we were trying to XXXXX?” Get student input on improvements! Some of my favorite questions were drafted by students! Get student input on improvements! Some of my favorite questions were drafted by students!