1. ASSESSMENT TOOL OR EDUTAINMENT TOY USING PERSONAL RESPONSE DEVICES FOR LIBRARY INSTRUCTION ASSESSMENT Patrick Griffis August 6, 2008

2. Outline  Overview of Personal Response Systems/Clickers  Scalable Options of Clicker Systems  Benefits/Drawbacks for Using Clickers  Using Clickers for Assessment  Experimenting with Clickers/Lessons Learned

3. Personal Response Systems ‘Clickers’  System allowing instructors to pose questions to students which students can answer anonymously  Anonymous response feature encourages participation from students which enhances student engagement in the classroom and provides instructors immediate feedback  Personal Response Systems were developed as a tool to increase student feedback and engagement in large class settings

4. Scalable Options: Common Model  Clicker Software + Receiver + Clicker Devices  Standard model  Classroom set of devices and receiver  Instructor responsible for receiver and software  Requires software download to instructor’s computer

5. Scalable Options: Clicker Only  Clicker Devices Only  LCD screen handheld receiver  Computer and projector not required  Answers from student keypads displayed and stored on handheld receiver and can be exported via USB to a computer  Easiest system to implement/Can be used anywhere

6. Scalable Options: Clicker Only

7. Scalable Options: Software Only  Clicker Software + Virtual Response Keypads  Receiver not required and Clicker Devices not required  Local Area Network and/or Wireless Network  Student computers connect via broadcasted IP Address  Requires software download to instructor computer and Virtual Clicker download to student computers

8. Scalable Options: Software Only

9. Alternative Web Polling Tools  DyKnow Vision (Licensed Software-One Time Cost)  SurveyMonkey (Subscription Web Survey Tool)  WordPress Polling (Ad On to WordPress Blog Software)

10. Tool: Clicker Benefits  Can increase student interactivity and engagement in bibliographic instruction with a ‘workshop’ feel  Can help instructor to assess what students already know coming into the session allowing instruction to be tailored to what students don’t know  Can help instructor assess what students have retained/learned in a session as well as areas that need reinforcement

11. Toy: Clicker Drawbacks  Learning curve for students can take time away from an instruction session which might already be hurting for time  Some instructors feel that Clickers are another of many slick gadgets/technologies that already serve to distract students in the classroom Clickers good-cell phones bad-mixed message for students

12. Toy: Clicker Drawbacks  Clicker use more expensive for Libraries which have the burden of purchasing Clicker Devices More feasible for classes that meet regularly to require students to purchase clicker devices  It costs more to use Clickers for Library Instruction Sessions than for Term Courses and Library Instruction Sessions have a much smaller window of opportunity to use these devices Nascent need to experiment for feasibility

13. View from Library Instructors: Tool  “A Personal Response System (PRS), or clickers, is an effective method for gathering assessment data during library instruction sessions” (Page 258)  “A PRS is a useful tool for gathering data and customizing instruction to student needs” (Page 260) Julian, Suzanne, and Kimball Benson. "Clicking your way to library instruction assessment: Using a Personal Response System at Brigham Young University." College and Research Libraries News 69, no. 5 (2008): 258-61.

14. View from Library Instructors: Toy  “New technology is entertaining but can quickly become the focus of a session if not used as part of an appropriate learning activity” (Page 260)  “We discovered it was important to carefully monitor the amount of time being devoted to clicker questions and evaluate if their use enhanced the instruction” (Page 260) Julian, Suzanne, and Kimball Benson. "Clicking your way to library instruction assessment: Using a Personal Response System at Brigham Young University." College and Research Libraries News 69, no. 5 (2008): 258-61.

15. Critical Question  Can the use of clickers enhance library instruction enough to be worth the time needed to use them?  5-10 minutes?  10-15 minutes?

16. Critical Response  Effective Clicker use allows library instructors to identify a few key areas needing most emphasis for student needs.  Reduced need to cover everything  How much time would be saved in a 50 minute session if session student learning objectives were cut by a third or even by half? Clickers could actually enhance effectiveness of sessions to save time.

17. An Unexpected Use of Clickers for Library Instruction Assessment  Julian & Benson claim that they serendipitously discovered that Clickers can be an effective tool for instructional assessment.  Original intention was to add interactivity to instruction sessions to increase student engagement and retention.  Authors began considering using Clickers for instructional assessment.  This purpose will become a new area of consideration and experimentation. Julian, Suzanne, and Kimball Benson. "Clicking your way to library instruction assessment: Using a Personal Response System at Brigham Young University." College and Research Libraries News 69, no. 5 (2008): 258-61.

18. Critical Question  Is a Clicker System an Effective, Sustainable, Practical Tool for Assessing Library Instruction?  Depends on the type/purpose of Instructional Assessment Formative Assessment Summative Assessment

19. Formative vs. Summative Assessment  Formative assessments are on-going assessments, reviews, and observations in a classroom. Teachers use formative assessment to improve instructional methods and student feedback throughout the teaching and learning process. The results of formative assessments are used to modify and validate instruction.  Summative assessments are typically used to evaluate the effectiveness of instructional programs and services at the end of an academic year or at a pre-determined time. The goal of summative assessments is to make a judgment of student competency--after an instructional phase is complete. Summative evaluations are used to determine if students have mastered specific competencies and to identify instructional areas that need additional attention.

20. Critical Questions  Is Instruction Assessment more Formative or Summative?  Are Clicker Systems Designed more for Formative or Summative Assessment?

21. ACRL Information Literacy Standards [Bloom’s Taxonomy] “ In implementing these standards, institutions need to recognize that different levels of thinking skills are associated with various learning outcomes--and therefore different instruments or methods are essential to assess those outcomes. For example, both "higher order" and "lower order" thinking skills, based on Bloom’s Taxonomy of Educational Objectives, are evident throughout the outcomes detailed in this document. It is strongly suggested that assessment methods appropriate to the thinking skills associated with each outcome be identified as an integral part of the institution’s implementation plan.”

22. Bloom’s Taxonomy OriginalRevised Knowledge Remember Recalling the information Comprehension Understand Explain the ideas and/or concepts Application Apply Using the newly acquired knowledge in another familiar situation Analysis Analyze Comparing and differentiating between constituent parts. Evaluation Evaluate Justifying a decision or course of action Synthesis Create Generating new ways of creating products, ideas or ways of viewing things

23. View from Clicker Instructors Higher Order & Lower Order Thinking Skills  “Different questions elicit different responses and require different levels of cognitive engagement. Knowledge-level questions ask for simple recall of facts and data without assessing them critically, whereas analysis, synthesis and evaluation questions require critical thinking and judgment.”(Page 5)  “When faculty are simply assessing students’ basic understanding, a knowledge-level question may be appropriate. But when faculty wish to engage students in thinking critically about course content, a knowledge-level question may fall short of reaching the goal.”(Page 5) Zhu, Erping. "Teaching With Clickers." Center for Research on Learning and Teaching Occasional Papers (2007): 1-7. http://www.crlt.umich.edu/publinks/CRLT_no22.pdf. http://www.crlt.umich.edu/publinks/CRLT_no22.pdf

24. Critical Questions  Are Clickers an Effective Instrument for Measuring Information Literacy Outcomes Requiring Higher Order Thinking Skills? What Type of Questioning Requires Higher Order Thinking Skills? What Type of Questioning is Possible with Clickers?

25. Questioning Capabilities of Clickers  Initial Models supported Fixed Response Questions  Multiple Choice  True/False  Newer Models support Alpha-Numeric Answers  10-15 Character Text Entry  LCD Display Screen  What About…..  Matching Answers  Short Answer/Essay Questions

26. Will Clickers Evolve to Fully Support Summative Assessment  Applications are being developed to enhance the Assessment capabilities of Clickers.  TurningPoint’s ‘TestingPoint’ Application Higher Education Assessment Tool With TestingPoint, professors and lecturers can create and administer self-paced assignments, quizzes and tests with ease, allowing for real-time summative and formative assessment.

27. TestingPoint  TestingPoint supports questions in multiple formats:  Multiple Choice  Multiple Response  Numeric Response  Matching  Fill in the Blank  Bimodal  True/False  Short Answer  Essay http://www.turningtechnologies.com/k12studentresponsesystem/testingpoint.cfm

28. Summative Assessment Considerations  Clickers were initially designed for Formative Assessment  Instructors have serendipitously experimented with using Clickers for Formative and Summative Assessment  Clicker developers are taking the need for Summative Assessment capabilities to heart  Library Instructors will have opportunity to experiment fully with Clicker Summative Assessment soon

29. My Own Experience  Chaired a campus committee tasked with evaluating Clicker systems and selecting one to be the campus standard.  Developed knowledge about Clicker systems and a desire to try them in library instruction to increase interactivity and student engagement.

30. Experimentation Issues: Trial Kit Size  Vendor ‘Trial Kits’ limited in scale  Trial Kits typically no larger than 10 Clicker Devices  Could not experiment with using Clickers in large class settings for which Clicker Systems are intended  Could not negotiate to have a site license trial of a Virtual Clicker  Trial Kits allow for familiarity with a Clicker System before purchase but does not allow for real class feasibility experimentation

31. Experimentation Issues: Trial Duration  Vendor Trials Limited in Time  Typically one to three month duration  Limited time for experimentation after learning curve Only enough time to learn how to use the Clicker System leaving little time for significant feasibility experimentation

32. Experimentation Recommendation  Collaborate with Professors Already Using Clickers  Simply visit their classroom and use their Clicker System Requires familiarity with their Clicker System  Students have Clickers and know how to use them No learning curve for Students taking away instruction time  No cost ideal solution to experimentation issues Real classroom feasibility experimentation

33. Further Considerations  Experiment with a Personal Response System which is widely used on campus  Which system is used most on campus?  Has your campus adopted a standard system?  If your campus is in the process of adopting a campus standard, try to arrange for a representative from your library system to be on the campus working group/committee

34. Favorable Conditions for my Experimentation with Clickers in Library Instruction  COM 101 Public Speaking Course has requirement for library instruction  Director of COM 101 Program was on Clicker Standardization Committee  I have conducted library instruction for these sessions numerous times

35. Challenges for my Experimentation with Clickers in Library Instruction  COM 101 library instruction sessions have a graded quiz assignment which is a summative assessment of information literacy learning outcomes  The quiz takes considerable time from instruction sessions already  Quiz questions are in formats not widely supported by Clicker systems

36. A Silver Lining  COM 101 Course Director considering making the required quiz take home for students  Future Clicker enhancements might fully support a quiz like this making the back end grading work for library instruction assessors much more efficient and sustainable without compromising effectiveness

37. QUESTIONS/COMMENTS Patrick Griffis Business Librarian University of Nevada, Las Vegas 4505 Maryland Parkway Las Vegas, NV 89154-7014 (702) 895-2231 patrick.griffis@unlv.edu patrick.griffis@unlv.edu