Determining the Most Appropriate Type(s) of Assessment in Library Instruction Bonnie L. Fong, Physical Sciences Librarian, Rutgers-Newark VALE / NJ ACRL / NJLA CUS 14 th Annual Users' Conference Friday, January 4, 2013

Assessment

Assessment Difficulties Many possibilities to consider Many factors to consider Who, What, When, Where, Why, How

Why Assess For someone else? –College/university –Academic department –Professor –Library For ourselves? –What are we trying to accomplish?

What to Assess Evaluation of library instruction session: –Students (e.g., useful, confusing, enjoyable) –Professor (e.g., fitting, effect on assignments) –Librarian (e.g., timing, activities that worked) Students’ information literacy skills: –Objective knowledge –Higher-order thinking –Application of knowledge (e.g., paper) Assessment tool

How to Assess Sessions Student self-evaluation –Typical one-minute paper: Questions: –What did you learn today? –Is there anything that is still confusing? Distribution? –Paper / index card –Online Quick Self-reflective

How to Assess Sessions Student self-evaluation –My modified version (for one-shot sessions): Did you learn something new? Do you think the session has helped you with doing the kind of research your ENG 102 assignment requires? What did you find most helpful about this session? What would you have liked more of? What would you have liked less of? Additional Comments

How to Assess Sessions Student self-evaluation –Survey in a science writing-intensive course: Did you find our discussion on XYZ helpful? a)Yes, I found it very helpful and used it for my assignment(s). b)Yes, it was helpful, but I did not wind up using it for my assignment(s). c)No, it was not helpful - I found it unclear. d)N/A - I already knew everything that was discussed. e)N/A - I was not in class that day. Which of the Reserve items did you use? Did you find the “Reserve items” handout helpful?

How to Assess Sessions Student self-evaluation –Survey for graduate students in mini-course: After our mini-course, how confident are you about using the following (17) resources? How often do you now use library resources, after our mini-course? Did you find our mini-course helpful? What did you find most helpful? What would you have liked more of? What suggestions do you have for improvement?

How to Assess Sessions Student self-evaluation –Survey in a mixed-enrollment seminar course: Beginning & end: –How confident are you about doing/using the following? (24) –What are some of the challenges or difficulties you are still encountering when looking for or using information? End: –How helpful do you think the library sessions were/will be for XYZ? –Which resources did you use to do complete your oral presentation project? –What was the most difficult part of the course?

How to Assess Sessions Professor evaluation –How did the library instruction session go? Did I cover everything you were hoping I would? Did anything seem unclear? Might you have any suggestions for how I could improve the session? –How did the assignment(s) turn out? Did student assignments indicate retained knowledge? Did student assignments show improvement?

How to Assess Sessions Professor evaluation –When: Right after library instruction session Within a week after library instruction session After the end of the semester –How: In-person Survey

How to Assess Sessions Librarian (self-)evaluation –Observations during class: Walking around during student work Students’ faces – confusion / boredom Student questions –Activities: Effectiveness Student interest –Timing

Assessing Information Literacy What are we trying to assess? –What are the learning objectives? –What is being taught? When can we assess? How can we measure skills/knowledge? –Identify appropriate type(s) of assessment –Develop / identify suitable assessment tool(s): Content (e.g., test questions, assignments) Device (e.g., software, hardware)

How to Assess Info. Literacy Objective knowledge: –True/False –Multiple choice –Choose all that apply –Matching –Matrix –Short answer completion

How to Assess Info. Literacy Higher-order thinking –Typically: Essay Long answer Oral response Student demonstration –Maybe: True/False Multiple choice Choose all that apply

Ex: Multiple Choice

Ex: Choose All That Apply Choose all that apply: C3. Where should you look to find spectra (e.g., IR, NMR, UV) information? a)CRC Handbook of Chemistry and Physics b)PubMed c)Reaxys d)SciFinder e)Web of Science

Ex: Matrix / Likert Scale

Ex: Short / Long Answer 1.Find all the items in SciFinder that’s been published in English by your assigned seminar speaker. a)What is your search strategy? Please be specific. i.Which section of SciFinder are you searching in? ii.What did you type in the search box? iii.If applicable, which of the “Author Name Candidates” did you choose? b)Identify the one that was cited the most. i.How many times was it cited? ii.Provide the citation in Nature formatting. c)How many of the references are journal articles? d)What has your assigned seminar speaker’s research focused on since 2007? How did you determine this? Please be specific. 2.Identify ONE person who has been doing research on the same topic as your assigned seminar speaker and might be considered a potential collaborator or competitor. This person cannot be someone s/he has co-written a paper with since How did you determine this (i.e., where did you look)?

How to Assess Info. Literacy Application of knowledge –In-class exercises –Evaluate assignment(s): Annotated bibliographies Papers Posters Oral presentations –Obstacles: Permissions Time

Ex: Rubric / Checklist Oral presentation:

How to Assess Info. Literacy Application of knowledge –Process: Demonstration / record thought process Search history Journal –Reflection –Choices made –Obstacles: Time Permissions

When to Assess Info. Literacy Pre- & post session During session Pre-session: –Before session –At start of session Post-session –At end of session –Shortly afterwards –Some time later

Limitations to Assessment No magic bullet –Different assessment tools for measuring different things –May need more than 1 method General limitations: –Time –Resources –Some things are difficult to measure

Notes Suggested further reading: Radcliff, C. J., Jensen, M. L., Salem Jr., J. A., Burhanna, K. J., & Gedeon, J. A. (2007). A practical guide to information literacy assessment for academic librarians. Westport, Conn: Libraries Unlimited. Unless otherwise noted, all images come from Microsoft Office’s clip art

Questions? Thank you!