1. Collaborative Development of a Library Instruction Assessment Plan for Continuous Improvement
Sarah Lucchesi
Learning Services Librarian
University of Southern Maine

2. University of Southern Maine
Four year public university
7,800+ students
3 campuses
Portland
Gorham
Lewiston/Auburn
- Before we jump in, wanted to share brief background info on USM to help you contextualize our instruction and assessment process.
- We work with many courses on campus across a wide range of majors and colleges.

3. Assessment Background
No formalized university learning goal for info lit
Little coordination among librarians who teach
Developed internal self-assessments to as a stepping-stone to larger assessment effort
Rubric assessments only allow us to evaluate our work in course-integrated sessions where students produce easily assessable work products such as written papers or presentations.
This talk will focus on the development of our continuous improvement processes and benefits we have seen so far.

4. Assessment Goals Self-Assessment Program Assessment
Improve teaching practice
Determine value of IL instruction to students
Develop new pedagogical skills
Determine benefits to the university
Looks at individual classes and librarians
Looks at entire instruction program - higher level

5. Program Development April 2018
Listed data that could be collected from four groups
Students
Faculty
Each other
Ourselves
Examples of data included session worksheets, surveys, final papers or projects, informal conversations, lesson plan feedback from colleagues, workshop attendance, and resource usage.
(c) someecards.com

6. Program Development Settled on four methods: Peer observation
Student work analysis
Self-assessment
Student and/or faculty feedback
Retreat – focused on developing outcomes, some academic year planning, and developing continuous improvement processes. Got off campus! Very important.

7. Peer Observation
Each librarian observed once/semester Pre-meeting and post-meeting Open-ended discussion questions vs. rubrics Both parties benefit
Pre-meeting no more than a week in advance. Discuss any specific areas of focus, as well as any off-limits areas. Post-observation meeting no more than a week after.
Elected to use an open-ended form rather than a rubric because we wanted to encourage reflection and discussion rather than a summative evaluation. The score is not the point.
Librarians give input to coordinator on the type of sessions or skills they are interested in observing. For example, if one librarian is very good at providing context and another librarian is interested in developing her skills in that area, that would be a good match for observation. Librarian being observed feels that they are modeling a good practice as well as being evaluated.

8. Student Work Analysis
Focus on in-session work Formative or summative One course per semester Determine whether in-session activities are useful
Example: collect student worksheets Same class over multiple semesters How well do students understand concepts based on worksheet answers?
While rubric assessments focus on end-of-semester student work products, this method focuses on in-session student work such as worksheets or answers to online forms, etc.
Are students struggling with in-session work? Conversely, is it too easy?
Goal is to modify future sessions to better fit student needs

9. Self Assessment
Based on ACRL Proficiencies for Instruction Librarians Each librarian modifies for own responsibilities Once/year Most personal and flexible of new methods
Uses include:
Setting annual goals
Identifying professional development
Basis for journaling
Identifying skills to observe in peer observation.
ACRL Proficiencies include instructional design, administrative, communication, teaching, and others. Not every teaching librarian has administrative duties, and some do very little instructional design. Each librarian therefore modified the list of proficiencies to reflect their own job duties.
Each year, in the spring. Then summer is spent on an improvement plan.
Since this could touch on sensitive areas, librarians can use this tool however they like. No requirement to share with the group or discuss with supervisor.

10. Student and Faculty Feedback
Students:
If you previously had a library session, how much of today’s session was still new to you?
The librarian was approachable and I left comfortable asking questions (if I had them) (agree/disagree scale)
What was the most/least useful thing about today’s session? (free response)
Name two differences between a scholarly/academic journal and a magazine. (free response)
Faculty:
The library session met the information needs of my students (agree/disagree scale)
Student participation was encouraged (agree/disagree scale)
The pedagogical techniques used during this session were effective and appropriate to the information covered. (free response)
How do you anticipate that the information covered in the library session will help your students achieve the course goals? (free response)

11. Challenges Team building needed to develop trust
Long history of working independently
Fear of how assessment data will be used
Slower start than I had hoped for

12. Next Steps Survey faculty again at end of semester
Bring more formalization to processes
Close the loop on self-assessment
Program-level assessment
Continue pedagogical sessions for teaching librarians
Surveying faculty at the end of the semester to see if library instruction helped with final products - don’t want to be too annoying sending them multiple surveys. Also not sure if they will respond after end of semester, but we’re going to try it.
More formalization – tried to do too much all at once with the implementation of these different methods. Improve processes for collecting and assessing in-session student work, as well as some type of close the loop activity on self-assessment. Hope to do another retreat this August and talk about improvements.
Currently we don’t survey students who have been in an instruction session – many studies have shown that students have difficulty determining whether a particular class or lesson was useful to their learning so this has not been a priority for us, but we do want to try it.
All of the librarians agreed that once per semester pedagogical sessions focused on a teaching technique would be very helpful. Last summer we talked about developing session outcomes, and over the winter we had a team lunch where we discussed ways to encourage student participation in class. Ideas for these pedagogical sessions can now come from the continuous improvement processes.

13. Sarah Lucchesi sarah.lucchesi@maine.edu
Thank you!
Sarah Lucchesi