1. Assessing Library Instruction for Military Students: A Study for Assessment in Action Initiative
Elizabeth Torres, Librarian (Team Leader); Timothy Martin, Head of Reference & Instruction Services; Farris Sanders, Librarian; Nori Leong, Director of Libraries & Learning Commons; Sheryl Sunia, Instructor of Justice Administration & Curriculum Area Liaison for Military Campus Programs; Han Nee Chong Wester, Instructional Design & Technology Specialist

2. Poster presentation - June 2016 ALA Orlando
This project is part of the program “Assessment in Action: Academic Libraries and Student Success” which is undertaken by the Association of College and Research Libraries (ACRL) in partnership with the Association for Institutional Research and the Association of Public and Land-grant Universities. The program, a cornerstone of ACRL's Value of Academic Libraries initiative, is made possible by the Institute of Museum and Library Services.
HPU Libraries was part of the third year cohort along with 54 other Academic Libraries
Participating institutions were divided into smaller cohorts of 9-10 institutions
The project included 14 months of webinars, collaborations, homework, and conducting the study
4 days of in-person workshops - June 2015 ALA San Francisco, January 2016 ALA Boston
Poster presentation - June 2016 ALA Orlando

3. Assess
What
You Value
Not Value
What You
Assess

4. Our Focus
Information Literacy, one of HPU’s ILOs, is the primary contribution of Reference & Instruction towards student success.
Military Campus Programs students are a unique and vital component of our student body.
Hawaii Pacific University’s Military Campus Programs (MCP) is offered through the College of Extended and Interdisciplinary Education. HPU was once again named a “Best for Vets” college and “Military Friendly School” for 2016.

5. Measurable changes that benefited students
Our Outcome
MCP students participating in library sessions regardless of method of delivery (face-to-face or hybrid) will demonstrate the ability to formulate complex search strategies and select appropriate sources to defend their position in their final essay. 
WRI 1200 ILO
Bloom’s Taxonomy
Impact of experience
Aligned with institutional values- we used specific language consistent with language used in WRI ILOs
Used active verbs that demonstrated the desired level of student achievement
Measurable changes that benefited students

6. Pre-study We applied for an expedited level of review within
our Institutional Review Board - IRB
Created a consent form
Protected the identity of students by creating codes
Created a debriefing form
Completed FERPA training
Created tools to use for collecting data
Pre- & post test
In-class search assignment with feedback
Rubric for assessing students final papers
Collaborated with AiA campus partners
Determined which MCP courses would participate
Determined how to use Blackboard & Google Forms

7. Our Study
Research Question: Does the method of delivery (face-to-face vs. hybrid) impact the effectiveness of library instruction for off campus military students?  
Study Population: Four WRI 1200 courses (two face-to-face, two hybrid), two MCP faculty, 28 students.

8. Our Assessment
Our study compared two library interventions:
Traditional MCP face-to-face 50 minute library instruction session at military bases.
Hybrid intervention that included 3-5 minute library tutorials in addition to a 50 minute library session at military bases.
Our assessment tools:
Pre- & post tests: 10 multiple Qs
Search assignment: 16 open-ended Qs
Final argumentative paper: Evaluated by 2 librarians with in-house rubric
We assessed 76 artifacts from 24 students, 11 from face-to-face and 13 from hybrid.
Unexpected Challenge: Students were not signing consent forms prior to instruction session. We had to ask for consent verbally, we signed up students during class and provided codes which impacted the amount of time we had to deliver instruction.

9. Our Tools Pre- & Post Test
Short questionnaire (ten questions) – due to time constraints with students.
Questions ranged from selecting proper resources for searching, to confidence in doing library research.

10. Our Tools Library Assignment 16 open ended questions
Questions prompted students to search specific databases, employ advanced search strategies, cite a relevant source and provide feedback on our instruction.

11. Our Tools Rubric Two librarians reviewed the final papers
We focused on selection of sources used in paper (variety and quality).
How were the sources used to support arguments?
Were students providing proper credit for their sources?

12. Results
Overall post-test scores improved for both intervention modes. Hybrid students had a higher pre-test score overall. We attribute this to the fact that 50% of students in the hybrid model took the pre-test after viewing the library tutorials.
Unexpected Challenge:
Rubric scores were higher for face-to-face. We attribute this to the unanticipated difference between the two instructors’ argumentative essay requirements.

13. Results
Most students found appropriate sources in the search activity, yet still had problems implementing some advanced search techniques such as truncation.
Hybrid students were more likely to express being confident in their research skills than face-to-face students.

14. Conclusions
Overall, there was an improvement in the students' search skills and their ability to find and use appropriate sources regardless of the intervention type. There was not enough statistical difference to prove one model was better than the other. Due to class sizes, our sample was small. We hope more MCP writing courses will be part of our future assessment efforts, including courses that are online only.
We acknowledge the uniqueness of MCP students and learned through the project that some elements of our instruction should be modified:
Spending additional time highlighting the benefits of the eBook collection for distance learners
Providing citation guidelines for resources
Emphasizing complex search strategies such as truncation
Adding more tutorials and expanding embedded services as teaching assistants

15. Our AiA Experience Used a variety of assessment tools
Flexibility - expect the unexpected
Sharing our findings within our community to cement future collaborations
Leadership - acting as facilitators within our projects/community/institution
Strong outcomes go a long way
Pressure on presenting at ALA - use not only as a deadline but as motivation
Assess what we value…

16. Value: What is of value to your institution/community, what is your library's added value to it?
Outcomes: How do you want patrons to be changed as a result of the library’s activities/actions?
Criteria: How will you know you have succeeded in proving value? What will be happening?
Actions: What will you do to make this happen?
Evidence: How will you collect results? What will you observe, judge or measure?

17. Thank You
Any Questions?