U SING ASSESSMENT TECHNIQUES TO GUIDE LIBRARY SERVICE IMPROVEMENTS Diane Wahl Beth Avery Annie Downey University of North Texas

I SSUES WITH RESEARCH VALIDITY Theory problems Single Method problems Researcher problems Data problems

W HAT IS T RIANGULATION ? Validation or not Method 1 Method 2 Method 3

W HY T RIANGULATION ? Validation Balance Widen Deepen

W HERE W E S TARTED LibQUAL+ Survey 2009 Results Assessed problem areas by looking at items with largest superiority gaps Superiority gap = the difference between what is perceived and what is desired

L ARGEST O NLINE U SER S UPERIORITY G APS RankLibQUAL+ Survey ItemMean Score 1A library Web site enabling me to locate information on my own Easy-to-use access tools that allow me to find things on my own Availability of online help when using my library’s electronic resources Print and/or electronic journal collections I require for my work The electronic information resources I need-1.15

F OLLOW U P F OCUS G ROUPS Summer 2009 Distance learners Improvements to online services will benefit on campus users as well as remote users Graduate students Supports university-wide focus on research

Online Student Focus Group Recommendations Increase the Libraries’ presence and visibility in Blackboard. Make the UNT Libraries’ website easier to search. Obtain, publicize, and provide instruction in the use of a discovery tool. Provide library instruction that doesn’t require being at the physical library. Communicate and collaborate more effectively with faculty members that teach online classes. C OMPARING L IB QUAL+ S URVEY AND F OCUS G ROUP R ESULTS FOR O NLINE S TUDENTS RankLibQUAL+ Survey Items with Largest Superiority Gaps – Online Students 1A library Web site enabling me to locate information on my own 2Easy-to-use access tools that allow me to find things on my own 3Availability of online help when using my library’s electronic resources 4Print and/or electronic journal collections I require for my work 5The electronic information resources I need

C OMPARING L IB QUAL+ S URVEY AND F OCUS G ROUP R ESULTS FOR G RADUATE S TUDENTS RankLibQUAL+ Survey Items with Largest Superiority Gaps – Graduate Students 1A library Web site enabling me to locate information on my own 2Print and/or electronic journal collections I require for my work 3Easy-to-use access tools that allow me to find things on my own 4Availability of online help when using my library’s electronic resources 5The electronic information resources I need Graduate Student Focus Group Recommendations Make information of interest to graduate students easier to find on the library website Improve communication with and outreach to graduate students. Increase graduate student participation in library workshops targeted to them. Customize reference services for each discipline.

E THNOGRAPHIC A NALYSIS BY A NTHROPOLOGY G RADUATE S TUDENTS Fall 2010 Graduate level Anthropology Dept. qualitative analysis classes One on campus and one online Used a variety of methods: Focus groups Interviews Observations

C OMPARING L IB QUAL+ S URVEY, F OCUS G ROUP & E THNOGRAPHIC R ESULTS FOR O NLINE S TUDENTS RankLibQUAL+ Survey Items with Largest Superiority Gaps – Online Students 1 A library Web site enabling me to locate information on my own 2 Easy-to-use access tools that allow me to find things on my own 3 Availability of online help when using my library’s electronic resources 4Print and/or Electronic journal collections I require for my work 5The electronic information resources I need Online Student Focus Group Recommendations Increase the Libraries’ presence and visibility in Blackboard Make the UNT Libraries’ website easier to search Obtain, publicize, and provide instruction in the use of a discovery tool Provide library instruction that doesn’t require being at the physical library Communicate and collaborate more effectively with faculty members that teach online classes Anthro Class Recommendations Improve user experience with the UNT Libraries website; (includes problems with access tools) Improve library instruction Improve communication

C OMPARING L IB QUAL+ S URVEY, F OCUS G ROUP & E THNOGRAPHIC R ESULTS FOR O NLINE S TUDENTS Rank Spring 2009 LibQUAL+ Survey Items with Largest Superiority Gaps – Online Students 1 A library Web site enabling me to locate information on my own 2 Easy-to-use access tools that allow me to find things on my own 3 Availability of online help when using my library’s electronic resources 4 Print and/or Electronic journal collections I require for my work 5 The electronic information resources I need Summer 2009 Online Student Focus Group Recommendations Increase the Libraries’ presence and visibility in Blackboard Make the UNT Libraries’ website easier to search Obtain, publicize, and provide instruction in the use of a discovery tool Provide library instruction that doesn’t require being at the physical library Communicate and collaborate more effectively with faculty members that teach online classes Fall 2010 Anthro Class Recommendations Improve user experience with the UNT Libraries website (includes problems with access tools) Improve library instruction Improve communication Rank Spring 2011 LibQUAL+ Survey Items with Largest Superiority Gaps – Online Students 1 Online course support (readings, links, references) 2 Easy-to-use access tools that allow me to find things on my own 3 Making me aware of library services 4 A library Web site enabling me to locate information on my own 5 The electronic information resources I need

C OMPARING L IB QUAL+ S URVEY, F OCUS G ROUP & E THNOGRAPHIC R ESULTS FOR G RADUATE S TUDENTS RankSpring 2009 LibQUAL+ Survey Items with Largest Superiority Gaps – Graduate Students 1 A library Web site enabling me to locate information on my own 2Print and/or electronic journal collections I require for my work 3Easy-to-use access tools that allow me to find things on my own 4 Availability of online help when using my library’s electronic resources 5The electronic information resources I need Summer 2009 Graduate Student Focus Group Recommendations Make information of interest to graduate students easier to find on the library website Improve communication with and outreach to graduate students Increase graduate student participation in library workshops targeted to them Customize reference services for each discipline Fall 2010 Anthro Class Recommendations Improve user experience with the UNT Libraries website Improve library instruction Improve communication

C OMPARING L IB QUAL+ S URVEY, F OCUS G ROUP & E THNOGRAPHIC R ESULTS FOR G RADUATE S TUDENTS Rank Spring 2009 LibQUAL+ Survey Items with Largest Superiority Gaps – Graduate Students 1 A library Web site enabling me to locate information on my own 2 Print and/or electronic journal collections I require for my work 3 Easy-to-use access tools that allow me to find things on my own 4 Availability of online help when using my library’s electronic resources 5 The electronic information resources I need Summer 2009 Graduate Student Focus Group Recommendations Make information of interest to graduate students easier to find on the library website Improve communication with and outreach to graduate students Increase graduate student participation in library workshops targeted to them Customize reference services for each discipline Fall 2010 Anthro Class Recommendations Improve user experience with the UNT Libraries website Improve library instruction Improve communication Rank Spring 2011 LibQUAL+ Survey Items with Largest Superiority Gaps – Graduate Students 1 A library Web site enabling me to locate information on my own 2 Quiet space for individual activities 3 Print and/or electronic journal collections I require for my work 4 Easy-to-use access tools that allow me to find things on my own 5 The electronic information resources I need

W ILLIS L IBRARY O BSERVATION S TUDY Pilot study 1 st floor only One week in February times per day Basic Findings Average of 100 students on first floor at a time Low: 20 High: 172 Roughly ¼ of students using laptops at any given time Average # of workstations in use: 30 Low: 8 High: 59

T OTAL P ATRONS BY T IME OF D AY

P ATRONS A LONE OR I N G ROUPS – W EEKDAYS ONLY

C OMPARISON OF L IB Q UAL 2011 TO W ILLIS P ILOT O BSERVATION S TUDY 2011 LibQUAL+ Survey Items with Largest Superiority Gaps A getaway for study, learning, or research Library space that inspires study and learning Quiet space for individual activities A library Web site enabling me to locate information on my own Modern equipment that lets me easily access needed information A comfortable and inviting location

B RAINSTORMING AFTER O BSERVATIONS Recommended by Nancy Foster (Library Anthropologist) Brainstorm possible implications based on patterns of what students DO rather than what they say they want Under utilized areas – W136 and Forum Glassed in Poor light Don’t feel as comfortable; less feeling of student ownership Highly used areas Obviously for them + workstations = high use Relaxed feel Good light; open Outlets