“22 items and a box” Martha Kyrillidou Association of Research Libraries NELA Boston, MA September 28, 2004 old.libqual.org

The Association of Research Libraries www. A SSOCIATION OF R ESEARCH L IBRARIES Mission:Shaping and influencing forces affecting the future of research libraries in the process of scholarly communication. Members:123 major research libraries in North America. Ratios:4% of the higher education institutions providing 40% of the information resources. Users:3 million students and faculty served. Expenditures:$2.48 billion annually, $954 million for acquisitions of which 20% is invested in access to electronic resources.

old.libqual.org Assessment “ The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable— i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.” Sarah Pritchard, Library Trends, 1996

old.libqual.org The Challenge of Assessment in Libraries Traditional statistics emphasize inputs, expenditures, acquisitions, holdings, etc. Helping funding agencies understand success of investment No demonstrable relationship between expenditures and service quality Lack of metrics describing outcomes: success from the user’s point of view Need to redesign library services to better meet changing patterns of use Building the climate, tools, and skill set for library assessment

old.libqual.org ARL New Measures Initiative Collaboration among member leaders with strong interest in this area Specific projects developed with different models for exploration Intent to make resulting tools and methodologies available to full membership and wider community

old.libqual.org ….only customers judge quality; all other judgments are essentially irrelevant” Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.

old.libqual.org LibQUAL+™ Goals Improve mechanisms and protocols for evaluating libraries Develop web-based tools for assessing library service quality Identify best practices in providing library service Support libraries seeking to understand changes in user behavior Assist libraries seeking to re-position library services in the new environment

old.libqual.org LibQUAL+™ Outcomes Securing information that contributes meaningfully to planning and improvement efforts at a local level Providing analytical frameworks that institutional staff can apply without extensive training or assistance Helping decision-makers understand success of investments Finding useful inter-institutional comparisons

old.libqual.org LibQUAL+™ Resources An ARL/Texas A&M University joint developmental effort based on SERVQUAL. LibQUAL+ ™ initially supported by a 3-year grant from the U.S. Department of Education’s Fund for the Improvement of Post-Secondary Education (FIPSE) Initial project established a expert team, re-grounded SERVQUAL concepts, and designed survey methodology Survey conducted at over 500 libraries resulting in a data base of over half a million user responses NSF funded project to refocus LibQUAL+™ on the National Science Digital Library (NSDL)

old.libqual.org “22 items” items56-items25-items22-items Affect of Service Service Affect ReliabilityLibrary as Place ReliabilityPersonal Control Information Control Provision of Physical Collections Self-RelianceInformation Access Access to Information

old.libqual.org

Survey Structure – Page 2 (Detail View)

old.libqual.org Key to Radar Charts

old.libqual.org

Key to Bar Charts

old.libqual.org LibQUAL+™ 2004 Summary Colleges or Universities American English (n = 69,449)

old.libqual.org LibQUAL+™ Growth

old.libqual.org 2004 LibQUAL+™ Survey Activity

old.libqual.org LibQUAL+™ 2004 by the Numbers Num. of InstitutionsCompleted SurveysValid Surveys Total202112,551107,069 American English16986,19581,852 College or University12174,66070,839 Academic Law234,2444,160 Academic Health Sciences124,0773,761 Hospital71,1131,024 Community College Family History Smithsonian Institution Public18786 British English2222,03921,530 College or University1820,51720,056 European Business41,5221,474 Swedish53,0342,430 Swedish English29286 French Canadian Continental French Dutch Dutch English17066 ARL5733,03531,494

old.libqual.org Rapid Growth in Other Areas Languages –American English –British English –French –Dutch –Swedish Consortia –Each may create 5 local questions to add to their survey Types of Institutions –Academic Health Sciences –Academic Law –Academic Military –College or University –Community College –European Business –Hospital –Public –State Countries - U.S., U.K., Canada, the Netherlands, Sweden, France, Australia

old.libqual.org The Box About 40% of participants provide open-ended comments, and these are linked to demographics and quantitative data.

old.libqual.org Library values are reflected in: physical environment (Library as Space) warmth, empathy, reliability and assurance of library staff (Affect of Service) ability to control the information universe in an efficient way (Information Control) and are unifying and powerful forces for: Overcoming language and cultural barriers Bridging the worlds of our users Improving library services Advancing the betterment of individuals and societies Library Values

old.libqual.org In Closing LibQUAL+™ methodology focuses on success from the users point of view (outcomes) Demonstrates that a web-based survey can handle large numbers; users are willing to fill it out; and survey can be executed quickly with minimal expense LibQUAL+™ requires limited local survey expertise and resources Analysis available at local and inter-institutional levels Many opportunities for using demographics to discern user behaviors

old.libqual.org Multiple methods Transactional surveys* Mystery shopping New, declining, and lost-customer surveys Focus group interviews Customer advisory panels Service reviews Customer complaint, comment, and inquiry capture Total market surveys* Employee field reporting Employee surveys Service operating data capture *A SERVQUAL-type instrument is most suitable for these methods Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000). Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.

old.libqual.org Discussion Understand how to interpret radar charts, bar charts, and identifying differences among user groups Identify immediate actions your library can take to improve the service quality scores Identify long-term actions that your library can take to improve library services

old.libqual.org Short-term actions Improve DD/ILL, scan articles Proxy servers, marketing, db of the month, report to faculty, communication, usability Other methods to talk/listen to users Publicize data and use of information Outreach/communication Share info with the staff Use library faculty communication channels

old.libqual.org LibQUAL+™ Resources LibQUAL+™ Website: Publications: Events and Training: LibQUAL+™ Bibliography: LibQUAL+™ Procedures Manual:

old.libqual.org

Martha Kyrillidou Director ARL Statistics and Measurement Program