LibQUAL+ ® in South Africa Sherrie Schmidt, Arizona State and ARL President Martha Kyrillidou, ARL Stellenbosch, South Africa August 13, 2007 old.libqual.org

Overview Introduction and Overview LibQUAL+® in South Africa The importance of LibQUAL® for ARL Statistics and Measurement Program LibQUAL+® in the UK LibQUAL+® at Arizona State LibQUAL+® at the U. of Virginia Share Fair What have we learned?

old.libqual.org

World LibQUAL+ ® Survey

old.libqual.org Mission:Shaping the future of research libraries in the changing environment of public policy and scholarly communication. Members: 123 major research libraries in North America. Ratios: 4 percent of the higher education institutions providing 40 percent of the information resources. Users: Three million students and faculty served. Expenditures: 37 percent is invested in access to electronic resources.

old.libqual.org ARL Mission Non-profit organization of the libraries of research institutions in North America Forum for exchange of ideas Agent for collective action

old.libqual.org... chart a course for every endeavor that we take the people's money for, see how well we are progressing, tell the public how we are doing, stop the things that don't work, and never stop improving the things that we think are worth investing in. –President William J. Clinton, on signing the Government Performance and Results Act of 1993 Assessment – the Imperative

old.libqual.org Assessment – the Challenge “The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.” Sarah Pritchard, Library Trends, 1996

old.libqual.org Library Assessment at ARL What are some of the developments with library assessments efforts? ARL StatsQUAL ™ E-Metrics LibQUAL+ ® DigiQUAL ™ MINES for Libraries ™ Where are the most critical needs and opportunities? What are the lessons learned?

old.libqual.org LibQUAL+ Brief History LibQUAL+ TM Brief History Experience with SERVQUAL in many libraries in the 1990s Texas A&M SERVQUAL assessment in the 1990s New Measures Initiative called a meeting of interested ARL libraries (ALA Midwinter 2000) Pilot with 12 ARL libraries (spring 2000) External funding through FIPSE, U.S. Department of Education (September 2000) Participation and endorsement by consortia (Greater Midwestern Big 12 Plus Consortium, OhioLINK, etc.) Consortia and related associations interest 2003 UK participation 2005 South Africa More than 200 institutions every year

old.libqual.org Rapid Growth Languages –American English –British English –Chinese (2007) –French –Dutch –Swedish –Norwegian –Finnish –Danish Consortia –Each may create 5 local questions to add to their survey Countries –Australia, Canada, Denmark, Egypt, Finland, France, Hong Kong, New Zealand, the Netherlands, Norway, South Africa, Sweden, Switzerland, UAE, U.K., U.S. Types of Institutions –Academic Health Sciences –Academic Law –Academic Military –College or University –Community College –Electronic –European Business –Family History –FFRDC –High School (2007) –Hospital –National Health Service England –Natural Resources –New York Public –Public –Smithsonian –State –University/TAFE

old.libqual.org LibQUAL+ ® Participants * 2007 data reflects Session I data only

old.libqual.org LibQUAL+ ® Languages

old.libqual.org Why LibQUAL+? To base decisions on data –Internally –Externally –Comparative –Longitudinally Cost effective Timely data Support available Community building Sharing of experiences Learning and improvement

old.libqual.org 13 Libraries English LibQUAL+™ Version 4000 Respondents QUAL QUAN QUAL QUAN QUAL PURPOSE DATA ANALYSIS PRODUCT/RESULT Describe library environment; build theory of library service quality from user perspective Test LibQUAL+™ instrument Refine theory of service quality Refine LibQUAL+™ instrument Test LibQUAL+™ instrument Refine theory Unstructured interviews at 8 ARL institutions Web-delivered survey Unstructured interviews at Health Sciences and the Smithsonian libraries to survey administrators Web-delivered survey Focus groups Content analysis: (cards & Atlas TI) Reliability/validity analyses: Cronbachs Alpha, factor analysis, SEM, descriptive statistics Content analysis Reliability/validity analyses including Cronbachs Alpha, factor analysis, SEM, descriptive statistics Content analysis Vignette Re-tooling Iterative Emergent Libraries English, Dutch, Swedish, German LibQUAL+™ Versions 160,000 anticipated respondents LibQUAL+ ® Project Case studies 1 Valid LibQUAL+™ protocol Scalable process Enhanced understanding of user-centered views of service quality in the library environment 2 Cultural perspective 3 Refined survey delivery process and theory of service quality 4 Refined LibQUAL+™ instrument 5 Local contextual understanding of LibQUAL+™ survey responses 6

old.libqual.org Multiple Methods of Listening to Customers Transactional surveys* Mystery shopping New, declining, and lost-customer surveys Focus group interviews Customer advisory panels Service reviews Customer complaint, comment, and inquiry capture Total market surveys* Employee field reporting Employee surveys Service operating data capture *A SERVQUAL-type instrument is most suitable for these methods Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000). Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.

old.libqual.org

Premises Three Seminal Quotations

old.libqual.org PERCEPTIONS SERVICE “….only customers judge quality; all other judgments are essentially irrelevant” Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press. LibQUAL+ ™ Premise #1

old.libqual.org LibQUAL+ ™ Premise #2 “Il est plus nécessaire d'étudier les hommes que les livres” —FRANÇOIS DE LA ROCHEFOUCAULD

old.libqual.org “We only care about the things we measure.” --Bruce Thompson, CASLIN, 2006 LibQUAL+ ™ Premise #3

old.libqual.org Extended GAPS Model Organizational Barriers to SQ Customers’ Assessment of SQ Poor Upward Communication Poor Horizontal Communication Poor Tech - Job Fit Perception of Infeasibility GAP 1 GAP 2 GAP 3 GAP 4 GAP 5 Reliability Responsiveness Assurance Empathy Tangibles

old.libqual.org Dimensions items56 items25 items22 items Affect of Service Service Affect Library as Place Reliability Personal Control Information Control Provision of Physical Collections Self-Reliance Information Access Access to Information

old.libqual.org Survey Structure (Detail View)

old.libqual.org Interpreting Service Quality Data Three Interpretation Frameworks

old.libqual.org Benchmarking Against Peer Institutions --1,000,000 Users; 1,000 Institutions! NORMS! NORMS! NORMS! Interpretation Framework #1

old.libqual.org Score Norms Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample. LibQUAL+™ norms have been created at both the individual and institutional level

old.libqual.org Benchmarking Against Self, Longitudinally “Nobody is more like me than me!” --Anonymous Interpretation Framework #2

old.libqual.org Interpreting Perceived Scores Against Minimally-Acceptable and Desired Service Levels (i.e., “Zones of Tolerance”) Interpretation Framework #3

old.libqual.org

“22 Items and The Box….” Why the Box is so Important –About 40% of participants provide open- ended comments, and these are linked to demographics and quantitative data. –Users elaborate the details of their concerns. –Users feel the need to be constructive in their criticisms, and offer specific suggestions for action.

old.libqual.org “…and Five Ancillary Items” Either Zero or Five Ancillary items are selected to address local or consortial concerns –Items from the initial LibQUAL+ TM item pool. –Items written by previous consortial groups.

old.libqual.org LibQUAL+ ® Resources LibQUAL+ ® Web site: Publications: Events and Training: Gap Theory/Radar Graph Introduction: LibQUAL+ ® Procedures Manual: