ARL Statistics and Measurement Library Assessment Thessaloniki, Greece June 13-15, 2005 Brinley Franklin, University of Connecticut Martha Kyrillidou, Association of Research Libraries

The Association of Research Libraries Mission: influences the changing environment of scholarly communication and the public policies that affect research libraries and the communities they serve. Members: 123 major research libraries in North America. Ratios: 4% of the higher education institutions providing 40% of the information resources. Users: 3 million students and faculty served. Expenditures: $3 billion annually of which 25% is invested in access to electronic resources.

The Association of Research Libraries Began in Members –US Academic –Canadian Academic –Non-university Statistics was early program –collected beginning 1961/62 –descriptive and quantitative data gathered since 1908

ARL Roles

ARL Statistics and Measurement …To describe and measure the performance of research libraries and their contribution to teaching, research, scholarship and community service …

ARL Statistics and Measurement …To describe and measure the performance of research libraries and their contribution to teaching, research, scholarship and community service …

Web Presence

Descriptive and Quantitative Measures Inputs –Collection size –Expenditures –Staffing Outputs –Services –People served Ratios (inputs  outputs) –e.g., expenditures per FTE

Traditional Statistics Longest and consistent data series (1908- ) Describe level of investment Define membership for the Association of Research Libraries Used for budget justification purposes Used for benchmarking with other peer institutions Describe trends in scholarly communication and for research libraries in general

The Trends

Copyright © 2004 Association of Research Libraries Source: ARL Statistics , Association of Research Libraries, 2003

Copyright © 2004 Association of Research Libraries Source: ARL Statistics , Association of Research Libraries, 2003

Copyright © 2004 Association of Research Libraries Source: ARL Statistics , Association of Research Libraries, 2003

Copyright © 2004 Association of Research Libraries Source: ARL Statistics , Association of Research Libraries, 2003

Printed Book Circulation: All ARL Libraries Note. M. Kyrillidou and M. Young. (2002). ARL Statistics Washington, D.C.: ARL, p.7.

Research Behavior: Personal Control When searching for print journals for research: Only 13.9% ask a librarian for assistance Only 13.9% ask a librarian for assistance Only 3.2% consider consulting a librarian a preferred way of identifying information Only 3.2% consider consulting a librarian a preferred way of identifying information Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment.

Reference Decrease: All ARL Libraries Note. M. Kyrillidou and M. Young. (2002). ARL Statistics Washington, D.C.: ARL, p.7.

The Challenge of Assessment in Academic Libraries Traditional statistics emphasize inputs, expenditures, acquisitions, holdings, etc. National Rankings are often misleading No demonstrable relationship between expenditures and service quality The lack of metrics describing outcomes: success from the user’s point of view

Assessment “The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.” Sarah Pritchard, Library Trends, 1996

Measures that Matter Input Output Outcome or Impact Consistent with organizational mission, goals and objectives and integration with program review Establish accountability Collection and use of reliable and valid data –Benchmarking with peers –Over time within own library

Why New Measures for ARL Increased demand for quality instead of quantity measures Growing customer and stakeholder service expectations and need to balance customer, stakeholder, and employee interests and needs Increasing demand for libraries to demonstrate outcomes/impacts in areas of importance to institution Exploding growth in use and applications of technology Competition for scarce resources Increasing pressure to maximize use of resources through benchmarking or best practice resulting in: –Cost savings –Reallocation

ARL New Measures Initiative Collaboration among member leaders with strong interest in this area January 1999 retreat to define topics (resulted in 8 areas for investigation) Specific projects developed with different models for exploration -- most self-funded Intent to make resulting tools and methodologies available to full membership and wider community

8 Areas of Investigation and Current ARL Projects Market penetration Customer satisfaction Extent of access Teaching and Learning Research Cost effectiveness Space Organizational Capacity LibQUAL+  E-Metrics Learning Outcomes Research Outcomes Cost Drivers

Future of New Measures Initiative Incorporation of some new data elements in traditional surveys Development of workshops and consulting services for performance measures Establishment of data gathering and statistical analysis tools

A vision

StatsQUAL™

New Measures LibQUAL+™ E-metrics … Project COUNTER MINES™ DigiQUAL™ SAILS Learning Outcomes Working Group Task Force on New Ways of Measuring Collections

Measuring Collections In an environment with increasing emphasis on digital resources, what are the metrics that are appropriate for describing and characterizing collections, and what collection trends are particularly important to identify and track over time?

The Value of the Qualitative Perspectives “Only with in-depth, local, qualitative, ‘culture’ studies can libraries know and understand what compels some to remain as far away from the library as possible, while others refrain from engaging library staff in their own search for proficiency and self-reliance” (Lincoln, p. 15).

“If Foucault is correct that we in the West live in surveilled societies, then what function does self-reliance serve? …the library user who wishes to navigate resources with as little help as possible – seeks a kind of privacy from the surveillance of librarian help …Having found the relative anonymity of cyberspace and a virtual world, this self-reliant user now seeks the same independence and lack of surveillance in the text-based and digitized universe of information resources known as the library” (Lincoln, p. 12). Cultural Perspective – Self-reliance

“…the library needs to welcome them in. It needs to make them feel like this is a place where they can be in almost a haven, a refuge” (Business professor) “…It’s beyond the ease [with] which you can find information, just because the library experience is something like Greece or Athens…” (Undergraduate) “writing an undergraduate thesis with this big dome over his head…he felt really like a scholar” (Linguistics professor) Writing a dissertation in a particular library for another scholar “was an emotional experience” Cultural Perspective – Library as Place

“In the physical [vs. virtual] reality, ‘texture’ has become important. Density of collections becomes important, and, if collections are not complete, users want to know where they can find missing volumes, journal articles, and/or how swiftly interlibrary loan will work for them” (Lincoln, p. 11). Cultural Perspective - Collections

“…as users have metamorphosed from penitents to self-reliant information surfers, the rules of engagement have changed. Service is not something dispensed; rather, it is enacted as an elaborate cultural ritual, the texture and fabric of which is changing in front of us. Service may now embody multiple overlays of meaning, many too dense for anything but an anthropological fieldwork study to uncover” (Lincoln, p. 15). Service as Performance

Think outside of the box survive