1. www.arl.org ARL Statistics and Measurement Library Assessment Thessaloniki, Greece June 13-15, 2005 Brinley Franklin, University of Connecticut Martha Kyrillidou, Association of Research Libraries

2. www.arl.org The Association of Research Libraries Mission: influences the changing environment of scholarly communication and the public policies that affect research libraries and the communities they serve. Members: 123 major research libraries in North America. Ratios: 4% of the higher education institutions providing 40% of the information resources. Users: 3 million students and faculty served. Expenditures: $3 billion annually of which 25% is invested in access to electronic resources.

3. www.arl.org The Association of Research Libraries Began in 1932 123 Members –US Academic –Canadian Academic –Non-university Statistics was early program –collected beginning 1961/62 –descriptive and quantitative data gathered since 1908

4. www.arl.org ARL Roles

5. www.arl.org ARL Statistics and Measurement …To describe and measure the performance of research libraries and their contribution to teaching, research, scholarship and community service …

6. www.arl.org ARL Statistics and Measurement …To describe and measure the performance of research libraries and their contribution to teaching, research, scholarship and community service …

7. www.arl.org Web Presence

8. www.arl.org Descriptive and Quantitative Measures Inputs –Collection size –Expenditures –Staffing Outputs –Services –People served Ratios (inputs  outputs) –e.g., expenditures per FTE

9. www.arl.org Traditional Statistics Longest and consistent data series (1908- ) Describe level of investment Define membership for the Association of Research Libraries Used for budget justification purposes Used for benchmarking with other peer institutions Describe trends in scholarly communication and for research libraries in general

10. www.arl.org The Trends

11. www.arl.org

12. Copyright © 2004 Association of Research Libraries Source: ARL Statistics 2002-03, Association of Research Libraries, 2003

13. www.arl.org Copyright © 2004 Association of Research Libraries Source: ARL Statistics 2002-03, Association of Research Libraries, 2003

14. www.arl.org Copyright © 2004 Association of Research Libraries Source: ARL Statistics 2002-03, Association of Research Libraries, 2003

15. www.arl.org Copyright © 2004 Association of Research Libraries Source: ARL Statistics 2002-03, Association of Research Libraries, 2003

16. www.arl.org Printed Book Circulation: All ARL Libraries Note. M. Kyrillidou and M. Young. (2002). ARL Statistics 2000-01. Washington, D.C.: ARL, p.7.

17. www.arl.org Research Behavior: Personal Control When searching for print journals for research: Only 13.9% ask a librarian for assistance Only 13.9% ask a librarian for assistance Only 3.2% consider consulting a librarian a preferred way of identifying information Only 3.2% consider consulting a librarian a preferred way of identifying information Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment.

18. www.arl.org Reference Decrease: All ARL Libraries Note. M. Kyrillidou and M. Young. (2002). ARL Statistics 2000-01. Washington, D.C.: ARL, p.7.

19. www.arl.org The Challenge of Assessment in Academic Libraries Traditional statistics emphasize inputs, expenditures, acquisitions, holdings, etc. National Rankings are often misleading No demonstrable relationship between expenditures and service quality The lack of metrics describing outcomes: success from the user’s point of view

20. www.arl.org Assessment “The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.” Sarah Pritchard, Library Trends, 1996

21. www.arl.org Measures that Matter Input Output Outcome or Impact Consistent with organizational mission, goals and objectives and integration with program review Establish accountability Collection and use of reliable and valid data –Benchmarking with peers –Over time within own library

22. www.arl.org Why New Measures for ARL Increased demand for quality instead of quantity measures Growing customer and stakeholder service expectations and need to balance customer, stakeholder, and employee interests and needs Increasing demand for libraries to demonstrate outcomes/impacts in areas of importance to institution Exploding growth in use and applications of technology Competition for scarce resources Increasing pressure to maximize use of resources through benchmarking or best practice resulting in: –Cost savings –Reallocation

23. www.arl.org ARL New Measures Initiative Collaboration among member leaders with strong interest in this area January 1999 retreat to define topics (resulted in 8 areas for investigation) Specific projects developed with different models for exploration -- most self-funded Intent to make resulting tools and methodologies available to full membership and wider community

24. www.arl.org 8 Areas of Investigation and Current ARL Projects Market penetration Customer satisfaction Extent of access Teaching and Learning Research Cost effectiveness Space Organizational Capacity LibQUAL+  E-Metrics Learning Outcomes Research Outcomes Cost Drivers

25. www.arl.org Future of New Measures Initiative Incorporation of some new data elements in traditional surveys Development of workshops and consulting services for performance measures Establishment of data gathering and statistical analysis tools

26. www.arl.org A vision

27. www.arl.org StatsQUAL™

28. www.arl.org New Measures LibQUAL+™ E-metrics … Project COUNTER MINES™ DigiQUAL™ SAILS Learning Outcomes Working Group Task Force on New Ways of Measuring Collections

29. www.arl.org Measuring Collections In an environment with increasing emphasis on digital resources, what are the metrics that are appropriate for describing and characterizing collections, and what collection trends are particularly important to identify and track over time?

30. www.arl.org The Value of the Qualitative Perspectives “Only with in-depth, local, qualitative, ‘culture’ studies can libraries know and understand what compels some to remain as far away from the library as possible, while others refrain from engaging library staff in their own search for proficiency and self-reliance” (Lincoln, p. 15).

31. www.arl.org “If Foucault is correct that we in the West live in surveilled societies, then what function does self-reliance serve? …the library user who wishes to navigate resources with as little help as possible – seeks a kind of privacy from the surveillance of librarian help …Having found the relative anonymity of cyberspace and a virtual world, this self-reliant user now seeks the same independence and lack of surveillance in the text-based and digitized universe of information resources known as the library” (Lincoln, p. 12). Cultural Perspective – Self-reliance

32. www.arl.org “…the library needs to welcome them in. It needs to make them feel like this is a place where they can be in almost a haven, a refuge” (Business professor) “…It’s beyond the ease [with] which you can find information, just because the library experience is something like Greece or Athens…” (Undergraduate) “writing an undergraduate thesis with this big dome over his head…he felt really like a scholar” (Linguistics professor) Writing a dissertation in a particular library for another scholar “was an emotional experience” Cultural Perspective – Library as Place

33. www.arl.org “In the physical [vs. virtual] reality, ‘texture’ has become important. Density of collections becomes important, and, if collections are not complete, users want to know where they can find missing volumes, journal articles, and/or how swiftly interlibrary loan will work for them” (Lincoln, p. 11). Cultural Perspective - Collections

34. www.arl.org “…as users have metamorphosed from penitents to self-reliant information surfers, the rules of engagement have changed. Service is not something dispensed; rather, it is enacted as an elaborate cultural ritual, the texture and fabric of which is changing in front of us. Service may now embody multiple overlays of meaning, many too dense for anything but an anthropological fieldwork study to uncover” (Lincoln, p. 15). Service as Performance

35. www.arl.org Think outside of the box survive