1. Assessing Library Performance:
New Measures, Methods, and Models
24th IATUL Conference • 2-5 June 2003 • Ankara, Turkey
Julia C. Blixrud, ARL Assistant Executive Director, External Relations

2. Measures that Matter Input Output Outcome Impact
Consistent with organizational mission, goals and objectives
Integration with program review
Balance customer, stakeholder, and employee interests and needs
Establish accountability
Collection and use of reliable and valid data

3. Forces and Challenges
Increasing demand for libraries to demonstrate outcomes/impacts in areas of importance to institution
Increasing pressure to maximize use of resources through benchmarking resulting in:
Cost savings
Reallocation

4. ARL New Measures Projects
Demonstration project for service effectiveness measures (LibQUAL+™)
Project to define usage measures for electronic information resources (includes institutional outcomes)
Identification of measures that demonstrate a library’s contribution to student learning outcomes
Investigation of role libraries play in support of the research process
Identification of cost-drivers and development of cost-benefit studies (ILL/DocDel)

5. User Surveys
Obtain direct responses to a series of questions from the community surveyed
Identify user issues, concerns and needs
Measure library performance from the user perspective, including satisfaction
Acquire quantifiable data that can be statistically analyzed and generalizeable for the larger population
Improve or change services
Increase library visibility and marketing
Contribute to broader institutional assessment/accreditation

6. LibQUAL+™ Project Goals
Establishment of a library service quality assessment program at ARL
Development of web-based tools for assessing library service quality
Development of mechanisms and protocols for evaluating libraries
Identification of best practices in providing library service

7. Survey Instrument
3 scales:
Minimum
Desired
Perceived

8. Dimensions of Library Service Quality

11. Percentages of Acquisitions Dollars Devoted to Electronic Resources

12. ARL E-Metrics Project Three phases:
Initial Phase (May-October 2000): What do we know? Inventory of current practices at ARL libraries as to statistics, measures, processes, and activities that pertain to networked resources and services.
Second Phase (November 2000-June 2001): What can we collect? Identified and field tested an initial draft set of statistics and measures
Final phase (July 2001-December 2001): What difference does this make? Build linkages to: educational outcomes/impact, research, technical infrastructure

13. Recommended Statistics & Measures
Patron Accessible Electronic Resources (R1-3)
Use of Networked Resources & Services (U1-5)
Expenditures for Networked Resources & Related Infrastructure (C1-3)
Library Digitization Activities (D1-3)
Performance Measures (P1-3)

14. Characteristics of Each Recommended Measure
Definition
Rationale
Unit of Measure
Data source
Frequency
Process
Related Issues

15. Focus on Electronic Resources
Project COUNTER
ICOLC
Measuring Impact (Franklin/Plum)

16. Cost-Effectiveness ILL/DD study Technical Services Cost Study
Activity Based Costing
Balanced Scorecard
Baldrige Quality Award

17. Outcomes
Institutional
Learning
Research

18. Learning Outcomes
Develop strategy to involve library in campus assessment activities to demonstrate the value of the library to the learning community
Move from content view (books, subject knowledge) to competency view (what students are able to do)
Understand learning outcomes of academic degree programs
Develop curriculum segments through which the library achieves outcomes
Information Literacy Competency Standards for Higher Education approved by the Association of College and Research Libraries in January 2000

19. Standardized Assessment of Information Literacy Skills
Base Based on outcomes as defined by ACRL information literacy competency standards d on ACRL standards
Contains items not specific to a particular institution or library
Is easily administered
Has been proven valid and reliable
Assesses at institutional level
Provides for both external and internal benchmarking

20. Research Outcomes (Surrogates)
R&D expenditures
PhDs awarded/year
Post-doctoral appointments
Research awards per year
Academy memberships
Faculty quality rankings

21. New Measure Challenges
Resources (i.e., time and money)
Buy-in (commitment to a culture of assessment
Access to individuals to evaluate
Expertise to conduct evaluation
Project management experience
Appropriate benchmarks
Conceptual clarity
Measurement & design requirements
Instrument validity and reliability

22. Julia C. Blixrud
Assistant Executive Director, External Relations
Association of Research Libraries
21 Dupont Circle, Ste 800
Washington, DC 20036
ext. 133
(fax)
(cell)