Roswitha Poll Münster, Germany Ten years after: „Measuring Quality“ revised

2/21 Measuring Quality 1st edition 1996 English, German, French, Russian, Italian only academic libraries 17 indicators

3/21 Measuring Quality 2nd revised edition 2007 academic and public libraries 40 indicators Authors: Peter te Boekhorst, Roswitha Poll

4/21 Measuring Quality: Why? So what are you doing at the moment? Writing a book about performance indicators for libraries What’s that, performance indicators? Methods for seeing whether the library is good And how do you know whether you are good? By asking users, by measuring whether we are doing things right For example? That all the books are always in their right place Yes, that’s important. And such things show you whether you are good? Well, not quite, but I can compare with other libraries And if they are worse, you know that you are good? That’s about it Okay, sounds easy But why do you need a book for that?

5/21 Changes in the library world electronic services predominant the teaching library as new role cost-effectiveness crucial competition of the institutions library as physical place widens to meeting and communication

6/21 Trends in performance measurement stakeholders‘ views quantitative and qualitative measures standardisation has proceeded - ISO will comprise e-measures user surveys find broad acceptance impact/outcome measures

7/21 More use of performance measures? More data available For electronic services - resources - expenditure - usage In national statistics - reference transactions - data for library space - attendances at user training and events But: Visits still only physical visits Digitisation missing

8/21 More use of performance measures? Joint benchmarking projects BIX –Library Index German public and academic libraries CASL (Council of Australian State Libraries) public libraries Swedish Quality Handbook all types of libraries HELMS (UK Higher Education Library Management Statistics) academic libraries Netherlands University Libraries

9/21 More use of performance measures? Joint benchmarking projects Projects: 5 Indicators: 55 Corresponding to ISO 11620: 25 Used in more than one project: 20 Individuality obvious

10/21 More use of performance measures? Interactive national statistics allow combinations of data (e.g. per capita) Less reports from individual libraries But: service level agreements with performance indicators on websites Performance measures have been integrated Time to re-shelve: 100% in 6 hours

11/21 The handbook: structure Balanced Scorecard chapter on impact/outcome bibliography for each indicator examples for possible scores

12/21 The handbook: How to choose indicators The set of indicators should cover the full range of library services consider traditional and electronic services if possible „merged“ indicators include indicators for „potentials and development“ consider the view of different stakeholders

13/21 Library quality: Stakeholder views Users Access to information worlwide Delivery of information to the desktop Speed and accuracy of delivery Good in-library working conditions Responsiveness of staff Reliability of services

14/21 Library quality: Stakeholder views Financing authorities Cost-effectiveness Clear planning, effective organisation Positive outcome on users Benefits for the institution’s goals Effective cooperation High reputation of the library

15/21 Library quality: Stakeholder views Staff Good working conditions Clear planning, straight processes Systematic staff development High reputation of the library

16/21 The handbook: How to choose indicators Only one indicator for one question? Example: How intensively is the collection used? Indicators: collection turnover loans per capita percentage of stock not used Each indicator gives a different picture and might serve different purposes of the evaluating library

17/21 The indicators A. Resources, infrastructure: What does the library offer? Library as place for learning and research 1. User area per capita 2. Seats per capita 3. Opening hours compared to demand Collections4. Expenditure on information provision per capita 5. Availability of required titles 6. Percentage of rejected sessions 7. Ratio of requests received to requests sent out in interlibrary lending 8. Immediate availability Staff9. Staff per capita Website10. Direct access from the homepage

18/21 Opening hours compared to demand Definition The actual number and time of opening hours compared to opening hours as desired by users Method: Survey Possible formula: A/B A = the number of present opening hours B = the number of hours which the users state that they need (present hours + additional hours) If a library opens 60 hours per week and in the survey users ask for 10 hours more, the score would be 60:70 = 0.86

19/21 Immediate availability Definition The percentage of immediate loans of total loans Method: Immediate loans = loans minus reservations Total loans = loans plus ILL loans received Question: The probability that a user’s loan request will be fulfilled immediately

20/21 Direct access from the homepage Definition The availability of the most frequently used resources and services via the homepage of the library’s website, measured by the number of clicks necessary and the comprehensibility of the terms used Method Cognitive walk-through: A small group of experts simulates user behaviour Question Whether the homepage leads directly or very quickly to the most frequently needed information

21/21 Direct access from the homepage Academic libraries Address Way to the library Opening times Online catalogue Lending service User card User account ILL/document delivery ReferenceDatabasesE-journals Subject access User training News, events

22/21 Direct access from the homepage Public libraries Address Way to the library Opening times Online catalogue Lending service User card User account Electronic collection Link collection ReferenceFees Services for children Branch libraries Community services News, events

ServiceClicksPoints direct information on the homepage, e.g. times open, address, catalogue search 010 unmistakable term on the homepage 33 0 ambiguous term on the homepage12 11 0 Access via the homepage

24/21 The indicators B. Use: How are the services accepted? General1. Market penetration 2. User satisfaction 3. Library visits per capita Library as place 4. Seat occupancy rate Collections5. Number of content units downloaded per capita 6. Collection use (turnover) 7. Percentage of stock not used 8. Loans per capita 9. Percentage of loans to external users Information services 10. Attendances at training lessons per capita 11. Reference questions per capita Cultural activities 12. Attendances at events per capita

25/21 Library visits per capita Definition The total number of library visits per year, either physical or virtual, by members of the population, divided by the number of persons in the population Method Physical visits: turnstile Virtual visits: web browser visits IP visits homepage visits

26/21 Library visits per capita: A method for counting homepage visits Counting the deliveries of a Transparent GIF placed on the home page for counting purposes placed on the home page for counting purposes The number of deliveries of the pixel is identical to the number of page deliveries. to the number of page deliveries. The advantage of this method is that it easy to use. The disadvantage is that access to other pages is not counted. is not counted. German Benchmarking Project BIX

27/21 The indicators C. Efficiency: Are the services offered cost-effectively? General1. Cost per user 2. Cost per visit 3. Cost per use 4. Ratio of acquisitions costs to staff costs Collection costs5. Cost per database session 6. Cost per download Processes - speed 7. Acquisition speed 8. Media processing speed 9. Employee productivity in media processing 10. Lending speed 11. Interlibrary loan speed Processes - reliability 12. Correct answer fill rate 13. Shelving accuracy

28/21 Cost per use Definition The total operating or recurrent expenditure of the library during the reporting year divided by the number of loans + in-house use + downloads from the electronic collection Method loans = lending system in-house use = sampling downloads = suppliers and library servers Question The indicator assesses the library costs per case of collection use and therewith the cost-efficiency of library services

29/21 The indicators D. Potentials and development: Is the library fit for future? Electronic services 1.Percentage of acquisitions expenditure spent on the electronic collection 2. Percentage of library staff providing end developing electronic services Staff development 3. Attendances at training lessons per staff member Budget4. Percentage of library means received by special grants or income generation 5. Percentage of institutional means allocated to the library

30/21 Attendances at training lessons per staff member Definition The number of attendance hours of staff members at formal training lessons during one year divided by the total number of library staff Method - number of attendants at each training - duration of a training A training of 3 hours with 12 attendants would be 36 attendance hours. Question What priority does the library give to staff training and therewith to the library’s ability to cope with development?

31/21 Staff training: What has been counted? Hours of training per staff member Days of training per staff member (hours calculated in days) Percentage of staff members who received training during the year Number of training lessons per staff member Training hours as percentage of total staff working hours Expenditure for staff training

32/21 Measuring Quality: What we tried to do indicators for all services new indicators practical examples for each indicator „merged“ indicators find the ultimate set of measures ?? bibliography for each indicator

It is difficult to catch a black cat in a dark room especially when it is not there Chinese Proverb