1. Derek Whitehead CAUL January 2003 Cheap, Useful, Fairly Valid – Next Generation Statistics

2. 2 Summary of the Session 1 Why statistics are useful / not useful 2 Why cheap, useful, fairly valid? 3 What library statistics should do 4 Some random examples 5 Issues for CAUL/CONZUL

3. 3 Why Statistics are Useful Determine how well we are going in relation to like institutions and to ourselves, over time Understand and improve service delivery Provide a basis for resource allocation and budgeting What isn’t counted isn’t valued Demonstrate extent and nature of complex changes over time Provide a more objective backing for judgement and opinion Enhance understanding of customers and their demands Opinion/user data provides an objective insight into the customer Statistics are an important way to communicate upwards

4. 4 Why Statistics are a Waste of Time They can be extremely time-consuming – libraries put more resources into them than anyone except Finance – look at ITS They can be spectacularly inaccurate and meaningless – reference statistics, for example They can be extremely misleading Comparisons almost always have flaws because of the difficulty of comparing like and like in complex service environments Much statistical information collected is NEVER used Opinion/user data is notoriously unreliable Consistency is harder to achieve than you think No-one believes them.

5. 5 Cheap, Useful, Fairly Valid Why these three characteristics? Library statistics are a part of the cost of managing, nothing more – keeping cost/effort down is essential. Useful should go without saying, but it is worth saying – if you don’t use the data, why keep it? Fairly valid is a corrective both ways – you need SOME level of validity, but you can go too far.... and of course, excessive validity can undermine the usefulness of statistics

6. 6 What Library Statistics Should Do Statistics which universities keep and make available could do the following a lot better than they do: Give an idea of actual outputs Indicate changes in type/balance of outputs Convey what we do Relate what we do to what others do – benchmarking Understand the customer better Help us to plan Provide very large numbers

7. 7 Some Other Statistics Photocopying and printing – copies Active borrowers In-house use Catalogue searches Online reserve downloads Student online sessions Loans – self-service to assisted ratio Library website hits CAUL E-Metrics survey: Electronic resources; Use of electronic resources; Expenditure; Ratios

8. 8 Some large numbers Data from various years, 2000/2002 Library website hits 2,511,659 Catalogue searches695,200 Loans (including renewals)565,786 Photocopies and prints2,229,000 Documents downloadedNA Traffic (turnstile)1,196,223 In house use of collections300,000

9. 9 Example: Who uses the library? Number who have borrowed one or more item from the library over the preceding 12 months (“active borrowers”): Higher Education students80.12% TAFE students40.39% Academic staff63.49% Other63.76% TOTAL58.03%

10. 10 Example - HEEFTSU Cost drivers allocate funds from TAFE/Higher Education and based on  Loans/info literacy classes formula  survey EFTSU counts Higher Education students FTE counts TAFE students (720 SCHs = 1 FTE) HEEFTSU combines TAFE and Higher Ed students  A precedent in copyright – 100:22  Swinburne ratio – 100:38

11. 11 Example – “hits” Here is some information on hits. Or we could use downloads, sessions or searches. Total hits 2,511,659 Less php hits-679,450 Less menu hits, images, temp-113,020 Net of misc. hits1,700,730 Less other directional-901,661 Total799,069

12. 12 Hits, what hits? Most downloaded file types Htm1,101,831 Php683,312 Gif255,949 Html136,945 Pdf112,103 Jpg20,875 Swf8,507 TOTAL2,334,868

13. 13 Example: mobile phones Question: Do you think that students should be allowed to use MOBILE PHONES in the library? Never1,69131% Yes, but only in non-silent areas1,14221% Yes, but must ring and talk quietly1,19722% Yes, any time1,45826% TOTAL5,488100%

14. 14 Example – “reference” questions 3453 in-person questions over a week, 100,000 per year Directional – university 105 (3.0%) Directional – library & hours 347 (10.1%) Referrals (all) 91 (2.6%) Service enquiries & bookings 1275 (36.9%) Photocopying and printing 395 (11.4%) Other IT-related enquiries 381 (11.0%) Catalogue enquiries 255 (7.4%) Other information enquiries 520 (15.1%)

15. 15 Issues for CAUL 1. The deemed list – unique in the world – does it work, is it useful? 2. The current statistical set – is there too much? are some things just included because they exist? 3. Are there missing statistics? – data we need, don’t have 4. If so, which statistics should we add? 5. Is the presentation format adequate, or could it do more for us? – see www.arl.org./statswww.arl.org./stats

16. 16