1. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk UKOLN is supported by: Pitfalls and blind alleys: perspectives on quality measures Ann Chapman Interoperability Focus

2. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk Quality measurement Accuracy – is the data correct? Consistency – is the data consistent? Functional – does the data support users? Currency – is data available when needed

3. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk Measuring tape or string? Surveys –Sampling frame, sample sizes –What is being surveyed? Questionnaires –Leading questions, response scales –User perceptions Toolkits –Limitations

4. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk Survey samples Sample sizes –Minimum 100 (BNB 1440 over a year) –Split samples, bigger overall sample Sampling frame –Public and academic libraries –Staff and students –Subject areas

5. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk Quality = type of record BNB Currency Survey 1980 – 2005 Hit split into record types –Full record from BL –Formerly CIP –CIP Assume records found are correct Sometimes spot error

6. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk Errors in data LDLSCP Audit 1994 – 100 records BL, 9 each LD library Checklist of error types –008/260 dont match, spelling, indicators, added entries, consistent data in related records Limited by not seeing items

7. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk Edits = errors BLCMP (Talis) study 1992-1993 –% of records edited –Analysis of 1310 paired before/after records – but no verification of need to edit –Member libraries submitted a small set of records edited and gave their reasons for editing

8. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk Quality = user understanding Cambridge University Study 1992 300 interviews –Random (91 UG, 209 PG/staff) –Users not grouped by area of study Main problems found –Misunderstanding data (jargon) –Difficulty with OPAC labels –Misinterpreting abbreviations

9. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk Decide on parameters Cat-Assess Tool (1999-2001) –Record cataloguing policies first –Random sample book/record check BNB Currency Survey –BNB Exclusions policy LDLSCP Audit –Assessment limited by not seeing item –To assess consistency, some related records must be in sample

10. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk Re-using data BNB Currency Survey –Acquisitions trends over time –Retention of stock –Comparison with other record suppliers (1996-1999) –Monitor effect of BL policies

11. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk So remember … 1.What do you want to measure? 2.Do you already collect data you can use? 3.If not, choose an appropriate method of data collection 4.Write up methodology and results, especially for repeated studies

12. A centre of expertise in digital information management www.ukoln.ac.uk www.bath.ac.uk UKOLN Visit the UKOLN website at: http://www.ukoln.ac.uk/ Bibliographic Management http://www.ukoln.ac.uk/bib-man/ Collection Description Focus http://www.ukoln.ac.uk/cd-focus/