1. Università di Bologna Dr. Luciana Sacchetti Dr. Ida D’Attoma University of Bologna Library of the Department of Statistics Attention to the end user: how to measure quality in an ISO certified Library 7° Northumbria International Conference Performance Measurement in Libraries and Information Services Spier, South Africa 13-16 August 2007

2. The Library in 2006 Established in 1982 as a research Library in Statistics 32,000 collection 2,024 mt. of open shelves 6,39 FTE Circulation 3082 (loans, DD, ILL) Main users : Researchers of the Department of Statistics & students in Statistics and Economics ISO 9001:2000 certified by DNV since 2004

3. Why adopting ISO 9001:2000 1.360° management System for quality 2.A host of operational tools tailored specifically to the library’s profile 3.Attention to the end user 4.incremental and tangible improvements 5.The ISO “lesson learned” can be applied to different organizational context

4. The ISO QMS at work ISO 9001:2000 QMS TOOLS QUALITY POLICY START/END WORKING BY OBJECTIVES 1.STATISTICAL INDICATORS NONCONFORMITY CORRECTION PROCEDURES & DESK INSTRUCTIONS 2.CUSTOMER SATISFACTION SURVEY INTERNAL & EXTERNAL AUDIT FOCUS GROUP 2.1 Cluster Analysis on Principal Component (work in progress)

5. Measuring/monitoring/evaluating quality Working by measurable objectives 22 statistical indicators. Six of them monitor back-office procedures Three others measure our user’s abilities and habits The remaining are taken from the Customer Satisfaction Survey The treatment of nonconformities consists of two steps: time taken to process new material the average usage of the physical journals & review section 1.monitoring and solving recurrent or minor nonconformities 2. discovering, analyzing and solving unpredictable or major ones

6. Measuring/monitoring/evaluating quality with QMS The Customer Satisfaction Annual Survey finds out what the users know (or perhaps more importantly don’t know) about the Library, in terms of relevance, usage and satisfaction Internal & External audit The Focus group is annual qualitative survey to evaluate strategic topic with our main stakeholder group, the Professors and researchers in the Department of Statistics 10-15 minute interviews with students and junior researchers over a thirty day period. The Questionnaire is divided in nine sections, the Satisfaction section being the most detailed The INTERNAL - conformity to procedures and specifications The EXTERNAL (DNV)- conformity of the System to the ISO standard and the improvement achieved by the library ” how best to evaluate our current journal collections ?” (FC2004)

7. The library among Statisticians: an ideal context The researchers of the Department of Statistics were very helpful during the implementing phase of the ISO QMS (statistical indicators, questionnaire set up etc.) Now that the System in fully implemented, we co-work to sharpen our measurement tools mainly focusing on the users expectations. Here comes the statistical research made with Dr. Ida D’Attoma I will present you today.

8. The ISO QMS at work (work in progress) Cluster Analysis on Principal Component

9. Descriptive vs Qualitative indicators UNIVERSITY LIBRARY SYSTEM QUANTITATIVE INDICATOR Loans total per year  Source: Data warehouse  Calculus: centralized query OUR OUTPUT 2754 loans in 2006 OUR RANKING 11° among 15 Social Science libraries of the University QUALITATIVE INDICATOR OF THE LIBRARY The “appeal” of the library’s books collection per year (%)  Source: QMS  Calculus: ad hoc indicator  Methodology: total of borrowable books with at least one loan / total of borrowable books OUTPUT 1077/15,072 *100= 7% in 2006 BENCHMARK Univ. Library of Economics Sciences 9% Univ. Library of Management Sciences 14%

10. Comments &corrections Loans total per year  From the University System point of view, the Library seems poorly efficient in meeting the information demands of the users – if compared to others with bigger collections and much more users & loans  The Image of the library is penalized UNIVERSITY QUANTITATIVE INDICATOR The “appeal” of the Library’s books collection per year  Since the average “collection’s appeal” is 10 % the Library 7% score seems good enough Considering that:  50 % of our collection is not borrowable (others libraries >90 %)  the indicator is not fully efficient since it doesn’t show which kind or part of the collection is more “appealing” (new books, specific matters or disciplines…). QUALITATIVE INDICATOR OF THE LIBRARY

11. Economics THE CUSTOMER SATISFACTION SURVEY The users’ segmentation by Faculty Library’s users are mainly students from the Faculties of Statistics and Economics

12. THE CUSTOMER SATISFACTION SURVEY The users' group In 2006 we interviewed 145 individuals, with some sets of questions being targeted at specific user groups: students, PhDs, Junior researchers Sample is mainly made up of students and so the analysis will only concern this users’ group

13. GLOBAL SATISFACTION FREQUENCY % 1. Unsatisfied00 2. Not much unsatisfied00 3. Nor unsatisfied nor satisfied00 4. Not much satisfied65,71 5. satisfied3735,23 6. much satisfied5047,61 7. absolutely satisfied1211,42 A high percentage of users has a high level of satisfaction The Customer satisfaction Survey

14. THE CUSTOMER SATISFACTION SURVEY The users have attributed an average score greater than 4 to all services In the measurement scale this corresponds to a higher importance, with not the same related frequency The importance attributed to the services allows us to believe that exists a balance between the service’s offer and the user’s requirements

15. THE CUSTOMER SATISFACTION SURVEY The average satisfaction scores toward general library’s aspects Scores belong to high range of satisfaction 1 = Library’s opening Hours 2 = cloakroom service convenience 3 = reading room seating 4 = PC availability 5.= reading room neatness 6 = reading room clinatization 7 = reading room neatness in the basement 8 = lighting in the reading room 9 = reading room clinatization in the basement 10 = clear information delivered 11= respect for the library's regulations 12 = librarian’s professional competence 13 = librarian’s helpfulness

16. THE CUSTOMER SATISFACTION SURVEY Our users are mainly satisfied of: Bibliographic online databases Web site Wireless access These aspects satisfy the demand of Library remote accessibility 1 = the books stock meets my requirements 2 = copies of textbooks available 3 = teachers notes availability 4 = the borrow duration appropriateness 5.= number of book that can be borrowed simultaneously 6 = OPAC 7 = the Library’s web site 8 = Wi-Fi Access 9 = Photocopying facilities

17. THE PRINCIPAL COMPONENT ANALYSIS Dr. Ida D’Attoma We analyze user’s satisfaction questionnaire with cluster analysis on principal components obtained from a previous PCA procedure The cluster analysis application to the Library is a “work in progress” aimed to go deeper into understanding the user satisfaction and his changing needs and requirements Objectives are: to discover or to reduce the variables’ dimensionality of the data set to identify new meaningful underlying variables Eventually, to eliminate redundancy of some items of the questionnaire

18. THE PRINCIPAL COMPONENT ANALYSIS How it works PCA reduces the variables’ number that describe units (the users) Represents them by a smaller number of new uncorrelated variables called PRINCIPAL COMPONENT (or dimensions) PC - being uncorrelated by construction - are a linear combination of original variables: a weighted sum of the original variables So PC are unidentified variables that allow us to attribute new scores to units

19. PCA The reduction’s procedure  How many PC to retain?  We have started with 22 original variables of the Questionnaire  it is reasonable to retain 7 dimensions, because they accounted for 50% of data variability (…) continue 14 to 22 Number Eigenvalue % explained variance % cumulated explained variance 12,768412,58 22,405010,9323,52 31,79008,1431,65 41,63827,4539,10 51,44326,5645,66 61,24805,6751,33 71,14225,1956,52 81,01834,6356,52 91,01834,5565,70 100,96244,3770,08 110,88414,0274,10 120,80433,6677,75 130,77823,5481,29

20. PCA Examining the correlations Once decided the number of component to retain, we assign a semantic name to each of them by examining the correlation between variables and components: 1.Efficiency of the lending service and professional competence of the Library staff 2.Efficiency of reading room and the web site 3.Library accessibility 4.Logistics & Infrastructures 5.Protection for the user’s property 6.Availability of textbooks 7.Quality of the book collection

21. Explained variance: 12,58% Efficiency of the lending service and professional competence of the library staff POSITIVE CORRELATION Teacher's notes availability0,40 The duration of borrow meets my needs0,37 There are enough copies of textbooks0,36 The number of book that can be borrowed simultaneously is sufficient 0,35 NEGATIVE CORRELATION The Library Staff competence-0,52 The Respect for the Library's regulations-0,51 Clear information delivered at the Library's desk-0,46 Helpfulness of the library staff-0,46 PRINCIPAL COMPONENT 1

22. Explained variance: 10,93% Efficiency of the Reading Room and the web site POSITIVE CORRELATION The Reading room neatness0,43 The Reading room climate0,38 The Reading room lighting0,33 The Reading room seating0,19 NEGATIVE CORRELATION The Web site-0,64 PRINCIPAL COMPONENT 2

23. PRINCIPAL COMPONENT 3 and 4 Component 3 Explained variance : 8,14% Library accessibility POSITIVE CORRELATION The Opening hours0,53 The Reading room seating0,28 The Wireless access0,21 NEGATIVE CORRELATION The Reading room climate-0,41 The PC availability-0,38 The Lighting in the Reference rooms in the basement -0,31 Component 4 Explained variance: 7,45% Logistics & Infrastructures POSITIVE CORRELATION The PC availability0,50 Photocopying facilities0,29 The Cloakroom service convenience 0,25 NEGATIVE CORRELATION The Electronic journals-0,57 The Opening hours of the L.-0,51

24. PRINCIPAL COMPONENT 5,6 and 7 Component 5 Explained variance: 6,56% Protection for the user's property POSITIVE CORRELATION Cloakroom service convenience 0,59 NEGATIVE CORRELATION Helpfulness of the Library Staff -0,23 Respect for the Library's regulations -0,19 Component 6 Explained variance: 5,67% Availability of textbooks POSITIVE CORRELATION There are enough copies of textbooks0,45 NEGATIVE CORRELATION Photocopying facilities-0,34 Teacher's notes availability-0,29 The duration of borrow meets my needs-0,27 Component 7 Explained variance: 5,19% Quality of the books’ collection POSITIVE CORRELATION The book stock meets my requirement0,27 Web site0,25 The duration of borrow meets my needs0,18 NEGATIVE CORRELATION Electronic journals-0,27 There are enough copies of textbooks-0,23

25. PCA & REGRESSION ANALYSIS PC/global satisfaction Based on PCA, we estimate the relation between each new PC and the Student’s GLOBAL SATISFACTION To do this we have implemented a REGRESSION ANALYSIS The PREDICTORS are the 7 component retained, and the DEPENDENT variable is the global satisfaction

26. Analysis of variance SourceDFSum of squares Mean square F valuePr>F Model712.044871.720703.480.0023 Error9747.917030.49399 Corrected Total 10459.96190 Root MSE0.70284R-Square0.2009 Dependent mean5.64762Adj R-Square0.1432 Coeff var12.44497 Despite the low R-square’s value the model satisfaction is on the whole significant The predictors explain the global user’s satisfaction PC/global satisfation REGRESSION ANALYSIS

27. Significative PC’s on global satisfaction PARAMETER ESTIMATES VariableDF Parameter estimate Standard error t ValuePr > | t | Intercept15.647620.0685982.34<.0001 PC 110.081460.041221.980.0510 PC 210.091140.044232.060.0420 PC 31-0.090430.05127-1.760.0809 PC 410.089060.053591.660.0997 PC 510.097330.057101.700.0915 PC 610.029060.061400.470.6371 PC 710.172550.064182.690.0084 Only three components are statistical significant & influence the global satisfaction:  PC1 Efficiency of lending service and professional competence of the library staff  PC2 Efficiency of reading room and web site  PC7 Quality of Library’s books collection MANAGERIAL IMPLICATIONS FOR QUALITY IMPROVEMENT IMPORTANT!

28. 20.00 % 41.90 % 38.10 % CLUSTER ANALYSIS ON PC Finally we implemented a cluster analysis procedure on the principal components obtained. The result is groupings of like observations that are different from other groupings. The choice of how many cluster retain depends of analysis purpose, managerial goal, budget constrain. Our goal is to generate a cluster solution that results in a number of subgroups that generates useful information for the Library management, and with cluster characteristics easy to describe. CA DENDOGRAM

29. CLUSTER ANALYSIS ON PC Once the specific solution set is selected, we also describe the cluster obtained in terms of other variables that did not directly participate to the clustering procedure. We asked what are variables that characterize the clusters. We asked if the frequency or the mean of variables considered is higher or lower in the cluster than in overall sample.

30. CLUSTER 1 “THE DO IT YOURSELF” Users are not from the Faculty of Economics, they live off the City Campus, go to Library 3 times a month. They think our book stock doesn't meet their needs, rating this service as less satisfying. They are less satisfied than the overall sample of our web site, neatness of lecture rooms, helpfulness and professional competence of the Library staff, respect for the Library's regulations, information delivered at the Information desk. But they are most satisfied than the overall sample of teacher's note availability, reading room seating, PC availability, journals of any support. They rate physical & electronic journals and internet access as the most important services. Photocopies is the most used service, the reading room the less used. They rate global service quality less than overall sample's rate 41,90% of the overall sample

31. Cluster 2 “THE CRAMMER” Users are mainly from the Faculty of Economics, they don't live in the City Campus, describe Library as a place where they can study, not as a place with Pcs or internet facilities. They think our book stock meet their needs, rating this service as most satisfying. They are more satisfied than the overall sample of: lending service, number of copies of textbooks, the OPAC & web site, number of borrowing books, helpfulness and professional competence, respect for the library' s regulation, information delivered at the Info desk. They are less satisfied than the overall sample of: teacher's notes availability, cloakroom service convenience, library's opening hours, reading room seating, lighting & climate, pc availability & wireless access. They rate physical & electronic journals, library's staff assistance as the less important services. Otherwise, they rate ”the open shelf” as the more important. Lending and Italian Statistical sources are the less used. 38,10% of the overall sample

32. Cluster 3 “THE GREGARIOUS ” They are not Economics students, but from Political Science. They go to the library once a week, they describe it as a meeting place, not as a place where they can borrow books and other materials. They are more satisfied than overall sample of: reading room climate neatness, lightning and seating, cloakroom service convenience, helpfulness and professional competence, respect for the library' s regulations, information delivered at the Information desk. They are less satisfied than overall sample of: lending service, number of copies of textbooks, number of borrowing books. They rate availability of lecture rooms and library's staff assistance as the most important services. Photocopies is the less used service, and both teacher's note service and the Italian Statistical sources are the most used. 20,00% of the overall sample

33. FINAL REMARKS (the Statistician) ADVANTAGES Applying the Cluster Analysis to identify underlying groups of observations within the data, allows us to account for user’s heterogeneity We can answer the questions like: “For what kind of people does quality system better work?” “ Who are our the students attending the library?” DISADVANTAGES An important disadvantage of the proposed analysis stems from the clustering procedure That is, the clustering procedure can result in many plausible solutions, with no apparent justification for choosing one over another

34. FINAL REMARKS (the Librarian) ADVANTAGES The research basically confirms our POLICY STATEMENT FOR QUALITY which main points are: staff competence & proactive attitude, collection growth according to the user’s needs, facility management The “do it yourself” browser of electronic information, cohabits with a more traditional user - “the crammer” for who books are still valuable objects DISADVANTAGES CA on PC needs a high statistical competence to be applied How much statistical competence is required to improve an ISO QMS? How many Ida D’Attoma are available in libraries? QUALITY ACTIONS  CA on PC outcomes will be discussed at the library’s annual assessment meeting  The Questionnaire’s variables could be reduced

35. Grazie  Quality System Management luciana.sacchetti@unibo.it  Statistical procedures Ida.dattoma2@unibo.it  Video clips Michele.cardamone@studio.unibo.it Images from the libraries of the University of Bologna Social Science Division and Library “Sala Borsa” Municipality of Bologna Dipartimento di Scienze Statistiche copyright 2007