Presentation on theme: "1. Sam Kalb (Queen’s University). Using the LibQUAL +® notebooks and other LibQUAL+ services to analyze and present your results (50 minutes) 2. Eun-ha."— Presentation transcript:
1. Sam Kalb (Queen’s University). Using the LibQUAL +® notebooks and other LibQUAL+ services to analyze and present your results (50 minutes) 2. Eun-ha Hong (Wilfrid Laurier University). Using statistical tools to further analyze your LibQUAL+® data (15 minutes) 3. Questions/discussion (25 minutes)
Any LibQUAL participant can access the results of all other libraries and consortia who participated in the same year “In an example of collaboration, LibQUAL+® participants are sharing their results within the LibQUAL+® community with an openness that nevertheless respects the confidentiality of each institution and its users.” Martha Kyrillidou, 2010 LibQUAL +® survey, introduction.
Use the LibQUAL +® Notebooks & analytic utilities to perform some simple analyses of your LibQUAL+ ® survey results Present the results to your stakeholders Use the data to target areas for improvement From: presentation by R. Bowlby and M. Kyrillidou,, LibQUAL+® Canada Workshop, Ottawa, Ontario, Canada, October 24-25, 2007
¾ of members chose the Lite format
2010 LibQUAL+® Lite: 61.7% 2010 LibQUAL+® Lite: 61.7% 2010 LibQUAL+® full: 54.3% 2010 LibQUAL+® full: 54.3% 2007 LibQUAL+® full: 48.8% 2007 LibQUAL+® full: 48.8% Completed Survey. The user has supplied a rating for all items on the survey.
absolute 1. There are no absolute high or low scores. relative indicators 2. Scores are relative indicators comparison with other scoresthe same survey another yearotherlibraries 3. Scores are only meaningful in comparison with other scores in the same survey, your survey from another year, other individual libraries and the consortial totals.
1. Zone of tolerance Perceptions vs. expectations meeting users minimum expectations (Adequacy Gap) Approaching users’ desired expectations (Superiority Gap) 2. My scores over time (longitudinal) Am I doing better or worse compared to last time I measured my performance 3. Peer comparisons From presentation by M. Kyrillidou, ALA, June 2007
User Assigned: value Desired. How highly do I value it? least Minimum. What is my least expectation? actual Perceived. My actual rating Calculated Scores: Adequacy Gap. Perceived – Minimum Superiority Gap. Desired – Perceived
important 1. Desired. How important is a service relative to other services; relatively to same services among other libraries above or below Minimum 2. Adequacy Gap. How far above or below Minimum expectation is the service delivered.
Focus mainly on your results by User Group The major academic user groups (faculty, grads and undergrads) have quite different perspectives and expectations of library services. Overall totals from different libraries or different years could be skewed by variations in user groups ratios. 5 most and the 5 least desired Look at the 5 most and the 5 least desired questions by each user group 5 highest and the 5 lowest gap scores Look at the services with 5 highest and the 5 lowest gap scores (Bear in mind that Superiority Gap scores are usually negatives. correlation Look for any correlation between the high/low Desired and the high/low Gap scores average (mean) each service dimension Look at the average (mean) scores for each service dimension by user group
Queen's 2007Queen’s 2010 Canadian Consortium (Universities) 2010 Dim.QuestionScoreDim.QuestionScoreDim.QuestionScore Overall IC-1Making electronic resources accessible from my home or office 8.51 IC-1Making electronic resources accessible from my home or office 8.51IC-1Making electronic resources accessible from my home or office 8.22 IC-8Print and/or electronic journal collections I require for my work 8.48 IC-2A library Web site enabling me to locate information on my own 8.38IC02A library Web site enabling me to locate information on my own 8.17 IC-4The electronic information resources I need 8.39 IC-8Print and/or electronic journal collections I require for my work 8.32IC-8Print and/or electronic journal collections I require for my work 8.16 IC-2A library Web site enabling me to locate information on my own 8.38 IC-4The electronic information resources I need 8.16IC-6Easy-to-use access tools that allow me to find things on my own 8.06 IC-6Easy-to-use access tools that allow me to find things on my own 8.30IC-6Easy-to-use access tools that allow me to find things on my own 8.15AS-3Employees who are consistently courteous 8.06
In addition to analyzing your results by User Group, you can analyze your results by Discipline using ARL’s Analytics utility ( to be covered later in the presentation ). Standard Disciplines Focus on LibQUAL +®’s Standard Disciplines › Facilitates comparison with other libraries › Most useful for Information Control dimension
Repor t - results notebook (PDF) Comments - respondents’ free-text comments (CSV or TXT) Consortial Notebooks – Consortium, CARL, CREPUQ, OCUL (PDF) Raw Data – individual records of every respondent’s survey in CSV format (spreadsheet) Data Keys & SPSS Syntax – description of the raw data elements or fields (MS Word)
Sections for Overall, Undergraduates, Graduates, Faculty, Staff, Library Staff include: › Demographic Summary › Core Questions Summary › Dimensions Summary › Local Questions › General Satisfaction Questions › Information Literacy Outcomes Questions › Library Use Summary Appendix describing changes in the dimensions and the questions included in each dimension.
Tables & Charts Individual questions – average scores & standard deviations Dimensions summary
AS-1 Desired as 7.64, the SD would be 0 How closely does a mean score in a notebook represent all the individual respondents scores for the particular item. If all respondents rated AS-1 Desired as 7.64, the SD would be 0
Desired Perceived Minimum Zone of tolerance Target
Range of Mean scores is relatively narrow (6.27 – 8.03) Range of Mean scores is relatively narrow (6.27 – 8.03) Affect of Service – Customer/client service Information Control – Collections & access to collections Library as Place – Physical facilities
Potential areas for improvement or further investigation
Survey YearMinimum Mean Perceived Mean Desired Mean Adequacy Gap current previous Minimum Adequacy Gap rising expectations Always check the Minimum mean score when evaluating the rise or fall in your Adequacy Gap score. A lower Gap score just may result from rising expectations
From presentation by M. Kyrillidou, ALA, June 2007
Local Questions General Satisfaction Questions Information Literacy Outcomes Questions Library Use Summary
LibQUAL Canada Item 3.4. Location Questions Summary You can compare your individual local question results with: Your past surveys Consortial or peer library results (assuming sufficient number of respondents, e.g. 100+)
without These are simple indicators without the context of minimum and desired ratings LibQUAL – General Satisfaction LibQUAL+ Canada Queen's In general, I am satisfied with the way in which I am treated at the library In general, I am satisfied with library support for my learning, research, and/or teaching needs How would you rate the overall quality of the service provided by the library?
From presentation by M. Kyrillidou, ALA, June 2007
Almost ½ of respondents fill in the Comments box Provide valuable insights and suggestions for improvements Essential component in understanding the reasons behind the survey scores. Available on LibQUAL+ ® Web site Data Repository
Download file in csv(Excel) or txt (text) Record Content: › ID: › UserGroup: › Discipline: › Branch: › Age: › Sex: › Comment:
Skim the comments Load csv file into text analysis program, e.g. ATLAS.ti, InMagic, etc. Conduct analysis Identify major themes, e.g. study space, library catalogue, noise, etc. ›
Analytics Norms Tables
Institutional Explorer (peer comparison) Representativeness graphs Radar chart Library Use chart Thermometer chart Cumulative percentile distribution Longitudinal Analysis (by survey year) Cumulative percentile distribution px
2010 Queen’s LibQUAL +® results Undergraduate Business students
Heath Sciences Graduate Students
ARL provides templates to assist you in preparing customized radar and thermometer charts to create custom analyses
Identify your score Compare it to a relevant norms table Year Subgroup Dimension Norms are stable and are not calculated on an annual basis anymore (last calculated 2005) Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample. From: presentation by R. Bowlby and M. Kyrillidou,, LibQUAL+® Canada Workshop, Ottawa, Ontario, Canada, October 24-25, 2007
Identify all of the stakeholders or constituents who want and need to know about the survey results Consider the “stake” of each of the above; what specific aspect of LibQUAL+ ® will be of most interest / concern Determine how to communicate with each identified stakeholder From: presentation by R. Bowlby and M. Kyrillidou,, LibQUAL+® Canada Workshop, Ottawa, Ontario, Canada, October 24-25, 2007
1 st Priority. Particularly those whom you asked to participate in the survey. As soon as you can: 1.Announce incentive award winners 2.Inform users of highlights of survey results challenges and opportunities 3.Present weak areas as challenges and opportunities not as negatives 4.Most importantly, what the library intends to do. Describe action items begun and planned Example: From: presentation by R. Bowlby and M. Kyrillidou,, LibQUAL+® Canada Workshop, Ottawa, Ontario, Canada, October 24-25, 2007
Compare your results with the corresponding consortial results and those of peer Canadian libraries Compare your results over multiple LibQUAL +® surveys (longitudinal analysis) Look at results to determine if users are not aware of what the library already does Explore one question by discipline and user group Probe the questions that had meaningful gaps between perceived results and minimum expectations (Adequacy Gap) From presentation by M. Kyrillidou, ALA, June 2007
From all of the data, determine what can and should be addressed Prioritize some action items Align with mission, vision and goals of parent organization Address users’ top priorities, by user group Improve areas of strong user dissatisfaction Build on strengths, if they are truly user needs and priorities Identify work that can be de-emphasized and resources that can be reallocated From: presentation by R. Bowlby and M. Kyrillidou,, LibQUAL+® Canada Workshop, Ottawa, Ontario, Canada, October 24-25, 2007
Carefully crafted Executive Summary Library performance (strengths & opportunities for improvement/ enhancements in services and priorities Present results via explanatory text with supporting tables and charts in an appendix. Don’t rely on tables and chart to convey your point.
Example from Library Web Site: 2007/issues&actions.htmlhttp://library.queensu.ca/webir/libqual- 2007/issues&actions.html 2007 Findings Actions
Queen’s University LibQUAL Core Questions Summary by User Group User Group Minimum (Mean/Avg) Desired (Mean/Avg) Perceived (Mean/Avg) Adequacy Gap Queen'sCanConQueen'sCanConQueen'sCanConQueen'sCanCon Faculty Graduate Undergraduate Example: Queen’s University 2010 user group scores compared with the LibQUAL Canada Consortial results
Completed Report discussed at Management Team; consultation plan developed Report, comments database, and plan distributed to all staff All-Staff information session Units and functional teams Meetings of individual units and functional teams identify the issues in their areas of responsibilities and recommend appropriate actions. Management Team Reviewed the compilation of issues and objectives in developing the 2008/09 Budget Report. Compiled and approved action items prepared by the functional teams and units.
old.libqual.org Engage the staff in reviewing the results and developing solutions Focus on assessment of services not people; don’t make the survey a staff assessment tool Keep the experience a positive one for the staff
The academy is an intensive five-day program that focuses on both qualitative and quantitative methods for collecting and analyzing library service quality data. Next scheduled for March 14-18, 2011, Toronto, Ont. Deadline for application: December 15, 2010
LibQUAL Canada: arl-libqual.htm arl-libqual.htm LibQUAL + ®: Other library web sites es es