Presentation on theme: "LIBQUAL+® AND THE EVOLUTION OF LIBRARY AS PLACE AT RADFORD UNIVERSITY, 2002-2008 Eric Ackermann Radford University Library Assessment Conference University."— Presentation transcript:
LIBQUAL+® AND THE EVOLUTION OF LIBRARY AS PLACE AT RADFORD UNIVERSITY, Eric Ackermann Radford University Library Assessment Conference University of Washington August 3-7, 2008
Introduction Me, the 15% Assessment Librarian Radford University McConnell Library
Background Changes to Library as Place continuous since 2002 Examples Coffee shop Designated quiet areas Group study areas No cell phone zones New furniture Improved signage
Goal of this study Are these physical changes reflected in our LibQUAL+® survey results?
Study design Data types: Score (rankings) & comments Data source: 2002, 2005, 2006, 2008 LibQUAL+® surveys Respondent groups: Faculty Graduate students Undergraduates Level of analysis: Library as Place dimension
Methodology Meta-analysis A data reduction method used to combine results across studies using Effect size meta-analysis Statistical procedure that converts incomparable metrics into a common effect size metric.
Data analysis Organize the data & respondent types into comparison groups Faculty: 2006 v Grad students: 2005 v. 2002, 2008 v Undergraduates: 2005 v. 2002, 2008 v All groups: 2005/6 v. 2002, 2008 v. 2005/6
Data analysis (cont) Determine effect size for each comparison group using mean adequacy gap (score data) odds ratio (positive: negative comments) Determine final weighted average effect size d for each respondent group for Results reported as d,.95 CI, and BESD
How did we measure success? (aka, Practical significance)
Practical significance Success LevelSignificance Action criteria Action dBESD Complete Users satisfied 0.31 or more 15.1% or more Celebrate! Partial Users somewhat satisfied 0.3 to % to -15% Look for improvements; implement changes Un- successful Users unsatisfied or less -15% or less High Priority review/change
What did we expect to find?
In the literature… Nothing found about specifically using LibQUAL+® to measure satisfaction with changes in Library as Place All reports found about measuring change due to drastic overhaul of building or new construction.
Conclusion… If dramatic changes to the Library as Place yield dramatic, immediate increases in user satisfaction, then incremental changes will yield modest gains in user satisfaction over time.
What did we actually find?
Response rate & representativeness Response rate (average of ) Faculty: 15.7% Grad students: 22.4% Undergraduates: 26.4% Representativeness (average of ) Faculty: 8.2% Grad students: 7.4% Undergraduates: -13.1%
So what? (Practical significance again)
Practical significance Partial success (Users somewhat satisfied) Action Criteria: d (or BESD + 15%) Faculty (0.01 or 0.6%) Grad students (0.19 or 9.2%) Undergrads (0.08 or 4.1%) Action Looked for improvements (e.g., examined Suggestion coded comments, conducted focus groups) Implemented changes (see next slide…)
Review/change Improvements since 2008 LibQUAL+ survey Created Front Desk from Media Services and Circ desks Reorganized quiet study floor Reduced noise level on quiet study floor More improvements planned New furniture for the lobby Walls to be painted Reference area reorganization
Conclusion Effective Method is statistically defensible Told us what we did (and did not) want to hear Practical Metrics have practical meaning at the local level Provide criteria for actionable results Sustainable Commitment to assessment by library administration Funded by university
Contact information Eric Ackermann Reference/Instruction and Assessment Librarian Box 6881, McConnell Library, Radford University, Radford, VA 24142