1. Forging Forward: Using Evaluation as a Stepping Stone Joe Matthews SLA – San Diego Fall Seminar October 30, 2015

2. Evaluation is the process of … Identifying and collecting data Focusing on specific services or activities Establishing criteria to gauge success Determining the degree to which stated goals and objectives are achieved

4. Types of Evaluation Activities Program or service planning Program or service monitoring Impact assessment (outcomes) Economic efficiency

5. Evaluation Questions How much?How many?How economical?How prompt? Magnitude % of change % of overall change Costs Magnitude Change Resources used Units processed Cycle times Turnaround time Anticipatory Library Controls

6. Evaluation Questions How valuable?How reliable?How accurate?How well? Effort expended Cost-benefit obtained Dependability Access Accuracy Currency Completeness Comprehensiveness Accuracy Expertise Performance meeting expectations Library & Customer Decide

7. Evaluation Questions How courteous?How responsive?How satisfied? Welcoming Attentive Anticipatory Helpful Empathetic Expectations met Materials obtained Personal interaction Ease of use Equipment used Facilities Willingness to return Customers Decide

8. Evidence-Based Library and Information Practice 1.Formulate a clearly defined question 2.Find the best evidence to answer the question 3.Critically appraise the evidence 4.Combine the appraisal of the evidence with professional judgment to make a decision 5.Evaluate the effectiveness of the results 6.Disseminate the results

9. Library Metrics Resources CapabilityOutputsOutcomes Impacts Input Measures Process Measures Output Measures Outcome Measures Impact Measures

10. Library Metrics National Center for Education Statistics Academic Library Survey Integrated Postsecondary Educational Data System (IPEDS) ACRL Academic Library Trends and Statistics survey ACRLMetrics IMLS Public Libraries Survey PLAMetrics

12. Perspectives & Tools LibraryCustomer QualitativeExpert opinion Checking lists Ethnographic QuantitativeUsage analysis Peer comparisons Citation studies ILL analysis Loss rate analysis Shelf availability Document delivery Curriculum analysis Collection development Life-cycle costs Surveys Value

13. Internal Evaluation Facilities Technologies Library Personnel Collections Services Other Input, Process, and Output Measures

14. External Evaluation Library’s Perspective Demographics and use Collection strengths Library personnel Library deployed technologies Facilities Services provided Unique programs and services

15. External Evaluation Customer Perspective – Outcomes ACRL – Megan Oakleaf’s Value report ACRL – Assessment in Action The LibValue Project – Carol Tenopir PLA – Project Outcome

16. Measuring Satisfaction Customer Satisfaction In-person interview Focus group interview Exit interview Comment cards Surveys Complaint/ Compliment analysis Mystery shopping

17. Satisfaction Surveys Gaps Model of Service Quality – LibQUAL –Affect of service –Information control –Library as place Counting Opinions – LibSat DIY

18. Customer Satisfaction Customer comments Net Promoter Score (NPS) Opportunity index Resolution rate

19. Value of the Library Value of the Library Personal & Organizational Benefits Economic Benefits Direct Indirect Nonuse Indirect

20. Value of the Special Library Cognitive results Affective results Meeting expectations Task accomplishments Time aspects Money aspects

21. Value of the Academic Library StudentFacultyInstitution Recruitment/Enrollment Retention & Graduation Success Achievement Learning Experience, Attitude & Perception of Quality Research Productivity Grants Teaching Reputation & Prestige

22. Value of the Public Library Generic Learning Outcomes Knowledge & Understanding Attitudes & Values Activity, Behavior & Progression Enjoyment, Inspiration & Creativity Skills

23. Communicating the Results Library staff members Friends groups Prospective donors Board of Trustees Community governing boards Institutional stakeholders Library administrative agencies State legislatures Manager/Partner Customers –Faculty –Graduate students –Undergraduate students –Others

25. 1.Accountability is part of librarianship 2.Austerity increases the need for evaluation 3.Librarians are engaged in organizational change management 4.Evaluation is not brain surgery

26. 5.Meaningful evaluation focuses on customers and demonstrates the value of the library to them 6.The hardest thing is to start – “Just do it” 7.Play with the various concepts

27. 8. Make evaluation a part of your regular work 9. Share the results 10. Implement a results reporting system

28. Joe AT JoeMatthews.Org