1. Data Matters for User Service: Acquiring & Assessing the Impact of NYPL’s Money Matters Grant-Funded Initiative Marie L. Radford Chair, Department of Library & Information Science & Associate Professor Rutgers University, NJ Kristin McDonough Director, Science, Industry, and Business Library, The New York Public Library Library Research Round Table American Library Association Conference Chicago, Illinois June 27 th -July 2 nd, 2013

2. The New Normal

3. Agencies Mandated to Develop Consumer Education Programs

4. Pre-Training Survey of Branch Staff Why How What We Found

5. Grants: Acquisition & Execution Not about the money…but the meaning Opportunity lookout Chat before submission Pitch & pivot Define scope Manage RFP process Negotiate and collaborate with contractor Engage internal stakeholders

6. Evaluating Money Matters Goal: To create a corps of frontline staff equipped with the skills & confidence to meet the growing need among their patrons to access timely, accurate & trustworthy information about personal finance. Evaluation IMLS requirement RFP by NYPL to recruit outside evaluator Measure MM’s success in meeting goals for: Individual workshops (content & format) Instructors Overall program

7. Money Matters Pro Program Staff Training Portal http:// bit.ly/NYPLMoneyMatters

8. Evaluation Design Mixed-Methods Quantitative & Qualitative Instruments designed with NYPL input Surveys Pre-Training Survey Immediate Post-Training Survey Final Post-Training Survey Observation Focus Groups Website Analytics

9. Surveys Mix of paper & online (Survey Monkey) Staff cooperation vital Understand need & purpose Paper successful for individual session evals (100% return rates) Online useful for Pre & Post surveys Incentive for Final Survey (gift card raffle)

10. Money Matters Workshops Workshop Titles (3 core, 8 electives, …..10 eLearning Modules) Banking (core) Retirement Planning (core) Credit & Debt 1 (core) Credit & Debt 2 Identity Theft Investing 1 Investing 2 Paying for College Income Tax Insurance Living on the Financial Edge

11. Immediate Post-Training Survey Results Completed at end of each training workshop Nearly 100% return rate (administered on- ground) Uniformly excellent results (means 5.0 to 4.0) Quality of instruction Content/scope Activities/scenarios Resources/handouts Likert scale = Strongly Agree (5) to Strongly Disagree (1)

12. Immediate Post-Training Survey Results “Before this workshop, how prepared were you to answer patron questions? 3.0 to 2.3 (mean, on several PF topics) “After completing this workshop how well prepared are you to answer patron questions?” 5.0 to 4.0 (mean, on same PF topics) Reported all workshops practical, relevant, enjoyable, excellent instructors, & useful in daily interactions with NYPL patrons.

13. Survey Results: Pre-Training (n= 90, 69%) vs. Final Post-Training (n= 95, 73%) Confidence in knowledge & understanding Personal Finance 2.6 (Pre-Training mean) rose to 3.9 (Post- Training mean) Adequate training in locating quality PF resources 2.7 (Pre) rose to 4.2 (Post) Adequate training in evaluating PF resources 2.7 (Pre) rose to 4.1 (Post) Would welcome PF questions 2.8 (Pre) rose to 3.8 (Post)

14. Staff Voices on Money Matters Program “Suggested to a single mom that starting to save now for her child's college education is beneficial because of the time value of money.” “Helped a patron who was a grandfather find information on how to pay his granddaughter's college costs and how to ask financial questions.” “Helping immigrants at the adult learning center set up bank accounts and helping them to improve their financial knowledge.” “I understand investment company information and reports better since I'm more comfortable with financial terms.”

15. Conclusion Critical factors in successful MM program Clear vision of need & goals Talented/qualified instructors NYPL involvement in content & design Staff cooperation in evaluation Feedback & adjustments Early observation Mid-point Focus Groups Interim reports

16. Acknowledgements & Contact This is one of the outcomes from the project Money Matters Financial Education Program Funded by the Institute for Museum and Library Services (IMLS) and NYPL Special thanks to: Marzena Ermler, T.J. Woods, Gretchen Smith, Brandy McNeil, Barbara O’Neill, & Carole Glade Staff training portal Money Matters Pro http://bit.ly/NYPLMoneyMatters http://bit.ly/NYPLMoneyMatters Money Matters Web Site: http://www.nypl.org/moneymatters http://www.nypl.org/moneymatters