1. CSI Cyberspace: A Multiple Case Study Investigation of the Untimely Demise of Seven Virtual Reference Services Marie L. Radford, Ph.D., Rutgers University mradford@scils.rutgers.edu M. Kathleen Kern, MLS, University of Illinois at Urbana-Champaign katkern@uiuc.edu New Reference Research: 11 th Annual Reference Research Forum, ALA, Chicago June 25, 2005

2. Problem Statement Number of chat reference services growing 3000+ chat & IM (Coffman & Arret, 2004) Alive & Clicking! Many services are successful and viewed as integral Clicked the Bucket! However, a number of chat services have died untimely deaths. Why?

3. Research Questions What were the deciding factors for ending these services? Who was involved in making the decision to end the services? How could these failures/false starts have been prevented? What has been the impact of discontinuing these services? What are the similarities and differences among the cases? What lessons can be learned to ensure the success of existing services or newly developing services?

4. Modus Operandi - Method Multiple Case Study Method Selection of Cases & Informants 7 Cases Data Collection Email Surveys Telephone Interviews Examination of Documents

5. Method Continued Analysis of Results Case Descriptions of Each Case Comparison of Cross – Case Findings Similarities Differences Iterative Process Examination of Documents

6. Results Despite our original & advertised title we had 7 cases not 6. 5 Academic 1 Public 1 Consortium One surprise, all 7 agreed to be interviewed & share their experiences. Another surprise – Most (5-7) had no grant funding!

7. Starting & Operating the Service – Common Themes No additional staff hired. Broad target audience – same as for in- person service. Distance education students a focus. Limited hours of operation. Average 22.4 hours per week User response positive.

8. Postmortem - 5 Major Reasons for Discontinuing Services Funding – Budget Cuts Low Use Staffing Technical Problems Institutional Culture

9. Reasons for Discontinuing Service Low Use FundingStaffingTech. Prob. Inst. Cult Primary 3 3 1 1 0 Note: Totals to 8 since Case B listed 2 primary factors (Technical Problems and Low Use).

10. Reasons for Discontinuing Service Low Use FundingStaffingTech. Prob. Inst. Cult Primary 3 3 1 1 0 Secondary 2 0 2 0 1 Total 5 3 3 1 1

11. CSI Cyberspace Mystery Why did these services have such low volume?

12. Exhibit A - Volume CaseMonthly Qs% of total Qs A35<1% B351.5% C30<1% D363% (less if all branches included) E69<1% F120-160Unknown G72%

13. Exhibit B – Marketing of Service CaseAmountComments AModerate Web, Displays, Flyers, Lib. Instr, Newspaper BExtensive Web, Branding, Articles, Lib. Instruction CModerate Target Pop., Flyers, Bookmarks, Library Instruction, Plan to Increase DExtensive Web, Newspaper, Ads, Signs, Posters, Table Tents ENone Limited hours precluded marketing push. FLittle Pilot training & exploration period. GExtensive Email Announcement, Meetings, Giveaways, E-Newspaper, Library Instruction

14. Exhibit C – Marketing of Service –Low Vol. CaseAmountComments AModerate Web, Displays, Flyers, LUI, Newspaper BExtensive Web, Branding, Articles, LUI CModerate Target Pop., Flyers, Bookmarks, LIU, Plan to Increase DExtensive Web, Newspaper, Ads, Signs, Posters, Table Tents ENone Limited hours precluded marketing push. FLittle Pilot training & exploration period. GExtensive Email Announcement, Meetings, Giveaways, E-Newspaper, LIU

15. Exhibit D – Hours of Service CaseHours of Service A M-F 3-8pm 20 hrs. per week (changed hours) B M-Th 1-7pm, F 1-5pm 28 hrs. per week C M-Th 5-8pm 12 hrs. per week D M-F 1-5pm 20 hrs. per week E Su-Th 7-10pm 18 hrs. per week F M-Th 2-8pm 24 hrs. per week G M-F 10am-5pm 35 hrs. per week

16. Exhibit E – Length of Service Giving VR a Fair Trial CaseTrial/PilotLength of service A1/2002 – 4/200426 months B1/01 – 8/029/2001 – 12/20027 months pilot 15 months C5/2002 – 4/200421 months D10/2001 – 4/20026 months E1/02 - 5/021/2003 – 5/20045 months pilot 14 months F9/2004 – 3/20036 months G7/2002 – 2/200420 months

17. Giving VR a Fair Trial … Shortest services ran 6 months (2 services) Longest run was 26 months Average of 17 months

18. Exhibit F – Service Evaluation User Eval? Type of Eval? Users Satisfd? Comments A YSurvey & Trans. YesMixed librarian perceptions B YSurvey & Assmt. YesLow (19%) return rate. C N ?Planned 2 nd yr. survey D N ? E YSurvey Yes68% “very satisfied” 40% return rate F YSurvey YesStaff satisfied too G YSurvey YesUsers highly satisfied, low return rate.

19. Impact of Closing Services Common Trends Demise had limited impact. Users did not notice. Email reference (phone service) more promotion, more use. Unique Responses Staff relieved, less pressure.

20. Lessons Learned If doing a multiple case study, don’t put a number in your title for conference papers. Consortium necessities: Full-time coordinator Variety of libraries Some reference questions are better answered by email.

21. More Lessons Learned… Critical factors: Staff training & enough volume for staff to stay primed, comfortable staffing patterns User needs assessment What hours for service? What software? User evaluation Marketing Design prominent & easily identifiable link

22. The Mystery Remains Unsolved Ongoing Investigation… Season 2? Future Research Directions Low volume, what are critical factors? Have identified 3 additional cases. Perhaps more will be identified today. More rigorous examination of documents. Research on successful services – what factors ensured their success? User expectations of service – what hours, days? More on users. More research on non-users

23. Future Directions for Services Investigate free & simpler software IM or refTracker Consortia Share Cost Share Staff Continue to scan landscape

24. There’s Hope… Future Directions for Cases CaseReinstate?Comments AMaybe In consortium, if users want it BMaybe Simple Chat, IM, or RefTracker, Summer ’05 CYes If budget & staffing restored DMaybe Possible if consortium EMaybe Possible if consortium (doubtful) FYes Reorganization in process, hope to reopen in Jan. ’06 GMaybe Something free like IM

25. Conclusion PPT slides will be posted at: RUSA Reference & Statistics Committee Site http://www.ala.org/RUSA Thanks & Questions??? Marie L. Radford, Ph.D., Rutgers University mradford@scils.rutgers.edu M. Kathleen Kern, MLS, University of Illinois at Urbana-Champaign katkern@uiuc.edu