1. ALISE Philadelphia 9 January 2008 Users and Librarians Engaging in Virtual Spaces: Using Critical Incidents to Inform Practice and Education in Chat Reference Marie L. Radford, Ph.D. Associate Professor, Rutgers, The State University of New Jersey Lynn Silipigni Connaway, Ph.D. Senior Research Scientist, OCLC

2. Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives $1,103,572 project funded by: Institute of Museum and Library Services $684,996 grant Rutgers, The State University of New Jersey and OCLC, Online Computer Library Center, Inc. $405,076 in kind contributions Project duration: 2 ½ Years (10/05-3/08)

3. Four phases: I.Focus group interviews II.Analysis of 850 QuestionPoint live chat transcripts III.Online surveys 176 VRS librarians 184 VRS non-users 60 VRS users (in progress, 200 targeted) IV.300 telephone interviews Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives

4. Critical Incident Technique (Flanagan, 1954) Qualitative technique Focuses on most memorable event/experience of participants. Allows categories or themes to emerge rather than be imposed.

5. Critical Incident Technique: Online Survey Questions Think about one experience in which you felt a chat reference encounter achieved (or did not achieve) a positive result. Describe the circumstances and nature of the reference query. Describe why you felt this encounter was a success (was not a success).

6. Critical Incident Technique: Online Survey Questions To Elicit Positive & Negative CI for Non-users Think about one experience in which you felt you achieved (did not achieve) a positive result after seeking library reference services in any format. Describe the circumstances and nature of your question. Describe why you felt the encounter was successful/unsuccessful. Did the format (face-to-face, telephone, email, or text messaging) help your experience to be successful/contribute to your lack of success? If yes, how?

7. Interpersonal Communication Analysis: Results Relational Facilitators Interpersonal aspects of the chat conversation that have a positive impact on the librarian-client interaction and that enhance communication. Relational Barriers Interpersonal aspects of the chat conversation that have a negative impact on the librarian-client interaction and that impede communication.

8. Relational Theory & Approach to Interpersonal Communication Every message has dual dimensions – both content and relational (Watzlawick, Beavin, & Jackson, 1967)

9. Dual Dimensions Content The “WHAT” of the message Information exchange Relational “HOW” message is to be taken Relationship of participants

10. Librarian Demographics Gender Female132 Male 42 Age 21-3034 31-4039 41-5050 51-6041 61+10 Ethnicity Caucasian152 African American 5 Other 5 Asian or Pacific Islander 2 Hispanic/Latino 1 Native American 0

11. Librarian Demographics Location Urban94 Suburban52 Rural26 Library Type Academic104 Public54 Special 7 Consortium 2 School 0

12. Librarians: Positive Result (CI N=143) Number % Primarily Relational 3 2% Primarily Content 56 39% Both Relational & 84 59% Content

13. Librarians: Positive Result (CI N=143) Relational Themes* Number % Attitude70 49% Relationship quality30 21% Familiarity 4 3% *The percentages do not total to 100% because each CI can be coded into more than one theme

14. Librarians: Positive Result (CI N=143) Content Themes* Number % Providing information 12587% Demonstrating knowledge 1611% Providing instruction 50 35% Convenience/multi- 11 8% tasking/ time saving/ money saving

15. Librarians: Negative Result (CI N=126) Number % Primarily Relational 46 36.5% Primarily Content 45 35.5% Both Relational & 35 28% Content

16. Librarians: Negative Result (CI N=126) Relational Themes* Number % Attitude68 54% Relationship quality32 25% Approachability 2 2% Impact of technology 3 2%

17. Librarians: Negative Result (CI N=126) Content Themes* Number % Information 75 60% Lack of knowledge 6 5%

18. User Demographics Age 12-14 4 15-18 7 19-2814 29-35 6 36-4518 46-55 6 56-65 3 65+ 2 Gender Female29 Male31

19. User Demographics Location Suburban37 Urban21 Rural 2 Ethnicity Caucasian45 Asian or Pacific Islander 7 African American 3 Other 3 Hispanic/Latino 2 Native American 0

20. Users: Positive Result (CI N=57) Number % Primarily Content 38 67% Both Relational & 16 28% Content Primarily Relational 3 5%

21. Users: Positive Result (CI N=57) Content Themes* Number % Providing information 53 93% Demonstrating knowledge 10 18% Providing instruction 3 5% Convenience/multi- 9 16% tasking/time saving/ money saving

22. Users: Positive Result (CI N=57) Relational Themes* Number % Attitude 22 39% Relationship quality 8 14%

23. Users: Negative Result (CI N=30) Number % Primarily Content 20 67% Both Relational & 6 20% Content Primarily Relational 4 13%

24. Users: Negative Result (CI N=30) Content Themes* Number % Information 20 67% Lack of knowledge 7 23%

25. Users: Negative Result (CI N=30) Relational Themes* Number % Attitude8 27% Relationship quality3 10% Impact of Technology1 3%

26. Non-user Demographics Age 12-1418 15-1842 19-2862 29-3511 36-4518 46-5519 56-6510 65+ 4 Gender Female125 Male 59

27. Non-user Demographics Location Suburban107 Urban 67 Rural 10 Ethnicity Caucasian131 Asian or Pacific Islander 30 African American 10 Other 7 Hispanic/Latino 4 No Response 0

28. Non-users: Positive Result (CI N=149) Number % Primarily Content96 64% Both Relational & 50 34% Content Primarily Relational 3 2%

29. Non-users: Positive Result (CI N=149) Content Themes* Number % Providing information123 83% Demonstrating knowledge 47 32% Providing instruction 25 17%

30. Non-users: Positive Result (CI N=149) Relational Themes* Number % Attitude 43 29% Relationship quality 21 14% Approachability 3 2%

31. Non-users: Negative Result (CI N=106) Number % Primarily Content 53 50% Primarily Relational 33 31% Both Relational & 20 19% Content

32. Non-users: Negative Result (CI N=106) Content Themes* Number % Information 66 62% Lack of knowledge 20 19% Instruction 7 7%

33. Non-users: Negative Result (CI N=106) Relational Themes* Number % Attitude36 34% Relationship quality12 11% Approachability18 17% Impact of technology 1 1%

34. Implications for Education Important to VRS Users & Non-Users Accuracy of answers/information Knowledge of sources & systems Positive attitude Good communication skills User education needed to give more realistic expectations for VRS Marketing to attract Non-Users

35. Future Directions Online survey results have informed development of telephone interviews. 100 Librarians (completed, analysis in progress) 100 Users (in progress) 100 Non-users (in progress)

36. End Notes This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives Funded by IMLS, Rutgers University, & OCLC Online Computer Library Center, Inc. Special thanks to Patrick Confer, Timothy Dickey, and Jocelyn DeAngelis Williams Slides available at project web site: http://www.oclc.org/research/projects/synchronicity/ http://www.oclc.org/research/projects/synchronicity/