1. Creating Chat Connections: E-valuating Virtual Reference Transcripts Marie L. Radford ACRL Delaware Valley Chapter November 2, 2007

2. Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives Project duration: 2 ½ Years (10/05-3/08) Four phases: I.Focus group interviews II.Analysis of 850 QuestionPoint live chat transcripts III.600 online surveys IV.300 telephone interviews

3. Phase II: Transcript Analysis Random sample  7/04 to 11/06 (18 months)  500,000+ pool of transcripts  30-50 per month = 850 total sample 746 usable transcripts  Excluding system tests & technical problems 372 classified by age/educational level  146 “Screenagers” (Middle & High School)  226 “Others” (College/Adult)

4. 6 Analyses Geographical Distribution –Originating library –Librarian respondents Type of Library Wait Time & Session Time Type of Questions –Katz/Kaske Classification Subject of Questions –Dewey Decimal Classification Interpersonal Communication –Radford Classification

5. VRS Session Times Wait time –Mean – 1.87 Minutes –Median – 1 Minute –Minimum – 1 Second –Maximum – 67 Minutes Session time –Mean – 12.42 Minutes –Median – 12 Minutes –Minimum – 12 Seconds –Maximum – 71 Minutes

6. VRS Transactions by Library Type

7. VRS Questions by Location of Originating Library

8. VRS Questions by Location of Librarian Respondents

9. Wait Time for VRS Users

10. VRS Mean Wait Time by Library Type

11. VRS Mean Session Times by Library Type

12. VRS Questions by Type

13. VRS Questions by Subject

14. Interpersonal Communication Analysis Theoretical Framework Watzlawick, Beavin & Jackson (1967) Pragmatics of Human Communication –All messages have both content & relational dimension. Content = Information (WHAT) Relational = Relationship Aspects (HOW)

15. Method Qualitative Analysis of Transcripts Development of category scheme Careful reading/analysis Identification of patterns

16. Interpersonal Communication Research Questions What relational dimensions are present in chat transcripts? Are there differences in relational dimensions/patterns of chat users & librarians? If so, what are they?

17. Results Relational Facilitators –Interpersonal aspects of the chat conversation that have a positive impact on the librarian- client interaction and that enhance communication. Relational Barriers –Interpersonal aspects of the chat conversation that have a negative impact on the librarian- client interaction and that impede communication.

18. Transcript Examples – Relational Facilitators “The Size of an Atom” Question Type: Subject Search Subject Type: Life Sciences, Biology (DDC:570) Duration: 40 min. “Diabetes” Question Type: Subject Search Subject Type: Business Duration: 43 min., 15 sec.

19. Transcript Example – Relational Barriers “Mesopotamian Government” Question Type: Subject Search Subject Type: History of Ancient World (DDC:930) Duration: 27 min. “Telekinetic Powers” Question Type: Subject Search Subject Type: Parapsychology & Occultism Duration: 7 min., 29 sec.

20. Facilitators – VRS Users Screenagers (n=146) vs. Others (n=226) Lower numbers/percentages per transcript SO Thanks 21% (75) vs. 77% (175) Agreement to try what 32% (46) vs. 51% (116) is suggested Closing Ritual 32% (47) vs. 49% (111) Self Disclosure 42% (61) vs. 55% (125) Seeking Reassurance39% (57) vs. 49% (111) Admit lack knowledge 19% (13) vs. 21% (47)

21. Facilitators – VRS Users Screenagers (n=146) vs. Others (n=226) Similar numbers/percentages per transcript S O Alternate Spelling/ 28% (41) vs. 27% (60) Abbreviated Words Informal Language 9% (13) vs. 9% (21) Offering Confirmation 8% (11) vs. 8% (13) Empathy 3% (4) vs. 4% (8)

22. Barriers – VRS Users Screenagers (n=146) vs. Others (n=226) Higher numbers/percentages per transcript SO Impatience 8% (12) vs. 6% (13) Rude or Insulting 6% (9) vs. 4% (9)

23. Facilitators - Librarians Screenagers (n=146) vs. Others (n=226) Lower numbers/percentages per transcript L to S L to O Offering Opinion/Advice 29% (43) vs. 37% (83) Explaining Search Strategy 6% (9) vs. 14% (31) All Lower Case 11% (63) vs. 18% (43) Encouraging Remarks 12% (18) vs. 17% (39)

24. Facilitators - Librarians Screenagers (n=146) vs. Others (n=226) Higher numbers/percentages per transcript L to S L to O Seeking Reassurance 61% (89) vs. 51% (115) Greeting Ritual52% (76) vs. 48% (108) Asking for Patience39% (57) vs. 35% (80) Explaining Signing off 5% (8) vs. 1% (2) Abruptly

25. Facilitators - Librarians Screenagers (n=146) vs. Others (n=226) Similar numbers/percentages per transcript L to S L to O Polite Expressions57% (83) vs. 56% (127) Inclusion 33% (48) vs. 34% (76) Thanks22% (32) vs. 23% (51) Makes Sure User Has 18% (27) vs. 20% (45) No More Questions Interjections 8% (11) vs. 9% (20)

26. Barriers - Librarians Screenagers (n=146) vs. Others (n=226) Higher numbers/percentages per transcript L to S L to O Abrupt Endings 16% (23) vs. 9% (20) Limits Time 6% (9) vs. 0% (1) Sends to Google 5% (8) vs. 0% (0) Reprimanding 4% (6) vs. 0% (1) Failure/Refusal to 5% (7) vs. 2% (5) Provide Information

27. Strategies that Work! All Modes of Reference Basic interpersonal skills Recognizing that user may need reassurance –Providing reassurance Awareness of appropriate self-disclosure –When to disclose –Acknowledgment of user’s self-disclosure Humor – importance of acknowledgment

28. More Strategies Greetings & Closings. –Beware negative closure! –Beware robotic scripts! Inclusion (use of we, let’s, etc.). Mirror relational strategies. Don’t b afraid 2 use informal language, abbreviations & emoticons as appropriate :)

29. Boost Satisfaction Collaborate across generations End encounter on a positive note. Ask “Have I answered your question completely?” Avoid “Negative Closure” Invite to return to desk or e-service if further help needed.

30. Bottom Line Communication critically important! –Difficult process –Generational differences add to complexity!! –Use your experience & intuition as guides.

31. Questions? Marie L. Radford, Ph.D. –Email: mradford@scils.rutgers.edumradford@scils.rutgers.edu –www.scils.rutgers.edu/~mradfordwww.scils.rutgers.edu/~mradford

32. End Notes This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives Funded by IMLS, Rutgers University, & OCLC Online Computer Library Center, Inc. Special thanks to Lynn Silipigni Connaway, Patrick Confer, Timothy Dickey, Jocelyn DeAngelis Williams, Julie Strange, Janet Torsney, & Susanna Sabolski-Boros. Slides available at project web site: http://www.oclc.org/research/projects/synchronicity/ http://www.oclc.org/research/projects/synchronicity/