Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Daniel Avrahami – Doctoral Symposium – UIST06 Who, What, and When: Supporting Interpersonal Communication over Instant Messaging Daniel Avrahami Carnegie.

Similar presentations


Presentation on theme: "1 Daniel Avrahami – Doctoral Symposium – UIST06 Who, What, and When: Supporting Interpersonal Communication over Instant Messaging Daniel Avrahami Carnegie."— Presentation transcript:

1 1 Daniel Avrahami – Doctoral Symposium – UIST06 Who, What, and When: Supporting Interpersonal Communication over Instant Messaging Daniel Avrahami Carnegie Mellon University www.cs.cmu.edu/~nx6

2 2 Daniel Avrahami – Doctoral Symposium – UIST06 Illustration John is making final changes to a presentation for a client visit. His team member Anne, working at a different site, sends him an instant message asking for some urgent information.

3 3 Daniel Avrahami – Doctoral Symposium – UIST06 Illustration John is making final changes to a presentation for a client visit. His team member Anne, working at a different site, sends him an instant message asking for some urgent information. Since John is pressed for time, he decides to ignore all incoming messages until after hes done, leaving Anne unable to finish her task.

4 4 Daniel Avrahami – Doctoral Symposium – UIST06 Illustration (cont) Consider now if we were able to: Accurately predict, based on his activity, that John was not likely to respond to Annes message for some time Predict, based on past communication patterns, that Anne and John are co-workers Such models could be used, for example, to increase the salience of the alert, indicating to John that Annes message may deserve his immediate attention

5 5 Daniel Avrahami – Doctoral Symposium – UIST06 Research goals The two main goals of my research work are to provide a better understanding of factors affecting IM interaction in its context, and to use this understanding for the creation of predictive statistical models and tools that support IM communication. In order to achieve these goals my research will use three complementary steps:

6 6 Daniel Avrahami – Doctoral Symposium – UIST06 Research goals Create accurate models that predict responsiveness to incoming IM, and investigate the factors affecting responsiveness (when) Investigate the effect of interpersonal relationships on IM interaction, and create statistical models that use this knowledge to predict relationships (who) Use basic properties of human dialogue to provide support for balancing of responsiveness and performance (what)

7 7 Daniel Avrahami – Doctoral Symposium – UIST06 Background Instant Messaging, or IM, is one of the most popular communication mediums today 12 billion instant messages are sent each day. Nearly 1 billion messages are exchanged by 28 million business users [IDC Market Analysis05] Useful in many ways: from quick questions and clarifications, coordination and scheduling, to discussions of complex work [Bradner99; Nardi00; Handel02; Herbsleb02; Isaacs02] IM has a number of shortcomings (Asynchrony + Limited aware + Low cost for sending)

8 8 Daniel Avrahami – Doctoral Symposium – UIST06 When: Predicting responsiveness to IM [Presented at CHI06]

9 9 Daniel Avrahami – Doctoral Symposium – UIST06 Background Wanted to answer the following question: If an instant message were to arrive right now, would the user respond to it? In how long?

10 10 Daniel Avrahami – Doctoral Symposium – UIST06 Data Collection

11 11 Daniel Avrahami – Doctoral Symposium – UIST06 Data collection Created a plugin for Trillian Pro (written in C) Non-intrusive collection of: IM events desktop events

12 12 Daniel Avrahami – Doctoral Symposium – UIST06 Participants 16 participants to date Nearly 5200 hours recorded Over 90,000 messages Over 400 buddies 4 participants provided full text On average, participants exchanged a message every: 3.4 minutes (researchers avg=8.1)

13 13 Daniel Avrahami – Doctoral Symposium – UIST06 Responsiveness 92%

14 14 Daniel Avrahami – Doctoral Symposium – UIST06 Defining Session Initiation Attempts session used two subsets: 5 minutes (similar to Isaacs02) and 10 minutes 92%

15 15 Daniel Avrahami – Doctoral Symposium – UIST06 What are we predicting? Generate a set of features for every message: IM state and desktop state Seconds Until Response computed, for every incoming message from a buddy, by noting the time it took until a message was sent to the same buddy Examined five responsiveness thresholds 30 seconds, 1, 2, 5, and 10 minutes

16 16 Daniel Avrahami – Doctoral Symposium – UIST06 Modeling method Weka ML toolkit Features selected using a wrapper-based selection technique AdaBoosting on Decision-Tree models 10-fold cross-validation 10 trials: train on 90%, test on 10% Next I report combined accuracy

17 17 Daniel Avrahami – Doctoral Symposium – UIST06 Results (full feature-set models) All significantly better than the prior probability (p<.001) (Graph shows 5-minute subset only)

18 18 Daniel Avrahami – Doctoral Symposium – UIST06 Results (buddy-independent models) Previous models used information about the buddy (e.g., time since messaging that buddy) Can predict different responsiveness for different buddies But what if you wanted just one level of responsiveness? Built models that did not use any buddy- related features

19 19 Daniel Avrahami – Doctoral Symposium – UIST06 Results (buddy-independent models) all significantly better than the prior probability (p<.001) BUT not sig. diff. from previous set

20 20 Daniel Avrahami – Doctoral Symposium – UIST06 Some practical considerations Preserving plausible deniability Making predictions about the receiver, visible to the receiver Multiple concurrent levels of responsiveness

21 21 Daniel Avrahami – Doctoral Symposium – UIST06 Who: Relationships and IM Communication [To be presented at CSCW06]

22 22 Daniel Avrahami – Doctoral Symposium – UIST06 Relationships and IM communication People use IM for both work and social communication Prior research shows that relationship type has significant effect on fact-to-face and other voice communication (Duck91) Wanted to investigate the effect of relationship on basic communication patterns

23 23 Daniel Avrahami – Doctoral Symposium – UIST06 Co-worker (Senior) Co-worker (Peer) Co-worker (Junior) Co-worker (Other) Friend Family Spouse Significant Other Acquaintance Friend & Co-worker Self Bot [Unknown/Unused] Buddy Coder

24 24 Daniel Avrahami – Doctoral Symposium – UIST06 Session-level measures #TimeMessage Text 117:42:45B:Hey [Participants name] 217:42:56B:what time does your group get in the AM? 317:42:57P:hey 417:43:01P:usually around 10 517:43:25B:ok 617:43:38B:i want to start circulating the card in the AM 717:43:58P:ok, good idea 817:44:02P:that's for coordinating this 917:44:13B:no problem 1017:44:27P:thanks :-) 1117:44:35P:sorry bout the typo 1217:44:38B:is ok VariableValue GroupStudent RelationshipWork Duration1.88minutes Message Count12 Turn Count7 Character Count232 Messages per Minute6.4 Messages per Turn1.71 Characters per Message19.3 Seconds Until First Reply1seconds Minimum Gap (between turns)1seconds Maximum Gap (between turns)24seconds Average Gap (between turns)12.2seconds Time of Day5:44pm

25 25 Daniel Avrahami – Doctoral Symposium – UIST06 The effect of relationships Used a repeated-measures ANOVA Relationship Category (Work, Mix, Social) and Group (Researchers, Interns, Students) were repeated Participants and BuddyID modeled as random effects Participants nested in Group BuddyID nested first in Participants, then in Group N=3297 sessions

26 26 Daniel Avrahami – Doctoral Symposium – UIST06 Results

27 27 Daniel Avrahami – Doctoral Symposium – UIST06 Summary of Results Sessions with Social contacts were longer and with more messages BUT at a significantly slower pace Maybe giving less attention to these sessions? Sessions with Work contacts were at a faster pace with longer messages Grounding? Complex concepts?

28 28 Daniel Avrahami – Doctoral Symposium – UIST06 Results: Session length Significant effect on Session Duration (p<.001) Social significantly longer sessions than both Mix and Work (Work and Mix n.s.) Similar effects for Number of Turns Number of Messages Number of Characters (Duration correlated at >.85)

29 29 Daniel Avrahami – Doctoral Symposium – UIST06 Results: Messaging rate Significant effect on Messaging Rate (p<.01) Social significantly slower than Mix (p=.003) Social marginally slower than Work (p=.078) Maximum-Gap (p<.05) Social longer than Work (p=.013)

30 30 Daniel Avrahami – Doctoral Symposium – UIST06 Results: Length of messages Significant effect on Message Length (Characters- per-Message) (p<.001) Work significantly longer than both Social (p<.001) and Mix (p=.002)

31 31 Daniel Avrahami – Doctoral Symposium – UIST06 Predicting relationships

32 32 Daniel Avrahami – Doctoral Symposium – UIST06 Predicting relationships How can it be used? Augmenting IM systems Indicators of unavailability Differential alerts Shared with other mediums E.g. Email Provide organizational overview

33 33 Daniel Avrahami – Doctoral Symposium – UIST06 Models performance 2-step Logistic regression model with 16-fold cross validation Results from pairs with 2 sessions or more (78% of the data) Significantly better than the prior probability Classified as WorkSocial Work 40.9% (83) 5.9% (12) Social 14.8% (30) 38.4% (78) Accuracy: 79.3% Classified as WorkMixSocial Work 25.3% (74) 5.1% (15) 2.0% (6) Mix 8.2% (24) 14.7% (43) 7.8% (23) Social 9.6% (28) 17.1% (50) 10.2% (30) Overall Accuracy: 50.2% Work vs. Rest: 75.1% Social vs. Rest: 63.5%

34 34 Daniel Avrahami – Doctoral Symposium – UIST06 What: Using content to balance responsiveness and performance [Presented at CSCW04]

35 35 Daniel Avrahami – Doctoral Symposium – UIST06 Responsiveness / Performance Tradeoff Users often multitask when using instant messaging [Nardi00, Isaacs02, Voida02] Users often have to choose between Staying on task and being responsive to IM Current solutions typically force users to choose one or the other: Update away messages Turn off IM client

36 36 Daniel Avrahami – Doctoral Symposium – UIST06 Quick response - do you have the figures? Leisurely response - check out www.cnn.com Politely deferred - ru busy? No response - going to meeting. ttyl Expectations for responsiveness

37 37 Daniel Avrahami – Doctoral Symposium – UIST06 The approach: QnA Users ignore, to the best of their ability, the alerts of incoming messages Transitioning (internally) to being unavailable By observing the content of messages, QnA automatically highlights incoming messages that may deserve their attention In particular, potential questions and answers

38 38 Daniel Avrahami – Doctoral Symposium – UIST06 Why questions and answers? A question and an answer form an Adjacency pair (Schegloff & Sacks73) From Arenas of Language Use Given a first pair part, a second pair part is conditionally relevant, that is, relevant and expectable, as the next utterance. Once A has asked the question, it is relevant and expectable for B to answer in the next turn. (Clark 1992, p. 157)

39 39 Daniel Avrahami – Doctoral Symposium – UIST06 How does it work? QnA listens to incoming and outgoing messages when an outgoing messages is sent if it is a question remember that expecting a response when an incoming messages arrives if it is a question and/or we are expecting an answer wait x seconds to see if user attends to the message if did not attend then show QnA notification

40 40 Daniel Avrahami – Doctoral Symposium – UIST06 is_a_question? Match to list of questions that can be politely deferred (are|r) (you|u) there busy? Go through list of rules and look for match (?|/) at end of sentence what (is|are|r|were|does|do|did|should|can) did(|nt|nt) (i|u|you|he|she|they|we) (are|r) (you|u) huh

41 41 Daniel Avrahami – Doctoral Symposium – UIST06 QnA summary QnA: A tool that allows users to stay on task, but still seem responsive to buddies who expect it Allows users to transition between work modes Sits quietly in the background when the user attends to messages Only notifies when the user ignores messages Download from www.cs.cmu.edu/~nx6

42 42 Daniel Avrahami – Doctoral Symposium – UIST06 Conclusions & Future Work

43 43 Daniel Avrahami – Doctoral Symposium – UIST06 Conclusions I have presented work on analysis and generation of predictive modeling in support of interpersonal communication over IM: Work on predictions of responsiveness to IM communication (specifically to session initiation attempts) Work on analysis and predictions of interpersonal relationships and their effect on communication Work on the use of basic properties of human dialogue to allow users to balance responsiveness and performance

44 44 Daniel Avrahami – Doctoral Symposium – UIST06 Current and Planned Work Understanding Responsiveness Investigate in detail the contribution of specific features Investigate distribution of responsiveness over time Content Analysis Determining the Communication Goals Content-based transcript segmentation

45 45 Daniel Avrahami – Doctoral Symposium – UIST06 this work was funded in part by NSF Grants IIS-0121560, IIS-0325351, and by DARPA Contract No. NBCHD030010 thank you for more info visit: www.cs.cmu.edu/~nx6 or email: nx6@cmu.edu

46 46 Daniel Avrahami – Doctoral Symposium – UIST06 Process diagram

47 47 Daniel Avrahami – Doctoral Symposium – UIST06 Process diagram

48 48 Daniel Avrahami – Doctoral Symposium – UIST06 Process diagram

49 49 Daniel Avrahami – Doctoral Symposium – UIST06 Process diagram

50 50 Daniel Avrahami – Doctoral Symposium – UIST06 Related work Interruptions and disruptions [Gillie89, Cutrell01, Hudson02, Dabbish04] Interruptibility and cost of interruption [Horvitz99, Horvitz03, Hudson03, Begole04, Horvitz04, Fogarty05, Iqbal06] Models of presence [Horvitz02, Begole03] Responsiveness to Email [Horvitz02, Tyler03]

51 51 Daniel Avrahami – Doctoral Symposium – UIST06 Participants 16 participants to date Researchers: 6 full-time employees at an industrial research lab (mean age=40.33) Interns: 2 summer interns at the industrial research lab (mean age=34.5) Students: 8 Masters students (mean age=24.5)

52 52 Daniel Avrahami – Doctoral Symposium – UIST06 How can such models help? senderreceiver intercept alert mask enhance awareness message

53 53 Daniel Avrahami – Doctoral Symposium – UIST06 sender How can such models help? message receiver intercept alert mask enhance

54 54 Daniel Avrahami – Doctoral Symposium – UIST06 sender How can such models help? message receiver intercept alert mask enhance

55 55 Daniel Avrahami – Doctoral Symposium – UIST06 sender How can such models help? message receiver intercept alert mask enhance

56 56 Daniel Avrahami – Doctoral Symposium – UIST06 sender How can such models help? awareness receiver intercept alert mask enhance shhhh

57 57 Daniel Avrahami – Doctoral Symposium – UIST06 sender How can such models help? awareness receiver intercept alert mask enhance (carefully) not now

58 58 Daniel Avrahami – Doctoral Symposium – UIST06 Data collection (cont.) Privacy of data Masking messages for example, the message:This is my secret number: 1234 :-) was recorded asAAAA AA AA AAAAAA AAAAAA: DDDD :-). Temporary masking Alerting buddies Hashing buddy-names

59 59 Daniel Avrahami – Doctoral Symposium – UIST06 Who: Relationships and IM Communication [To be presented at CSCW06]

60 60 Daniel Avrahami – Doctoral Symposium – UIST06 Relationships and IM communication People use IM for both work and social communication Availability might depend on relationship Wanted to investigate the effect of relationship on basic communication patterns

61 61 Daniel Avrahami – Doctoral Symposium – UIST06 Background Relationship type has significant effects on communication, including the quality, purpose and perceived value [Duck91] Cues, such as tempo, pauses, speech rates and the frequency of turns, affect the way in which conversation partners perceive each other [Feldstein94] Frequency affects communication [FTF:Whittaker94, IM:Isaacs02]

62 62 Daniel Avrahami – Doctoral Symposium – UIST06 Relationships distribution

63 63 Daniel Avrahami – Doctoral Symposium – UIST06 Results

64 64 Daniel Avrahami – Doctoral Symposium – UIST06 Predicting relationships Cross-validation with 16 models (omitting one participant each time) Nominal Logistic Regression

65 65 Daniel Avrahami – Doctoral Symposium – UIST06 What: Using content to balance responsiveness and performance [Presented at CSCW04]

66 66 Daniel Avrahami – Doctoral Symposium – UIST06 Issues Determining that a message contains a question or an answer can be difficult interleaved conversations many short messages that comprise a single turn loose grammar and spelling Gives buddies a way to increase the salience of their messages. what if they abuse it?

67 67 Daniel Avrahami – Doctoral Symposium – UIST06 Future work Collect feedback from users A few users who have used QnA for over 2 years now But would like more users Please download QnA from my homepage Improve question identification Implement ignore list

68 68 Daniel Avrahami – Doctoral Symposium – UIST06 Conclusions & Future Work

69 69 Daniel Avrahami – Doctoral Symposium – UIST06 Contributions This works contribution to the HCI field will span both theoretical and applied aspects. From a theoretical point of view, this work will provide insights into the factors that influence interpersonal communication patterns and responsiveness. At the applied level, this work will provide predictive statistical models that can be used in many applications. Finally, this work promotes the creation of tools that use knowledge and predictive models generated from naturally occurring interaction.


Download ppt "1 Daniel Avrahami – Doctoral Symposium – UIST06 Who, What, and When: Supporting Interpersonal Communication over Instant Messaging Daniel Avrahami Carnegie."

Similar presentations


Ads by Google