1. Technology and Effective Communication

2. “Micro” Social Theory Much work occurs in groups or teams of 2+ people
E.g., lab groups, project teams, classes
Teamwork varies along a number of dimensions, e.g.:
Synchronous/asynchronous
Timing (fast/slow)
Nature of artifacts being manipulated (documents, objects, etc.)
Interdependence of roles
Assumption: designing technology to support remote group interaction requires knowing how face-to-face teams coordinate their language and actions to achieve their goals

3. Coordination Mechanisms
In face-to-face settings, team members use a variety of coordination mechanisms:
Conversation
Nonverbal cues (e.g., facial expressions, gaze)
Gestures/pointing
Observation of partners’ actions and task status
Technologies for remote collaboration are unlikely to be able to implement all of these coordination mechanisms
Need: a theory of group interaction that will allow us to predict what features of face-to-face interaction should be implemented in tools for remote collaboration and how those features should be implemented
Predictions must be specific to the types of tasks work teams are performing
Remote teams have to rely on a reduced set of coordination mechanisms (e.g., telephone) or substitutions for their usual mechanisms (e.g., cursors instead of hands for pointing)
How can technology support remote group interaction?
Selections among existing technologies
Design of new CSCW technologies

4. Clark’s Theory of Common Ground
Interpersonal communication is more efficient when people share more common ground
Common ground = mutual knowledge, beliefs, goals, attitudes that people know that they share
Grounding = The interactive process in a conversation by which communicators exchange evidence about what they do or do not understand.
Presentation phase: Speaker presents utterance to addressee
Acceptance phase: Addressee accepts utterance by providing evidence of understanding
People ground utterances to the extent necessary “for current purposes”
Principle of least collaborative effort – the pair should do the minimum necessary for successful grounding

5. Grounding in a Bicycle Repair Task
Example of grounding
Grounding in a Bicycle Repair Task
h, Pick up the seat please.
w, Okay.
w, Got the seat.
h, And have it upside down like you couldn't sit on it- trust me.
w, I can't sit on it.
h, Okay.
h, Now- uh- closer to your right hand there are these things called rails.
w, Got em.

6. Unpacking Mutual Knowledge/Common Ground
Communication rests on mutual knowledge or common ground:
The knowledge the parties to a communication hold in common and know they have in common
Speakers are hypothesis testers.
“If I say ‘X’, will listener understand ‘X’?”
“If I say ‘Did you see the game?’ will listener understand ‘Did you see the Pirates/Cub’s wildcard play-off game last night?’”
Speaker does hypothesis testing at two points:
Presentation phase — “What should I say?”
Acceptance phase — “Did the listener understand what I meant or should I elaborate?”

7. Name these objects B A
100%: Circle
70%: Star
30%: Adjective Star

8. Name these objects B C A
80%: Circle
20%: White Circle
0%: Star
100%: Adjective Star
60%: Star
40%: Adjective Star
Speakers take into account what they expect their partners to know
Name objects to distinguish among similar objects which a listener (a) has in mind and (b) is likely to confuse

9. But speakers can fail to anticipate what their partner will understand
Subjects compose sentence-long statement about two topic
Sarcastic vs sincere
vs voice
Replication
vs voice vs in-person
Convey phrase with an emotion (sarcasm, seriousness, anger, sadness)
Partner was stranger vs friend
Friendship made no difference
Kruger, J., Epley, N., Parker, J., & Ng, Z.-W. (2005). Egocentrism over Can we communicate as well as we think? Journal of Personality and Social Psychology, 89(6),

10. Referential communication task
Referring to things is basic to communication
Stylized game to understand reference: One person (the director) tells another (the worker) in what order to place these Tangram figures

11. Demo of a Referential Communication Task
Form 3 person teams: director, worker, observer
Director tells workers how to arrange figures
Workers arranges the figures
Observer observes & records:
How fast & accurately the director/worker team performed.
How the pair coordinated naming conventions.
How director knew if the worker understood a direction.
What they did to get better over time.

12. Observer record sheet Trial Correct? Time to complete (Min:Sec)
Notes: How did director know that the worker understood a direction? 1
Y N
____:____ 2 3 4

13. Partners are learning
Communicators come to agree on a pair-specific description of objects
With a new partner, words per object returns to close to original level

14. What evidence did you use to improve in the demo?

15. What evidence do people use for grounding?
Personal knowledge
Group membership
Linguistic co-presence
Explicit feedback
Physical co-presence

16. Personal Knowledge
Encoders describe colors or figures for self or for other.
Study 1: Other is “another student”
Decoders get own descriptions (self) descriptions for another (social), or someone else’s self-description (non-social)
Study 2: Other is friend in experiment or “another student”
DV=% Accurate

17. Stimuli for Expert vs. Novice Study
Empire state
Chrysler
Flatiron
Woolworth
Metlife
RCA Victor 1 2 3 4 5 6

18. Partners can partially accommodate to differences in others knowledge
Task: Order postcards of NYC landmarks
Experts: New Yorkers
Novices: Mid-westerns & others
Experts talking to experts are more efficient than novices talking to novices
Work with resources at hand
Mixed pairs learn from each other
Novices learn to use names
Experts learn to use descriptions
But adjustments are incomplete

19. Role of technology

20. Applying Grounding Theory To Technology
Clark & Brennan (1991): “People should ground with those techniques available in a medium that lead to the least collaborative effort.”
Hypothesis: Objective characteristics of different communication media change the costs of conversational grounding and strategies people use.
Some key types of costs:
Production/Reception costs: costs of producing/receiving messages
Start-up costs: costs of initiating conversation
Asynchrony costs: costs of timing utterances
Speaker change costs: costs of turn-taking
Repair costs: costs of correcting misunderstandings
Should allow us to predict in advance what features new technologies should have to meet different collaborative purposes

21. Affordances of Communication Media (Clark & Brennan, 1991)
Co-Presence
Participants share physical environment, including a view of what each other is doing and looking at
Visibility
Participants can see one another but not what each is doing or looking at
Audibility
Participants can hear one another
Cotemporality
Messages are received close to the time that they are produced, permitting fine-grained interactivity
Simultaneity
Multiple participants can send/receive messages at the same time, allowing backchannel communication
Sequentiality
Participants take turns in an orderly fashion in a single conversation
Reviewability
Messages do not fade over time
Revisability
Messages can be revised before being sent

22. Technology changes strategies and costs of grounding
Exactly how conversationalist achieve common ground depends up the details of the technology available
Features of communication setting
Change
Needs for & costs of
formulation
production
reception
understanding
start-up
delay
asynchrony
speaker change
display
fault
repair
co-presence
visibility
audibility
co-temporality (no lag)
simultaneity (full duplex)
sequentiality
reviewability
revisability

23. Affordances of Conventional Media
Face-to-Face
Video Conf.
Phone
Copresence ++ ? --
Visibility +
Audibility
Cotemporality
Simultaneity
Sequentiality
Reviewability
Revisability

24. Direction giving exercise
Volunteer to describe a simple figure to the class, with and without feedback A B

25. Interactivity improves communication
encoding
Message
decoding
Sender
Receiver
Response/
Backchannel
What does the traditional model leave out? 2 headed arrows.
Want to briefly demonstrate that interactivity has major effects on communication outcome
Early research I did shows that if 2 people are hearing the same story fm a speaker, but only one can provide back channel feedback & other has feedback blocked, that the active listener understand the story better, agrees more with the speaker.
Speaker's story is more tailored to what the active listener's knowledge. E.g., efficient where the active listener was previously briefed, and more repetitive where active listener made confused.
decoding
encoding
Feedback and interactivity is one effective way of achieving common ground
Feedback tailors communication to an audience, making it more effective

26. Effects of technology on partner-specific learning
Need feedback to learn from each other
Some technology can disrupt the feedback
Half-duplex (speakerphone) vs full-duplex audio (telephone)
Even msecs reduces coordination
Video that desynchronizes audio & video channels disrupts lip-reading

27. Role of Visual Information

28. Referential Communication Results
Multiple 2-person referential communication tasks
E.g., Find nearest doctor on map
E.g., Build trash cart
Common results:
Voice speeds solutions compared to typing
Faster times
More turns
More words
Visual channel doesn't help (in a talking head set-up)
(fm Chapanis, 1972)

29. "Talking-heads" video doesn't improve referential communication
Tacit assumption in using video
Since face-to-face communication is successful, communication media that are more like face-to-face are better
Most of the content is in the words
Gestures may be pre-verbal, rather than illustration
For emotion, video and audio channel can be redundant
Rich media are useful for handling ambiguous and controversial topics
E.g., Images change lie-detection, but help liar over the lie-detector
Seeing your partner doesn't improve ability to communicate about objects in the world:

30. Physical Co-presence Provides Visual Cues for Grounding
Physical co-presence: People are present at the same time in the same place
Provides several types of visual information that can be used to ground utterances.
Task-oriented video systems vs. “talking heads” video:
Advantages: View of others’ hands and of task objects allows monitoring of partner’s attention and comprehension; shared view of objects allows for efficient reference.
Disadvantages: No feedback from others’ facial expressions

31. Bicycle Studies: Method
Bicycle repair task
25 pairs of participants
“Helper” provides instructions
“Worker” repairs the bike
Communications Media
Side-by-Side
Audio/Video: head-mounted camera showing workers’ hands and field of view
Audio-Only
Within-subjects design
Dependent measures
Task completion times, observer ratings of work and communication quality, surveys, content-coded transcripts

32. In the Video Condition Pairs Used Visual Space
Grounding: Used visual space to disambiguate reference, establish common ground, & identify task elements
Situational awareness: To know when worker was ready for next instruction
Example(219_s_v.mov)
But technical flaws limited its utility

33. Bicycle Study: Key Findings
Performance is best with full physical co-presence (Side-by-Side)
Communication with video system was more efficient than audio-only.
Workers use deictic pronouns for task objects/locations when they think the helper can see them
The head-mounted video system used in this study did not adequately support shared visual space.
The top figure shows performance times in minutes by media condition. As can be seen, the side-by-side condition (left bar) is significantly faster than the other two conditions, which did not differ from one another.
The second figure shows a portion of our conversational analysis, focuses on deictic reference.
Reference: A small but critical proportion of overall messages. Objects had to be identified before instructions for working with them could be given. Ability to see participants’ behaviors and task objects should lead to more efficient reference. The results show that helpers used deictic references very little unless they were side-by-side with the worker. Workers, however, used deictic references almost as much when they knew the helper could see them via video as when they were side by side.

34. Exploring the Role of Shared Visual Information
What features of physical space influence its value?
Fidelity of views
Hypotheses: Delay, small windows, rotation & other factors that make views dissimilar will degrade collaborative performance
When is shared visual context most important?
Visual complexity
Hypothesis: When task is complex enough that language itself is insufficient to efficiently describe events

35. New Research Paradigm to Test Those Questions
Stylized referential communication task
To increase control
To systematically vary task attributes
Construct artificial shared visual environment
Allows independent manipulation of features of a shared space that co-occur in the real world => identify which are important

36. Cooperative Jigsaw Puzzle Task
Helper has picture of target and gives instructions to worker, who moves pieces to match target
Subjects communicate via audio & shared computer screens
Target
Shared view
Work area
Staging area

37. Task visual complexity
Manipulations
Task visual complexity
Visual fidelity
None: Audio only
Partial
Shared screen with a 3-second delay
Shared screen with rotation
Shared screen with a small view port
Immediate: Full shared screens with no delay & no rotation
Simple
Complex
Primary colors
Tartan plaids
Static colors
Changing colors
Pieces abutted
Pieces overlapped

38. Summary of Multiple Experiments
Task performance
Shared visual space improved task performance (speed & accuracy) in all experiments
Improved performance most for visually complex tasks
Shifted conversational strategies
Shared visual space improved improved efficiency of reference (e.g., words/reference)
Lack of shared visual space forced many workarounds

39. Experimental Manipulations
Fidelity of the Visual Space
Immediate
Delayed (3 seconds)
None
Other studies
Rotation of the spatial perspectives
Discontinuous, “push to see” images
Visual difficulty:
Static vs. Dynamic Tasks
Spatially easy vs. difficult puzzles
Easy versus difficult to name objects
Same vs. different visual perspective
Immediate condition
TALK ABOUT THE ADVANTAGES TO EACH OF THESE!!!
While Hypothesis 1 might seem straightforward, remember that in the previous study Fussell et al. found no difference between those performing the task with the head-mounted camera in comparison to those in the audio-only condition
Hypothesis 2 and 3 In cases were the scene can easily be described with language (e.g., simple objects that are easily discernable) there is less value to the SVS, however, when the objects or environment are changing or difficult to describe there should be an added value to having the SVS
Hypothesis 4 By seeing the partner perform some task, the Helper gets immediate feedback about whether the partner understood a directive. If the visual feedback were delayed (e.g., as caused by video compression or network lags), the value of the visual information may be diminished. Delay in updating the display may diminish the value of the visual information both for the ability to compare the current work state to the goal state (as described in the earlier scenario) and for the team’s ability to coordinate language (as discussed here).
Hypothesis 5 As suggested by Clark’s theory, we should expect to see structural and content changes in the discourse as the pairs attempt to communicate using the least collaborative effort
Visual feedback, however, may be less necessary if the task is simple enough (e.g., a game of tic-tac-toe) or if the pair has an efficient, well-practiced vocabulary to describe events (e.g., routine communication between pilot and air traffic controllers). In these cases, a shared visual display provides little new information.
No SVS condition

40. How Does SVS Change Communication
How are pairs communicating differently when they have a shared visual space?
Communication is more efficient with a shared visual space
Helper uses actions to assess worker’s comprehension
Both helper and worker use more efficient referring expressions and diexus (e.g., this/that/pointing)
Helper can precisely time interruptions and corrections
SVS facilitates an awareness of the task state
Without shared visual space
Worker is responsible for updating helper on state of the task
Worker verbally indicates understanding of an instruction, and is sometimes wrong
Examined Details of Discourse to Explore Predictions
We examined word rate (the number of words, controlling for time)
Explored details of Helper and Worker utterances separately to understand how overall communication was changing

41. Shared Visual Space Is More Important in Changing Environments
Shared visual space improved performance
Immediate significantly faster than delay and no SVS (p<.0001)
3 second delay led to significant decrease in the value of the shared space
Shared visual space was more important when the objects were changing (i.e., hard to describe)
Immediate affected significantly less than delay (p=.05) and no SVS (p=.0002)
Shared visual space is less important when words can easily describe the objects and environment
Puzzle Difficulty
Easy vs. Difficult (LS Means: 61.9 sec vs sec, p=0.002)
Color Drift
Stable vs. Drifting (LS Means: 54.3 sec vs sec, p<0.001)
Interaction:
Visual Space * Puzzle Difficulty Interaction
Not significant (F (2,256) = 1.06, p > .35)
Instructive in that visual complexity itself did not raise the value of a shared view of the work.
It was primarily when the task was dynamic and the environment was changing that having the display was most beneficial

42. Rate of Word Production: Workers Increase Their Speech Rate
Pairs increased speech rate when fidelity decreased (all p<.02)
The fidelity of the SVS influenced Workers more than the Helpers (F(2,110) = 10.80, p < .0001)
The pairs adjusted their use of language to accommodate for lack of shared space
Workers increased their speech rate to compensate
The accommodation was insufficient in comparison to when the pairs made use of the shared visual space
Fidelity of Shared Space
Immediate vs. Delay (t=-2.55, p=.01)
Immediate vs. No SVS (t=-4.84, p<.0001)
Delayed vs. No SVS (p<.017)
(LS Means: Immed=19.4; Delay=30.1; None=45.0 words per unit time)
Fidelity of Shared Space * Speaker Role Interaction
F (2,110) = 10.80, p < .0001
The fidelity of the SVS influenced the Worker’s efficiency much more than the Helper’s
Changes in Worker behavior reflect their accommodation to differences in the Helper’s view of the workspace
Adjust use of language, but still not as good as when they have SVS
*** NOTE ***
THIS IS VERY INTERESTING!!
The Worker always sees the same screen. They have their staging area on the right, and the work area on the left.
However, they adjust their speech pattern (they begin to speak more, while the Helpers stay approximately the same) in order to facilitate the efficiency of the group as whole. This provides support for Clark’s theory of least collaborative effort.

43. Conversation Issuing Acknowledgements
Workers took a more active role in ensuring messages were understood when there was no shared visual space (i.e., when the helper could see them).
Immediate SVS
No SVS
H: The, the right hand, the top right hand corner of the blue block touches the bottom left hand corner of the first orange block.
W: [Positioned piece correctly]
W: Like that?
H: Yeah.
H: All right that's good.
H: And that's gonna be on top of the red one but only the right side of the red is going to be showing.
H: You know what I mean?
W: OK, so it's like...
H: Oh, like, put it on the left side of the red.
W: ...side of it and you see half of the red block.
H: Right, of the red, Yeah.
W: OK.

44. Typically the Helper gave directives and the Worker moved pieces
Acknowledgements of Understanding: Pairs Use SVS to Monitor Comprehension
Pairs were most explicit in stating their understanding when they had no shared visual space
Workers were more explicit in stating their understanding when there was no shared visual space available
Typically the Helper gave directives and the Worker moved pieces
Used SVS to monitor understanding
Reserved language for breakdowns
Without the SVS the pairs substitute language to confirm understanding
Shared Visual Space
Immediate vs. No SVS (p<.0001)
Immediate vs. Delay (ns)
The pairs were most explicit in stating their understanding when they had No SVS
Shared Visual Space * Speaker Role Interaction
F(2,107)=8.752, p=.0003
Use SVS to monitor comprehension
Follow cycle of Helper giving directives and reserve language for breakdowns
When No SVS it is important for Workers to explicitly state their understanding
When SVS Available
More efficient to follow cycle of Helper giving Worker instructions and reserving speech for when things go wrong
When SVS Fidelity Decreases
Workers must be more explicit in communicating their level of understanding
Bigger difference for the No SVS (t=4.05, p<.0001) than for the Delayed (t=1.36, p=.1777) in comparison to the Immediate
The interaction between the shared visual space and the speaker demonstrates that it was more important for the Workers to explicitly state their understanding when the shared visual space was of lower fidelity (for the interaction F(2,107)=8.752, p=.0003).
Closer examination reveals that this is even more evident in the No Shared Visual Space pairs than it is for the Immediate (t=4.05, p<.0001), while there appeared to be less difference between the Immediate and Delayed Shared Visual Space (t=1.36, p=.1777).
The Helpers typically produce about the same portion of acknowledgements regardless of the degree of the shared visual space. However, the Workers significantly increase their rate of production.

45. Using visual info for grounding vs situational awareness
ImmedDiffStable PM-B3.mpg
Grounding: Visual info helps parties to assess whether the language used is mutually understood or needs to be elaborated
D:: Do the red first
W: Which red?
D: Dark red (W drags out a red block).
D: Darker (W drags out a red block)
D: Is there a lighter one? (W drags out a red block)I
D: Yeah, that’s it
Situational awareness: Visual info allows parties to assess the current state of a task and plan future actions
D: Now you want a darker blue on top of the lighter blue.
W: (moves the correct tile into position)
D: (realizes W finished the last instruction & D can offer a new instruction)
D: OK, now an orange touching the top right hand corner …

46. Cisco Telepresence Application
Will this technology improve distributed work?

47. Feelings of connection may be more important than information flow

48. What About Feelings of Connection? (Garau, Slater, Bee, Sasse, 2001)
Study comparing f/f, avatar w/ realistic gaze, avatar w/ random gaze & audio communication
Negotiation task – avoid a scandal
DVs
Face-to-face feel, Involvement, co-presence, partner evaluation:

49. Yee et al Meta-Analysis
Performance & subjective evaluations improved with avatar
Realism only influence subjective evaluations
Comparison of 25 experiments
People interact via text or voice
Presence of avatar
None
Unrealistic (cartoon)
Photorealistic
Outcomes
Performance
Subjective evaluations of experience or partner