The Effect of Interface on Social Action in Online Virtual Worlds Anthony Steed Department of Computer Science University College London

Avatar Puppet Systems From the very early systems common behaviours emerged –Customisation of representation –Spatial group-forming behaviours –Social reactions –“Presence” or not in your avatar (i.e. being at the keyboard) needs to be signalled with gestures, otherwise difficult to interact

Outline: Transparency & Boundaries Puppeteering systems take effort to express activity and motion Can be difficult for users to understand intentions and actions Immersive systems alleviate some of these barriers by making the interface transparent Do this by engaging the the user(s) in the virtual world, but we can envisage mixed-reality systems that break the boundary in a different way

Immersive Interfaces

Effectiveness of Immersive Interaction Subjective Report Tasks Cognitive & Emotional Behaviour Physiological Autonomic Social

Effectiveness of Immersive Interaction Subjective Report Tasks Cognitive & Emotional Behaviour Physiological Autonomic Social

Effectiveness of Immersive Interaction Subjective Report Tasks Cognitive & Emotional Behaviour Physiological Autonomic Social

Effectiveness of Immersive Interaction Subjective Report Tasks Cognitive & Emotional Behaviour Physiological Autonomic Social “I felt as if I was being watched”

Effectiveness of Immersive Interaction Subjective Report Tasks Cognitive & Emotional Behaviour Physiological Autonomic Social

Effectiveness of Immersive Interaction Subjective Report Tasks Cognitive & Emotional Behaviour Physiological Autonomic Social

Why is Collaboration So Effective? Tracked gestures are immediately communicative It is very easy to interpret gaze and pointing of the other Immersed users spend very little time “manipulating” the interface Indeed in other experiments, users with immersive interfaces emerged as leaders over desktop interfaces users

Forgetting Which Hand is Which

Capturing the User

Eyecatching: Eyetracking Generation 2

Experiment Outline Task –Grab-Instructions –Position-Instructions Measures –Time to do both types of instruction –Errors in both types of instruction –Conversational analysis Conditions –No eye movement –Modelled eye gaze –Tracked eye gaze

Results

Conversational Analysis Success –“OK, can you pick this cube” –“This one?” –“Yes” Look at speaker Look from head to cube Point at cube Look at speaker Mutual gaze

Look at speaker Conversational Analysis Failure –“OK, can you pick this cube” –“This one?” –“No, this one”, “This one?” Look at speaker Look from head to wrong cube Point at wrong cube Look at speaker Mutual gaze Look at cube

Technical Challenges Avatar representation Lip synchronisation End-to-end latency Frame rate Motion capture Capture real world so you can talk about it

Eye Gaze

Key Aspects Comparison with video as benchmark Subjects answer as series of questions to a confederate Stage 1: Do users exhibit characteristic gaze, blink and pupil dilation when they talk to a video or avatar-mediated representation of a questioner? Stage 2: Can independent observers detect lying when it is presented as an avatar?

Stage 1: Key Findings Participants have similar behaviour when speaking to an avatar or a video

Stage 2: Key Findings

Breaking Boundaries

32 Capture Destination Capture Visitor Display Visitor Display Destination \\\\\ Asymmetric Collaboration

Immersion is great but … Immersive interfaces are expensive You are bound to the metaphor where there is a virtual place you go to Bring the avatar to you –Make it aware of the user and the space around you –Interpret the real world and interact with it

34 Demo Highlights Panoramic Camera (PointGrey Ladybug 3)

Conclusions

Collaboration can be very fluid with immersive interfaces –Several challenges remain concerning capture –Desirable to bring more of those capabilities to non- immersive (passive capture) systems Many rules that can be applied to agents We expect that asymmetric collaboration situations will be more common in the future and this deserves further attention

Acknowledgements Eyecatching –William Steptoe, Robin Wolff, Alessio Murgia, Estefania Guimaraes, John Rae, Wole Oyekoya Presenccia –Wole Oyekoya BEAMING –Will Steptoe, Wole Oyekoya, Tim Weyrich, Fabrizio Pece, Jan Kautz –Partners at UB, Jean-Marie Normand, Mel Slater –