Presentation on theme: "Work in Collaboration with Isbister (NYU-Poly), Jeffery Ventrella, Leslie Bishko, Jacki Morie EMIIE Lab."— Presentation transcript:
Work in Collaboration with Isbister (NYU-Poly), Jeffery Ventrella, Leslie Bishko, Jacki Morie EMIIE Lab
Nonverbal Behavior: From real to virtual Figure from A. Vinciarelli, M. Pantic, H. Bourlard, and A. Pentland, “Social signals, their function, and automatic analysis: a survey,” ICMI, 2008.
Enhance Communication Express Emotions and Reactions Social Signals : agreement, confusion, alignment, turn-taking Applications: Virtual Meetings (IBM, …) Online Education Performances Social Hang Out Nonverbal Behavior: Importance
CVEs, Virtual Worlds support for Social Signals
Social Com Software: Nonverbal Behavior (Tiny Chat)
Social Com Software: Nonverbal Behavior (Mingleverse)
Social Com Software: Nonverbal Behavior (Shami et al.) N. S. Shami, L.-T. Cheng, S. Rohall, A. Sempere, and J. Patterson, "Avatars Meet Meetings: Design Issues in Integrating Avatars in Distributed Corporate Meetings," in Group, 2010.
Funded by AFRL
Research Goal Understand how people use Social Signaling in Second Life: Proximity Gaze Appearance Gestures Facial Expressions
Social Signaling Study in Second Life 28 Participants in 7 sessions 10 males 18 females, average age 40 (range 20-60) Belonging to the following communities: Experimental Performance Arts 1 group, 6 participants Theatrical Arts2 groups 14 participants Building / Animating1 group 3 participants Music Composition1 group 4 participants Role Play1 group 2 participants Educators1 group 2 participants
Social Signaling Study in Second Life (SL) Participant Recruitment Online Recruiting in SL and Ning sites More than 50 friends made visiting 40 locations Specific community and location targeted Attended events & distributed bulletins via and SL Participants also made referrals to their friends What sort of activity are they doing? Focus Group Show and Tell on SS topics derived from lit review Sessions took place on Lakamaka Islands (owned by SRI) ▪ Privacy and performance concerns wearing scripts ▪ Participants joined the Lakamaka Community David’s Avatar
Social Signaling Study in Second Life Data Collection Background Survey Semi-Structured Interview (during focus group) Real Time Proximity and Chat logging Sample Screenshots on Lakamaka
Focus Group – Sample Questions Physical Appearance How do you make your character different from the other characters in the virtual world? Gesture and Posture Do you use gestures as part of your online communication? Face and Eye Behavior Do you change facial expressions and direct your avatar gaze? Voice How do you use voice and text chat to communicate? Space and Environment Do you have any sense of a “personal bubble” around your avatar? Technological Affordances What scripts do you use if any?
No standard way of puppeteering avatars Few research studies: what behavioral cues are important how they are used
Participants: groups of 2-3 11 participants Situation Setup: an argument Video Tapped the Session Logged all button pushing and interaction Total: minutes
"Being able to visually interact with other Skype users through emotions is great, when you do not wish to use you camera or do not have access to one." "bring the avatar back to the screen having fun making the avatar move around" "Can’t see the point of the system while there is webcam and chat options.”
“It difficult to concentrate for both moving the avatar around and talking at the same time.” “As time goes on the content of the conversation becomes more important, the visuals becomes unnecessary.”
Context is very important Important Signals: Thumbs Up and down Turn taking Applause Proximity Gaze
Social Signaling is Important Identified some important behavioral cues depend on context It is still unclear: The right UI and control The right semantic buttons Conscious vs. non-conscious
FLAME (1998) – modeling Emotions and Learning in believable characters ELE (2003) – Expressive Lighting Engine: system that allocates and adjusts lights to accommodate interaction and aesthetic goals Mirage (2002) – Interactive Narrative Architecture using theatre acting and filmmaking principles Player modeling (2002) – figuring out style of play and adapting an interactive narrative accordingly
Adaptive Systems Believable characters Visual Design and Presentation User Experience Research Games User Research Group (IGDA) Assassin’s Creed II Study (Ubisoft) Façade studies Triangulation of eye tracking, physio, game metrics (Electronic Arts) Co-op patterns (Bardel Entertainment) Virtual world monitoring through visualization and Metrics (God Mode Games and Relic Entertainment) Laban + Delsarte for modeling expressive behavoior