Human-Centered Computing Telepresence, AR, tangible interfaces John Canny UC Berkeley

HCC overview Telepresence, AR, tangible interfaces John Canny PRoPs (Personal Roving Presences) and gesturing avatars. UPM - a universal planar manipulator. Bearable computers - turning laptops into augmented reality systems and wearable computers.

HCC overview PRoPs: Personal Roving Presences Space Browser + Eric Paulos Surface cruiser

HCC overview PRoPs: Toward being there A PRoP is a Personal Roving Presence. PRoPs seek to bridge the gap between telepresence and being there. There are two essential aspects of remote presence that only telerobotics can provide:  Non-verbal communication, i.e. gaze, proxemics, gesture etc. which are highly spatialized.  A gamut of human behaviors, such as chance encounters, informal meetings, showing or being shown, browsing, exploring etc, which are affordances of a physical presence.

HCC overview Terrestrial PRoPs: Carts Carts are a good match to human capabilities. Our strategy was to list important physical communication skills, and to implement as many as possible.

HCC overview The rest of the story: A taxonomy of interaction behaviors Verbal: symbolic and prosodic Nonverbal communication theory (psychology, linguistics, anthropology). Main cues are:  Gaze  Proxemics  Facial expression  Body and hand gesture  Posture  Touch Example: A Handshake depends on all the cues.

HCC overview Differences between verbal and nonverbal communication Adam Kendon (83) summarizes the difference very well:...gesture has properties different from speech. In particular, it employs space as well as time in the creation of expressive forms, whereas speech can use only time. We find, therefore, that the way information may be preserved in speech, as compared to gesture, tends to be very different.

HCC overview Why nonverbal skills are spatial Gaze - very sensitive to mutual direction. Center-right-left. Proxemics - Depends on distance to others and the surroundings. Facial expression - the least spatial, directed by gaze. Gesture - deictic gesture is completely spatial, so are other pictographic forms. Posture - Directional. Touch - Cant fake it without space.

HCC overview You need social context as well Communication. Persuasion. Trust-building. These are radically different contexts that rely on different non-verbal cues. We need to study telepresence in representative scenarios.

HCC overview The two-way street: PRoPs provide selective control over particular cues. They can be turned on and off at will. So they provide a potentially interesting testbed for fine-grained study of non-verbal cues. We can create a rich variety of media and study their influence on the overall perception of the other person.

HCC overview The input side How do you achieve rich, expressive input for PRoPs/avatars with today’s technology? Cross-modal mapping from pen gesture to body gesture. Both have symbolic and expressive character:

HCC overview Future research? Gaze cue repair: Face to face is not perfect. There are common breakdowns in face-to-face communication, such as gaze aversion. Beyond Being There: It may be possible to rectify gaze aversion using image processing and/or a synthetic gaze cue.

HCC overview Project 2: UPM: A universal planar manipulator A smart desk that can move a large number of objects placed on it independently. Provides a tangible interface in both directions for 2D layouts: floor plans, landscapes, stage directions etc. Uses a small numbers of motors (4) and the non-linearity of friction.

HCC overview UPM: A universal planar manipulator First prototype used linear voice-coils from ancient disk drives, capable of about 2g acceleration. New prototype should manage about 50gs.

HCC overview Bearable Computing (new project) An exploration of issues in personal, persistent computing (augmented reality, worn interfaces) using ordinary laptop computers. Avoid head-mounted displays (expensive and low- res) head-tracking, and cables. The approach: use optics to overlay computer images on reality, but use laptop or pocket- mounted displays. Testbed: Grad course in HCC next semester.

HCC overview An augmented reality classroom Virtual image with notes, questions, private chat space Physical space Glasses to rotate the laptop image

HCC overview An AR classroom Students work in groups of 5-7, communicating silently via pen or keyboard chat. Each group has one main note-taker, the others add their own comments or questions to the transcript. Students can mark up the group transcript, or the lecturer’s notes. There is non-archived chat also. One student per group works as facilitator or TA, posing questions to the others, and testing understanding.

HCC overview Bearable computers on the go The laptop can serve as a “bearable” computer, attached wirelessly to its owner. Laptop in briefcase with wireless TV and keyboard TXs. Pocket-worn portable LCD TV Double-mirror element in glasses Heads-up virtual image Wireless chordal keyboard or Palmpilot in pocket for input.

HCC overview Goals of this research Can we enhance attention in class with live group chat/note-taking? Do we get a richer transcript with collaborative note-taking? Does the head-up display make note-taking less distracting ? Will people do more tasks with a mobile bearable computer because the threshold for computer use is lower?