1. Gaze Awareness for Videoconferencing: A Software Approach Nicolas Werro

2. Table of contents Introduction Main problems The software approach Rendering Eye manipulation Altering head pose Computer vision Conclusion

3. Introduction Importance of gaze awareness in face-to- face communication Most video conferincing systems make it impossible for participants to make eye contact

4. Main problems People stare into their displays rather than into the camera In addition to inhibiting eye contact, participants have no sense of spatial relationship

5. The software approach The aim is to develop an inexpensive videoconferencing system that work with cheap, commonly available hardware. The architecture of the video subsystem :

6. Rendering On the receiving end, the system uses the vision information to : extract the head from the video frame correct the gaze This is achieved in two steps: the system replaces the eyes with synthetic eyes to aim the gaze the head pose is adjusted

7. Eye manipulation New synthetic eyes focused on the desired direction are rendered The system uses segmentation data to decide, for each pixel, whether to use the pixel from the original face image or from the eyeball image

8. Altering head pose The head orientation is changed using texture mapping: First, the system creates a 3D model in the shape of the subject’s head Then, it executes texture mapping, projecting the face image from the video frame onto the model  The system can rotate the model to any desired orientation

9. Altering head pose

10. Computer vision The vision component must track the head pose, segment the eyes, and determine gaze direction

11. Computer vision Computer vision in general is very difficult, but several factors make the problem more tractable in this problematic : We’re looking only for a head, not arbitrary objects We deal only with head poses that permit the subject to gaze directly at the screen The system only needs to detect gaze that is directed at the screen

12. Conclusion The results of this work appear promising: Accurate vision data can be extracted from each video regarding head pose, eye segmentation and gaze direction The pose of the head can be arbitrarily oriented The eyes can be synthesized with the appropriate gaze direction The resulting videoconferencing system supports a sense of space and gaze awareness