1. Virtual Characters

2. Overview What is a digital character? What is a digital character? Why do would we want digital characters? Why do would we want digital characters? Scope of the problem Scope of the problem What we want to tackle What we want to tackle How we want to tackle it How we want to tackle it

3. What is a Virtual Character? Virtual character - a character who represents an executing system Virtual character - a character who represents an executing system In TRON (1982), humans and humans that represents software interacted within a world that represented the hardware. In TRON (1982), humans and humans that represents software interacted within a world that represented the hardware.

4. What is a Virtual Character We look to to have humans and human the represent software interact in the real world. We look to to have humans and human the represent software interact in the real world. http://movies.yahoo.com/shop?d=hv&id=1807432839&cf=trailer

5. Digital Characters A new medium to interact with system information A new medium to interact with system information Spectrum from the paper clip -> AIBO Spectrum from the paper clip -> AIBO Examples: Examples: Agents (AI, NLP) Agents (AI, NLP) Robots (Robotics) Robots (Robotics)

6. Life-Sized Virtual Characters Virtual Characters as a way to interact with information Virtual Characters as a way to interact with information If virtual characters are presented with an adequate level of realism, would people respond to them as they would to other people? If virtual characters are presented with an adequate level of realism, would people respond to them as they would to other people? Effective Interaction Effective Interaction Natural (> than keyboard and mouse) Natural (> than keyboard and mouse) 3D, Dynamic (augmentable) 3D, Dynamic (augmentable) Effective Collaboration Effective Collaboration Non-verbal communication (60%) Non-verbal communication (60%) High impact? High impact?

7. Comparing Human-Human Interaction Modalities IMPhon e Video Phone Digital Characte r 3D Tele- Immsn Face to Face InputVerbalLHHHHH Non-VerbalLNH? (M)HH PrivacyHHLHLL OutputVerbalLHHHHH Non-VerbalLNH? (M)HH Dynamic?MNNHL/?L EfficacyImpactLMM? (H)HH InteractionLLL? (M)HH Logistic s BandwidthLLMLHN/A CostLLMMH[ N – H ] AvailabilityHHLN/AL[ N – H ] N- None, L – Low, M – Medium, H – High

8. Digital Characters Advantages: Advantages: Input: Non-Verbal, privacy Input: Non-Verbal, privacy Output: Non-Verbal, dynamic Output: Non-Verbal, dynamic Efficacy: Impact, interaction Efficacy: Impact, interaction Logistics: Bandwidth, Cost Logistics: Bandwidth, Cost Disadvantages: Disadvantages: Input: Non-verbal fidelity Input: Non-verbal fidelity Output: Non-verbal fidelity Output: Non-verbal fidelity Efficacy: Interaction fidelity Efficacy: Interaction fidelity

9. Interaction Each participant in a communication has three stages: perception, cognition, and response Each participant in a communication has three stages: perception, cognition, and response Investigate: Display, perception, efficacy Investigate: Display, perception, efficacy Thinking Participant Thinking PerceivingResponding Virtual Character Responding PerceivingResponding Perceiving Interaction

10. What we plan on studying There are many components in digital characters There are many components in digital characters We will focus only on interaction We will focus only on interaction Input (perception) Input (perception) How can we have the system capture non-verbal information? (emotion, gestures, eye-gaze, etc.) How can we have the system capture non-verbal information? (emotion, gestures, eye-gaze, etc.) Proposed solution: Initially straps -> advanced computer vision techniques Proposed solution: Initially straps -> advanced computer vision techniques Output (response) Output (response) How do we display the digital character? How do we display the digital character? Proposed solution: Evaluate different display technologies Proposed solution: Evaluate different display technologies Eventually we will look at efficacy (future research) Eventually we will look at efficacy (future research)

11. How we plan on studying it Perception: Perception: STRAPS: Track head, arms, upper body, and objects (telephone). STRAPS: Track head, arms, upper body, and objects (telephone). Develop techniques to transmit eye-gaze and expressions Develop techniques to transmit eye-gaze and expressions User Studies: Evaluate effectiveness of each component User Studies: Evaluate effectiveness of each component Response: Response: Evaluate the display modality. User Studies: Evaluate the display modality. User Studies: Immersive vs Non-Immersive Immersive vs Non-Immersive Life-sized vs. Non-life sized Life-sized vs. Non-life sized 2D vs 3D 2D vs 3D Goal: TV + webcams + laptop Goal: TV + webcams + laptop

12. Projects underway Interpersonal communication Interpersonal communication Distributed acting rehearsal (DAS student) Distributed acting rehearsal (DAS student) Teaching Teaching Deaf students Deaf students Future work: Universal Access Future work: Universal Access Disabled Disabled Minorities Minorities Rural communities Rural communities

13. Recruiting Looking for Looking for MS and PhD students MS and PhD students Equipment Equipment Wide area 12’x12’ tracker Wide area 12’x12’ tracker V8 HMD V8 HMD 42” plasma TV 42” plasma TV Cyberware Scanner Cyberware Scanner 6 data projectors (stereo) 6 data projectors (stereo) Firewire cameras Firewire cameras High-end PCs, video High-end PCs, video Work with Work with Cognitive Psychologists Cognitive Psychologists Computer Scientists Computer Scientists Computational Geometry Computational Geometry Aesthetic computing Aesthetic computing Simulation/modeling Simulation/modeling Digital Arts and Science Digital Arts and Science