Virtual Reality and Digital Characters: New Modalities for Human Computer Interaction G2V2 Talk September 5 th, 2003 Benjamin Lok

Welcome! My name is Benjamin Lok. My name is Benjamin Lok. Pronounced like “lock” as in lock the door. Pronounced like “lock” as in lock the door. Not Loke as in Tone Lōc. Not Loke as in Tone Lōc. Feel free to call me Ben. Feel free to call me Ben. A bit about myself A bit about myself Background Background Interests Interests

Quick Research Bio University of Tulsa University of Tulsa Medical Informatics Medical Informatics University of North Carolina University of North Carolina Tracking in Virtual Environments (MS) – 3rdTech HiBall Tracking in Virtual Environments (MS) – 3rdTech HiBall Avatars in Virtual Environments Avatars in Virtual Environments Interaction in Virtual Environments Interaction in Virtual Environments University of North Carolina at Charlotte (Postdoc) University of North Carolina at Charlotte (Postdoc) Digital Characters Digital Characters Virtual Reality Virtual Reality Computer Vision Computer Vision Research Areas Research Areas Interactive Computer Graphics (Rendering) Interactive Computer Graphics (Rendering) Virtual Reality Virtual Reality Human Computer Interaction Human Computer Interaction Digital Characters Digital Characters Assistive Technologies Assistive Technologies Computational Geometry / Computer Vision / Image Processing Computational Geometry / Computer Vision / Image Processing

Human Computer Interaction Q: What is the field of HCI? Q: What is the field of HCI? A: Discipline concerned with the study, design, construction and implementation of human-centric interactive computer systems A: Discipline concerned with the study, design, construction and implementation of human-centric interactive computer systems Not only Computer Scientists! Not only Computer Scientists! Comp Science – application and design of interfaces Comp Science – application and design of interfaces Sociology and Anthropology – how humans and technology adapt to each other Sociology and Anthropology – how humans and technology adapt to each other Ergonomics – safety Ergonomics – safety Psychology – cognitive processes and behavior of users Psychology – cognitive processes and behavior of users Linguistics – Development of human and machine languages Linguistics – Development of human and machine languages From:

Human Computer Interaction We are concerned about designing the interface between the user and the system. We are concerned about designing the interface between the user and the system. Examples of HCI topics: Examples of HCI topics: Graphical User Interface Graphical User Interface Widgets Widgets Interfaces themselves Interfaces themselves Mice, keyboard, joystick Mice, keyboard, joystick Evaluation of interfaces Evaluation of interfaces

Fun Question From From A right-handed user is known to be within 10 pixels of the exact center of a large, 1600 X 1200 screen. You will place a single- pixel target on the screen that the user must point to exactly. List the five pixel locations on the screen that the user can access fastest. For extra credit, list them in order from fastest to slowest access. A right-handed user is known to be within 10 pixels of the exact center of a large, 1600 X 1200 screen. You will place a single- pixel target on the screen that the user must point to exactly. List the five pixel locations on the screen that the user can access fastest. For extra credit, list them in order from fastest to slowest access. A: (Right handed user) A: (Right handed user) The pixel immediately at the current cursor location The pixel immediately at the current cursor location The bottom-right corner. The bottom-right corner. The top-left corner. The top-left corner. The top-right corner. The top-right corner. The bottom-left corner. The bottom-left corner.

Virtual Reality Computer generated synthetic environment Computer generated synthetic environment Virtual Worlds Virtual Worlds Immersive Immersive Sensory Feedback Sensory Feedback Interaction Interaction

Digital Characters A new medium to interact with system information A new medium to interact with system information Spectrum from the paper clip -> AIBO Spectrum from the paper clip -> AIBO Why would we want a digital character? Why would we want a digital character? Dynamic Dynamic Better than keyboard and mouse for some tasks Better than keyboard and mouse for some tasks Natural interaction Natural interaction

Current Research Combine HCI with VR Combine HCI with VR If all things in VR are virtual, how do you interact with the environment? If all things in VR are virtual, how do you interact with the environment? Combine HCI with digital characters Combine HCI with digital characters How do you interact with digital characters? How do you interact with digital characters? Interaction = input and output Interaction = input and output

Merging real and virtual spaces

Interacting with Real Objects in VR How important are real objects? How important are real objects? A: For cognitive (problem solving), very important! A: For cognitive (problem solving), very important! Real Space Purely Virtual Hybrid Vis. Faithful Hybrid

NASA Case Study

Videos of Task Having a hybrid environment provides substantial benefits in prototype design.

Locomotion in VR Another aspect of ‘natural interaction’ is locomotion (define). Another aspect of ‘natural interaction’ is locomotion (define). Most common way: Most common way: Use a ‘virtual walking’ metaphor. Use a ‘virtual walking’ metaphor. Does this reduce effectiveness? Does this reduce effectiveness? We can test this because of new wide-area tracking technologies. We can test this because of new wide-area tracking technologies.

Locomotion in VR Video Play video Play video

Results Natural interaction is better!

Avatars in VE Most virtual environments do not provide an avatar (user self- representation) Most virtual environments do not provide an avatar (user self- representation) Why? Because tracking the human body is difficult. Why? Because tracking the human body is difficult. Solution: Use simple computer vision to track colored markers to generate an avatar Solution: Use simple computer vision to track colored markers to generate an avatar

STRAPS Video Play Video Play Video

Digital Characters d= &cf=trailer d= &cf=trailer Thinking PerceivingResponding Thinking PerceivingResponding Digital Character Participant Interaction

DIgital ANimated Avatar (DIANA)

Upcoming Research How do we naturally interact with a digital character? How do we naturally interact with a digital character? Display Display Perception Perception Efficacy Efficacy Combine Straps with DIANA Combine Straps with DIANA Avatar Telephone Avatar Telephone Acting practice Acting practice

VR Interaction Getting real objects into virtual environments Getting real objects into virtual environments How do you naturally interact with virtual objects? How do you naturally interact with virtual objects?

Questions?

Thinking PerceivingResponding Thinking PerceivingResponding Digital Character Participant Interaction