1. Mixed Reality Trompe l’oëil in the 21st Century
Charles Hughes, School of Computer Science
Media Convergence Laboratory
School of Film and Digital Media
June 15, 2005

2. To deceive the viewer as to its reality.
Trompe l’oëil
To deceive the viewer as to its reality.
The most extravagant blending of painting, architecture and painting of architecture for an environment can be witnessed within the Sistine Chapel. Michelangelo intended to blur the lines between the architecture and celestial visions.

3. See-Through HMDs
Optical
Video

4. Extending Mixed Reality “Magic is behind the eyeballs”
Augmented Reality
Augmented Virtuality
VIRTUAL
REALITY
The success of any and all illusion is not with the painting techniques, technology or artist.
The real “Magic is behind the eyeballs”. How does the audience actually blur those lines with its imagination?
With thousands of years of experienced-based entertainment, we use any proportion necessary of reality and virtuality, the key is to engage the imagination to complete the effect.
Video Games
Dark Rides
IMAGINED
Rides

5. Examples from our Media Convergence Lab
MR Experiences
Examples from our Media Convergence Lab

6. Entertainment

7. Time Portal (AR) MR @ SIGGRAPH’03 Visual+audio+SFX Animatic
Play Time Portal Animatic (sets story line) and SIGGRAPH Excerpt – 3minutes + 35 seconds.
Animatic

8. Collaboration

9. Placing Real Entities into Virtual Environments
Augmented Virtuality
AUGMENTED VIRTUALITY
AV is about bringing real things into virtual spaces. AV is to Virtual Reality what AR is to reality. There are things that want to be presented in the virtual world, but virtual reality is not able to replicate or defeats its intrinsic purpose. For instance in a virtual museum where you want to showcase the actual object with all its rich detail and patina. A virtual model will defeat the purpose of an artifact. In the case of our project MS Isle, we wanted to bring real people and their natural gesture within a totally virtual fantasy world. The reason not to go completely virtual is the importance of human to human interaction.
Placing Real Entities into Virtual Environments

10. Demo Dome Characteristics
Relative to CAVE
Lightweight
Inexpensive
Can have distinct POVs
E.g., consider a city planner and an architect
Each wants specialized information
But both need to have a common context

11. Education

12. MR Sea OSC
MR ROVER
Artist’s Concept

14. Water’s Journey through …

15. Training

16. MR MINI-MOUT
Instead of going to the field first, we have created a mini-MOUT at RDEcom.
Instead of bringing it to the MOUT, we have brought the MOUT to us.
It has turned into a rich facility to test complex human performance issues.
Show MOUT2 Asset Planning – 45 seconds

17. Merging Realities VR Actors Vehicular Sniper VR Sets SFX Hostiles CGF
HUD
FFW
It includes critical items that we will be able to rapidly create scenarios for training and testing.
With current HMD, the resolution is dependent on the HMD Cameras.
Play MOUT2MR – 2 minutes.
Multi-Player
MOUT
VR Cues
Merging Realities

19. Research

20. MR Software Suite Platform neutral Story Engine Audio Engine
Agent-based
Plug-in architecture
Audio Engine
Constraint-based speaker usage
Graphics Engine
DMX Engine
Protocols
Command and control streams
Behavior, GUI and device interfaces

21. Visual Rendering Interactive-time and Realistic (GPU)
Global Illumination
Shading and shadows
Interaction of real and virtual
Ambient lighting
Color mapping so virtual react
Story-based rendering
Various HDR projects

22. Fire/Shadows
This starts three seconds after viewing slide. It runs for 1:06 and shows real-time illumination, simple fire (with a fan blowing it) and shadow casting (virtual on real) due to a virtual flashlight.

23. Pegasus in CSB
These start separated by one second within the current window. This allows the audience to see what the corrected colors just behind the uncorrected The corrected merely adjusts its colors and luminance to shift in the direction of the current background. That makes it appear as if the colors of the coke machine reflect off the Pegasus and the colors wash out when passing by the windows. That’s just pure deception. The videos are just 16 seconds long.
You need to point out that no actual illumination algorithms are used here; we just change the colors on the virtual character to react to those of its background, the real world video feed. Is this real-time? YES, and performance can be improved greatly on the new GPUs; this is a video done in May before release of nVidia GeForce 6800.Can we do better with the color blending? YES, and we will.

24. Charles E. Hughes
School of Computer Science & Media Convergence Lab & School of Film and Digital Media & Text and Technology Program (English)
Home Page: