1. Visual, auditory, and haptic displays Dr. Xiangyu Wang Acknowledgment of Dr. Doug Bowman’s lecture notes

2. Introduction to displays Display: device which presents perceptual information Often ‘display’ used to mean ‘visual display’ Goal: display devices which accurately represent perceptions in simulated world (i.e., higher levels of immersion)

3. Vision Stimulus: light of wavelengths ~350-750 nm

4. Visual displays for VEs Standard monitor (mono/stereo) Head-mounted/head-referenced Projected (usually stereo) –single-screen –multiple, surrounding screens Retinal display Volumetric displays

5. Characteristics of visual displays Field of regard (FOR), field of view (FOV) Brightness, contrast ratio Resolution Screen geometry Light transfer Refresh rate Ergonomics

6. VR with a monitor

7. Head-mounted displays (HMDs) full surround (FOR=360) simple stereo cumbersome tethering to computer often small FOV single user

8. Surround-screen displays less obtrusive headgear multi-user better stereo occlusion problem missing sides

9. Surround-screen displays Video:

10. Six-sided CAVE (Duke U. DiVE)

11. Tabletop displays direct manipulation “god’s-eye” view change orientation small FOR

12. Virtual retinal display (VRD) HIT lab / Microvision image scanned directly onto retina great potential

13. Volumetric display Pixels displayed in actual 3D space Multi-user correct viewing Size issues Can’t move/reach into display

14. Volumetric display Video

15. Which visual display to use? Consider lists of pros and cons Consider depth cues supported Consider level of visual immersion But this is a very hard question to answer empirically Instead of comparing actual displays, compare levels of immersion

16. Auditory displays Second most studied sense Second most common VE display Stimulus: disturbance of molecules in a medium (air) Perceptions: pitch, loudness, location

17. Simple VE audio Intensity fall-off (1/d2) Headphones also block out real-world noises Ambient sound (e.g. stream) Present speech instead of text

18. 3D auditory displays Technologies: –Speaker-based –Headphone-based Uses: –Virtual objects emitting sound –Sensory substitution

19. Auditory displays Video clip (1) shows the actual physical environment in which the virtual reality system was installed, and shows a participant during the course of the exploratory phase of the experiment. Video clip (2) shows the virtual environment during several phases of the experimental procedure. The virtual scene is shown from both the participants subjective view and also the overhead view. The virtual audio scene is also included and is rendered for the participant's position in the scene.

20. Haptic Displays Touch –Actually a range of skin senses –touch / pressure –hot / cold –pain Useful for object identification & understanding

21. Haptic displays Exoskeleton Robot arms Phantom Tactile devices Video: haptic Prototyping –http://www.youtube.co m/watch?v=- i9Wm2rTsao

22. SPIDAR haptic display

23. Passive haptics Use of props - “poor man’s” haptic Display –ARtoolkit –pen & tablet

24. Haptic displays Displaying to other skin senses Simple, special-purpose “displays” for temperature, air movement, etc. –Fan –Heat lamp

25. Olfactory displays Sense of smell not studied extensively - use in VEs? A few academic projects Smell synthesis still in the future

26. Olfactory displays Video: "Fragra": Entertainment System Utilizing Olfactory Display by Arito MochizukiArito Mochizuki

27. Vestibular/kinesthetic displays Sense of body, self-motion Virtual body representation Can “display” to these senses by: –Using motion platforms –Stimulating the proper parts of the brain

28. Conclusions Making VEs multi-sensory is becoming easier becoming cheaper But most VEs still use only visual display We need to study the effects of level of auditory and haptic immersion