1. User Issues in 3D TV & Cinema Martin S. Banks Vision Science Program UC Berkeley

2. Issues in 3D TV & Cinema Technical Issues Developing content Sufficient resolution over time: temporal aliasing Sufficient separation between two eyes’ images: “ghosting” User Issues Perceptual distortions due to incorrect viewing position Flicker & motion judder due to temporal sampling Maintaining depth across scene cuts Window violations Residual ghosting Visual discomfort due to vergence-accommodation conflict Appropriate blur relative to other depth signals Conflict between visually-induced motion & vestibular signals

3. Technical Issues Developing content Sufficient resolution over time: temporal aliasing Sufficient separation between two eyes’ images: “ghosting” User Issues Perceptual distortions due to incorrect viewing position Flicker & motion judder due to temporal sampling Maintaining depth across scene cuts Window violations Residual ghosting Visual discomfort due to vergence-accommodation conflict Appropriate blur relative to other depth signals Conflict between visually-induced motion & vestibular signals Issues in 3D TV & Cinema

4. Technical Issues Developing content Sufficient resolution over time: temporal aliasing Sufficient separation between two eyes’ images: “ghosting” User Issues Perceptual distortions due to incorrect viewing position Flicker & motion judder due to temporal sampling Maintaining depth across scene cuts Window violations Residual ghosting Visual discomfort due to vergence-accommodation conflict Appropriate blur relative to other depth signals Conflict between visually-induced motion & vestibular signals Issues in 3D TV & Cinema

5. Focal distance Vergence distance Vergence & Accommodation: Natural Viewing

6. 36 1.5 3 4.5 6 Vergence Distance (diopters) 0 0 Focal Distance (diopters) Focal distance Vergence distance 1.54.5 Vergence & Accommodation: Natural Viewing

7. 36 1.5 3 4.5 6 Vergence Distance (diopters) 0 0 zone of clear single binocular vision Focal Distance (diopters) Focal distance Vergence distance 1.54.5 Vergence & Accommodation: Natural Viewing

8. 36 1.5 3 4.5 6 Vergence Distance (diopters) 0 0 Percival's zone of comfort zone of clear single binocular vision Focal Distance (diopters) Focal distance Vergence distance 1.54.5 Vergence & Accommodation: Natural Viewing

9. Focal distance Vergence distance Vergence & Accommodation: Stereo Display

10. 36 1.5 3 4.5 6 Vergence Distance (diopters) 0 0 Focal Distance (diopters) 1.54.5 Focal distance Vergence distance Vergence & Accommodation: Stereo Display Percival's zone of comfort zone of clear single binocular vision

11. Displays with Nearly Correct Focus Cues Two multi-focal displays we’ve developed: 1.Fixed-viewpoint, volumetric display with mirror system & 3 focal planes (Akeley, Watt, Girshick, & Banks, SIGGRAPH, 2004). 2.Fixed-viewpoint, volumetric display with switchable lens & 4 focal planes (Love, Hoffman, Kirby, Hands, Gao, & Banks, Optics Express, 2009)

12. Multi-focal Display Akeley, Watt, Girshick & Banks (2004), SIGGRAPH.

13. Multi-focal Display

14. Akeley, Watt, Girshick & Banks (2004), SIGGRAPH. Multi-focal Display

15. Depth-weighted Blending Depth-weighted blending along lines of sight Weights dependent on dioptric distances to planes Akeley, Watt, Girshick, & Banks (2004), SIGGRAPH.

16. Do V-A Conflicts Cause Fatigue/Discomfort?

17. 600-ms stimulus at near or far vergence-specified distance Appeared at each focal distance Hoffman, Girshick, Akeley, & Banks (2008), Journal of Vision Do V-A Conflicts Cause Fatigue/Discomfort?

18. ** cues-inconsistent cues-consistent Severity of Symptom 1 3 5 7 9 How tired are your eyes? How clear is your vision? How tired or sore are your neck & back? How do your eyes feel? How does your head feel? ** = p < 0.01 (Wilcoxen test) Hoffman, Girshick, Akeley, & Banks (2008), Journal of Vision Do V-A Conflicts Cause Fatigue/Discomfort?

19. ** * Which session was more fatiguing? Which session irritated your eyes more? Which session gave you more headache? Which session did you prefer? cues-consistent much worse than inconsistent cues-inconsistent much worse than consistent no difference ** = p < 0.01 (Wilcoxen test) Hoffman, Girshick, Akeley, & Banks (2008), Journal of Vision Do V-A Conflicts Cause Fatigue/Discomfort?

20. Discomfort & 3D Cinema

24. Discomfort & 3D Cinema & TV

25. Technical Issues Developing content Sufficient resolution over time: temporal aliasing Sufficient separation between two eyes’ images: “ghosting” User Issues Perceptual distortions due to incorrect viewing position Flicker & motion judder due to temporal sampling Maintaining depth across scene cuts Window violations Residual ghosting Visual discomfort due to vergence-accommodation conflict Appropriate blur relative to other depth signals Conflict between visually-induced motion & vestibular signals Issues in 3D TV & Cinema

26. Almost never view pictures from correct position. Retinal image thus specifies different scene than depicted. Do people compensate, and if so, how? Viewing Pictures

27. Stimuli Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.

28. Stimulus: simulated 3D ovoid with variable aspect ratio. Task: adjust ovoid until appears spherical. Vary monitor slant S m to assess compensation for oblique viewing positions. Spatial calibration procedure. If compensate, will set ovoid to sphere on screen (ellipse on retina). Observation Point SmSm CRT Experimental Task Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.

29. Center of Projection Observation Point No compensation: set ovoid to make image on retina circular: retinal coordinates screen coordinates Predictions

30. Center of Projection Observation Point Compensation: Set ovoid to make image on screen circular: Predictions retinal coordinates screen coordinates

31. Aspect Ratio (screen coords) 1 1.2 1.4 -40-2002040 invariance predictions Viewing Angle S m (deg) SmSm Predictions

32. 1 1.2 1.4 -40-2002040 invariance predictions retinal predictions Aspect Ratio (screen coords) Viewing Angle S m (deg) SmSm

33. Results 1 1.2 1.4 -40-2002040 monoc-aperture invariance predicts retinal predicts Aspect Ratio (screen coords) Viewing Angle (deg) JLL Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.

34. Results 1 1.2 1.4 -40-2002040 monoc-aperture binoc-no aperture invariance predicts retinal predicts JLL Results Aspect Ratio (screen coords) Viewing Angle (deg) Vishwanath, Girshick, & Banks (2005), Nature Neuroscience.

35. Compensation for Incorrect Viewing Position Pictures not useful unless percepts are robust to changes in viewing position. People compensate for oblique viewing position when viewing 2d pictures. Two theories of compensation: pictorial & surface. Data clearly favor surface compensation. Two versions of surface method: global & local. Data clearly favor local slant.

36. 2D Pictures vs 3D Pictures Two eyes presented same image Binocular disparities specify orientation & distance of picture surface; hence useful for compensation 2D

37. Two eyes presented different images Binocular disparities specify orientation & distance of picture surface and layout of picture contents; hence not useful for compensation Two eyes presented same image Binocular disparities specify orientation & distance of picture surface; hence useful for compensation 3D Two eyes presented same image Binocular disparities specify orientation & distance of picture surface; hence useful for compensation 2D 2D Pictures vs 3D Pictures

38. Stereo (3D) Pictures For most applications, viewers will not be at correct position. Retinal disparities thus specify a different layout than depicted. Do people compensate? Is correct seating position for a 3D movie more important than for 2D movie?

39. Stereo Picture Geometry display surface stereo projectors

40. display surface stereo projectors depicted hinge Stereo Picture Geometry

41. display surface stereo projectors depicted hinge Stereo Picture Geometry

42. display surface stereo projectors depicted hinge disparity-specified hinge Stereo Picture Geometry

43. perceived dihedral angle? display surface stereo projectors depicted hinge disparity-specified hinge Stereo Picture Geometry

44. Predictions 02545 0 30 60 90 120 Viewing Angle (deg) 35° 17.5° 0°0° -17.5° -35° Hinge Setting (deg) Invariance: Hinge settings are 90° for all viewing angles and base slants Retinal disparity: Hinge settings vary significantly with viewing angle & base slant

45. Viewing Angle (deg) 025 45 30 60 90 120 Hinge Setting (deg) Results non-stereo pictures

46. Viewing Angle (deg) 025 45 30 60 90 120 stereo pictures 025 45 30 60 90 120 Hinge Setting (deg) Results non-stereo pictures

47. Viewing Angle (deg) 025 45 30 60 90 120 025 45 30 60 90 120 Hinge Setting (deg) non-stereo picturesstereo pictures Results

48. Summary User issues in 3D cinema & TV Vergence-accommodation conflicts cause visual fatigue & discomfort Can be handled by attending to viewer’s distance from screen & range of disparities presented relative to screen Perceptual distortions due to incorrect viewing position Compensation is good with non-stereo pictures Compensation is significantly poorer with stereo pictures suggesting that viewer position could be more important

49. Acknowledgments Kurt Akeley (Microsoft) Simon Watt (Univ. of Wales, Bangor) Ahna Girshick (NYU) David Hoffman (UC Berkeley) Robin Held (UC Berkeley) Funding from NIH, NSF, & Sharp Labs