1. Cosc 6326/Psych6750X Vision and Visual Displays

2. Why do we have two eyes? Binocular vision has several advantages including –increased field of view –redundancy –increased detection probability –depth perception from vergence stereopsis

3. Geometry of Binocular Correspondence Images in the two eyes differ (or are disparate) due to separation of vantage points Points of light project to the same retinal locations in the two eyes only on the horopter Points off the horopter have horizontal and/or vertical disparity

5. The Geometrical Point Horopter LERE Fixation Point Uncrossed disparity Crossed disparity

6. Even flat frontal surfaces have disparity Differential perspective Both horizontal and vertical disparities Gradient of vertical size disparity –Varies with distance –Affects depth, shape judgments … Left Eye Image Right Eye Image

7. Vergence

8. Vergence eye movements opposite (disjunctive) movements of the eyes a principle function is to align both high acuity foveae on target of interest vergence helps bring images of objects of interest into correspondence (so that the images fall on similar parts of both retinae)

9. since eyes are laterally separated in the head the optical axes of the two eyes are parallel only for fixation at infinity nearer targets require crossing one’s eyes (convergence)

10. Several catergories –tonic vergence –voluntary vergence or proximal vergence induced voluntarily or by an object that appears to be at a different distance –accommodative vergence when accommodating there is a link with vergence that drives a compatible vergence change; also vergence accommodation

11. disparity vergence –vergence in response to disparity in the images in the two eyes –allows for aligning the images of the target in the two eyes

12. Two main questions for us 1.Does vergence give a sense of depth? could theoretically triangulate depth of target from angle of convergence static vergence is probably a fairly weak indicator for distance judgements but indirectly important for size and shape perception, calibration of stereopsis e.g. micropsia in the wallpaper effect size and shape constancy

13. Dynamic vergence may play a role in signalling change in depth also controversial Regan (1986) found no changing depth from changing vergence in large stimuli but weak effect in small stimuli; substantial changes in apparent size

14. 2. Dissociation of accommodation and vergence Normally vergence, accommodation and pupil size are controlled in a tightly coupled manner (the ‘near triad’) In most binocular displays, the display is focused at a fixed distance and thus accommodation should be fixed

15. This fixed accommodation leads to vergence/accommodation mismatch –depth cue conflicts between vergence and accommodation; stereopsis and blur –difficulty in obtaining clear vision for stimuli requiring vergence nearer or further than the optical distance of the display OR –difficulty ‘fusing’ stimuli –contributes to eyestrain, simulator sickness

16. –adaptation, re-adaptation, visual stress (seminar next week Wednesday)

17. Stereoscopic vision

18. Binocular Disparity –Horizontal disparity arises from lateral separation of the eyes (parallax) –Relative depth is coded by disparity scaled inversely by the square of viewing distance  (ad) D 2 –Stereopsis (solid sight) refers to depth perception from disparity

19. Depth from Disparity Screen Fixation Point, F LeftEyeRightEye P F PRPRPRPR PLPLPLPL –Horizontal disparity arises from lateral separation of the eyes Screen Fixation Point, F LeftEyeRightEyePF PRPRPRPR PLPLPLPL D d a

20. Stereoscopic Voxels from Davis and Hodges 1995

23. http://www.pandigitalmedia.com/bgussin

24. Sensitivity to disparity People are extremely sensitive to relative disparity. Stereopsis is a hyperacuity –discrimination thresholds of a few seconds of arc –Upper limit: patent versus qualitative stereopsis much poorer sensitivity to absolute disparity thus very sensitive to depth differences between stimuli; poor ability to estimate absolute distance

25. Sensitivity to disparity Disparity has an inverse square dependence on distance Thus, stereopsis is most effective for estimating depth and surface shape at near distances

26. Sensitivity to disparity many people are stereo-anomalous –cannot see depth from disparity (about 5% of population) or –cannot see motion-in-depth from changing disparity or –have deficits in particular part of the visual field, or for crossed disparity, or for uncrossed disparity cannot rely on all subjects seeing stereoscopic depth

27. Two of the main issues in stereopsis are –matching of corresponding parts of the two images –estimation of depth from disparities

28. Keplerian projection Correspondence problem If features are similar a matching ambiguity arises Human stereopsis usually matches images very effectively Often the most difficult problem in stereoscopic computer vision

29. Random-dot stereograms

30. not all features in one eye’s image have a corresponding feature in the other image these monocular zones play an important role in perception of objects and surface properties in binocular vision –depth from monocular occlusion –luster –binocular rivalry

32. Fusion Typically a wide range of disparities present in a scene If disparities are modest people ‘fuse’ the images of the object in the left and right and perceive a unitary object Outside this range objects are seen doubly (diplopic) or one eye’s image is suppressed (binocular rivalry)

33. Fusion Only a range of disparities near the horopter are seen singly (Panum’s fusional area) Also a limited range of disparities for which depth can be obtained from disparity. But depth can be seen in diplopic images Fusion depends on size, contrast, disparity gradient, temporal factors …

34. Estimation of Depth from Disparity Relation between horizontal disparity and depth varies with distance and location of surface wrt head (recall the curvature of the horopter) Need to account for viewing system parameters of distance and eccentricity to judge –metric depth, relief, surface curvature, surface slant … –absolute retinal disparity does not provide viewing system parameters need: ‘sensed’ eye position, vertical disparities, other depth cues, …

35. Stereoscopic issues in displays

36. Types of display based on ‘ocularity’ –Monocular : Image presented to one eye only no stereopsis, rivalry with dark images –Biocular : same image presented to both eyes Stereopsis specifies a flat surface stereoscopic artefacts can arise with misalignment –Binocular : eye specific (dichoptic) images presented to each eye Stereoscopic imagery possible Autostereoscopic - binocular display that requires single display and no special eyewear

37. Methods of presenting stereoscopic displays –hmds –shutters –polaroid- linear, circular –anaglyphs –barrier, lenticular –holograms –….

39. From www.stereographics.com

41. Example - 3ality autostereoscopic display Many new autostereoscopic displays based on lenticular sheets or parallax barrier methods – trade spatial resolution for stereopsis Shutter and some other systems trade temporal resolution for stereopsis (flicker) 3ality display codes stereoscopic information in luminance, preserving spatial and temporal resolution (at expense of dynamic range).

43. Viewer wears glasses with a pair of orthogonal polarizing filters At each pixel, left eye should see a certain amount of light of one polarization, and the right eye a certain amount of the other Left and right eyes images mixed by summing luminance of the two images (vector sum) - ‘magnitude’ of the vector

44. Images from Kleinberger et al, Stereoscopic Displays and Applications X, SPIE vol #5006

45. An ‘extra’ liquid crystal panel twists the light for each pixel to the appropriate intermediate polarization - ‘angle’ of the vector Eyewear decomposes the combined light into appropriate amount of left and right eye polarization

48. Autostereoscopic version

49. When are stereo displays useful? –3D visualisation –ambiguous/poor monocular information –static displays –direct 3D manipulation –cluttered, complicated scenes

50. some issues –field of view –depth resolution –ghosting and crosstalk –refresh rate for time multiplexed techniques –distortion, alignment and calibration –focus –head tracking, view dependence –‘orthostereoscopy’ vs visual comfort

51. Generating Stereoscopic Displays A range of algorithms have been proposed to deal with generating stereoscopic imagery to deal with –fusion, focus problems –head movement –optimising range of depth portrayed we’ll look at these in next week’s seminars

52. Trade-offs in field of view in binocular displays –100% overlap stereoscopic vision over the FOV less distortion effects total FOV is same as monocular FOV

53. –Partial Overlap: stereopsis restricted to binocular overlap region parts of normal binocular field sees monocular view: rivalry possible, no stereopsis total FOV exceeds monocular FOV distortion introduces disparity even if optics for the two eyes are matched

54. Calibration and Alignment Calibration is critical for stereoscopic displays. Some parameters: –interpupillary distance (IPD) varies with subject 51mm ~ 76mm vergence changes effective IPD slightly (may be a small artifact in precise AR displays) –convergence of the optics –relative rotational alignment of the displays –interocular differences in linearity, distortion, image size

55. In stereoscopic displays relative misalignment may distort space Alignment is critical for tele-operation systems which fuse images from multiple sensors or in augmented reality

56. Distortion –Nonlinear mapping from object space to image space arising from lens or display systems –radial distortions are common e.g. pincushion or barrel distortions –for stereoscopic displays, distortion is particularly troublesome when not matched in the two eyes