5 Fleming, Torralba, Adelson Todd and colleagues Mingolla and Grossberg Koenderink and van Doorn Zucker and colleagues Zaidi and Li Malik and Rosenholtz
6 It is remarkable that we can recover 3D shape: No motion No stereo No shading No texture image consists of nothing more than a distorted reflection of the world surrounding the object Ideal mirrored surface Fleming et al. (2004). JOV Shape from Specularities
7 As the object moves from scene to scene, the image changes dramatically. Yet, somehow we are able to recover the 3D shape. Shape from Specularities
24 slightly curved Anisotropies in surface curvature lead to anisotropies in the image.
25 Stability across changes in surface reflectance A parametric space of glossy plastic materials (using Ward model) Diffuse Reflectance, d Specular Reflectance, s
26 Idea: Experiment 1 Rationale: measure stability of 3D shape across changes in surface reflectance Method: gauge figure task? Problem: costly to do full depth reconstruction for many shapes and materials Solution? Compare sparse gauge measurement? Alternative task?: locate depth extrema along given raster line (2D task)
27 Texture Anisotropic compression of texture depends on surface slant
28 Texture Anisotropic compression of texture depends on surface slant
32 Affine Transformation Shear: - does affect first derivatives - does NOT affect second derivatives
33 Shear: - does affect first derivatives - does NOT affect second derivatives Affine Transformation
34 Shear: - does affect first derivatives - does NOT affect second derivatives Affine Transformation
35 Shear: - does affect first derivatives - does NOT affect second derivatives Affine Transformation
36 Shear: - does affect first derivatives - does NOT affect second derivatives Affine Transformation
37 Shear: - does affect first derivatives - does NOT affect second derivatives Affine Transformation
38 Idea: Experiment 2 Rationale: use orientation fields to predict misperceptions of 3D shape Possible methods Gauge figure task? Matching task: subject adjusts shear of a textured object until it appears to match the shaded version of the same object Subject adjusts shear of one oject (shaded or textured) until it appears to match the degree of shear of another object? Sounds too strange?
39 Illusory distortions of shape Inspired by Todd & Thaler VSS 05
40 Illusory distortions of shape Inspired by Todd & Thaler VSS 05
41 Idea: Experiment 3 Rationale: use orientation fields to predict misperceptions of 3D shape Possible methods gauge figure task to reconstruct full 3D shape. Again, this is costly, but perhaps a few shapes are enough depth extrema task: locate depth extrema along raster line (this is what Todd and Thaler did). Potentially we could predict the locus directly from the orientation field
42 Idea: Experiment 3 Compare small and large changes in orientation field by using texture stretching along the line of sight Advantage: same infringement of isotropy assumption, different change in apparent 3D shape Unstretched Stretched 2:1 along line of sight
43 Uses biologically plausible measurements Orientation selectivity maps in primary visual cortex of tree shrew. After Bosking et al. (1997). Potential of Orientation Fields
44 No need for visual system to estimate reflectance or illumination explicitly. Classical shape from shading uses the reflectance map to estimate surface normals from image intensities Reflectance map is usually unknown and ambiguous Potential of Orientation Fields
45 Stable across albedo discontinuities. Breton and Zucker (1996), Huggins and Zucker (2001) Potential of Orientation Fields
46 Handle improbable combinations of reflectance and illumination. non-linear intensity transfer function normal shading weird shading Potential of Orientation Fields
47 We could measures shape estimates with these types of stimuli as well. non-linear intensity transfer function normal shading weird shading Link back to experiment 1 ?
48 May explain how images with no obvious BRDF interpretation nevertheless yield 3D percepts Potential of Orientation Fields Ohad Ben-Shahar
51 Conclusions Orientation fields are potentially a very powerful source of information about 3D shape For the early stages of 3D shape processing, seemingly different cues may have more in common than previously thought
52 Thank you Collaborators Ted Adelson Antonio Torralba Funding RF supported by DFG FL 624/1-1
53 What still needs to be explained? For Lambertian materials (or blurry illuminations), the reflectance map is so smooth that it is significantly anisotropic. Therefore shading orientation fields vary considerably with changes in illumination. sidefronttop
54 What still needs to be explained? Surprising prediction: 3D shape should actually be less stable across changes in illumination for diffuse than for specular materials. We found evidence for changes in 3D shape with changes in illumination Alternative: higher order invariants establish an equivalence between different orientation fields. Example: joint measures of orientation at different locations. sidefronttop
55 Note analogy to textures of different orientations Todd et al. (2004) What still needs to be explained?
56 Matte dark grey Rough metal Glossy light grey