Depth Cue Integration in Grasping Slanted Object Zhongting Chen & Jeffrey Saunders The University of Hong Kong APCV 2015

Perception vs. Action Visual processing for perception and control of actions may involve different mechanisms Evidence from brain deficits – Agnosia (ventral deficit) vs. Optic ataxia (dorsal deficit) Karnath et al., 2009, Figure 1

Perception vs. Action Another approach line: visual illusions on perception and action control – Action control: insensitive to visual illusion? – Controversial results (e.g. Franz, Bülthoff & Fahle (2003) vs. Ganel, Tanzer, & Goodale (2008) ) Current study: are depth cues used in similar way for 3D perception and control of action?

Depth cues for grasping 3D slant perception – Underestimation of depth with limited cues – Based on integrated information from binocular and monocular cues Question: are depth cues used in the same way for slant perception and natural grasping control? Weak cue Prior Combined Strong cue Prior Combined

Grasping slanted objects Grasping slanted objects is a natural situation where 3D information is needed for control of grasping Grip orientation must be aligned with the 3D slant of the object Contact points for stable grip depend on 3D shape of the object

Questions Does the slant of the grip axis depend on 3D cues in the same way as slant judgments? Do grasp points change in a way that is consistent with errors in perceived slant? When grasping real objects that are slanted in depth …

Experiment 1: slant of grip axis Free-hand grasping toward irregularly shaped planar 3D objects Binocular or monocular viewing Isotropic or vertically compressed (82%) – Isotropic: consistent monocular and binocular cues – Compressed: conflicting cues

Cue conflicting stimuli Isotropic vs. Compressed 82% Perspective projection Slant 45 ° Slant 55 ° Slant from projected contour: 55°

Procedure Sample size: n = 12

Monocular Binocular Slant Results: binocular vs. monocular Slant of grip axis Normalized distance Viewing effect Slant conditions Slant of grip axis

Results: cue conflict stimuli Slant of grip axis Normalized distance Isotropic, Slant 45° Compressed, Slant 45° Isotropic, Slant 55° ** *** Intermediate slant for cue conflict stimuli

Summary of Experiment 1 Slant of the grip axis during reaching depended on depth information Monocular vs. binocular – More frontal bias for monocular, consistent with slant underestimation Conflicting cues – more slanted for compressed objects, consistent with use of monocular cue Grasp point selection: – Not predictable enough to observe the systematic shift by varying depth information

Experiment 2 Goal: test the effect of depth cues on grasp points selection Regular objects with predictable grasp points Sample size: n = 10

Cue conflict stimuli Perspective views Original (Isotropic) Stretched Compressed Slant 45° Slant 35° Slant 45° Slant 55° Slant 45° Frontal views

Slant of grip axis Normalized distance Results: slant in cue conflict conditions Isotropic, Slant 35° Isotropic, Slant 45° Isotropic, Slant 55° Stretched, Slant 45° Compressed, Slant 45° Slant of grip axis Isotropic, Slant 35° Isotropic, Slant 45° Isotropic, Slant 55° Compressed, Slant 45° Stretched, Slant 45° *** **

Measurement: frontal angle of grip axis Grasp point selection

Predicted frontal angles Frontal- parallel 45 ° slanted grip axis shifting based on monocular information

Distortion effect (for right-tilted stimuli ) frontal angles of grip axis Normalized distance Results: grasp point selection Isotropic, Slant 35°(reference for mono) Isotropic, Slant 45°(reference for bino) Isotropic, Slant 55°(reference for mono) Compressed, Slant 45° Stretched, Slant 45° Frontal angles of grip axis Isotropic, Slant 35° Isotropic, Slant 45° Isotropic, Slant 55° Compressed, Slant 45° Stretched, Slant 45° *** **

Distortion effect (for left-tilted stimuli ) frontal angles of grip axis Normalized distance Results: grasp point selection Frontal angles of grip axis Isotropic, Slant 35° Isotropic, Slant 45° Isotropic, Slant 55° Compressed, Slant 45° Stretched, Slant 45° Isotropic, Slant 35° Isotropic, Slant 45° Isotropic, Slant 55° Compressed, Slant 45° Stretched, Slant 45°

Summary of Experiment 2 Slant of the grip axis – Binocular and monocular depth cues both affected the slant of the hand during reaching Grasp point selection – Binocular and monocular cues also contributed to selection of grasp points – only for right-tilted objects – left-tilted objects showed a ceiling effect

Conclusion Use of depth cues for control of hand movements consistent with slant perception: – Frontal bias when lacking depth information – Integration of monocular and binocular cues Cue integration demonstrated for both the slant of the hand and grasp point selection

Grasping illusions Shape of cue conflict stimuli were perceived incorrectly – an illusion Grasping was consistent with illusory percept NO dissociation between visual processing of perception and action in this situation 45° slant

Acknowledgement Ms Xing Xing, graduate student Dr. Jeff Saunders Supervisor Supported by Hong Kong Research Grants Council