Presentation is loading. Please wait.

Presentation is loading. Please wait.

The role of contours in form- based motion processing P.U. Tse, G. P. Caplovitz, and P.-J. Hsieh.

Similar presentations


Presentation on theme: "The role of contours in form- based motion processing P.U. Tse, G. P. Caplovitz, and P.-J. Hsieh."— Presentation transcript:

1 The role of contours in form- based motion processing P.U. Tse, G. P. Caplovitz, and P.-J. Hsieh

2

3 Form from motion or Structure from motion

4 Form from motion Motion from form

5 1.Transformational Apparent Motion

6 Form from motion Motion from form 1.Transformational Apparent Motion 2.Trackable features: Object shape can influence how fast it is perceived to rotate

7 Form from motion Motion from form 1.Transformational Apparent Motion 2.Trackable features: Object shape can influence how fast it is perceived to rotate 3.Emergent motion signals

8 Overview of talk Theoretical background to contour discontinuities The role of contour discontinuities in form/motion processing Transformational apparent motion Motion from feature tracking Emergent motion signals

9 Overview of talk Theoretical background to contour discontinuities The role of contour discontinuities in form/motion processing Transformational apparent motion Motion from feature tracking Emergent motion signals

10 Contour discontinuities What are they? Why are they important?

11 Attneave, 1954

12

13

14

15

16

17 Biederman, 1987

18

19

20 Curvature detectors

21 ( Wolfe, Yee & Friedman-Hill (1992).

22 Curvature detectors ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( Wolfe, Yee & Friedman-Hill (1992).

23 Curvature detectors ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( (  There exist filters tuned to curvature Wolfe, Yee & Friedman-Hill (1992).

24 Curvature as a cue to 3D shape Tse, 2002, Psych. Review

25

26

27

28

29

30

31

32

33 Theory: The visual system locates points of contour curvature discontinuity in order to generate shape-from-contours Tse (2002, Psych. Review)

34

35

36 Caveat: much more is going on in shape formation

37

38

39

40

41 Curvature as a cue to 3D shape

42

43

44

45

46

47 Curvature discontinuities pop out Kristjansson & Tse, 2001, P&P

48

49

50 Curvature discontinuities pop out Kristjansson & Tse, 2001, P&P

51 Curvature discontinuities pop out Kristjansson & Tse, 2001, P&P

52

53

54 Curvature discontinuities pop out Kristjansson & Tse, 2001, P&P

55 Curvature discontinuities pop out Kristjansson & Tse, 2001, P&P

56 Curvature discontinuities pop out Kristjansson & Tse, 2001, P&P

57 Curvature discontinuities pop out Kristjansson & Tse, 2001, P&P  There exist filters tuned to curvature discontinuities

58 Main theoretical point Contour discontinuities include: *corners *junctions *terminators *deep concavities and convexities These are highly informative when constructing 3D shape and 3D motion

59 Overview of talk Theoretical background to contour discontinuities The role of contour discontinuities in form/motion processing Transformational apparent motion Motion from feature tracking Emergent motion signals

60 Overview of talk Theoretical background to contour discontinuities The role of contour discontinuities in form/motion processing Transformational apparent motion Motion from feature tracking Emergent motion signals

61 Overview of talk Theoretical background to contour discontinuities The role of contour discontinuities in form/motion processing Transformational apparent motion Motion from feature tracking Emergent motion signals

62

63

64 Why did you brain match like this?

65 And not like this?

66 Does form influence matching?

67

68

69 ? ?

70 Form did not seem to play much if any role in matching. People assumed that motion- energy and the nearest neighbor principle dominated matching.

71 Tse, Nakayama, and Cavanagh argued that this was an artifact of using translational apparent motion as a probe.

72

73 Transformational apparent motion Tse, Nakayama, and Cavanagh, 1996, 1998

74 Transformational apparent motion

75

76

77

78

79

80

81

82

83

84 Tse, Nakayama, and Cavanagh, 1996, 1998

85 Transformational apparent motion Tse, Nakayama, and Cavanagh, 1996, 1998

86 Transformational apparent motion Tse, Nakayama, and Cavanagh, 1996, 1998

87 Transformational apparent motion Tse, Nakayama, and Cavanagh, 1996, 1998

88 Transformational apparent motion Tse and Logothetis, 2002

89 Transformational apparent motion Tse and Logothetis, 2002

90

91

92

93

94

95

96

97

98

99

100

101

102 High-level vs Low-level motion processing Low-level: Motion energy dependent, nearest neighbor matching

103 High-level vs Low-level motion processing Low-level: Motion energy dependent, nearest neighbor matching High-level: Matching takes place on the basis 3D form relationships. High-level motion paths are determined following a stage of segmentation, grouping, and 3D analysis that is particularly sensitive to contour relationships and occlusion among attended figures.

104

105

106

107

108

109

110

111

112 Mri data specs GE 1.5T signa 1-shot T2*-weighted gradient-recalled EPI block design, quasi-random order 20 second epochs, 8 vols TR=2.5s, 25 slices, FA=90degrees voxel size: 3.75x3.75x5mm EPI FOV 240x240x256mm Data analysis BV4.9, matlab Stimulus back-projected, FOV 22x16 degrees

113 Stimuli TCCCTT TCCTTC

114 TCCCTT TCCTTC Task: press button when fixation point transiently disappears for 17ms.

115 See TAM demo

116 Back Right Left Bottom right left rearbottom n=19

117 Back Right Left Bottom right left rearbottom hMT+ n=19

118 Back Right Left Bottom right left rearbottom hMT+ LOC n=19

119

120 ** ** * * *

121 * * * * * * *

122 * * * * * *

123 Ventral pathway Dorsal pathway V1 V2 thick stripe V3A MT V2 thin stripe V4v MST LOC V3vV3d

124 Ventral pathway Dorsal pathway V1 V2 thick stripe V3A MT V2 thin stripe V4v MST LOC V3vV3d

125 The key idea here Form analysis occurs in retinotopic and ventral non-retinotopic areas BEFORE the perception of motion. Perceived motion direction follows a spatiotemporal analysis of GLOBAL FORM relationships among contours.

126 The key idea here Form analysis occurs in retinotopic and ventral non-retinotopic areas BEFORE the perception of motion. Perceived motion direction follows a spatiotemporal analysis of GLOBAL FORM relationships among contours.

127 The key idea here Form analysis occurs in retinotopic and ventral non-retinotopic areas BEFORE the perception of motion. Perceived motion direction follows a spatiotemporal analysis of GLOBAL FORM relationships among contours.

128

129

130

131 Any model of motion processing that fails to explicitly incorporate a stage of global contour-based form analysis will be incomplete.

132 For example, high-level motion cannot just be a motion-energy detector that receives input from a salience map. There must be an explicit stage of form analysis.

133 Overview of talk Theoretical background to contour discontinuities The role of contour discontinuities in form/motion processing Transformational apparent motion Motion from feature tracking Emergent motion signals

134 Overview of talk Theoretical background to contour discontinuities The role of contour discontinuities in form/motion processing Transformational apparent motion Motion from feature tracking Emergent motion signals

135 Velocity Space The infinite number of motions that lead to the same component vector all lie on a “constraint line” in vector space

136 Percepts of Translational Motion The examination of Plaids –two oriented, overlapped, independently moving sinewave gratings will appear to move as a plaid –The direction and speed of motion can be predicted by various component-motion integration models

137 Integration Example Predicted by IOC or VS Example Predicted by IOC and not VS

138

139

140

141

142

143

144

145 Trackable Feature Solution to Aperture Problem Eyetracking and Neurophysiological Evidence suggests that certain form features (Endpoint Terminators) can be used by the visual system to disambiguate percepts of motion independent of component motion (Area MT: Pak and Born (2003)) (V1: Pak and Livingston (2003))

146 Components generated along a curved contour

147 IOC succeeds for Translational Motion

148

149

150

151

152 IOC Fails for Rotation

153

154

155

156

157

158 IOC for translating ellipse

159 IOC for rotating ellipse

160 See ellipse demo

161 Rigidity = Preservation of Shape Regions of High Curvature are Critical to the perception of shape

162 Component Motion will always be non-rigid Although Component Motion predicts variable speeds Some form of form analysis must occur for stimulus to remain rigid, presumably regions of high curvature are critical for rigidity

163

164 Experiment 1 Experiment 2 Experiment 3 Experiment 4

165 Experiment 1 Experiment 2 Experiment 3 Experiment 4

166 Caplovitz, Hsieh, and Tse, 2004

167

168 Experiment 1 Experiment 2 Experiment 3 Experiment 4

169

170

171

172

173 Paradox!

174 Experiment 1 Experiment 2 Experiment 3 Experiment 4

175

176

177

178

179 vs.

180 Experiment 1 Experiment 2 Experiment 3 Experiment 4

181

182

183

184 Determined equiluminant green for each subject

185 Experiment 1 Experiment 2 Experiment 3 Experiment 4

186 Experiment 1 Experiment 2 Experiment 3 Experiment 4

187

188 Experiment 1 Experiment 2 Experiment 3 Experiment 4

189

190 Now pit equiluminant ellipse against luminance-defined ellipse

191 vs.

192 Equiluminant ellipses appear to rotate more slowly than luminance-defined ellipses

193 There is an interaction: as the ellipse gets thinner, the perceived angular velocity of an equiluminant ellipse becomes closer to that of a luminance-defined ellipse.

194

195 vs.

196 Experiment 1 Experiment 2 Experiment 3 Experiment 4

197

198 Experiment 1 Experiment 2 Experiment 3 Experiment 4

199

200 Summary Global form-based processing of trackable features such as corners and regions of relatively high positive curvature serve as a motion cue in high-level motion processing. Perceived motion magnitude is a function of both low- and high-level inputs.

201 Question Which areas of the brain vary parametrically with contour curvature discontinuity points in motion?

202 Question Which areas of the brain vary parametrically with contour curvature discontinuity points in motion? All stimuli had the same area, but the sharpness of curvature discontinuities varied greatly across conditions.

203

204

205

206

207

208

209

210 Five types of stimuli were used, consisting of two half ellipses joined along their common major axis.

211

212

213

214

215 All stimuli rotated at the same speed.

216 See Bumps stimuli

217 Stimuli

218 In order to guarantee that they were maintaining fixation, observers carried out a simple task at the fixation point that was not directly related to issues of curvature processing (press button when fixation point blinks).

219

220

221

222

223

224

225

226

227

228 Hold area constant Hold max velocity constant Hold perceived angular velocity constant

229 Hold area constant Hold max velocity constant Hold perceived angular velocity constant

230 Hold area constant Hold max velocity constant Hold perceived angular velocity constant

231 Summary Significant parametric variation between contour curvature discontinuities and BOLD signal occurs in V3A/B. V3A/B may process curvature discontinuities, junctions, and terminators that provide hMT+ with unambiguously moving features to disambiguate motion energy signals.

232 Summary Significant parametric variation between contour curvature discontinuities and BOLD signal occurs in V3A/B. V3A/B may process curvature discontinuities, junctions, and terminators that provide hMT+ with unambiguously moving features to disambiguate motion energy signals.

233 Summary Significant parametric variation between contour curvature discontinuities and BOLD signal occurs in V3A/B. V3A/B may process curvature discontinuities, junctions, and terminators that provide hMT+ with unambiguously moving features to disambiguate motion energy signals.

234 Ventral pathway Dorsal pathway V1 V2 thick stripe V3A MT V2 thin stripe V4v MST LOC V3vV3d

235 Ventral pathway Dorsal pathway V1 V2 thick stripe V3A MT V2 thin stripe V4v MST LOC V3vV3d TAM results

236 Ventral pathway Dorsal pathway V1 V2 thick stripe V3A MT V2 thin stripe V4v MST LOC V3vV3d Bumps results

237 Ventral pathway Dorsal pathway V1 V2 thick stripe V3A MT V2 thin stripe V4v MST LOC V3vV3d Bumps results

238 Overview of talk Theoretical background to contour discontinuities The role of contour discontinuities in form/motion processing Transformational apparent motion Motion from feature tracking Emergent motion signals

239

240

241

242

243

244

245

246

247

248

249 Key idea: Form determines motion There exist two interacting form analyses Ventral (V4v etc)  3D surfaces, volumes Dorsal (V3A etc)  3D trajectories Both rely on contour discontinuities

250 Overview of talk Theoretical background to contour discontinuities The role of contour discontinuities in form/motion processing Transformational apparent motion Motion from feature tracking Emergent motion signals

251 Conclusions

252 Perceived motion direction follows global form analysis of contour relationships

253 Conclusions Perceived motion direction follows global form analysis of contour relationships This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+.

254 Conclusions Perceived motion direction follows global form analysis of contour relationships This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+. Perceived motion speed follows form analysis of trackable features (contour discontinuities)

255 Conclusions Perceived motion direction follows global form analysis of contour relationships This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+. Perceived motion speed follows form analysis of trackable features (contour discontinuities) This form analysis occurs in V3A/B

256 Conclusions Perceived motion direction follows global form analysis of contour relationships This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+. Perceived motion speed follows form analysis of trackable features (contour discontinuities) This form analysis occurs in V3A/B CDs play a key role in parsing and as TFs.

257 Conclusions Perceived motion direction follows global form analysis of contour relationships This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+. Perceived motion speed follows form analysis of trackable features (contour discontinuities) This form analysis occurs in V3A/B CDs play a key role in parsing and as TFs. Motion perception based on emergent motion

258 Thank you


Download ppt "The role of contours in form- based motion processing P.U. Tse, G. P. Caplovitz, and P.-J. Hsieh."

Similar presentations


Ads by Google