1. S EPTEMBER 2008, S LOVENIA 2 ND B ALKAN V ISION S CIENCE M EETING Aspects of face perception: Aspects of face perception: gaze direction and emotional expression Dejan Todorović Dejan Todorović Laboratory of experimental psychology Department of psychology University of Belgrade, Serbia

2. 2 Information carried by faces general features: gender race age specific, particular features: identity current, changeable features: facial expression gaze direction

3. Current features 3 Usually depend on the configuration profile of facial and head muscles: facial expression gaze direction

4. Current features 4 may also depend on: placement of inner facial features (eyes, nose, mouth) within face outline head orientation and inclination examples simple schematic faces more realistic faces

5. schematic faces 5 gaze direction straight ahead note what happens when all inner facial features (eyes, nose, mouth) are shifted horizontally leftwards/rightwards

6. schematic faces 6 gaze direction straight aheadleftwardsrightwards perceived gaze direction:

7. schematic faces 7 'Necker faces': bi-stable gaze direction red guy looks leftwards, green guy looks at you looks leftwardslooks at you

8. 8EXPERIMENTS face eccentricity STIMULI iris eccentricity

9. Experiment 1: design stimuli: 21 portraits х 8 repetitions = 168 trials (random order) preceded by 21 practice trials subject task: 'does the portrait look at your face or not?' response measures: percent response type ('at my face', 'not at my face') reaction times 15 subjects extension of research by Maruyama & Endo (1984)

10. Experiment 1: results Effects of face eccentricity and iris eccentricity on percentages of ‘looks at me’ answers face eccentricity shift in one direction … … is compensated by iris eccentricity shift in opposite direction

11. Experiment 2: design subject task: 'what is the gaze direction of the portrait, in degrees?' response measure: judged angle of gaze direction help: protractor scheme below the computer screen stimuli: 21 portraits х 4 repetitions = 84 trials 15 subjects obtained data converted from 0 to 180 range to -90 to 90 range negative: leftwards positive: rightwards zero: ‘looks at me’

12. Experiment 2: results Effects of face eccentricity and iris eccentricity on judged angle of gaze direction judged angle of gaze direction increases with iris eccentricity judged angle increases/decreases with positive/negative face eccentricity average effect: 12 degrees

13. Experiment 2: multiple regression independent variables: iris eccentricity (IE), face eccentricity (FE) dependent variable: judged angle of gaze direction (GD) regression equation on means: GD = 1.35 IE + 0.63 FE + 2.29, R 2 = 0.985 general model for perceived gaze direction: GD = iw*IE + fw*FE (in concrete case: iw=1.35, fw=0.63) special case: portrait looks at me GD = iw*IE + fw*FE = 0 this is true in two cases: (1) IE = FE = 0 meaning: centered irises in symmetrical face (2) IE/FE = -fw/iw = const in concrete case: -0.63/1.35 = -0.47 meaning: particular counterbalance of iris and face eccentricity

14. 14 REALISTIC FACES irises centralgaze at observergaze leftwardsirises left gaze rightwardsirises right Emery’s (2000) rule for perception of gaze direction: not quite true... none of depicted portraits are looking in the indicated directions! Do these findings apply for realistic faces? many authors claim that only IE affects perceived gaze

15. 15 REALISTIC FACES irises rightwards gaze rightwards gaze at observer head frontal head leftwards

16. 16 REALISTIC FACES irises centered gaze at observergaze rightwards head frontalhead rightwards

17. 17 REALISTIC FACES irises leftwards gaze leftwardsgaze at observer head leftwards head rightwards orientation of head affects perceived gaze direction

18. 18 REALISTIC FACES 'Necker faces': bi-stable gaze direction

19. Experiment: synthetic faces 24% 8% -24% -8% task: does portrait look at me or not? -25%-15%-5%5%15%25% head orientation iris eccentricity

20. Results 24% 8% -24% -8% Effects of head orientation and iris eccentricity on percentages of ‘looks at me’ answers results are similar as for schematic faces

21. Conclusions perceived gaze direction is based not only on iris position but also on head orientation cues head orientation cues: e.g. face eccentricity indicates head orientation relative to observer iris position: indicates gaze direction with respect to head 'gazing at me' percept based on a counter-balance of head and iris cues

22. 22 facial expression neutral mood schematic faces note what happens when all inner facial features (eyes, nose, mouth) are shifted vertically bottomwards/topwards

23. 23 facial expression neutral moodpoor moodgood mood perceived facial expression: schematic faces

24. Experiment 1: schematic faces three factors: 1. features placement 2. mouth curvature 3. iris location PRESENTATION: 18 faces on sheet of paper, randomized placement TASK: rating scale rate expression on 7-step bipolar sad-happy scale SUBJECTS: N=83

25. 25 Experiment 1: Results Effects of feature placement and mouth curvature on facial expression rating both feature placement and mouth curvature affect rated mood no effect of gaze direction

26. Experiment 2: more ‘realistic’ faces two factors: 1. feature placement 2. mouth curvature PRESENTATION: 18 faces on sheet of paper, randomized placement (mixed with Exp. 3 figures) TASK: rating scale rate expression on 7-step bipolar sad-happy scale SUBJECTS: N=71

27. 27 Experiment 2: Results Effects of feature placement and mouth curvature on facial expression rating both feature placement and mouth curvature affect rated mood

28. Hypothesis 28 possible explanation of effect of vertical features shift on perceived facial expression: feature location is associated with head inclination head is low / high: projection of features is low/high head inclination is associated with mood head is low / high: mood is poor / good thus feature location is associated with mood features low features high poor moodgood mood head lowhead high

29. Experiment 3: head inclination two factors: 1. head inclination 2. mouth curvature PRESENTATION, TASK, SUBJECTS: as in Experiment 2 CONFOUNDING EFFECT: projected mouth curvature depends on head inclination

30. Experiment 3: Results Effects of head inclination and mouth curvature on facial expression rating head inclination affects rated mood for ‘sad’ faces no clear effect for ‘neutral’ and ‘happy’ faces result structure replicated in two additional experiments

31. Conclusions 31 perceived gaze direction and facial expression do not depend only on registration of action of facial muscles they also depend on facial feature placement and head orientation / inclination horizontal feature placement is a cue of head turn vertical feature placement may be a cue of head inclination, which is associated with mood this explanation only applies for sad faces no effect for neutral and happy faces for such faces, there may exist a ‘facial constancy’ effect (independence from head inclination)

32. 32 THANK YOU!

33. Experiment 1: results (reaction times) Effects of face eccentricity and iris eccentricity on reaction times to ‘looks at me’ answers more frequent answers are faster reactions to eccentric faces are slower

34. -27% -18% -9% 0% 9% 18% 27% Experiment 1: fitting percentages -27% -18% -9% 0% 9% 18% 27% peak = 9.6% spread = 11.45 R 2 =0.979 a: peak b: spread peak = - 1.5% spread = 12.76 R 2 =0.958 peak = -11.3% spread = 11.26 R 2 =0.991 fit model: normal curve

35. Experiment 2: fitting judged gaze angles almost identical slopes for the three face eccentricities +/- 20% face eccentricity adds/subtracts 12 degrees fit model: straight lines

36. Comments 36 criticism: what is the relevance of studies with schematic faces for the perception of gaze direction in real faces? answers: visual system does not have special mechanisms for detection of gaze direction in schematic faces schematic faces probably activate same face perception mechanisms as real faces shown in EEG and fMRI studies schematic faces are much simpler and more easily controlled as stimuli than real faces shift of facial features is probably a cue of head turn nevertheless: it is of interest to study real(istic) faces

37. 37 Wollaston (1824) REALISTIC FACES

38. Experiment stimuli: synthetic faces software: Poser 5 model: default male head two factors: 1. iris eccentricity: 6 levels 24%8%-24%-8% 2. head turn: 4 levels -25%-15%-5%5%15%25%

39. Experiment stimulus presentation 24 portraits х 7 repetitions = 168 trials (random order) preceded by 24 practice trials subjects: three groups of 11 subject tasks: group 1: head orientation task 'is the head oriented leftwards or rightwards' group 2: iris location task 'are the irises located leftwards or rightwards from center?' group 3: gaze direction task 'does the portrait look at your face or not?' response measure: percentage of responses

40. Experiment results: 1. head task 'head rightwards' responses 24% 8% -24% -8% threshold for head orientation: 5% head turn equivalent to 1.8 degrees head turn angle agrees with Wilson et al. (2000)

41. Experiment results: 2. iris task 'irises rightwards' responses 24% 8% -24% -8% threshold for iris eccentricity: 5.3% iris shift equivalent to 2 minutes of arc within upper range of previous data

42. 42

43. 43

44. 44

45. Experiment: conclusions possible algorithms of perception of gaze: 1. analytic separate encoding of head and iris cues combining cues to compute gaze direction 2. holistic effective stimulus is the whole facial pattern match pattern to stored template associated with gaze direction note: both algorithms configurational do not depend on local information only

46. Experiment results: 3. gaze task 'gazes at me' responses 24% 8% -24% -8%

47. 47 'Necker face' schematic faces

48. 48 PRESENTATION: on computer screen pairs of faces from Exp. 3 TASK: which of the two faces looks happier? STIMULI all possible pairings of 9 faces 36 pairs x 4 replications = 144 stim. randomized, balanced for side SUBJECTS: n=18 DEPENDENT VARIABLE: happiness index for each face percentage of times face chosen as the happier one in the pair Experiment 4 RESULTS Head inclination affects rated mood for ‘sad’ faces No clear effect for ‘neutral’ and ‘happy’ faces replicated in another experiment