Presentation is loading. Please wait.

Presentation is loading. Please wait.

Doğu Erdener Auditory-visual speech perception in young children: language and age- specific 6 December 2007.

Similar presentations


Presentation on theme: "Doğu Erdener Auditory-visual speech perception in young children: language and age- specific 6 December 2007."— Presentation transcript:

1

2 Doğu Erdener Auditory-visual speech perception in young children: language and age- specific factors @ 6 December 2007

3 Development of Speech Perception  First 6 months   phonetically-based, language-general  Beyond 1 st year  phonologically-based, language-specific  Consonants  7-11 months (Best, 1995)  Vowels  4-6 months (Kuhl et al., 1992)  Tones  6-9 months (Burnham & Mattock, 2006)  A working heuristic (Burnham et al., 2002)  Acoustic / Phonetic Stage (0-6 months)  Phonological Stage (6-12 months)  Semantic Stage (12-24 months)  Orthographic Stage (6-8 years)

4  Amodal (e.g.,Schwartz, et al., 1997)  Phonetic (e.g.,Burnham & Dodd, 2004)  Phonological / Postcategorical (e.g.,Massaro, 1998)  none/some/all of the above? Pre-categorical Auditory-Visual Speech Perception (AVSP): Integration  in early infancy (till ~6 months) is phonetically based  beyond first year of life  phonological Recall that auditory-only speech perception …. AVSP integration research: We need differential and ontogentic data! AVSP integration research: We need differential and ontogentic data!

5 AVSP: Cross-Language (Differential) Studies  Cross-language phonemic differences  e.g., Werker et al. (1992)  Visual influence from /th/ in the McGurk stimulus [A]-/ba/ + [V]-/tha/ : anglophones > francophones more experienced francophones > less experienced francophones  Amount of visual speech influence across langauges:  American > Japanese (Sekiyama & Tohkura, 1993)  Japanese > Mandarin (Sekiyama, 1997)  Sekiyama studies  Possible reasons:  Cross-cultural studies: Less eye/face contact in some cultures + Japanese Ss tend to look at the eye area whereas American Ss on the mouth area (Yuki et al., 2007).  Linguistic 1: Tonal (e.g. Cantonese) and pitch-accent (e.g., Japanese) information are not visually salient.  Linguistic 2: Fewer visually distinct consonant clusters and vowels in Japanese than in English.

6 Development of Auditory-Visual Speech Perception  Infancy  Infants match auditory and visual speech information (Burnham & Dodd, 2004, Rosenblum et al., 1997)  Childhood  McGurk & MacDonald (1976)  3 to 8 years + adults  Massaro et al. (1986)  4-6 years + adults  Adulthood  AV integration  adults > children (Sekiyama & Burnham, in press; Massaro et al.,1986)  Late adulthood  older Ss > younger Ss on vısual speech information when auditory sensitivity is diminished (Cienkowski & Carney, 2002).

7 Focus: Developmental Data from Infants and Children  Why is there an age-related increase?  Infancy and childhood are marked with several novel language-relevant and language-specific challenges, such as  phonetic  phonological  semantic  vocabulary  reading

8  Japanese and Australian English Speakers  Three child (6-, 8-, and 11-year-olds) groups and a control group of adults.  Method  McGurk stimuli  DV: Visual Speech Index (VSI) Score based on the proportion of auditory-based responses  Identification Task: /ba/, /da/, or /ga/? Sekiyama & Burnham (in press): an ontogenetic + differential study

9 AV+ minus A Augmentation A minus AV- Interference Visual Speech Index (VSI) [AV+] minus [AV-] 6-yos 8-yos 11-yos Adults

10  Sekiyama and Burnham (in press) found:  Developmental increase in visual speech influence with age… In English speakers but Not in Japanese speakers  So….. What factors modulate the development of auditory-visual speech perception (measured by VSI), especially between 6 and 8 years of age?

11 Experiment 1: Factors Investigated  Language Specific Speech Perception (LSSP)  Effect of phonological experience in native language: Native Speech Perception minus Non-native Speech Perception (N-NN) (Burnham, 2003).  Reading  Reading ability is positively related to LSSP in children (Burnham, 2003).  Evidence for reading-lipreading link: (de Gelder & Vroomen, 1998; Cavé et al., 2007)  Articulation ability is positively related to visual speech influence  in children (Desjardins et al., 1997), and  in adults with cerebral palsy (Siva et al., 1995)

12 Experiment 1 AVSP Language-specific Speech Perception (LSSP) Reading Articulation Regression Correlation

13 Experiment 1: Method  Language Specific Speech Perception (LSSP) Test: (Native minus Non-native speech perception  N-NN) 18 N [voiced vs. voiceless] + 18 NN [voiced vs. prevoiced]  Reading Test  WRAT-3 reading subtest: 15 letters + 42 words = 57 items  Articulation Test  Queensland Articulation Test (QAT)  Naming task: 64 pictures  Aus English consonants: initial, medial, & final positions  Auditory-Visual Speech Perception (AVSP) Test  McGurk paradigm: A & V combinations of /ba/, /da/, and /ga/  48 McGurk stimuli: (24 AV, 12 AO, and 12 VO), 3 consonants x 4 talkers (2 Eng & 2 Jap) x 2 congruence types x 2 background noise (Clear & noisy [+4 dB]) versions  Visual Speech Index (VSI)  AV (+) minus AV (-)

14 Experiment 1: Method  Participants n=96; Australian English 5-, 6-, 7-, and 8-year-olds n=48; Australian English adults N=144  Procedure  Individual testing (~60 minutes)  Task orders counterbalanced  Clear & Noise (+4dB SN Ratio)  Each stimulus presented twice

15 Exp. 1 Results: AVSP Test  VSI scores: Linear increase with age (p <.01) + Quadratic  between 5 and 6 years (p <.01).  VO scores: Linear increase with age (p <.01).  AO scores: Linear increase with age (p <.01).

16 Exp. 1 Results: LSSP, reading and articulation  LSSP scores: No age-based differences (p >.01) but native > non-native (p<.01)  Reading scores: Increase with age (p<.01)  Articulation scores: Increase with age (p children (p <.01).

17  Children Age + AO + VO + LSSP + Articulation + Reading  VSI  Adults Age + AO + VO + LSSP +Articulation + Reading  VSI Exp. 1 Results: Regression Analyses (DV = VSI Scores)

18 Exp. 1 Results: Correlation Coefficients

19 Experiment 1: Discussion  AVSP and lipreading (VO) increase with age  This increase is sharp at around 6 years of age, around the onset of reading instruction.  No age-based LSSP differences  native and non-native difference is largest at 6 years  Reading instruction  a new challenge  new sources of speech information used?  Regression Analyses  Children: lipreading + LSSP  VSI  Adults : AO  VSI  Correlation Analyses  Articulation is correlated with AO, lipreading (VO), and reading.

20 Exp. 1 Discussion  Experiment 2  Experiment 1:  LSSP & Lipreading  AVSP Schooling and/or new language-specific challenges seem to increase visual speech influence.  Towards Experiment 2 (3- & 4-year-olds): ?  Status of LSSP  AVSP link in pre-orthographic stages?  Are AVSP and basic Cognitive abilities related?  Are AVSP and Vocabulary Knowledge related?

21 Experiment 2: Method  Participants: N=48; 3-year-olds & 4-year-olds  AVSP – material, stimuli and procedure  AVSP: AX discrimination task  Three conditions AV(36), AO(12), VO(12)  AX-VSI (/1): based on different trials  LSSP – material, stimuli and procedure  18 Native [pa-p h a] & 18 Nonnative [ba-pa] trials.  AX-based category change paradigm.  Vocabulary Test – material, stimuli, and procedure  PPVT (Dunn & Dunn,1997)  age-based word sets of 12 items  on average 5-6 sets.  Cognitive Flexibility – material, stimuli and procedure  FIST (Jacques & Zelazo, 2001)  15 items (shapes, sizes & colours)

22 Exp. 2 Results: AVSP Test  VSI-AX scores: 4-year-olds > 3-year-olds (p <.01)  VO scores: 4-year-olds ≈ 3-year-olds (p =.07)  AO scores: 4-year-olds > 3-year-olds (p <.05)

23 Exp. 2 Results: LSSP, vocabulary and cognitive flexibility  LSSP scores:  4-year-olds > 3-year-olds (p <.01)  No stimulus language effect or contrast/age interaction (p >.05)  Vocabulary scores: 4-year-olds > 3-year-olds (p <.01)  FIST scores: 4-year-olds > 3-year-olds (p <.01)

24 Exp. 2 Results: Regression Analyses  VSI scores as dependent variable Age + AO + VO + LSSP + Vocabulary + FIST  VSI-AX

25 Exp. 2 Results: Correlation Coefficients

26 Experiment 2: Discussion  AVSP, LSSP, vocabulary and FIST improve with age  AVSP improves between 3 and 4 years.  Lipreading  p=.07 – same age-based trend, but small age difference?  Regression Analyses  AO & FIST  VSI-AX  similar to adults in Exp. 1 (AO  VSI)  Why does AO predict VSI-AX?  AVSP  product of AO speech perception  Three and 4 years: object-word pairing is a challenge, but not as much as reading – so no need for extra info.  AVSP is determined by inherent abilities? e.g., AO speech perception and cognitive function.

27 Regression Analyses: LSSP, age, language tests regressed onto AVSP 5,6,7, 8 years LSSP + VO + AO + Articulation + Reading  AVSP AVSP is related to LSSP and lipreading Adults LSSP + VO + AO +Articulation + Reading  AVSP AVSP is related only to auditory speech perception 3, 4 years LSSP + VO + AO+ Cognitive + Vocab.  AVSP AVSP is related only to auditory speech perception

28 General Discussion  Visual speech influence develops with age  AV: e.g., Massaro et al. (1986)  VO: e.g., Desjardins et al. (1997) (though p=.07 in Exp.2)  AVSP follows a phonetic  phonological trend  AVSP is amodal and phonetic (Burnham & Dodd, 2004; Rosenblum et al., 1997)  AVSP is phonological (Massaro, 1998)  But more importantly, AVSP seems to occur at all levels of language processing.

29  AVSP- Articulation Link  Vocabulary and articulation  challenges in early childhood  Tests of AVSP, LSSP, and articulation for children (12-24 months), with and without speech disorder (Dodd et al.,in press).  AVSP-Reading Link  Burnham (2003): reading  LSSP  LSSP  AVSP  So, integrative ability  phonemic/phonological processing  This may be the case for English but not for other languages?  Around school age, language-relevant challenges emerge  Reading  phoneme-grapheme mapping  LSSP intensifies (orthographic stage: Burnham et al., 2002)  Children use all speech information available Hyperarticulated speech styles (Lees & Burnham, 2005). e.g., teacherese (Håkansson, 1987). Cognitive skills activated + maturation General Discussion

30 Past  Current  Future AVSP and reading link (Cave et al., 2007; de Gelder & Vroomen, 1998) Burnham (2003) Articulation-lipreading link (Desjardins et al.,1997)

31 AVSP (integrative module) Auditory-only speech perception Lipreading Speech Production (articulation) Reading

32 Some questions awaiting answer  Speech Production-Perception Link:  the perception-production link: a developmental perspective; e.g., testing children with speech disorders (Dodd et al., in press).  Visual Speech and Reading Link: Research on the link between study  strong correlations between visual aspects of speech and reading  This study  correlations between reading and AO, lipreading and articulation.  Cavé et al. (2007); de Gelder & Vroomen (1998).  Cross-language reading x AVSP studies.

33 Acknowledging……. Prof. Denis Burnham Prof. Barbara Dodd, Asc.Prof. Chris Davis, Prof.Kaoru Sekiyama MARCS Auditory Laboratories & MARCS Baby Lab. College of Arts, University of Western Sydney Australian Postgraduate Award Aprica Foundation, Osaka, Japan Participating schools, teachers, parents and kids in Sydney

34


Download ppt "Doğu Erdener Auditory-visual speech perception in young children: language and age- specific 6 December 2007."

Similar presentations


Ads by Google