1. Effects of Mispronunciation on Spoken Word Recognition in
Cochlear Implant Users & Typically Hearing Listeners
T. Ellis 1, K. Apfelbaum 2, H. L. Rigler 1, M. Seedorff 3, B. McMurray 1,4
1 Department of Psychological and Brain Sciences, University of Iowa 2 Department of Psychology, Ohio State University
3 Department of Biostatistics, University of Iowa 4 Delta Center
Background
Methods
Effect of Mispronunciation
Participants:
53 post-lingually deafened CI users (mean age 52.2y)
36 typically hearing age-matched controls (mean age 52.1y)
Subjects excluded if accuracy fell below 90% on correctly pronounced trials (N=3)
Auditory stimuli: Presented over loudspeakers in sound-attenuated booth
40 CVC English words
4 mispronounced tokens of each word
2 degrees of mispronunciation (single feature or multi-feature)
2 places of mispronunciation (onset or offset)
320 randomized trials
Procedure: 4 AFC, visual world paradigm
On screen: Target image + 3 phonetically unrelated images
Task: Hear a token, click on visual referent
Measure fixations to the target for trials on which correct referent is selected
Statistical Approach:
Looks to target as a function of time for correct trials only.
Bootstrapped Difference of Time Series (bdots): Estimates time window over which two time series are different (with appropriate correction for multiple comparisons). (Olseon et al., in press).
How much does mispronunciation impede word recognition?
Mispronunciation effect: difference in target fixations for correct and incorrect pronunciations.
Speech information arrives sequentially.
At early points in time, signal is
temporarily ambiguous.
Listeners immediately begin activating
many lexical candidates.
Visual World Paradigm (VWP)
Speech is highly variable within and between
talkers (e.g., rate, prosody, coarticulation, production errors).
Mispronunciations influence the lexical competition dynamic, and are a useful experimental tool for measuring how people respond to variable input.
bakery
Hear: “ba…
basic
barrier
barricade
bait
baby
Onset mispronunciation
----- CI
----- CI
Single feature
Multi-feature TH
Mispronunciation effect
200 ms 1 2 3 4 5
Trials
Time
% fixations
Target
Cohort
Rhyme
Unrelated
p =
p =
Time (msec)
Time (msec)
Onset MP
Offset MP
Target
Single feature
Multi-feature
beach
deach
heach
beaj
beag
fish
sish
nish
fiss
fid
jet
chet
thet
jek
jech
Offset mispronunciation
Mispronunciation effect
p =
----- CI
Mispron. Type
Time windows detected (msec)
Single feature onset
; > 1136
Multi-feature onset
; > 1496
Single feature offset
> 1804
Multi-feature offset
Not analyzed due to poor fit of data
Single feature TH
Swingley (2009) studied adults’ and children’s sensitivity to consonant mispronunciations in VWP.
Target fixations time-locked to location of mispronunciation
Time (msec)
Early in processing
Onset: CI users show less interference from mispronunciation.
Offset: CI users show similar interference effects.
Late in processing
Onset: CI users show more interference.
Offset: CI users show less interference
Discussion & Future Directions
People who use cochlear implants:
Can perceive speech accurately despite degraded input.
Develop compensatory listening strategies.
Altered weighting of acoustic cues (Moberly et al., 2014).
Modified time course of speech processing (Farris-Trimble et al,. 2014)
Adult CI users are slower to fixate the target; fewer overall looks.
CI users sustain looks to competitors longer than typically hearing listeners, suggesting a “wait and hedge bets” compensatory strategy.
Results
CI users are slower than TH listeners to activate correct target.
Both groups show similar time course of processing for onset and offset MP
 CI users show similar incremental processing to TH listeners.
CI users’ processing delay allows more information to accumulate before making strong commitments: more flexible strategy for dealing with uncertain input.
 They show less effect of mispronunciation and can recover better in some circumstances.
Correctly Pronounced Condition CI
---- TH
(msec)
Proportion fixations
Target
Cohort
Effect of CI use: Target fixations are delayed and reduced in CI users relative to TH listeners across conditions
Significant group differences between 252 and 3000 msec (yellow interval).
adult CI
adult control
Future analyses: within-group differences in time course of processing. Preliminary results suggest unexpected advantage for CI users without acoustic hearing, especially later during processing.
Time
p = 0.001
Question
References
Target fixations by location of mispronunciation
Do CI users’ compensatory listening strategies simultaneously help overcome surface variation (mispronunciations) in the speech signal?
Farris-Trimble, A., McMurray, B., Cigrand, N., & Tomblin, J. B. (2014). The process of spoken word recognition in the face of signal degradation. Journal of Experimental Psychology. Human Perception and Performance, 40(1), 308–27. doi: /a
Moberly, A. C., Lowenstein, J. H., Tarr, E., Caldwell-Tarr, A., Welling, D. B., Shahin, A. J., & Nittrouer, S. (2014). Do adults with cochlear implants rely on different acoustic cues for phoneme perception than adults with normal hearing? Journal of Speech, Language, and Hearing Research, 57(3), 566–582. doi: /2014
Oleson, J. J., Cavanaugh, J. E., McMurray, B., & Brown, G. (in press). Detecting time-specific differences between temporal nonlinear curves: Analyzing data from the visual world paradigm. Statistical Methods in Medical Research
Swingley, D. (2009). Onsets and codas in 1.5-year-olds’ word recognition. Journal of Memory and Language, 60, 252–269. doi: /j.jml
Proportion of Fixations
2015 ASHA Convention