1. Speech recognition with amplitude and frequency modulations: Implications for cochlear implant design
Fan-Gang Zeng
Kaibao Nie
Ginger Stickney
Ying-Yee Kong
Ashish Bhargave
Hongbin Chen
Michael Vongphoe
Janice Chang
What’re AM and FM?
What are their perceptual roles?
Where to find it?
Implications?

2. What is fine structure? Rosen’s definition: Hilbert’s definition:
Envelope (5-50 Hz)
Periodicity ( Hz)
Fine structure (500-10,000 Hz)
Hilbert’s definition:
Temporal envelope
Fine structure
Original AM
Fine Structure FM

3. Little math  Flanagan (1980) “Parametric coding of speech spectra” .
Discard absolute phase:
Discard relative phase (i.e., frequency modulation): . .

4. Zeng, Nie, Stickney et al. PNAS (2005)
Implementation
Combo of Dudley’s vocoder and Flanagan’s phase vocoder
Zeng, Nie, Stickney et al. PNAS (2005)

5. Spectra: What does FM encode?
Zeng, Nie, Stickney et al.
PNAS (2005)

6. Sentence, speaker, and tone recognition
Combo:
Target:
Masker:
Zeng, Nie, Stickney et al. PNAS (2005)

7. Comparison with previous studies
Shannon et al. 1995
Dorman et al. 1997
Zeng et al. 2005
Zeng, Nie, Stickney et al. PNAS (2005)

8. Spectral resolution and noise type
30-dB SRT
Zeng, Nie, Stickney et al.
PNAS (2005)

9. Speech recognition in combined hearing
HA+CI CI HA
10-dB SRT
Kong, Stickney, and Zeng JASA (2005)

10. FM detection in CIs: Results
Frequency
Time
Chen and Zeng JASA (2004)

11. Summary Using FM to improve auditory performance:
Speech cues are not redundant: FM complements AM in speech perception
FM is important for speech recognition with competing voice as maskers
FM is important for music and tonal language perception
FM is a slow version of fine structure that can be perceived and used to improve cochlear implant performance

12. Acknowledgements NIH - NIDCD Chinese NSF Advanced Bionics Corp
Cochlear Corp
Medel
Peter Assmann
Ann Bradlow
Keli Cao and CG Wei
Larry Feth
Ruth Litovsky
Jones Ackland