2. Hearing + Perception, part 2 April 10, 2013

3. Hearing Aids et al. Generally speaking, a hearing aid is simply an amplifier. Old style: amplifies all frequencies New style: amplifies specific frequencies, based on a listener’s particular hearing capabilities. More recently, profoundly deaf listeners may regain some hearing through the use of a cochlear implant (CI). For listeners with nerve deafness. However, CIs can only transmit a degraded signal to the inner ear.

4. Cochlear Implants A Cochlear Implant artificially stimulates the nerves which are connected to the cochlea.

5. Nuts and Bolts The cochlear implant chain of events: 1.Microphone 2.Speech processor 3.Electrical stimulation What the CI user hears is entirely determined by the code in the speech processor Number of electrodes stimulating the cochlea ranges between 8 to 22.  poor frequency resolution Also: cochlear implants cannot stimulate the low frequency regions of the auditory nerve

6. Noise Vocoding The speech processor operates like a series of critical bands. It divides up the frequency scale into 8 (or 22) bands and stimulates each electrode according to the average intensity in each band. This results in what sounds (to us) like a highly degraded version of natural speech.

7. What CIs Sound Like Check out some nursery rhymes which have been processed through a CI simulator:

8. CI Perception One thing that is missing from vocoded speech is F0. …It only encodes spectral change. A former honors student, Aaron Byrnes, put together an experiment testing intonation perception in CI-simulated speech for his honors thesis. Tested: discrimination of questions vs. statements And identification of most prominent word in a sentence. 8 channels: 22 channels:

9. The Findings CI User: Excellent identification of the most prominent word. At chance (50%) when distinguishing between statements and questions. Normal-hearing listeners (hearing simulated speech): Good (90-95%) identification of the prominent word. Not too shabby (75%) at distinguishing statements and questions. Conclusion 1: F0 information doesn’t get through the CI. Conclusion 2: Noise-vocoded speech might not be a completely accurate CI simulation.

10. Mitigating Factors The amount of success with Cochlear Implants is highly variable. Works best for those who had hearing before they became deaf. The earlier a person receives an implant, the better they can function with it later in life. Works best for (in order): Environmental Sounds Speech Speaking on the telephone (bad) Music (really bad)

11. Critical Period? For congentially deaf users, the Cochlear Implant provides an unusual test of the “forbidden experiment”. The “critical period” is extremely early-- They perform best, the earlier they receive the implant (12 months old is the lower limit) Steady drop-off in performance thereafter Difficult to achieve natural levels of fluency in speech. Depends on how much they use the implant. Partially due to early sensory deprivation. Also due to degraded auditory signal.

12. Practical Considerations It is largely unknown how well anyone will perform with a cochlear implant before they receive it. Possible predictors: lipreading ability rapid cues for place are largely obscured by the noise vocoding process. fMRI scans of brain activity during presentation of auditory stimuli.

13. Infrared Implants? Some recent research has shown that cells in the inner ear can be activated through stimulation by infrared light. This may enable the eventual development of cochlear implants with very precise frequency and intensity tuning. Another research strategy is that of trying to regrow hair cells in the inner ear.

14. One Last Auditory Thought Frequency coding of sound is found all the way up in the auditory cortex. Also: some neurons only fire when sounds change.

15. And now for something completely different… Q: What’s a category? A classical answer: A category is defined by properties. All members of the category exhibit the same properties. No non-members of the category exhibit all of those properties.  The properties of any member of the category may be split into: Definitive properties Incidental properties

16. Classical Example A rectangle (in Euclidean geometry) may be defined as having the following properties: 1.Four-sided, two-dimensional figure (quadrilateral) 2.Four right angles This is a rectangle.

17. Classical Example Adding a third property gives the figure a different category classification: 1.Four-sided, two-dimensional figure (quadrilateral) 2.Four right angles 3. Four equally long sides This is a square.

18. Classical Example Altering other properties does not change the category classification: 1.Four-sided, two-dimensional figure (quadrilateral) 2.Four right angles 3. Four equally long sides This is still a square. A. Is red. definitive properties incidental property

19. Classical Linguistic Categories Formal phonology traditionally defined all possible speech sounds in terms of a limited number of properties, known as “distinctive features”. (Chomsky + Halle, 1968) [d] = [CORONAL, +voice, -continuant, -nasal, etc.] [n] = [CORONAL, +voice, -continuant, +nasal, etc.] … Similar approaches have been applied in syntactic analysis. (Chomsky, 1974) Adjectives = [+N, +V] Prepositions = [-N, -V]

20. Prototypes The psychological reality of classical categories was called into question by a series of studies conducted by Eleanor Rosch in the 1970s. Rosch claimed that categories were organized around privileged category members, known as prototypes. (instead of being defined by properties) Evidence for this theory initially came from linguistic tasks: 1.Semantic verification (Rosch, 1975) Is a robin a bird? Is a penguin a bird? 2.Category member naming.

21. Prototype Category Example: “Bird”

22. Exemplar Categories Cognitive psychologists in the late ‘70s (e.g., Medin & Schaffer, 1978) questioned the need for prototypes. Phenomena explained by prototype theory could be explained without recourse to a category prototype. The basic idea: Categories are defined by extension.  Neither prototypes nor properties are necessary. Categorization works by comparing new tokens to all exemplars in memory. Generalization happens on the fly.

23. A Category, Exemplar-style “square”