2. More From Music music through a cochlear implant Dr Rachel van Besouw Hearing & Balance Centre, ISVR

3. What a cochlear implant (CI) is and who can have one What a CI sound processor does How CI hearing compares to normal hearing Why music is particularly challenging for CI users How to get More From Music… What will I learn in the next 25 minutes?

4. A CI is a surgically implanted device that electrically stimulates the auditory nerve fibres in the inner ear Image courtesy of AB What is a CI?

5. Ear canal Pinna Ear drum Ossicles (hammer, anvil & stirrup) Image courtesy of AB Cochlea Auditory nerve What is a CI?

6. Microphone(s) Sound processor Transmitter coil Receiver coil & stimulator Electrode array Image courtesy of AB What is a CI?

7. People with severe-to- profound hearing loss in both ears Due to abnormalities in the cochlea Image courtesy of AB Cochlea Who is it for?

8. Aging Noise Certain medicines Viral and bacterial infection (rubella, measles, meningitis) Ménière’s disease Genetic origin Premature birth Head injury Abnormal cochleae Micrographs: Keithley, E.M. in: Ryan, A.F. PNAS 2000;97:6939-6940 What causes this type of hearing loss?

9. What a cochlear implant (CI) is and who can have one What a CI sound processor does How CI hearing compares to normal hearing Why music is particularly challenging for CI users How to get More From Music… What will I learn in the next 25 minutes?

10. What does a CI sound processor do? microphone(s) sound processor transmitter coil

11. Sound processing stages Pre-emphasis (boosts frequencies important for speech) Automatic gain control (compresses the loudness range) Speech enhancement (reduces unwanted background noise)

12. Sound processing stages Splits the sound signal into frequency bands (the number of bands depends on the number of available electrodes)

13. Sound processing stages Generates the pulse sequences for each electrode

14. Sound processing stages Sets the amplitude range for each electrode (ensures that the amplitudes of the pulses are above threshold, but below the most comfortable loudness level)

15. Sound processing stages Turns the data into a radio frequency (RF) signal for transmission to the implant

16. What a cochlear implant (CI) is and who can have one What a CI sound processor does How CI hearing compares to normal hearing Why music is particularly challenging for CI users How to get More From Music… What will I learn in the next 25 minutes?

17. decibels 120 110 100 90 80 70 60 50 40 30 20 10 0 -10 Loudness & Dynamic Range loud soft How does a CI compare to normal hearing?

18. decibels 120 110 100 90 80 70 60 50 40 30 20 10 0 -10 Loudness & Dynamic Range loud soft ~20 dB range How does a CI compare to normal hearing?

19. Pitch How does a CI compare to normal hearing?

20. low pitch region in cochlea low pitch info high pitch info high pitch region in cochlea CI input CI output Pitch How does a CI compare to normal hearing?

21. What a cochlear implant (CI) is and who can have one What a CI sound processor does How CI hearing compares to normal hearing Why music is particularly challenging for CI users How to get More From Music… What will I learn in the next 25 minutes?

22. A visual analogy… Why is music particularly challenging? Image courtesy of Alan Olley

23. Over the Rainbow (Piano Only) Some where over the rainbow way up high… time amplitude Music…

24. Over the Rainbow (Piano Only) Some where over the rainbow way up high… frequency (high) (low) …is complex

25. Over the Rainbow (Piano Only) Over the Rainbow (Piano Only) Some where over the rainbow way up high… Filtered into >= 22 channels… frequency (high) (low)

26. … and turned into pulse sequences Some where ov - er the rain-bow way up high… Over the Rainbow (Piano Only)

27. The normal cochlea has ~16,000 hair cells and many more nerve fibres, which provide fine pitch cues at different locations Cochlear implants have between 12 and 22 electrodes and these can produce different pitches depending on their location (the further into the cochlea, the lower the pitch) Some electrodes may sound the same, resulting in even fewer place pitch cues Limited ‘place cues’ for pitch

28. The rate of the pulses delivered by the electrodes is fixed and is usually too high to provide timing or ‘temporal’ cues about pitch Temporal pitch cues from the envelope of the pulse sequences are there, but they are weak. Limited ‘temporal cues’ for pitch

29. More electrodes will not necessarily result in better pitch perception due to: –current spread (each electrode affects a large area) –the condition of the cochlea (auditory nerve fibres may have degenerated) What if we used more electrodes?

30. What a cochlear implant (CI) is and who can have one What a CI sound processor does How CI hearing compares to normal hearing Why music is particularly challenging for CI users How to get More From Music… What will I learn in the next 25 minutes?

31. Whilst it is true that pitch is important for melody, melody is not essential for music Rhythm is conveyed very well by the implant Loudness cues (although compressed) can convey the dynamics of music Gross changes in the frequency spectrum are also conveyed by the implant Music ≠ Melody

32. Music with clear changes in pitch Music with a clear rhythm Music will a simple arrangement Appropriate volume Quiet listening environment or direct input (TV/Hifi audio cable) Clues for interpreting music (visual aids, lyrics etc) Training & repetition… ? What can help?

33. Interactive music awareness programme 24 x 30 min structured sessions Interactive software applications enabling users to manipulate music to suit their implant Uses subtitled video tutorials and written instructions To be launched free online with a user forum ~Jan 2014 @ www.MoreFromMusic.org

34. More From Music Email us for further information… Rachel: rvb@isvr.soton.ac.uk (project info, research collaboration) Ben: b.oliver@soton.ac.uk (compositions, stems, software ) Sarah: s.m.hodkinson@soton.ac.uk (professionals’ training) Mary: mlf@isvr.soton.ac.uk (workshops for patients) or visit the music focus group website www.soton.ac.uk/mfg