1. Electro-haptic stimulation
Carl Verschuur and Mark Fletcher
National Cochlear Implant User Association annual meeting
8th June 2019

2. Haptics
Any form of interaction involving the sensation of touch (tactile sensation), including
Communication between people and/or animals using touch
Perception and recognition of objects in the world through touch
Limited amount of information they can transfer

3. Haptic technology Computer games Mobile phones Virtual reality
Surgical training
Robotics

4. Tactile aids- an earlier application of haptics to hearing impairment

5. Vs

6. Tactile aids were actually quite impressive-but well short of cochlear implants
They could tell the difference (100% correct) between “dob” and “bot”, “so” and “show”, “let” and “net”.
Brooks & Frost (1983) Evaluation of a tactile vocoder for word recognition. J. Acoust. Soc. Am.

8. Cochlear implant limitations
Speech-in-noise
Spatial hearing (particularly if only implanted in one ear) e.g. locating where a sound comes from
Music enjoyment
Listening effort
Limited amount of information they can transfer

9. You can use the sense of touch for complex perception
Frequency range is from 10 Hz to 1000 Hz
Most sensitive around 250 Hz
You can tell differences in frequency of just a few Hz
Dynamic range (weakest sensation you can detect to largest you can tolerate) is around 55 decibel
Some of the above compare favourably with cochlear implant hearing

10. Can we use haptics/touch to enhance cochlear implant hearing?+

11. Electro-haptic stimulation
Using haptic stimulation (touch) in addition to the cochlear implant itself (electrical hearing) to enhance the listening performance and experience of the cochlear implant user
Limited amount of information they can transfer

12. Our work so far…
Speech-in-noise
Spatial hearing

13. Our work so far…
Speech-in-noise
Spatial hearing

14. Aims
Boost CI performance in noise with additional haptic stimulation (electrohaptic)
“Real time”
Can get useful info from real world, e.g. noisy multi-talker scenario
Include some manageable auditory training

15. Recent work 10 cochlear implant users
The variations in level over time from a noise+ speech signal were used to stimulate via haptics
Wrist stimulation
Before and after 20 mins exposure to auditory training material (different speaker and material from testing)
Deception – “Audio enhancement”
Fletcher et al (2019, pre print). Electro-haptic hearing: Speech-in-noise performance in cochlear implant users is enhanced by tactile stimulation of the wrists.

16. Recent work No effect before training
8 % average benefit after training
2 participants got more than 20 % benefit
Nobody got worse
Fletcher et al (2019, pre print). Electro-haptic hearing: Speech-in-noise performance in cochlear implant users is enhanced by tactile stimulation of the wrists.

17. Our work so far…
Speech-in-noise
Spatial hearing

18. Localisation testing

19. Localisation using touch
Location converted to differences in level of vibration to two wrists
12 CI users, all implanted in one ear
Looking straight ahead
Before and after 40 mins of training
Tactile stimulus: stereo speech envelope ( ,000 Hz)
Conditions: CI only, electro-haptic, and haptic only

20. Localisation using touch
Haptic OR electro-haptic was much better than electrical (CI listening only)
Training improved performance
Chance performance

21. Electro-haptic localisation accuracy compared to other groups
Hearing preservation – single CI but bilateral low-frequency hearing
Dorman (2016) Sound Source Localization by Normal-Hearing Listeners, Hearing-Impaired Listeners and Cochlear Implant Listeners. Audiol Neurootol.

22. Electro-haptic localisation accuracy compared to other groups
Audio only
Hearing preservation – single CI but bilateral low-frequency hearing
Dorman (2016) Sound Source Localization by Normal-Hearing Listeners, Hearing-Impaired Listeners and Cochlear Implant Listeners. Audiol Neurootol.

23. Electro-haptic localisation accuracy compared to other groups
Audio only
Audio-haptic:
Before training
Hearing preservation – single CI but bilateral low-frequency hearing
Dorman (2016) Sound Source Localization by Normal-Hearing Listeners, Hearing-Impaired Listeners and Cochlear Implant Listeners. Audiol Neurootol.

24. Electro-haptic localisation accuracy compared to other groups
Audio only
Audio-haptic:
Before training
After training
Hearing preservation – single CI but bilateral low-frequency hearing
Dorman (2016) Sound Source Localization by Normal-Hearing Listeners, Hearing-Impaired Listeners and Cochlear Implant Listeners. Audiol Neurootol.

25. Summary
Promising evidence electro-haptic stimulation can benefit speech-in-noise performance
Promising evidence electro-haptic stimulation can benefit sound localisation

26. Future work (when the project starts…)

27. Improved processing
Account for tactile threshold variation (e.g. compression)
Noise reduction
Spatial (localisation) cue enhancement

28. Remote training

29. Music enhancement
Jeremy Marozeau (DTU), Rúnar Unnþórsson​ (University of Iceland), and Soren Riis (Oticon Medical)

30. Portable device (in development)
8 channels
Real-time implementation

31. The team 15 minutes talk at BSA 2017 Mark Fletcher Uni. Southampton
Ama Hadeedi
Uni. Southampton
Carl Verschuur
UoS Auditory
Implant Service
Ben Lineton
Uni. Southampton
Tobias Goehring
Uni. Cambridge
Ahmed Bin Afif
Uni. Southampton
Sean Mills
Uni. Southampton
15 minutes talk at BSA 2017
Robyn Cunningham
Uni. Southampton
Soren Riis
Oticon
Ian Wiggins
Uni. Nottingham
Sam Perry
Uni. Southampton
Rúnar Unnþórsson​
Uni. Iceland
Jeremy Marozeau
Danish Technical
University

32. References https://www.electrohaptics.co.uk/
Craig (1972) Difference threshold for intensity of tactile stimuli. Perception and Psychophysics
Gescheider et al. (1996) Effects of stimulus duration on the amplitude difference limen for vibrotaction. JASA
Gescheider et al. (2003) Temporal gap detection in tactile channels. Somatosensory & Motor Research
LaMotte et al. (1975) Capacities of humans and monkeys to discriminate vibratory stimuli of different frequency and amplitude: A correlation between neural events and psychological measurements. J. Neurophysiology
Rothenberg et al. (1977) Vibrotactile frequency for encoding a speech parameter. J. Acoust. Soc. Am.
Von Bekesy (1955) Human Skin Perception of Traveling Waves Similar to Those on the Cochlea. J. Acoust. Soc. Am.
Richardson & Frost (1976) Tactile localization of sounds: Acuity, tracking moving sources, and selective attention. J. Acoust. Soc. Am.
Jousmäki & Hari (1998) Parchment-skin illusion: sound-biased touch. Current Biology
Gick & Derrick (2009) Aero-tactile integration in speech perception. Nature Letters
Bach-y-Rita et al. (1969) Vision substitution by tactile image projection. Nature. 221:
Bach-y-Rita (2004) Tactile Sensory Substitution Studies. Annals New York Academy of Sciences.
Fletcher et al (2019, pre print) Electro-haptic hearing: Speech-in-noise performance in cochlear implant users is enhanced by tactile stimulation of the wrists.
Brooks & Frost (1983) Evaluation of a tactile vocoder for word recognition. J. Acoust. Soc. Am.
Dorman (2016) Sound Source Localization by Normal-Hearing Listeners, Hearing-Impaired Listeners and Cochlear Implant Listeners. Audiol Neurootol.
O’Connell et al. (2017) Hearing Preservation Cochlear Implantation: a Review of Audiologic Benefits, Surgical Success Rates, and Variables That Impact Success. Curr Otorhinolaryngol Rep
Huang et al. (2017) Electro-Tactile Stimulation Enhances Cochlear Implant Speech Recognition in Noise. Sci Reports
Meredith & Allman (2015) Single-unit analysis of somatosensory processing in the core auditory cortex of hearing ferrets. European Journal of Neuroscience.
Selected references. Ordered as they appeared (so vaguely grouped)