CNBH, PDN, University of Cambridge Roy Patterson Centre for the Neural Basis of Hearing Department of Physiology, Development and Neuroscience University of Cambridge Part II: Lent Term 2014: ( 4 of 4) Central Auditory Processing Lecture slides, sounds and background papers on Lecture slides on CamTools

CNBH, PDN, University of Cambridge Act III: The processing of communication sounds in the early stages of the auditory system, and hypotheses about the representation of communication sounds in later stages of the auditory pathway Act I: Communication sounds and the information in these sounds: message, S s and S f Overview: Act IV: Brain imaging evidence concerning the representation of communication sounds in auditory cortex Act II: Behavioural evidence for the role of these different forms of information in the perception of communication sounds

CNBH, PDN, University of Cambridge We have discussed a model of auditory perception that describes how sounds might be processed and represented at a sequence of stages in auditory system. So, this Auditory Image Model (AIM) predicts that we should find a hierarchy of processing modules in the auditory pathway. One representation is intended to simulate your initial Auditory Image of the incoming sound and it is central to the model. All of the stages are mandatory and the order is crucial. Sensations like pitch and loudness are summary statistics calculated from the auditory image after it has been constructed. Speech and music perception are thought to be based on the patterns that arise in the auditory image.

CNBH, PDN, University of Cambridge LL The correspondence between the perceptual model and the anatomy suggests that (1) AIM could be useful when designing brain imaging studies of the auditory system and (2) the brain imaging data could help us locate the auditory image. Overview 2 There is a sequence of neural centres in the auditory pathway. It looks like it could be a processing hierarchy. The centres are separated by distances that are large relative the resolution of functional brain imaging (fMRI).

CNBH, PDN, University of Cambridge Anatomy of the Auditory Pathway: 1 Basilar membrane motion in the cochlea }

CNBH, PDN, University of Cambridge } Neural activity pattern in the cochlear nucleus

CNBH, PDN, University of Cambridge Strobed temporal integration in the inferior colliculus? }

CNBH, PDN, University of Cambridge The initial auditory image in the MGB?? }

CNBH, PDN, University of Cambridge Auditory Image The normalized auditory image in primary auditory cortex???

CNBH, PDN, University of Cambridge So the brain imaging research focuses on finding evidence that the neural centres in the auditory pathway are involved in source segregation and normalization, and that the segregation and pulse- rate normalization come before the resonance scale normalization. Moreover, speech-specific analysis and music-specific analysis should occur in neural centres beyond, but not too far from, those associated with segregation of pulse- resonance sounds from noise and their normalization. The Auditory Image Model describes how the auditory system separates pulse-resonance sounds from noise, and how it normalizes and segregates the information about the pulse-rate (S s ) and the resonance scale (S f ) from the message.

CNBH, PDN, University of Cambridge Find two sounds that differ only in the perceptual property of interest (like pitch). Brain Imaging with Simple Contrasts Scan the brain while people are listening, first to one sound and then to the other sound. Compare the brain activity produced by the two sounds looking for places where one sound produces more activity than the other.

CNBH, PDN, University of Cambridge Brain imaging with Regular Interval Noise Copy a sample of random noise; delay it by N ms; add it to the original noise. The process emphasises time intervals of N ms in the sound and we hear a weak tone in the noise. As you repeat the delay and add process, the relative strength of the tonal component of the sound increases. RIN makes a good imaging stimulus because the sounds have similar distributions of energy over time and frequency. In the experiment the RIN had 8 iterations of the delay and add process.

CNBH, PDN, University of Cambridge Auditory Image B G J F A D Neural activity patterns of Noise and RIN Noise RIN

CNBH, PDN, University of Cambridge C H B G K E Initial auditory images of noise and RIN

CNBH, PDN, University of Cambridge Difference in sensitivity to stimulus: positive negative haemodynamic response to test stimulus haemodynamic response to scanner noise sparse imaging continuous imaging [original figure by D. Hall, IHR, Nottingham] Continuous Imaging vs Sparse Imaging

CNBH, PDN, University of Cambridge Imaging pitch and melody in the brain The sound has no pitch (a noise), a fixed pitch (boring melody) or changing pitch (proper melody). On a given scan, the listener is presented a sound with a pulsing rhythm. Asked to listen for pattern in the sound, but no response is required.

CNBH, PDN, University of Cambridge T value Axial view at level CN CN Parasagital view showing CN/ IC IC CN Coronal view showing MGB + superior temporal lobe MGB AC Coronal view showing IC + superior temporal lobe IC AC CN Sound minus silence contrast Griffiths et al. Nature Neuroscience (2001)

CNBH, PDN, University of Cambridge Left Hemisphere saggital axial Right Hemisphere structural coronal x Figure 2 noise-silence fixed-noise diatonic-fixed random-fixed 34.4° Group Analysis Patterson, Uppenkamp, Johnsrude and Griffiths (2002)

CNBH, PDN, University of Cambridge saggital axial structural coronal noise-silence fixed-noise tonic-fixed random-fixed 34.4° x Group analysis Patterson, Uppenkamp, Johnsrude and Griffiths (2002)

CNBH, PDN, University of Cambridge regular irregular Neural Activity PatternAuditory Image equal energy click trains Gutschalk, Patterson, Scherg, Uppenkamp, and Rupp, (2002) strong pitch no pitch

CNBH, PDN, University of Cambridge Effects of regularity and intensity in MEG anterior source: HGposterior source: PT Gutschalk, Patterson, Scherg, Uppenkamp, and Rupp, (2002) effect of level in posterior source effect of regularity in anterior source

CNBH, PDN, University of Cambridge auditory cortexprimary auditory cortexall soundstonal sounds fixed pitch lively pitch loudness Conjecture Conjecture Conjecture Proposed functional organisation of auditory cortex

CNBH, PDN, University of Cambridge Where does the auditory system segregate the information associated with S s, S f and the message?

CNBH, PDN, University of Cambridge pulse ringing A damped sinusoid (12-ms period)

CNBH, PDN, University of Cambridge 1000 Hz 100 Hz 6000 Hz pulse ringing Auditory image of a damped sinusoid

CNBH, PDN, University of Cambridge regular irregular onset timing irregular regular formant frequencies Stimuli for Phonology Study

CNBH, PDN, University of Cambridge Comparison of speech and music regions z = 4mm y = -24mm z = -5mm mpmr-silence nvdvpv-mpmr mpmr-nvdvpv y = -17mm z = 4mm z = -5mm noise-silence fixed-noise lively-fixed

CNBH, PDN, University of Cambridge Left Hemisphere saggital axial Right Hemisphere structural coronal noise-silence fixed-noise tonic-fixed random-fixed 34.4° x Group analysis pitch phonology vtl AudIm

CNBH, PDN, University of Cambridge auditory cortexprimary auditory cortexall soundstonal sounds fixed pitch lively pitch loudness Conjecture Conjecture Conjecture Proposed functional organisation of auditory cortex receptive phonology

CNBH, PDN, University of Cambridge Act III: the processing of communication sounds in the auditory system (signal processing) Act I: the information in communication sounds (animal calls, speech, musical notes) Done! Act IV: the processing of communication sounds (anatomy, physiology, brain imaging) Act II: the perception of communication sounds (the robustness of perception)

CNBH, PDN, University of Cambridge End of Act IV Thank you Gutschalk, A., Patterson, R.D., Rupp, A., Uppenkamp, S. and Scherg, M. (2002). Sustained magnetic fields reveal separate sites for sound level and temporal regularity in human auditory cortex. NeuroImage Kriegstein, K. Von, Smith, D. R. R., Patterson, R. D., Kiebel, S. J. and Griffiths, T. D. (2010). “How the human brain recognizes speech in the context of changing speakers,”J. Neuroscience 30(2) 629–638. Patterson, R.D., Uppenkamp, S., Johnsrude, I. and Griffiths, T. D. (2002). The processing of temporal pitch and melody information in auditory cortex. Neuron

CNBH, PDN, University of Cambridge Roy Patterson, David Smith, Tim Ives, Ralph van Dinther Centre for the Neural Basis of Hearing, Physiology Department, University of Cambridge Cast list MEG in Heidelberg: Andre Rupp, Alexander Gutschalk, Stefan Uppenkamp, Michael Scherg fMRI in Cambridge: Ingrid Johnsrude, Dennis Norris, Matt Davis, Alexis Hervais-Adelman, William Marslen-Wilson MRC Cognition and Brain Sciences Unit, 15 Chaucer Road, Cambridge MEG in Muenster: Katrin Krumbholz, Annemarie Preisler, Bernd Lutkenhoner

CNBH, PDN, University of Cambridge Vowels in voiced and whispered speech A) Anatomy /a/ - Long vocal tract GPR 120 Hz B) Glottal folds C) Vocal tract Speech-related Speaker-related /u/ - Long vocal tract Whispered Voiced GPR 200 Hz Amplitude (dB) Frequency (Hz) Amplitude (dB) Frequency (Hz) VTL – vocal tract length Filtering by vocal tractGlottal fold parameters GPR 0 Hz D) Speech/Speaker overlap GPR- glottal pulse rate GPR 120 Hz, VTL 14cm, /a/ GPR 200 Hz, VTL 14cm, /a/ GPR 0 Hz, VTL 14cm, /a/ Whispered Voiced GPR 120 Hz GPR 200 Hz GPR 0 Hz /a/ - Shorter vocal tract GPR 120 Hz, VTL 14cm, /u/ GPR 120 Hz, VTL 14cm, /a/ GPR 120 Hz, VTL 10cm, /a/ glottal folds vocal tract Kriegstein, Smith, Patterson, Kiebel, and Griffiths, T. D., J. Neuroscience 30(2), (2010) Voiced

CNBH, PDN, University of Cambridge Activation of voiced vs whispered speech voiced > whispered whispered > voiced GPR varies > VTL varies signal change (%) voiced > silence whispered > silence TE1.2 TE1.1 x=+51 y=-2 L ***

CNBH, PDN, University of Cambridge saggital axial structural coronal noise-silence fixed-noise tonic-fixed random-fixed 34.4° x Group analysis Patterson, Uppenkamp, Johnsrude and Griffiths (2002) Conjecture Conjecture Conjecture

CNBH, PDN, University of Cambridge Vocal Tract Length (red) vs Speech Recognition (green) VTL varies > VTL fixed Speech task > control task x=-58 L y=-37

CNBH, PDN, University of Cambridge saggital axial structural coronal noise-silence fixed-noise tonic-fixed random-fixed 34.4° x Group analysis Patterson, Uppenkamp, Johnsrude and Griffiths (2002) Conjecture Conjecture Conjecture

CNBH, PDN, University of Cambridge Auditory Neuroscience – Making Sense of Sound Jan SchnuppJan Schnupp, Eli Nelken, and Andy King,Eli NelkenAndy King published at MIT Press