Control of prosodic features under perturbation in collaboration with Frank Guenther Dept. of Cognitive and Neural Systems, BU Carrie Niziolek [carrien] 14 sept 2005

Introduction Speech prosody: patterns of intonation and vocal stress In English, conveys non-lexical information, such as emphasis. The pattern of F0, duration, and intensity affects interpretation of phrases. How are prosodic cues controlled? What are the neural bases of speech segmentation?

Purpose of study: To understand feedback-based control of phrase-level prosody. Prosodic features (F0, intensity, duration) integrated? controlled independently?

Motivation Prosody conveys differences in both linguistic and affective state Not subservient to speech segments, but the “scaffolding” that holds different levels of phonetic description together

F0 as a stress indicator Observation: possible to characterize stressed syllables in terms of prosodic features Longer duration and greater intensity than unstressed syllables (Somewhat) predictable pitch contour “BOB caught a dog” t F0 threshold

F0 as a stress indicator BOB caught a dogBob CAUGHT a dog F0 ≈ 160 Hz

Methodology “BOB caught a dog” t F0 “Bob caught a dog” t F0 Flatten curve by shifting down all F0s above the threshold value, making the syllable sound less stressed

Experimental Question Do subjects respond by compensating for such a perturbation, increasing the F0 on the syllable they want to stress? Are other compensations also evident? Determine the degree to which prosodic aspects of speech are controlled in an integrated or independent manner.

Who caught a dog? BOB caught a dog. What did Bill do to a kid? Bill BIT a kid.

Method Present modulated feedback in real time Track and shift pitch Compare subject’s output F0 in baseline and perturbed conditions baselineramp full-pert post-pert shift amount trial #

Normal and pitch-shifted speech “Bill BIT a kid”

Results: pitch Averaging across all syllable positions, peak F0 is higher during perturbation than during the baseline condition (before and after perturbation). Slight compensation. Separating by stress position suggests that the effect may be larger for second syllable stress.

Results: mean-energy intensity Average mean-energy intensity is slightly higher during full- pert condition than during baseline. Significant? (This could be a function of F0, not a separate phenomenon.)

Future work Subjects needed! Continued analysis of F0 and intensity data Brain imaging Determine what structures and neural circuits are responsible for this prosodic control Model simulations After incorporating mechanisms for prosodic control into the model, compare DIVA with human psychophysical data

Summary What are the mechanisms responsible for the control of prosody? Auditory perturbation study: prosody manipulated and presented as feedback Some degree of compensation in pitch occurs during the perturbation. No evidence for any adaptation effects.

Thanks Frank Guenther, BU Kevin Reilly, BU Rupal Patel, Northeastern 1. Streeter et al. (1983). Acoustic and perceptual indicators of emotional stress. J. Ac. Soc. Am. 73, Pierrehumbert, J.B. (1980). The Phonology and Phonetics of English Intonation. MIT PhD dissertation, Cambridge, MA. References