The Time Course of Processing Emotional Prosody: Behavioral and Electrophysiological Investigations Lauren Cornew, 1 Leslie J. Carver, 1 and Tracy Love 1,2 1 University of California, San Diego, 2 San Diego State University References Background Exp. 2 Preliminary Results Questions Acknowledgements This research was supported in part by an NSF Graduate Research Fellowship to the first author and NIH grants (DC00494 and DC03885) to the last author. Special thanks to Sarah Callahan, Hong Duong, Maxwell Moholy, Negin Khalifian, Hoa Nguyen, Uyen Pham, Sara Rust, Danny Sanchez, and Matt Walenski! Experiment 1 Discussion Exp. 1a Results 43 UCSD undergraduates (monolingual native English speakers, 27 female, mean age = 21) Stimuli: 48 Jabberwocky sentences (mean length = 2.7s), recorded by a Native English female speaker at a regular rate of speech (approx 5 syll/sec), were spoken in happy, angry, and neutral prosody Gating paradigm: 8 Sentences edited into successive clips, with duration increasing in 250ms increments; 5s of silence in between (Figure 1 below) 13 UCSD undergrads (right-handed monolingual native English speakers, mean age = 21; note that this experiment is part of a larger study that is still in progress.) Stimuli: Jabberwocky sentences from Experiment 1 (presented in their entirety; i.e., not spliced), alternating with tone sequences of ascending, descending, and constant pitch Target detection task: 6 blocks of trials; each emotion (happy, angry, neutral) served as a target in 3 blocks; order counterbalanced Participants instructed to press one button (right/left counterbalanced across subjects) following a Jabberwocky sentence conveying the target emotion, and the other button following any other type of audio clip Experiment 1a Methods Experiment 2 Methods Every clip judged to be happy, angry, or neutral Variables of interest: Percent correct Isolation point (length of the clip at which participants chose the correct emotion and did not subsequently change their decision) Mixed factorial design: all participants received all emotions, but no one participant heard any sentence more than once Participants randomly assigned to 1 of 3 groups: Is there enhanced processing of negative prosody? Or, is there an advantage for emotional (positive or negative) compared to non-emotional prosody? Two experiments, Behavioral (Exp. 1a & 1b) and Electrophysiological (Exp. 2), aim to explore these questions. Participants in Experiments 1a and 1b demonstrated a processing advantage for neutral prosody, which was identified more accurately and more rapidly than happy or angry prosody. This ‘neutral bias’ was not simply due to a greater ease in discriminating emotional from non-emotional prosody. Does the “neutral bias” observed in Experiments 1a and 1b reflect perception, attention, decision/response, or a language processing or acoustic parameter? Emotion interacts with cognition at many levels of processing, from basic perceptual 1 and attentional 2 stages to higher cognitive functions such as decision-making 3 and categorization. 4 Findings from studies of auditory emotion recognition are mixed and include reports of enhanced processing of negative 5, positive, 6 and neutral 7 prosody. 1 Phelps et al (2006). Psych. Science, 17, Carretié et al (2003). Psychophys., 40, Bechara, Damasio, & Damasio (2003). Ann. NY Academy Sciences, 985, Ito et al. (1998). J Personality and Social Psych., 75, Wambacq, Shea-Miller, & Abubakr (2004). NeuroReport, 15, Alter et al (2003). Speech Communication, 40, Schirmer & Kotz (2003). JCN, 15, Grosjean, F. (1980). Perception & Psychophys., 28, Greater accuracy for neutral prosody, F(2, 34) = 7.87, p =.001: Faster correct identification of neutral prosody, F(2, 34) = 24.67, p =.000: “The hessups ate pea chup after the sholt.” 250ms 500ms 750ms Entire Sentence... Figure 1. Schematic of a spliced sentence Figure 2 Figure 3 Experiment 1b Methods 24 UCSD undergrads (monolingual native English speakers, mean age = 22) Stimuli: same as in Experiment 1a Similar design as above: participants randomly assigned to 1 of 4 groups: Exp. 1b Results Greater accuracy for neutral prosody than happy prosody, t(11) = 3.52, p =.003, or angry prosody, t(10) = 4.48, p =.001. Faster correct identification of neutral prosody than happy t(11) = 2.23, p =.024, or angry prosody, t(10) = 3.06, p = – 0 window: significant effect of emotion maximal at Pz, P3/4, Oz, and O1/2 Amplitudes in neutral condition significantly more negative than in happy or angry conditions (all ps ≤.001), which did not differ from one another: Results (cont’d) 600 – 2000 window: significant effect of emotion maximal at FT7/8 Sustained positivity in neutral condition significantly greater than in happy or angry conditions (all ps ≤.001), which did not differ from one another: Figure 5 Figure 6 Experiment 2 Discussion General Discussion Behavioral and ERP data support a bias for processing neutral prosody over happy and angry prosody. Future directions include comparing task-relevant vs. task-irrelevant processing of emotional prosody. ERP data indicate a negative deflection in the 400ms preceding the isolation point, which likely reflects emotion recognition processes. Neutral prosody elicited a greater amplitude positivity between ms than emotional prosody. Figure 5 EEG amplified 100x with a SynAmps amplifier; electrodes referenced to linked mastoids Incorrect trials excluded, as well as time windows containing artifacts Participants whose data yielded >70% trial inclusion were retained for data analysis (n = 11). ERPs to Jabberwocky emotional prosody stimuli time-locked to the average isolation point for each emotion, as determined in Experiment 1a, in order to examine the recognition process (reference Fig. 3) Data analysis: Repeated measures ANOVAs; Greenhouse-Geisser adjustment applied to correct for sphericity violations Time windows of interest: -400 – 0ms, 600 – 2000ms EEG recording and ERP reduction: Quik-Cap (CompuMedics, Inc.) with 32 sintered Ag/AgCl electrodes arranged according to modified international system placement (Figure 5) Figure 4