Presentation is loading. Please wait.

Presentation is loading. Please wait.

Page 1 Monkey See, Monkey Do, Monkey… Talk? by Helen Zou July 23, 2010 Monkey See, Monkey Do, Monkey… Talk? by Helen Zou July 23, 2010.

Similar presentations


Presentation on theme: "Page 1 Monkey See, Monkey Do, Monkey… Talk? by Helen Zou July 23, 2010 Monkey See, Monkey Do, Monkey… Talk? by Helen Zou July 23, 2010."— Presentation transcript:

1 Page 1 Monkey See, Monkey Do, Monkey… Talk? by Helen Zou July 23, 2010 Monkey See, Monkey Do, Monkey… Talk? by Helen Zou July 23, 2010

2 Page 2 Using Vocalization Features to Identify Ethanol Intoxication in Rhesus Macaques by Helen Zou July 23, 2010 Using Vocalization Features to Identify Ethanol Intoxication in Rhesus Macaques by Helen Zou July 23, 2010

3 Page 3 Overview Introduction Background –Rhesus Macaques –Speech processing Literature review –Previous findings in humans –Macaque vocalizations Experiment procedure Data analysis –Segmentation and clustering –Extracting features Results Acknowledgments Introduction Background –Rhesus Macaques –Speech processing Literature review –Previous findings in humans –Macaque vocalizations Experiment procedure Data analysis –Segmentation and clustering –Extracting features Results Acknowledgments

4 Page 4 Introduction Duke University – Class of 2013 Biomedical Engineering major and Neuroscience minor ed Dr. Grant because of her work with primates and neuroscience Vocalization project Worked at both ONPRC and OGI Not under any specific program, except… Had to give a presentation anyway Duke University – Class of 2013 Biomedical Engineering major and Neuroscience minor ed Dr. Grant because of her work with primates and neuroscience Vocalization project Worked at both ONPRC and OGI Not under any specific program, except… Had to give a presentation anyway

5 Page 5 Background – Rhesus Macaques Alcohol drug discrimination and self- administration Predictors of heavy drinking (dominance- related?) BEC (Blood Ethanol Concentration) Need simpler way to measure intoxication in social settings Why not look at speech? Alcohol drug discrimination and self- administration Predictors of heavy drinking (dominance- related?) BEC (Blood Ethanol Concentration) Need simpler way to measure intoxication in social settings Why not look at speech?

6 Page 6 Background – Speech Processing Voiced, unvoiced, and noise For monkeys, we focused on voiced (coos and screams) Potential features –Frequency and pitch –Shimmer (amplitude) and jitter (pitch) –Spectral entropy –Root mean square (energy) Voiced, unvoiced, and noise For monkeys, we focused on voiced (coos and screams) Potential features –Frequency and pitch –Shimmer (amplitude) and jitter (pitch) –Spectral entropy –Root mean square (energy)

7 Page 7 Sample Wave Form

8 Page 8 Sample Voiced Region

9 Page 9 Sample Noise

10 Page 10 Sample Background

11 Page 11 Overview Introduction Background –Rhesus Macaques –Speech processing Literature review –Previous findings in humans –Exxon Valdez case –Macaque vocalizations Introduction Background –Rhesus Macaques –Speech processing Literature review –Previous findings in humans –Exxon Valdez case –Macaque vocalizations

12 Page 12 Prior Studies – Klingholz Recognition of low-level alcohol intoxication from speech signal (1988) Approach recognition of intoxication as speaker identification task Measure laryngeal and articulatory features –Laryngeal - fundamental frequency and signal-to-noise (SNR) –Articulatory – formants (F1/F2 ratio) Major findings –Increased FO variation –Decreased SNR –Did not change F1/F2 Limitation: small sample size Much more accurate than human recognition Approach recognition of intoxication as speaker identification task Measure laryngeal and articulatory features –Laryngeal - fundamental frequency and signal-to-noise (SNR) –Articulatory – formants (F1/F2 ratio) Major findings –Increased FO variation –Decreased SNR –Did not change F1/F2 Limitation: small sample size Much more accurate than human recognition

13 Page 13 Prior Studies – Hollien Effects of ethanol intoxication on speech suprasegmentals (2001) Measured several different features –Nonfluency increase is best measure –F0 increases and utterance duration increases (moderate measure) –F0 variability slightly increases (poor measure) –Vocal intensity had no change 20% of subjects exhibited no consistent changes Unfortunately, disagrees with the previous findings Measured several different features –Nonfluency increase is best measure –F0 increases and utterance duration increases (moderate measure) –F0 variability slightly increases (poor measure) –Vocal intensity had no change 20% of subjects exhibited no consistent changes Unfortunately, disagrees with the previous findings

14 Page 14 Exxon Valdez Court Case Acoustic Analysis of Voice Recordings from the Exxon Valdez by J. Tanford et al (1992) Oil tanker crashed in Alaska in 1989 Captain of ship denied intoxication Analysis of speech found: –Misspoken words –Slurred pronunciations –Slower speaking rate –Lower pitch –Increased f0 variability Characteristics were consistent with intoxication Oil tanker crashed in Alaska in 1989 Captain of ship denied intoxication Analysis of speech found: –Misspoken words –Slurred pronunciations –Slower speaking rate –Lower pitch –Increased f0 variability Characteristics were consistent with intoxication

15 Page 15 Previous Study – Weerts Primate vocalizations during social separation and aggression: effects of alcohol and benzodiazepines (1996) Focused on testing the effect of different social situations –Social separation: EtOH reduced isolation peeps –Aggression: EtOH increased aggression peeps Social context determines effect of drugs (potential confounding variable?) Focused on testing the effect of different social situations –Social separation: EtOH reduced isolation peeps –Aggression: EtOH increased aggression peeps Social context determines effect of drugs (potential confounding variable?)

16 Page 16 Summary of Previous Work Experiments done on the effect of intoxication on human speech have inconsistent findings Very few studies actually done on macaque vocalizations Many uncontrolled variables (long-term voice effort, social context, etc.) Definitely some effect of ethanol intoxication on speech features Experiments done on the effect of intoxication on human speech have inconsistent findings Very few studies actually done on macaque vocalizations Many uncontrolled variables (long-term voice effort, social context, etc.) Definitely some effect of ethanol intoxication on speech features

17 Page 17 The Question Will the vocalizations of monkeys change when intoxicated versus when sober?

18 Page 18 Methods Put recorders on the monkeys Gavage with water or alcohol (alternating) Measure BECs in one hour Take off recorders Analyze data for various features Identify differences in vocalization Draw conclusions from data and voila! But in reality… Put recorders on the monkeys Gavage with water or alcohol (alternating) Measure BECs in one hour Take off recorders Analyze data for various features Identify differences in vocalization Draw conclusions from data and voila! But in reality…

19 Page 19 Problems 1.Exceeding recorder threshold 2.Not enough vocalizations 1.Exceeding recorder threshold 2.Not enough vocalizations 1.Attenuate with rubber and foam 2.Switch to more vocal monkey 1.Attenuate with rubber and foam 2.Switch to more vocal monkey Solutions

20 Page 20 Clementine Example Waveform

21 Page 21 Data Analysis? Recordings had vocalizations, noise, silence, other monkeys, etc. How would we isolate the monkey of interest? Recordings had vocalizations, noise, silence, other monkeys, etc. How would we isolate the monkey of interest?

22 Page 22 Sample Spectrum Noise Vocalizations

23 Page 23 Clementine Example Spectrum

24 Page 24 Data Analysis 1.Cut the wave file into smaller segments 2.Isolate vocalization parts of speech 3.Extract features for vocalization regions 4.Compare features for intoxicated versus sober speech 1.Cut the wave file into smaller segments 2.Isolate vocalization parts of speech 3.Extract features for vocalization regions 4.Compare features for intoxicated versus sober speech

25 Page 25 Segmentation/Clustering Robust Speaker Change Detection by J. Ajmera et al. (2003) Originally created for separating speakers in news broadcasts Find likely change points Segment data with overlapping frames Cluster similar segments (by speaker) Originally created for separating speakers in news broadcasts Find likely change points Segment data with overlapping frames Cluster similar segments (by speaker)

26 Page 26 Segmentation and Clustering

27 Page 27 Data Analysis 1.Cut the wave file into smaller segments 2.Isolate vocalization parts of speech 3.Extract features for vocalization regions 4.Compare features for intoxicated versus sober speech 1.Cut the wave file into smaller segments 2.Isolate vocalization parts of speech 3.Extract features for vocalization regions 4.Compare features for intoxicated versus sober speech

28 Page 28 Spectrum – Human vs. Monkey

29 Page 29 Results – Human vs. Monkey Bandwidth of formants in monkey vocalizations is larger than for humans Humans have more formants (5+), monkeys have much fewer (2-4) Distance between the formants for monkeys is much larger than between human formants Shape of formants is curved for screams and straight for coos Bandwidth of formants in monkey vocalizations is larger than for humans Humans have more formants (5+), monkeys have much fewer (2-4) Distance between the formants for monkeys is much larger than between human formants Shape of formants is curved for screams and straight for coos

30 Page 30 Spectrum – Human vs. Monkey HumanNoiseCooScream

31 Page 31 Results - F0 graphs

32 Page 32 Results – Alcohol vs. Water Graphed all of the features F0 as x- variable produced most significant results F0 tends to be higher during intoxication Graphed all of the features F0 as x- variable produced most significant results F0 tends to be higher during intoxication

33 Page 33 Results – Rms vs. f0

34 Page 34 Results Root mean square (energy) vs. fundamental frequency Control vocalizations have larger variation in energy Intoxication has higher f0 Root mean square (energy) vs. fundamental frequency Control vocalizations have larger variation in energy Intoxication has higher f0

35 Page 35 Results – Rms vs. Spec entropy

36 Page 36 Results Spectral entropy vs. f0 Control vocalizations have larger variation in spectral entropy Intoxication has higher f0 Spectral entropy vs. f0 Control vocalizations have larger variation in spectral entropy Intoxication has higher f0

37 Page 37 Results Alcohol increases fundamental frequency (agrees with Hollien study) Alcohol decreases variation in energy and spectral entropy Consistent with alcohol impairing muscle control of vocal cords Alcohol increases fundamental frequency (agrees with Hollien study) Alcohol decreases variation in energy and spectral entropy Consistent with alcohol impairing muscle control of vocal cords

38 Page 38 Limitations Very small sample size Limited number of vocalizations Lots of silence and noise in recordings BEC was low (between.017 and.044) Monkeys were separated – may have different results in social setting Only paired comparisons Very small sample size Limited number of vocalizations Lots of silence and noise in recordings BEC was low (between.017 and.044) Monkeys were separated – may have different results in social setting Only paired comparisons

39 Page 39 In the Future Further study correlations between different vocalization features and intoxication Use recordings to correlate with other factors (such as stress, dominance, etc.) Find ways to increase vocalizations Pair vocal recordings with visual tracking Measure ethanol intake using vocalizations in social settings Expand studies to other species Further study correlations between different vocalization features and intoxication Use recordings to correlate with other factors (such as stress, dominance, etc.) Find ways to increase vocalizations Pair vocal recordings with visual tracking Measure ethanol intake using vocalizations in social settings Expand studies to other species

40 Page 40 Conclusion Added to the studies done on macaque vocalizations Used computer algorithms to separate and analyze data Found that formants are a good way to separate human and monkey vocalizations Alcohol increases f0 and decreases variability of energy and spectral entropy Eventually use vocalizations to measure intoxication in macaques in social settings Added to the studies done on macaque vocalizations Used computer algorithms to separate and analyze data Found that formants are a good way to separate human and monkey vocalizations Alcohol increases f0 and decreases variability of energy and spectral entropy Eventually use vocalizations to measure intoxication in macaques in social settings

41 Page 41 Acknowledgments Dr. Kathy Grant Dr. Izhak Shafran The Grant Lab (Kevin Nusser, Andrew Rau, Jessica Shaw, and Cara Candell) Meysam Asgari OGI and ONPRC staff and coworkers Dr. Kathy Grant Dr. Izhak Shafran The Grant Lab (Kevin Nusser, Andrew Rau, Jessica Shaw, and Cara Candell) Meysam Asgari OGI and ONPRC staff and coworkers

42 Page 42 Questions?

43 Page 43 Prior Studies - Klingholz Approach recognition of intoxication as speaker identification task 11 human test subjects and 5 controls Read a text segment in German Measure laryngeal and articulatory features –Laryngeal - fundamental frequency and signal-to-noise (SNR) –Articulatory – formants (F1/F2 ratio) Intoxication results –Increased FO variation –Decreased SNR –Did not change F1/F2 Correlation between BAL and F0 Long-term voice effort has similar effect Much more accurate than human recognition Approach recognition of intoxication as speaker identification task 11 human test subjects and 5 controls Read a text segment in German Measure laryngeal and articulatory features –Laryngeal - fundamental frequency and signal-to-noise (SNR) –Articulatory – formants (F1/F2 ratio) Intoxication results –Increased FO variation –Decreased SNR –Did not change F1/F2 Correlation between BAL and F0 Long-term voice effort has similar effect Much more accurate than human recognition

44 Page 44 Prior Studies - Hollien Speech samples at four levels of intoxication 35 human subjects Results –Nonfluency increase is best measure –F0 increases and utterance duration increases (moderate measure) –F0 variability increases (poor measure) –Vocal intensity had no change 20% of subjects exhibited no consistent changes Speech samples at four levels of intoxication 35 human subjects Results –Nonfluency increase is best measure –F0 increases and utterance duration increases (moderate measure) –F0 variability increases (poor measure) –Vocal intensity had no change 20% of subjects exhibited no consistent changes

45 Page 45 Prior Studies - Weerts 33 squirrel monkeys in two different social situations Social separation: EtOH reduced isolation peeps Aggression: EtOH increased aggression peeps Social context determines effect of drugs 33 squirrel monkeys in two different social situations Social separation: EtOH reduced isolation peeps Aggression: EtOH increased aggression peeps Social context determines effect of drugs


Download ppt "Page 1 Monkey See, Monkey Do, Monkey… Talk? by Helen Zou July 23, 2010 Monkey See, Monkey Do, Monkey… Talk? by Helen Zou July 23, 2010."

Similar presentations


Ads by Google