2. The role of prosody in dialect synthesis and authentication Kyuchul Yoon Division of English Kyungnam University Spring 2008 Joint Conference of KSPS & KASS

3. 2 Goals 1.Synthesize Masan utterances from matching Seoul utterances by prosody cloning 2.Examine the role of prosody in the authentication of synthetic Masan utterances (Listening experiment)

4. 3 Background Differences among dialects –Segmental differences Fricative differences in the time domain (Lee, 2002) –Busan fricatives have shorter frication/aspiration intervals than for Seoul Fricative differences in the frequency domain (Kim et al., 2002) –The low cutoff frequency of Kyungsang fricatives was higher than for Cholla fricatives (> 1,000 Hz) –Non-segmental or prosodic differences Intonation or fundamental frequency (F0) contour difference Intensity contour difference Segment durational difference Voice quality difference

5. 4 Synthesis Simulating (by prosody cloning) Masan dialect from Seoul dialect The simulated Masan utterances will have –the speech segments of Seoul dialect –the prosody of Masan dialect F0 contour Intensity contour Segmental duration

6. 5 Evaluation Through a listening experiment Stimuli consist of –#1. Authentic, but synthetic, Masan utterance –#2. Seoul utterance with Masan segmental durations (D) –#3. Seoul utterance with Masan F0 contour (F) –#4. Seoul utterance with Masan intensity contour (I) –#5. Seoul utterance with Masan durations and F0 contour (D+F) –#6. Seoul utterance with Masan durations and intensity contour (D+I) –#7. Seoul utterance with Masan F0 contour and intensity contour (F+I) –#8. Seoul utterance with Masan durations, F0 contour and intensity contour (D+F+I) (1) 동대구에 볼 일이 없습니다. (2) 바다에 보물섬이 없다

7. 6 Prosody transfer (PSOLA algorithm) Three aspects of the prosody –Fundamental frequency (F0) contour –Intensity contour –Segmental durations Pitch-Synchronous OverLap and Add (PSOLA) algorithm (Mouline & Charpentier, 1990) –Implemented in Praat (Boersma, 2005) –Use of a script for semi-automatic segment-by-segment manipulation (Yoon, 2007)

8. 7 Prosody transfer (PSOLA algorithm) Procedures for full prosody transfer –Align segments btw/ Masan and Seoul utterances –Make the segment durations of the two identical –Make the two F0 contours identical –Make the two intensity contours identical

9. 8 Prosody transfer (PSOLA algorithm) Align segments btw/ Masan and Seoul utterances Make the segment durations of the two utterances identical ㅂㅏㄹㅏㅁ “… 바람 …” Masan ㅏㅏ Seoul stretch shrink ㅂㄹㅁ

10. 9 Prosody transfer (PSOLA algorithm) ㅂㅏㄹㅏㅁ Masan Seoul ㅂㅏㄹㅏㅁ Masan F0 Seoul F0 Make the two F0 contours identical

11. 10 Prosody transfer (PSOLA algorithm) Seoul intensity ㅂㅏㄹㅏㅁ Masan Seoul ㅂㅏㄹㅏㅁ Masan intensity Make the two intensity contours identical

12. 11 Synthetic (simulated) Masan stimulus

13. 12 Synthetic authentic Masan stimulus

14. 13 Listening experiment 16 stimuli (8 + 8) Presented to 13 Masan/Changwon listeners –On a scale of 1 (worst) to 10 (best) –Used Praat ExperimentMFC object –Allowed repetition of stimulus: up to 10 times

15. 14 Listening experiment

16. 15 Results & Conclusion Histogram of listener responses

17. 16 Results & Conclusion F0 contour transfer 1 … listener responses … 10

18. 17 Results & Conclusion Seoul utterances with Masan prosody D F I DF DI FI DFI Masan

19. 18 Results & Conclusion Main effects of –Segmental durations; F(1,12)=11.53, p=0.005 –F0 contour; F(1,12)=141.12, p=0.00000005 Regression analysis

20. 19 Results & Conclusion Prosody cloning not sufficient for dialect simulation –(Sub)Segmental differences may be at work –Quality of synthetic stimuli F0 contour transfer (from Masan to Seoul) –Most influential on shifting perception from Seoul to Masan utterances

21. 20 References [1] Kyung-Hee Lee, “Comparison of acoustic characteristics between Seoul and Busan dialect on fricatives”, Speech Sciences, Vol.9/3, pp.223-235, 2002. [2] Hyun-Gi Kim, Eun-Young Lee, and Ki-Hwan Hong, “Experimental phonetic study of Kyungsang and Cholla dialect using power spectrum and laryngeal fiberscope”, Speech Sciences, Vol.9/2, pp.25-47, 2002. [3] Kyuchul Yoon, “Imposing native speakers’ prosody on non-native speakers’ utterances: The technique of cloning prosody”, Journal of the Modern British & American Language & Literature, Vol.25(4). pp.197-215, 2007. [4] E. Moulines and F. Charpentier, “Pitch synchronouswaveform processing techniques for text-to-speech synthesis using diphones”, Speech Communication, 9 5-6, 1990. [5] P. Boersma, “Praat, a system for doing phonetics by computer”, Glot International, Vol.5, 9/10, pp.341-345, 2005.