1. Some Research Activities in MIR Lab J.-S. Roger Jang ( 張智星 ) jang@cs.nthu.edu.tw http://www.cs.nthu.edu.tw/~jang Multimedia Information Retrieval Lab CS Dept, Tsing Hua Univ, Taiwan

2. -2- Outline zSpeech Assessment zSinging Voice Separation zAudio Music Annotation

3. -3- Demo: Practice of Mandarin Idioms of Length 4 ( 一語中的 ) yLevel (difficulty) of an idiom is based on it’s freq. via Google search: x 孤掌難鳴 ===> 260,000 x 鶼鰈情深 ===> 43,300 x 亡鈇意鄰 ===> 22,700 x 舉案齊眉 ===> 235,000 yCan be adapted for English learning yNext step: multi- threading, fast decoding via FSM

4. -4- Demo: Recitation Machine （唸唸不 忘） zSupport Mandarin & English zSupport user-defined recitation script zNext step: multithreading for recording & recognition

5. -5- Demo: Dialog Practice via Videos zDialog-based practice and evaluation

6. -6- Demo: Embedded Systems yChicken run ( 落跑雞 )Chicken run ( 落跑雞 yPenguin for Tang Poetry ( 唐詩企鵝 )Penguin for Tang Poetry ( 唐詩企鵝 ) yRobot Fighter ( 蘿蔔戰士 )Robot Fighter ( 蘿蔔戰士 ) ySinging Bass & Dog ( 大 嘴鱸魚和唱歌狗 )Singing Bass & Dog ( 大 嘴鱸魚和唱歌狗 )

7. -7- Speech Assessment: Current/Future Directions zOn-going work: yTone recognition and assessment yRetroflex & nonretroflex recognition yDetection of “ 兒化音 ” zResearch directions yIdentification of confusing phone/syllables yScore optimization schemeScore optimization scheme zDemo page: yhttp://mirlab.org/mir_main/demo.htmhttp://mirlab.org/mir_main/demo.htm

8. -8- Singing Voice Separation zChao-Ling Hsu, Jyh-Shing Roger Jang, and Te-Lu Tsai, "Separation of Singing Voice from Music Accompaniment with Unvoiced Sounds Reconstruction for Monaural Recordings", Proceedings of 125th AES Convention, San Francisco, USA, Oct. 2008.Chao-Ling Hsu, Jyh-Shing Roger Jang, and Te-Lu Tsai, "Separation of Singing Voice from Music Accompaniment with Unvoiced Sounds Reconstruction for Monaural Recordings", Proceedings of 125th AES Convention, San Francisco, USA, Oct. 2008.

9. -9- SVS: Current/Future Directions zAudio Melody Extraction yClose the loop: pitch  vocal  better pitch  better vocal  … zLack of a public-domain dataset yWe are preparing one… zMore error analysis is under way.

10. -10- Audio Music Annotation & Retrieval zZhi-Sheng Chen, Jia-Min Zen, Jyh-Shing Roger Jang, "Music Annotation and Retrieval System Using Anti-Models", Proceedings of 125th AES Convention, San Francisco, USA, Oct. 2008.Zhi-Sheng Chen, Jia-Min Zen, Jyh-Shing Roger Jang, "Music Annotation and Retrieval System Using Anti-Models", Proceedings of 125th AES Convention, San Francisco, USA, Oct. 2008

11. -11- Research Directions z“Glass ceiling” problem yPointed by Stephen Downie, “The music information retrieval evaluation exchange (2005– 2007):A window into music information retrieval research” yWe should go beyond spectral-based approaches to have more semantic models/representations zInterpretation of “Sad” and “Stong”: Probability of fuzziness?