1. Lyric alignment in popular songs Luong Minh Thang

2. Outline Project description & techniques Some important knowledge Base-line system

3. Outline Project description & techniques Some important knowledge Base-line system

4. Project description Given: textual transcription of lyrics acoustic musical signal of a song Purpose: find the timestamps for the beginning and ending points for each line of the song.

5. Techniques to be investigated A repetition based technique for detecting self-similarities in both the audio and the text Dynamic programming (also known as dynamic time warping) will be employed to align the repetition analysis of both media together to produce an alignment

6. Outline Project description & techniques Some important knowledge Base-line system

7. Chroma vectors 1 octaves with 12 semitones 12-dimensional chroma vector C C# D … B C C# D…B C C# D…B C C# D…B Tone heights 1 octaves with 12 semitones 0.3 0.5 0.1 0.7

8. Dynamic time warping algorithm Actually, dynamic programming

9. Word similarity using phoneme dictionary Using CMU phoneme dictionary Each word is decomposed into phoneme sequence ALIGNMENT: AH0 L AY1 N M AH0 N T ALGORITHMS: AE1 L G ER0 IH2 DH AH0 M Z Similarity of 2 words is the similarity of the 2 phoneme sequences

10. Outline Project description & techniques Some important knowledge Base-line system

11. Base-line system: overview Musical signal inputText input Chroma-vector calculations Simplification Text processing Music notation sequence Word sequence Word self- similarity matrix Mapping Aligning Phoneme dictionary Symbol Self- similarity matrix

12. Base-line system: overview Musical signal inputText input Chroma-vector calculations Simplification Text processing Music notation sequence Word sequence Word self- similarity matrix Mapping Aligning Phoneme dictionary Symbol Self- similarity matrix

13. Base-line system: overview Musical signal inputText input Chroma-vector calculations Simplification Text processing Music notation sequence Word sequence Word self- similarity matrix Mapping Aligning Phoneme dictionary Symbol Self- similarity matrix

14. Base-line system: overview Musical signal inputText input Chroma-vector calculations Simplification Text processing Music notation sequence Word sequence Symbol Self- similarity matrix Mapping Aligning Phoneme dictionary Word self- similarity matrix

15. Base-line system: overview Musical signal inputText input Chroma-vector calculations Simplification Text processing Music notation sequence Word sequence Symbol Self- similarity matrix Mapping Aligning Phoneme dictionary Word self- similarity matrix

16. Thank you !