1. Polyphonic Transcription Bruno Angeles McGill University - Schulich School of Music MUMT-621 Fall 2009 1/14

2. Outline Polyphonic vs Monophonic The Human Method Issues Methods Evaluation of Transcription Methods 2/14

3. Polyphonic vs Monotonic Transcription Audio signal with musical content  symbolic format Monophonic: a single instrument playing individual notes Polyphonic transcription: A single instrument playing several concurrent notes An instrument playing individual notes with other instruments An instrument playing several concurrent notes with other instruments Several instruments playing concurrent notes 3/14

4. The Human Method 1.Initial sketch  Sections and key phrases 2.Chord scheme or bass line 3.Melody and counter–melodies Instrument playback Musical knowledge Beat tracking Style detection Instrument identification Hainsworth and Macleod (2004) 4/14

5. Issues with Polyphonic Transcription Restrictions: instrument, genre, maximum polyphony Transcription: pitch + timing + attack + release Overlapping harmonics  difficult frequency–based analysis Solution: better understanding and reproduction of the human method? 5/14

6. A Few Definitions Musical note: “a discrete note pitch with a specific onset and an offset time” Melodies: “consecutive note sequences with organized and recognizable shape” Chords: “combinations of simultaneously sounding notes” Ryynänen and Klapuri (2005) 6/14

7. Methods Bottom–up methods: no high–level analysis Blackboard systems: develop hypotheses at various levels Model–based algorithms: high– and low–level analysis + parameter extraction Common method: 1.Preprocessing (e.g., lowpass filtering) 2.Event extraction 3.Classification: Support Vector Machines, Hidden Markov Models, Neural Networks, Gaussian Mixture Models, etc. 4.Postprocessing (e.g., remove outliers) Other methods: probabilistic note-based, matrix factorization 7/14

8. Evaluation of Transcription Methods N C : number of correct events detected N D : total number of events detected N: actual number of events Precision: P = N C / N D Recall: R = N C / N F–Measure: F = 2RP/(R+P) = 2 N C / (N + N D ) 8/14

9. Evaluation of Transcription Methods Gillet and Richard (2005) 2 drummers Different musical sequences Celtic & Groove 5/4: Many ghost notes Table 4 of Gillet and Richard (2005) 9/14

10. Evaluation of Transcription Methods Ryynänen and Klapuri (2005): R ≈ P ≈ 40% Multiple instruments Multiple concurrent notes Hainsworth and Macleod (2005): P = 78.7% Bass guitar Polyphonic context Gillet and Richard (2005): R ≈ P ≈ 84% Drums 10/14 Ryynänen and Klapuri (2005):

11. Conclusion Analogy with speech recognition The problem has not yet been solved Still easier for humans to do – although complex Combine with source separation? 11/14

12. Thank you! 12/14

13. References Bruno, I., S. L. Monni, and P. Nesi. 2003. Automatic music transcription supporting different instruments. In Proceedings of the Third International Conference on Web Delivering of Music. Leeds, UK. 37–44. Cemgil, A. T., B. Kappen, and D. Barber. 2003. Generative model based polyphonic music transcription. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. New Paltz, NY, USA. 181–4. Gillet, O. K., and G. Richard. 2005. Drum Track Transcription of Polyphonic Music Using Noise Subspace Projection. In Proceedings of the International Conference on Music Information Retrieval. London, UK. 92–99. Hainsworth, S. W. 2001. Analysis of musical audio for polyphonic transcription. In First Year Ph.D. Report. Hainsworth, S. W., and M. D. Macleod. 2001. Automatic bass line transcription from polyphonic music. In Proceedings of the International Computer Music Conference. Havana, Cuba. Hainsworth, S. W., and M. D. Macleod. 2004. The automated music transcription problem. In Cambridge University Engineering Department. Klapuri, A. 2004. Signal processing methods for the automatic transcription of music. In Ph.D. Dissertation. 13/14

14. References Lidy, T., A. Rauber, A. Pertusa, and J. M. Iñesta. 2007. Improving genre classification by combination of audio and symbolic descriptors using a transcription system. In Proceedings of the International Conference on Music Information Retrieval. Vienna, Austria. 61–6. Marolt, M. 2001. SONIC: Transcription of polyphonic piano music with neural networks. In Proceedings of the Workshop on Current Research Directions in Computer Music. Barcelona, Spain. 217–24. Nichols, E., and C. Raphael. 2007. Automatic transcription of music audio through continuous parameter tracking. In Proceedings of the International Conference on Music Information Retrieval. Vienna, Austria. 387–92. Niedermayer, B. 2008. Non-negative matrix division for the automatic transcription of polyphonic music. In Proceedings of the International Conference on Music Information Retrieval. Philadelphia, PA, USA. 544–9. Paulus, J. 2006. Acoustic modelling of drum sounds with Hidden Markov Models for music transcription. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. Toulouse, France. Pertusa, A., and J. M. Iñesta. 2003. Polyphonic music transcription through dynamic networks and spectral pattern identification. In IAPR International Workshop on Artificial Neural Networks in Pattern Recognition. 19–25. Ryynänen, M., and A. Klapuri. 2006. Transcription of the singing melody in polyphonic music. In Proceedings of the International Conference on Music Information Retrieval. Victoria, BC, Canada. 14/14