1. Instrument Classification in a Polyphonic Music Environment Yingkit Chow Spring 2005

2. Objective To develop and expand on processing techniques for classification of instruments in a polyphonic music environment Incentive Classification of instruments can prove useful for automatic music transcription.

3. General Background Most previous research with instrument classification works with monophonic audio Work in Polyphonic music usually deals with “Blackboard” system that accesses knowledge sources and works with a top-down approach Difficulties with polyphonic music: Overlap of tones can be detrimental to extracting the correct identifying features Confusion of octave

4. Features Used for Classification in Monophonic Music Analysis [1] Features Used: RMS Envelope 60 % Accuracy, 60% Reliability CQT Frequency Spectrum (PCA) 66 % Accuracy, 68% Reliability MSA Trajectories (PCA) 75 % Accuracy, 76 % Reliability Combined: 82% Accuracy, 83% Reliability

5. Results from [1], Monophonic Audio CONFUSION MATRIX NNC Combined: k=5 WEIGHTED MAJORITY, Confusion Matrix weighted (reliability) ACCACC BASBAS CELCEL CLACLA DBBDBBB FLUFLU GL O GUIGUI HORHOR MARMAR OBOOBO ORGORG PIAPIA SaxSax TROTRO TRUTRU TUBTUB VIOVIO XYLXYL Accur acy 79846586828810095938967898457759610 0 7375 Relia bility 909410 0 69707986837187868791847875838883

6. Missing Feature Approach[2], For Polyphonic music Jana Eggink and Guy J. Brown Find fundamental frequency and compare against harmonic sieves Spectral peaks of fundamental and harmonics are the features used for classification (50- 6kHz, 60 Hz window) Instruments tested in this paper: (flute, clarinet, violin, cello, oboe)

7. Missing-Feature Approach, Special Conditions[2] Frequency regions with energy from interfering tones are excluded from classification process. Cepstral coefficients are not used as features since they correspond to all frequencies. Local spectral features are used. Cannot handle drum or “untuned percussion instrument”

8. Results with Missing Feature Approach to monophonic music Confusion Matrix for the 5 instruments in a Monophonic music environment The identification of the family of instrument (String, Woodwind) is about 85% FluteClarinetOboeViolinCello Flute7715008 Clarinet15620815 Obo0156988 Violin00155431 Cello0015 69

9. Missing Feature Approach, Instrument Classification in 2 tone samples FluteClarinetOboeViolinCello Flute7560109 Clarinet134932212 Oboe2013252816 Violin34105725 Cello39163637 Confusion Matrix for the 5 instruments in a polyphonic music environment (2 simultaneous tones)

10. Project Goal I will test the Missing Feature Model against a different set of instruments: Instruments to be selected based on available sample set and to provide a variety of instruments from different families. Alternative source would be taking input from a synthesized version of the instrument (MIDI to Wav). Add system over the missing feature model to include information from neighboring frames (in time) and use information of partials in the classification scheme

11. Testing I will test my system first, within monophonic music, to get an upper bound on the capabilities of the system for each instrument. Secondly, I will test the system within a 2 note environment

12. References 1.“Multi-feature Musical Instrument Sound Classifier”, I. Kaminskyj, http://www.mikropol.net/volume6/kaminskyj_i/kaminskyj_i.html 2.“A Missing Feature Approach to Instrument Identification in Polyphonic Music”, by Jana Eggink and Guy J. Brown http://www.dcs.shef.ac.uk/%7Ejana/egginkICASSP03.pdf 3.“Instrument Recognition in Acompanied Sonatas and Concertos”, by Jana Eggink and Guy J. Brown http://www.dcs.shef.ac.uk/%7Ejana/egginkICASSP04.pdf 4.Music Samples from the University of Iowa, http//:theremin.music.uiowa.edu/