1. Aspects of Music Information Retrieval Will Meurer School of Information University of Texas

2. Music Information Retrieval (MIR) MIR Overview Challenges in MIR Current MIR Technology Possibilities & Concerns Recommendations Final Remarks

3. MIR Overview Currently MIR is chiefly Bibliographic How is Music so different? Downie’s 7 Facets Pitch, Temporal, Harmonic, Timbral, Editorial Textual* and Bibliographic* Representations Visual (musical scores, manuscripts) Aural (digital music) Text Hybrid (visual representation of an audio file) * Used in current mainstream MIR systems

4. MIR Overview Facets

5. MIR Overview Visual Representations E---------------0---------3-3------3--1-1-------------3-1-1---------------- B---1-------------3-2-----2-2------2--3-3-----3-2---2---3-3------------1--- G---0--0--0h1-------0-----0-0------0--2-2---2---0-0-----2-2------------2--- D---2--2------------------------------0---0-------------0----0-1-2-3-3---3- A-3-------------------0-0-----0--0--------------0-------------------------- E-------------0------------------------------------------------------------ Common Music Notation Tablature

6. MIR Overview User Groups General music listeners Music students, performers, composers, and conductors; music therapists; musicologists; music librarians and library patrons; audio engineers; scholars; researchers, and; intellectual property lawyers

7. Challenges in MIR Began in the 1950’s, still an “emerging discipline” Subjectivity and Versioning Many levels of music knowledge No standardization No standard test collection (HNH Naxos) No standardized sets of performance No standardized evaluation metrics Lack of bibliographic control (Downie’s site)Downie’s site No communication among interested disciplines

8. Current MIR Technology Aural Queries Query By Humming (QBH) systems Input: aural melody Matches interval sequences to index terms Musart (Bartsch et al., 2003) matches melody, harmony, and rhythm

9. Current MIR Technology Indexing for Aural Queries Thematic melodies are extracted from the source (Beginning of Beethoven’s 9 th Symphony) Translated into text representations of intervals, pitch, and harmony (e.g. EEFGGFEDCCDEEDD) Text versions shrink index size. Audio indexing is expensive and involves more processing to match queries Musart extracts thematic material automatically by finding common passages N-gramming

10. Current MIR Technology N-gramming “Chunks” search terms Compares search “chunks” to indexed “chunks” Example:  Indexed melody is CCGGAAG-FFEEDDC- (Twinkle, Twinkle, Little Star)  Searcher hummed CCGGAAG-FFECDDC  N-gramming this query would match the CCGGAAG even though FFECDDC was incorrectly hummed Provides fault tolerance

11. Current MIR Technology Polyphonic Focus Monophonic/polyphonic queries Doraisamy and Rüger (2002)  Evaluated monophonic queries against a polyphonic database  Results were “promising” Polyphonic/polyphonic queries Musart  Flattens chord tones into text codes  Does not account for timbral aspects  Not suitable for large databases where more matches are made per query. The more fault tolerance, the more results

12. Current MIR Technology Fusing the Representations and Formats Need to synchronize data in all formats and representations Allows one system to serve many different types of users Arifi et al. (2003) synchronized Score (visual), MIDI (digital), and PCM (digital audio)

13. Possibilities & Concerns Another Facet? Jan LaRue “SHMRG” G is the overall “form” and how parts of a piece Affect the Effect of a piece Growth may be useful to index and search LaRueDownie SoundTimbral HarmonyHarmonic MelodyPitch RhythmTemporal Growth???? Editorial, Bibliographic, and Textual work within and between LaRue’s S, H, M, and R.

14. Possibilities & Concerns Further effects of copyright laws Interfaces and usability Current focus is on technology, not usability Dixon, Pampalk & Widmer (2003) Browse multiple views simultaneously Unnatural, awkward interface

15. Possibilities & Concerns Navigation?

16. Recommendations Downie & Olson (2003), Chopin Early Editions Content-based search features Symbolic content search Optical Music Resolution (OCR for music) Version distinguishing

17. Recommendations Focus must be on why Complex problems? Simple solutions. Base the fault tolerance level on searcher’s aural query precision from past queries Results should display multiple facets: bibliographic, textual, pitch (what key), etc. Results should offer different formats: score, mp3, MIDI Display all versions from the database within each search result

18. Final Remarks Music is a complicated form of information and requires special retrieval systems Demand for MIR will increase, and research and funding will follow Copyrights and lack of standardization may prevent fast growth of MIR development MIR technology is improving, application is lacking Interface design and usability must develop as the technology advances