THE 2007 MUSIC INFORMATION RETRIEVAL EVALUATION EXCHANGE (MIREX 2007) RESULTS OVERVIEW The IMIRSEL Group led by J. Stephen Downie Graduate School of Library.

Slides:

Advertisements

Similar presentations

Speaker Associate Professor Ning-Han Liu. What’s MIR  Music information retrieval (MIR) is the interdisciplinary science of retrieving information from.

Advertisements

Note-level Music Transcription by Maximum Likelihood Sampling Zhiyao Duan ¹ & David Temperley ² 1.Department of Electrical and Computer Engineering 2.Eastman.

LYRIC-BASED ARTIST NETWORK Derek Gossi CS 765 Fall 2014.

Institute of Information Science Academia Sinica 1 Singer Identification and Clustering of Popular Music Recordings Wei-Ho Tsai

Review Mining for Music Digital Libraries: Phase II J. Stephen Downie, Xiao Hu The International Music Information Retrieval Systems Evaluation Lab (IMIRSEL)

Outline Introduction Music Information Retrieval Classification Process Steps Pitch Histograms Multiple Pitch Detection Algorithm Musical Genre Classification.

tracking beat tracking beat Mcgill university :: music technology :: mumt 611>>

Data Mining and Text Analytics in Music Audi Sugianto and Nicholas Tawonezvi.

/25 Singer Similarity A Brief Literature Review Catherine Lai MUMT-611 MIR March 24,

Evaluation of the Audio Beat Tracking System BeatRoot By Simon Dixon (JNMR 2007) Presentation by Yading Song Centre for Digital Music

Berenzweig - Music Recommendation1 Music Recommendation Systems: A Progress Report Adam Berenzweig April 19, 2002.

Retrieval Evaluation: Precision and Recall. Introduction Evaluation of implementations in computer science often is in terms of time and space complexity.

1 Content-based Music Information Retrieval Student: DENG Jie. Supervisor: Prof. LEUNG, Clement Department of Computer Science Hong Kong Baptist University.

GCT731 Fall 2014 Topics in Music Technology - Music Information Retrieval Introduction to MIR Course Overview 1.

Hadoop Team: Role of Hadoop in the IDEAL Project ●Jose Cadena ●Chengyuan Wen ●Mengsu Chen CS5604 Spring 2015 Instructor: Dr. Edward Fox.

CS 5604 Spring 2015 Classification Xuewen Cui Rongrong Tao Ruide Zhang May 5th, 2015.

NM7613: Music Signal Analysis and Retrieval 音樂訊號分析與檢索 Jyh-Shing Roger Jang ( 張智星 ) CSIE Dept, National Taiwan University.

2015/9/111 Introduction to ISMIR/MIREX J.-S. Roger Jang （張智星） Multimedia Information Retrieval (MIR) Lab CSIE Dept, National Taiwan Univ.

Educational Software using Audio to Score Alignment Antoine Gomas supervised by Dr. Tim Collins & Pr. Corinne Mailhes 7 th of September, 2007.

Survey Of Music Information Needs, Uses, and Seeking Behaviors Jin Ha Lee J. Stephen Downie Graduate School of Library and Information Science University.

August 12, 2004IAML - IASA 2004 Congress, Olso1 Music Information Retrieval, or how to search for (and maybe find) music and do away with incipits Michael.

Usability Fujinaga Uitdenbogerd, A., and R. Schyndel A review of factors affecting music recommender success. International Symposium on.

Per Møldrup-Dalum State and University Library SCAPE Information Day State and University Library, Denmark, SCAPE Scalable Preservation Environments.

Music Information Retrieval -or- how to search for (and maybe find) music and do away with incipits Michael Fingerhut Multimedia Library and Engineering.

Developing a Concept Extraction Technique with Ensemble Pathway Prat Tanapaisankit (NJIT), Min Song (NJIT), and Edward A. Fox (Virginia Tech) Abstract.

Aspects of Music Information Retrieval Will Meurer School of Information University of Texas.

1 A Collaborative Music Information Retrieval / Music Digital Library Research Model: A Briefing J. Stephen Downie, Ph.D. Graduate School of Library &

Exploiting Recommended Usage Metadata: Exploratory Analyses Xiao Hu, J. Stephen Downie, Andreas Ehmann THE ANDREW W. MELLON FOUNDATION The International.

MIREX 2014 Evaluation Results J. Stephen Downie & IMIRSEL University of Illinois at Urbana-Champaign Classical Composer Identification.

Automatic music classification and the importance of instrument identification Cory McKay and Ichiro Fujinaga Music Technology Area Faculty of Music McGill.

Polyphonic Transcription Bruno Angeles McGill University - Schulich School of Music MUMT-621 Fall /14.

Timbre and Modulation Features for Music Genre/Mood Classification J.-S. Roger Jang & Jia-Min Ren Multimedia Information Retrieval Lab Dept. of CSIE, National.

2016/6/41 Recent Improvement Over QBSH and AFP J.-S. Roger Jang （張智星） Multimedia Information Retrieval (MIR) Lab CSIE Dept, National Taiwan Univ.

IR Homework #3 By J. H. Wang May 4, Programming Exercise #3: Text Classification Goal: to classify each document into predefined categories Input:

Singer similarity / identification Francois Thibault MUMT 614B McGill University.

FACTORS AFFECTING RESPONSE RATES FOR REAL-LIFE MIR QUERIES Jin Ha LeeM. Cameron JonesJ. Stephen Downie Graduate School of Library and Information Science.

McGill University > Schulich School of Music > Music Technology > MUMT 611 j j MusicMetaManager j j Cory McKay Jason A. Hockman part of the jMIR software.

THE 2005 MUSIC INFORMATION RETRIEVAL EVALUATION EXCHANGE (MIREX 2005) RESULTS OVERVIEW Graduate School of Library and Information Science J. Stephen DownieKris.

IR Homework #3 By J. H. Wang May 10, Programming Exercise #3: Text Classification Goal: to classify each document into predefined categories Input:

QBSH Corpus The QBSH corpus provided by Roger Jang [1] consists of recordings of children’s songs from students taking the course “Audio Signal Processing.

Ask a Librarian: The Role of Librarians in the Music Information Retrieval Community Jenn Riley, Indiana University Constance A. Mayer, University of Maryland.

Audio Train/Test: Mood Classification - MIREX08 Dataset SubIDParticipantsAccuracy JJ1Ming-Ju Wu, Jyh-Shing Roger Jang0.68 PP1Renato Panda, Rui Pedro Paiva0.68.

THE 2006 MUSIC INFORMATION RETRIEVAL EVALUATION EXCHANGE (MIREX 2006) RESULTS OVERVIEW The IMIRSEL Group led by J. Stephen Downie Graduate School of Library.

Discussions on Audio Melody Extraction (AME) J.-S. Roger Jang ( 張智星 ) MIR Lab, CSIE Dept. National Taiwan University.

R ESEARCH P ROGRESS R EPORT – C OVER S ONGS I DENTIFICATION Ken.

BASS TRACK SELECTION IN MIDI FILES AND MULTIMODAL IMPLICATIONS TO MELODY gPRAI Pattern Recognition and Artificial Intelligence Group Computer Music Laboratory.

Metamidi: a tool for automatic metadata extraction from MIDI files Tomás Pérez-García, Jose M. Iñesta, and David Rizo Computer Music Laboratory University.

Automatic Music Transcription: Employing Hidden Markov Models to assist with Multiple Fundamental Pitch Estimation ASHWIN D. D’CRUZ Acting Supervisor:

3D Motion Classification Partial Image Retrieval and Download Multimedia Project Multimedia and Network Lab, Department of Computer Science.

A content-based System for Music Recommendation and Visualization of User Preference Working on Semantic Notions Dmitry Bogdanov, Martin Haro, Ferdinand.

IR Homework #2 By J. H. Wang May 9, Programming Exercise #2: Text Classification Goal: to classify each document into predefined categories Input:

A shallow description framework for musical style recognition Pedro J. Ponce de León, Carlos Pérez-Sancho and José Manuel Iñesta Departamento de Lenguajes.

Genre Classification of Music by Tonal Harmony Carlos Pérez-Sancho, David Rizo Departamento de Lenguajes y Sistemas Informáticos, Universidad de Alicante,

Audio Processing Mitch Parry. Resource! Sound Waves and Harmonic Motion.

Big Data Processing of School Shooting Archives

Introduction to Music Information Retrieval (MIR)

Introduction to ISMIR/MIREX

Large-Scale Music Audio Analyses Using High Performance Computing Technologies: Creating New Tools, Posing New Questions J.

The Greek Audio Dataset

David Sears MUMT November 2009

MIR Lab: R&D Foci and Demos （ MIR實驗室：研發重點及展示）

Audio Segmentation, Classification, and Retrieval

Artist Identification Based on Song Analysis

Usability Fujinaga 2003.

Closing Remarks on MSAR-2017

Introduction to Music Information Retrieval (MIR)

Aspects of Music Information Retrieval

Introduction to Music Information Retrieval (MIR)

SVM Based Learning System for F-term Patent Classification

Music Signal Processing

Presentation transcript:

THE 2007 MUSIC INFORMATION RETRIEVAL EVALUATION EXCHANGE (MIREX 2007) RESULTS OVERVIEW The IMIRSEL Group led by J. Stephen Downie Graduate School of Library and Information Science University of Illinois at Urbana-Champaign MIREX Task Results MIREX Task Results Non-compliance with input/output formats Robustness issues: Tolerance for silence, graceful failing, scalability issues (dealing with larger data sets), parallelizability, writing out results incrementally (useful for failures) Library dependency issues Pre-compiled MEX files not running JAVA object serialization/de-serialization issues Low quality and trustworthiness of the metadata for music General shortage of new test sets Effectively managing results output: From standard out to Wiki MIREX 2007 Challenges MIREX 2007 Challenges Audio Genre Classification RankParticipant Avg. Raw Accuracy 1IMIRSEL (svm)68.29% 2 Lidy, Rauber, Pertusa & Iñesta 66.71% 3Mandel & Ellis66.60% 4Mandel & Ellis (spec)65.50% 5Tzanetakis, G.65.34% 6Guaus & Herrera62.89% 7IMIRSEL (knn)54.87% Audio Mood Classification RankParticipant Avg. Raw Accuracy 1 Tzanetakis, G.61.50% 2 Laurier, C.60.50% 3 Lidy, Rauber, Pertusa & Iñesta 59.67% 4 Mandel & Ellis57.83% 5 Mandel & Ellis (spec)55.83% IMIRSEL (svm)55.83% 7 Lee, K. (1)49.83% 8 IMIRSEL (knn)47.17% 9 Lee, K. (2)25.67% Audio Cover Song Identification RankParticipant Avg. Precision 1 Serrà & Gómez Ellis & Cotton Bello, J Jensen, Ellis, Christensen & Jensen Lee, K. (1) Lee, K. (2) Kim & Perelstein IMIRSEL0.017 Audio Onset Detection RankParticipant Avg. F- Measure 1 Zhou & Reiss Lee, Shiu & Kuo (0.3) Lee, Shiu & Kuo (0.4) Lee, Shiu & Kuo (0.2) Lee, Shiu & Kuo (lp)0.796 Röbel, A. (1)0.796 Röbel, A. (3) Röbel, A. (2)0.793 Röbel, A. (4) Stowell & Plumbley (cd) Stowell & Plumbley (som) Stowell & Plumbley (pow) Stowell & Plumbley (rcd) Stowell & Plumbley (wpd) Lacoste, A Stowell & Plumbley (mkl) Stowell & Plumbley (pd)0.565 Query by Singing/Humming (Jang’s Collection) RankParticipantMRR 1 Wu & Li (2) Wu & Li (1) Jang, Lee & Hsu (2) Jang, Lee & Hsu (1) Lemström & Mikkilä Gómez, Abad-Mota & Ruckhaus Ferraro, Hanna, Allali & Robine Uitdenbogerd, A. (1) Uitdenbogerd, A. (3) Uitdenbogerd, A. (2)0.093 Query by Singing/Humming (ThinkIT Collection) RankParticipant MRR 1 Wu & Li 2 (Wu pitches) Wu & Li 1 (Wu notes) Wu & Li 2 (Jang pitches) Gómez, Abad-Mota & Ruckhaus (Wu notes) Lemström & Mikkilä (Wu notes) Jang, Lee & Hsu 2 (Jang pitches) Ferraro, Hanna, Allali & Robine (Wu notes) Jang, Lee & Hsu 2 (Wu pitches) Jang, Lee & Hsu 1 (Jang pitches) Jang, Lee & Hsu 1 (Wu pitches) Audio Classical Composer Identification RankParticipant Avg. Raw Accuracy 1 IMIRSEL (svm)53.72% 2 Mandel & Ellis (spec)52.02% 3 IMIRSEL (knn)48.38% 4 Mandel & Ellis47.84% 5 Lidy, Rauber, Pertusa & Iñesta 47.26% 6 Tzanetakis, G44.59% 7 Lee, K.19.70% Audio Artist Identification RankParticipant Avg. Raw Accuracy 1 IMIRSEL (svm)48.14% 2 Mandel & Ellis (spec)47.16% 3 Mandel & Ellis40.46% 4 Lidy, Rauber, Pertusa & Iñesta 38.76% 5 Tzanetakis, G.36.70% 6 IMIRSEL (knn)35.29% 7 Lee, K.9.71% Audio Music Similarity RankParticipant Avg. Fine Score 1Pohle & Schnitzer Tzanetakis, G Barrington, Turnball, Torres & Lanskriet Bastuck, C. (1) Lidy, Rauber, Pertusa & Iñesta (1) Mandel & Ellis Lidy, Rauber, Pertusa & Iñesta (2) Bastuck, C. (2) Bastuck, C. (3) Bosteels & Kerre (1) Paradzinets & Chen Bosteels & Kerre (2)*0.178 Multi F0 Estimation RankParticipantAccuracy 1Ryynänen & Klapuri (1) Zhou & Reiss Pertusa & Iñesta (1) Yeh, C Vincent, Bertin & Badeau (2) Cao, Li, Liu & Yan (1) Raczyński, Ono & Sagayama Vincent, Bertin & Badeau (1) Poliner & Ellis (1) Leveau, P Cao, Li, Liu & Yan (2) Egashira, Kameoka & Sagayama (2) Egashira, Kameoka & Sagayama (1) Cont, A. (2) Cont, A. (1) Emiya, Badeau & David (1) Multi F0 Note Tracking RankParticipantAvg. F- Measure 1Ryynänen & Klapuri (2) Vincent, Bertin & Badeau (4) Poliner & Ellis (2) Vincent, Bertin & Badeau (3) Pertusa & Iñesta (2) Egashira, Kameoka & Sagayama (4) Egashira, Kameoka & Sagayama (3) Pertusa & Iñesta (3) Emiya, Badeau & David (2) Cont, A. (4) Cont, A. (3)0.087 Symbolic Melodic Similarity RankParticipantADRFine 1Gómez, Abad-Mota & Ruckhaus (1) Gómez, Abad-Mota & Ruckhaus (2) Ferraro, Hanna, Allali & Robine Uitdenbogerd, A. (3) Uitdenbogerd, A. (1) Uitdenbogerd, A. (2) Pinto, A. (1) Pinto, A. (2) Special Thanks to: The Andrew W. Mellon Foundation, the National Science Foundation (Grant No. NSF IIS and No. NSF IIS ), the content providers and the MIR community, all of the IMIRSEL group – M. Bay, A. Ehmann, A. Gruzd, X. Hu, M. C. Jones, J. H. Lee, M. McCrory, K. West, X. Xiang and all the volunteers who evaluated the MIREX results, and the ISMIR 2007 organizing committee * The results for Bosteels & Kerre (2) were interpolated from partial data due to a runtime error.