Presentation is loading. Please wait.

Presentation is loading. Please wait.

Scott McCaulay Joe Rinkovsky Pervasive Technology Institute Indiana University Music Information Retrieval With Condor.

Similar presentations


Presentation on theme: "Scott McCaulay Joe Rinkovsky Pervasive Technology Institute Indiana University Music Information Retrieval With Condor."— Presentation transcript:

1 Scott McCaulay Joe Rinkovsky Pervasive Technology Institute Indiana University Music Information Retrieval With Condor

2 Overview PFASC is a suite of applications developed at IU to perform automated similarity analysis of audio files PFASC is a suite of applications developed at IU to perform automated similarity analysis of audio files Potential applications include organization of digital libraries, recommender systems, playlist generators, audio processing Potential applications include organization of digital libraries, recommender systems, playlist generators, audio processing PFASC is a project in the MIR field, an extension and adaptation of traditional Text Information Retrieval techniques to sound files PFASC is a project in the MIR field, an extension and adaptation of traditional Text Information Retrieval techniques to sound files Elements of PFASC, specifically the file by file similarity calculation, have proven to be a very good fit with Condor Elements of PFASC, specifically the file by file similarity calculation, have proven to be a very good fit with Condor

3 What We’ll Cover Condor at Indiana University Condor at Indiana University Background on Information Retrieval and Music Information Retrieval Background on Information Retrieval and Music Information Retrieval The PFASC project The PFASC project PFASC and Condor, experience to date and results PFASC and Condor, experience to date and results Summary Summary

4 Condor at IU Initiated in 2003 Initiated in 2003 Utilizes 2350 Windows Vista machines from IU’s Student Technology Clusters Utilizes 2350 Windows Vista machines from IU’s Student Technology Clusters Minimum 2GB memory, 100 Mb network Minimum 2GB memory, 100 Mb network Available to students at 42 locations on the Bloomington campus 24 x 7 Available to students at 42 locations on the Bloomington campus 24 x 7 Student use is top priority, Condor jobs are suspended immediately on use Student use is top priority, Condor jobs are suspended immediately on use

5 Costs to Support Condor at IU Annual marginal annual cost to support Condor Pool at IU is < $15K Annual marginal annual cost to support Condor Pool at IU is < $15K Includes system administration, head nodes, file servers Includes system administration, head nodes, file servers Purchase and support of STC machines are funded from Student Technology Fees Purchase and support of STC machines are funded from Student Technology Fees

6 Challenges to Making Good use of Condor Resources at IU Windows environment Windows environment –Research computing environment at IU is geared to Linux, or to exotic architectures Ephemeral resources Ephemeral resources –Machines are moderately to heavily used at all hours, longer jobs are likely to be preempted Availability of other computing resources Availability of other computing resources –Local users are far from starved for cycles, limited motivation to port

7 Examples of Applications Supported on Condor at IU Hydra Portal (2003) Hydra Portal (2003) –Job submission portal –Suite of Bio apps, Blast, Meme, FastDNAml Condor Render Portal (2006) Condor Render Portal (2006) –Maya, Blender video rendering PFASC (2008) PFASC (2008) –Similarity analysis of audio files

8 Information Retrieval - Background Science of organizing documents for search and retrieval Science of organizing documents for search and retrieval Dates back to 1880s (Hollerith) Dates back to 1880s (Hollerith) Vannevar Bush, first US presidential science advisor, presages hypertext in “As We May Think” (1945) Vannevar Bush, first US presidential science advisor, presages hypertext in “As We May Think” (1945) The concept of automated text document analysis, organization and retrieval was met with a good deal of skepticism until the 1990s. Some critics now grudgingly concede that it might work The concept of automated text document analysis, organization and retrieval was met with a good deal of skepticism until the 1990s. Some critics now grudgingly concede that it might work

9 Calculating Similarity The Vector Space Model Each feature found in a file is assigned a weight based on the frequency of its occurrence in the file and how common that feature is in the collection Each feature found in a file is assigned a weight based on the frequency of its occurrence in the file and how common that feature is in the collection Similarity between files is calculated based on common features and their weights. If two files share features not common to the entire collection, their similarity value will be very high Similarity between files is calculated based on common features and their weights. If two files share features not common to the entire collection, their similarity value will be very high This vector space model (Salton) is the basis of many text search engines, and also works well with audio files This vector space model (Salton) is the basis of many text search engines, and also works well with audio files For text files, features are words or character strings. For Audio files, features are prominent frequencies within frames of audio or sequences of frequencies across frames. For text files, features are words or character strings. For Audio files, features are prominent frequencies within frames of audio or sequences of frequencies across frames.

10 Some Digital Audio History Uploaded to Compuserve 10/1985 Uploaded to Compuserve 10/1985 –one of the most popular downloads at the time! 10 seconds of digital audio 10 seconds of digital audio Time to download (300 baud): 20 minutes Time to download (300 baud): 20 minutes Time to load: 20 minutes (tape) 2 minutes (disk) Time to load: 20 minutes (tape) 2 minutes (disk) Storage space: 42K Storage space: 42K From this to Napster in less than 15 years From this to Napster in less than 15 years

11 Explosion of Digital Audio Digital audio today similar to text 15 years ago Digital audio today similar to text 15 years ago Poised for 2nd phase of the digital audio revolution? Poised for 2nd phase of the digital audio revolution? –Ubiquitous, easy to create, access, share –Lack of tools to analyze, search or organize

12 How can we organize this enormous and growing volume of digital audio data for discovery and retrieval?

13 What’s done today Pandora - Music Genome Project Pandora - Music Genome Project –expert manual classification of ~ 400 attributes Allmusic Allmusic –manual artist similarity classification by critics last.fm – Audioscrobbler last.fm – Audioscrobbler –collaborative filtering from user playlists iTunes Genius iTunes Genius –collaborative filtering from user playlists

14 What’s NOT done today Any analysis (outside of research) of similarity or classification based on the actual audio content of song files Any analysis (outside of research) of similarity or classification based on the actual audio content of song files

15 Possible Hybrid Solution Automated Analysis User Behavior Manual Metadata Classification/Retrieval system could use elements of all three methods to improve performance Classification/Retrieval system could use elements of all three methods to improve performance

16 Music Information Retrieval Applying traditional IR techniques for classification, clustering, similarity analysis, pattern matching, etc. to digital audio files Applying traditional IR techniques for classification, clustering, similarity analysis, pattern matching, etc. to digital audio files Recent field of study, has accelerated with the inception of the ISMIR conference in 2000 and MIREX evaluation in Recent field of study, has accelerated with the inception of the ISMIR conference in 2000 and MIREX evaluation in 2004.

17 Common Basis of an MIR System Select very small segment of audio data, 20-40ms Select very small segment of audio data, 20-40ms Use fast Fourier transform (FFT) to convert to frequency data Use fast Fourier transform (FFT) to convert to frequency data This ‘frame’ of audio becomes the equivalent of a word in a text file for similarity analysis This ‘frame’ of audio becomes the equivalent of a word in a text file for similarity analysis The output of this ‘feature extraction’ process is input to various analysis or classification processes The output of this ‘feature extraction’ process is input to various analysis or classification processes PFASC additionally combines prominent frequencies from adjacent frames to create temporal sequences as features PFASC additionally combines prominent frequencies from adjacent frames to create temporal sequences as features

18 PFASC as an MIR Project Parallel Framework for A udio S imilarity C lustering Parallel Framework for A udio S imilarity C lustering Initiated at IU in 2008 Initiated at IU in 2008 Team includes School of Library and Information Science (SLIS), Cognitive Science, School of Music and Pervasive Technologies Institute (PTI) Team includes School of Library and Information Science (SLIS), Cognitive Science, School of Music and Pervasive Technologies Institute (PTI) Have developed MPI-based feature extraction algorithm, SVM classification, vector space similarity analysis, some preliminary visualization. Have developed MPI-based feature extraction algorithm, SVM classification, vector space similarity analysis, some preliminary visualization. Wish list includes graphical workflow, job submission portal, use in MIR classes Wish list includes graphical workflow, job submission portal, use in MIR classes

19 PFASC Philosophy and Methodology Provide an end-to-end framework for MIR, from workflow to visualization Provide an end-to-end framework for MIR, from workflow to visualization Recognize temporal context as an critical element of audio and a necessary part of feature extraction Recognize temporal context as an critical element of audio and a necessary part of feature extraction Simple concept, simple implementation, one highly configurable algorithm for feature extraction Simple concept, simple implementation, one highly configurable algorithm for feature extraction Dynamic combination and tuning of results from multiple runs, user controlled weighting Dynamic combination and tuning of results from multiple runs, user controlled weighting Make good use of available cyberinfrastructure Make good use of available cyberinfrastructure Support education in MIR Support education in MIR

20 PFASC Feature Extraction Example Summary of 450 files classified by genre, showing most prominent frequencies across spectrum Summary of 450 files classified by genre, showing most prominent frequencies across spectrum

21 PFASC Similarity Matrix Example Hip Hop FolkRock Folk Rock Audio file summarized as a vector of feature values, similarity calculated between vectors Audio file summarized as a vector of feature values, similarity calculated between vectors Value is between 0.0 and 1.0, 0.0 = no commonality, 1.0 = files are identical Value is between 0.0 and 1.0, 0.0 = no commonality, 1.0 = files are identical In the above example, same genre files had similarity scores times higher than different genre files In the above example, same genre files had similarity scores times higher than different genre files

22 Classification vs. Clustering Most work in MIR involves classification, e.g. genre classification, an exercise that may be arbitrary and limited in value Most work in MIR involves classification, e.g. genre classification, an exercise that may be arbitrary and limited in value Calculating similarity values among all songs in a library may be more practical for music discovery, playlist generation, grouping by combinations of selected features Calculating similarity values among all songs in a library may be more practical for music discovery, playlist generation, grouping by combinations of selected features Calculating similarity is MUCH more computationally intensive than categorization, comparing all songs in a library of 20,000 files requires ~200 million comparisons Calculating similarity is MUCH more computationally intensive than categorization, comparing all songs in a library of 20,000 files requires ~200 million comparisons

23 Using Condor for Similarity Analysis Good fit for IU Condor resources, a very large number of short duration jobs Good fit for IU Condor resources, a very large number of short duration jobs Jobs are independent, can be restarted and run in any order Jobs are independent, can be restarted and run in any order Large number of available machines provides great wall clock performance advantage over IU supercomputers Large number of available machines provides great wall clock performance advantage over IU supercomputers

24 PFASC Performance and Resources A recent run of 450 jobs completed in 16 minutes. Time to run in serial on a desktop machine would have been about 19 hours A recent run of 450 jobs completed in 16 minutes. Time to run in serial on a desktop machine would have been about 19 hours Largest run to date contained 3,245 files, over 5 million song-to-song comparisons, completed in less than eight hours, would have been over 11 days on a desktop Largest run to date contained 3,245 files, over 5 million song-to-song comparisons, completed in less than eight hours, would have been over 11 days on a desktop Queue wait time for 450 processors on IU’s Big Red is typically several days, for processors it would be up to a month Queue wait time for 450 processors on IU’s Big Red is typically several days, for processors it would be up to a month

25 Porting to Windows

26 Visualizing Results

27

28 PFASC Contributors Scott McCaulay (Project Lead) Scott McCaulay (Project Lead) Ray Sheppard (MPI Programming) Ray Sheppard (MPI Programming) Eric Wernert (Visualization) Eric Wernert (Visualization) Joe Rinkovsky (Condor) Joe Rinkovsky (Condor) Steve Simms (Storage & Workflow) Steve Simms (Storage & Workflow) Kiduk Yang (Information Retrieval) Kiduk Yang (Information Retrieval) John Walsh (Digital Libraries) John Walsh (Digital Libraries) Eric Isaacson (Music Cognition) Eric Isaacson (Music Cognition)

29 Thank you!


Download ppt "Scott McCaulay Joe Rinkovsky Pervasive Technology Institute Indiana University Music Information Retrieval With Condor."

Similar presentations


Ads by Google