MusicSense: Contextual Music Recommendation using Emotional Allocation Modeling Rui Cai, Chao Zhang, Chong Wang, Lei Zhang, and Wei-Ying Ma Proceedings of the 15th international conference on Multimedia, MULTIMEDIA '07 Keywords: MusicSense, contextual music recommendation, emotional allocation modeling, moods Short papers poster session 2 - arts, content, applications

2 劇情 Introduction (1/2) automatically recommendation now has becoming an important role in music sales –Last.fm - collaborative filtering –Pandora - content similarity passive mode – queries -> recommendation active way –Google AdSense deliver contextual advertises matched to websites’ content

3 劇情 Introduction (2/2) MusicSense –contextual music recommendation –automatically deliver music pieces (or their thumbnails) which are relevant to the context of a Web page when users read it –measure the context relevance between music songs and Web pages –emotion as the bridge for such a relevance matching –music is all about conveying composers’ emotions –lots of Web pages such as Weblogs also express sentiments of writers

4 小小的 paper review –Lu et al. utilized Gaussian mixture models to classify music songs into four emotion categories, using acoustic content features like intensity, timbre, and rhythm –Cui et al. have comparatively studied various supervised classifiers to classify online product reviews into positive and negative opinions –Leshed et al. tried to categorize Weblogs into ten most frequently used moods with SVM most works can handle only a few mood categories difficult to collect enough high quality training data for all possible emotions cross-modal emotion mapping is still an open problem

5 MusicSense 怎麼辦 A probabilistic modeling called Emotional Allocation to characterize songs and Web pages as distributions in a common emotion space –leverages both the statistics from a large-scale Web corpus and guidance from psychological studies –keeps the inference ability of generative models relevance matching –songs and Web documents are respectively represented by a collection of word terms, based on which their emotion distribution parameters are optimized in an iterative way –Kullback-Liebler divergence

6 Framework

7 Three steps –emotional allocation modeling –Web-based music description generation and Web document analysis –probability inference and relevance matching refer to knowledge of psychology, to get a relatively complete and well-structured emotion vocabulary (40 basic emotion vocabulary) –R. Cowie, E. Douglas-Cowie, and et al. Emotion recognition in human computer interaction. IEEE Signal Processing Magazine, 18(1):33–80, 2001

8 content-based music analysis lyrics and reviews to describe the semantic of a song use search engines to retrieve more information from the Web to characterize songs –we just prepare two queries in the form of “title + lyrics” and “title + reviews”, respectively –first page returned by the first query, and the top 20 pages returned by the second query –retrieved pages are merged as a virtual document after removing HTML tags and stop words –tf × idf is computed as its weight –top N terms with the highest weights are selected out as the description (N = 100 the most informative terms) Web documents – Blog – 100 highest tf × idf are kept as salient words

9 Emotional Allocation Modeling Assume that –given a language and its vocabulary –different emotions should have different distributions over the terms in this vocabulary –the frequencies of a term under different emotions are also different Given a collection of terms (e.g. a document), –we can suppose it is generated by sampling a mixture of various emotions, –as terms in this collection can be considered as controlled by different emotions. The parameters of such a sampling can be computed in a maximum likelihood manner In such a way, a term collection would have a certain allocation of emotions, in form of a probability distribution.

10 Model LDA 模型 k 種 emotion, m 種字 latent Dirichlet allocation

11 β

12 Relevance Matching Distance 越短越好

13 實驗 Evaluation collected 100 songs and 50 Weblogs Five college students were then invited to label the ground truth –Each labeler was asked to listen each song and then tag it with one or more words from the forty emotions in the Basic English Emotion Vocabulary [8]. The Weblog posts were also tagged in the same way. –For each Weblog, each labeler was asked to find out 3 ∼ 5 songs, which are the most ideal candidates in his (her) mind for listening when reading that blog post, from all the 100 songs.

14 關於 Emotion Allocation 的結果 0.71 on this song. The average correlation coefficient over the whole 100 songs is about 0.48

15 關於 Music Recommendation 的結果 Merge first - on average, there are around 5.75 such suggestions for each post top N ranked songs are selected as recommendations

16 Conclusion music recommendation (MusicSense) –new probabilistic model called Emotional Allocation Modeling –each song (or a Weblog) is generated with a distribution over the mixture of emotions –Emotion acts as a bridge for the relevance matching between blogs and songs –Preliminary experiments Future –deeply investigate some current implementation details to improve the performance –also try to utilize more information besides emotion to measure the relevance between music and documents –carry out more user study to design an idea UI to deliver the contextual music recommendation