Presentation on theme: "Music Recommendation by Unified Hypergraph: Music Recommendation by Unified Hypergraph: Combining Social Media Information and Music Content Jiajun Bu,"— Presentation transcript:
Music Recommendation by Unified Hypergraph: Music Recommendation by Unified Hypergraph: Combining Social Media Information and Music Content Jiajun Bu, Shulong Tan, Chun Chen, Can Wang, Hao Wu, Lijun Zhang and Xiaofei He Zhejiang University 1
Multi-type Media Fusion Content analysis – text – Image – Audio – Video – …… Social analysis – Friendship – Interest group – Resource collection – Tag – …… 2 Hypergraph
Outlines Music Recommendation Social media information Unified Hypergraph Model Music Recommendation on Hypergraph (MRH) Experimental results 3
Music Recommendation 4 We have huge amount of music available in music social communities It is difficult to find music we would potentially like Music Recommendation is needed! Recommended music by the Last.fm.
Traditional Music Recommendation Traditional music recommendation methods only utilize limited kinds of social information Collaborative Filtering (CF) only uses rating information Acoustic-based method only utilizes acoustic features Hybrid method just combines these two 5
Music Social Community 6 [www.pandora.com] Actions which users can do on resources Social activities between users
Users can also make friends. Users can bookmark resources by tags. Users can listen to music. These are music tracks this user likes best. Introduction to Last.fm 7 Users can join in groups
Social Media Information in Last.fm 8 Friendships Memberships Tagging relations Listening relations Inclusion relations
Social Media Information The rich social media information is valuable for music recommendation. To build the users’ preference profiles. To predict users’ interests from their friends. To recommend music tracks by albums or artists. …… 9
How About Graph Model? Use traditional graph to model social media information but fail to keep high-order relations in social media information 10 (u 1, t 1, r 1 ) (u 1, t 2, r 2 ) (u 2, t 2, r 1 ) It is unclear whether u 2 bookmarks r 1, r 2, or both.
Unified Hypergraph Model Using a unified hypergraph to model multi-type objects and the high-order relations Each edge in a hypergraph, called a hyperedge, is an arbitrary non-empty subset of the vertex set Modeling each high-order relation by a hyperedge, so hypergraphs can capture high-order relations naturally 11 (u 1, t 1, r 1 ) (u 1, t 2, r 2 ) (u 2, t 2, r 1 ) The high-order relations among the three types of objects can be naturally represented as triples.
Unified Hypergraph Construction 12 The six types of objects form the vertex set of the unified hypergraph. Each type of relations corresponds to a certain type of hyperedges in the unified hypergraph. tag album
Hyperedges Construction Details 13 :a hyperedge corresponding to each pairwise friendship :a hyperedge corresponding to each group : a hyperedge for each user-track listening relation :a hyperedge corresponding to each tagging relation :a hyperedge for each album or artist : a hyperedge for each track-track similarity relation
Ranking on Unified Hypergraph 14 Setting a user as the query … Track List Tracks have more strong “hyperpaths” to the query user will get higher ranking scores
Notation A unified hypergraph : Vertex-hyperedge incidence matrix 15
Notation-2 : the degree of a hyperedge is the number of vertices in the hyperedge : : the degree of a vertex is the weight sum of all hyperedges the vertex belongs to: D v, D e and W : diagonal matrices consisting of hyperedge degrees, vertex degrees and hyperedge weights 16
Problem Definition Given some query vertices from, rank the other vertices on the unified hypergraph according to their relevance to the queries. : the ranking score of the i-th object : the vector of ranking scores : the query vector 17
Cost Function Vertices contained in many common hyperedges should have similar ranking scores Obtained ranking scores should be similar to pre-given labels 18 The optimal ranking result is achieved when Q(f) is minimized
Matrix-vector Form 19
Optimal Solution 20 Requiring that the gradient of Q(f) vanish gives the following this equation We define Note: all the matrices are highly sparse!
Music recommendation on Hypergraph (MRH) The offline training phase: Constructing matrix H and W Computing matrix D v and D e Calculating, where The online recommendation phase: Building the query vector y Computing the ranking results f* Recommending top tracks which not listened 21
General Ranking Framework 22 Setting a user as the query … User List … Group List … Tag List … Track List … Album List … Artist List For friend recommendation For artist recommendation For group recommendation For album recommendation For topic recommendation For music recommendation
Personalized Tag Recommendation 23 Setting a user and an resource as the queries … Tag List Personalized Tag recommendation for the target user and resource
Objects and Relations in Our Dataset 24 Objects Relations
Compared Algorithms AlgorithmsInformation Used User-based Collaborative Filtering (CF)R3R3 Acoustic-based music recommendation (AB)R 3, R 9 Ranking on Unified Graph (RUG)R 1, R 2, R 3, R 4, R 5, R 6, R 7, R 8, R 9 Our proposed music recommendation on Hypergraph method (MRH) MRH-hybridR 3, R 9 MRH-socialR 1, R 2, R 3, R 4, R 5, R 6, R 7, R 8 MRHR 1, R 2, R 3, R 4, R 5, R 6, R 7, R 8, R 9 25 R 1 : friendship relations R 2 : membership relations R 3 : listening relations R 4 : tagging relations on tracks R 5 : tagging relations on albums R 6 : tagging relations on artists R 7 : track-album inclusion relations R 8 : album-artist inclusion relations R 9 : similarities between tracks
Performance Comparison 26 It is clear that our proposed algorithm significantly outperforms the other recommendation algorithms Comparison of recommendation algorithms in terms of MAP and F1. Comparison of recommendation algorithms in terms of NDCG.
CF algorithm does not work well too. This is probably because the user-track matrix in our data set is highly sparse Acoustic-based (AB) method works worst. That is because acoustic- based method incurs the semantic gap and similarities based on acoustic content are not always consistent with human knowledge The superiority of MRH over RUG indicates that the hypergraph is indeed a better choice for modeling complex relations in social media information Our proposed method alleviates these problems. MRH-hybrid only uses similarity relations among music tracks and listening relations, but it works much better than AB and CF Precision-Recall Curves 27 Comparing to MRH-social, MRH uses similarity relations among tracks additionally. We find that using this acoustic information can improve the recommendation result, especially when we only care top ranking music tracks.
The baseline is MRH only using listening relations Social Information Contribution 28 Comparison of MRH on different subsets of social media information in terms of MAP and F1. MRH using listening relations and inclusion relations MRH using listening relations and tagging relations MRH using listening relations, and social relations There is a little improvement at lower ranks obtained by social relations. Intuitively, the users’ tastes may be inferred from friendship and membership relations. Using inclusion relations among resources, we can recommend music tracks in the same or similar albums, as well as the tracks performed by the same or similar artists. So the performance is improved greatly. Tagging relations do not improve the performance. That is because there is a strong correlation between listening relations and tagging relations, and thus the usage of tagging relations is limited
System of a Down War? An Example 29 No. 793 Top 5 Recommended Tracks for No.793 Dirty WindowDeer DanceSpit It OutKnow System of a Down Slipknot Metallica System of a Down From: The reason these three tracks are recommended is that user No. 793 likes music come from System of a Down best. User No. 793 joins in the groups about metal and named Slipknot. Users in these groups are fans of Metallica (one of the four most popular heavy metal band. ) and Slipknot
Conclusion We use the unified hypergraph model to fuse multi-type media, includes multi-type social media information and music content. Social media information is very useful for music recommendation. Hypergraphs can accurately capture the high- order relations among various types of objects. 30