1. Context-Aware Mobile Music Recommendation for Daily Activities
Xinxi Wang, David Rosenblum, Ye Wang
School of Computing, National University of Singapore

2. A simple question
Before we start, I would like to ask you a simple question.
Do you listen to the same or different music when running or sleeping?
After they raise up their hands:
Thanks.
Do you prefer the same or different music when running or sleeping?

3. Motivation - short-term needs
Users’ short-term music information needs are influenced by users’ activities [A. C. North 2004]
Traditional recommender systems model user long-term preferences only
Running
Sleeping
Girlfriend
Sleepsong
Girlfriend
Most of you prefer different music for different activities. This has been studied by some music psychologist.
From this study, we know users’ short-term music information needs are influenced by users’ activities.
However, most traditional recommender systems are based on collaborative filtering. And they model user long-term preferences only. They don’t know what a user is doing. So they can make mistakes.
For example, if I’am this user and I’m going to sleep. According to CF, Girlfriend will be recommended to me. This is obviously a mistake, and I don’t think I can fall asleep with this song.
This song is generally good, but I’m going to sleep. It’s too noisy!
Collaborative filtering (CF) . ..

4. Motivation - the cold-start problem
How to recommend
this new song?
new song problem
How to recommend
songs to this new user?
new user problem
Collaborative Filtering (CF) cannot handle both
Content-based filtering cannot solve the new user problem

5. The Main Idea
Our system detects users’ daily activities in real-time and recommends suitable music automatically
Running
Sleeping
The main idea of our system is like this:
Our system detects users’ daily activities in real-time and recommends suitable music automatically
Specifically, we first predict every song’s suitable activities by examine the audio features of very song. For example, we predict this song is suitable for running, but this song is not suitable for running. We do the same prediction for other activities.
On the other hand, we predict a user’s current activity by examine the low level sensor data collected from the mobile phone. Then we use the predicted activity to lookup this table and find out the best song and then recommend it to the user.
Since this process is identical to every user, to make the recommendation adaptive and personalized, we also utilize users’ feedback.
Let’s do a short demo of our system. … … … …
Audio content analysis
Sensor based activity detection
Personalization and adaptation

6. System Architecture Backend Frontend Music audio feature extraction
Binary classifiers
(Adaboost)
Running
Walking
Sleeping
Music
Database
Working
Studying
Shopping
Classification results
Recommendation
ACACF
Probabilistic Graphical Model
Let’s see how our system works in detail.
Our system is based heavily on machine learning. It’s consist of two parts: the backend and the frontend. The backend is built in a powerful cluster. And the frontend is all inside a mobile phone.
In the backend, we have a large music database. We first extract music audio features for every song, and then we use a set of binary classifiers to predict whether a song is suitable for an activities or not. The classification results will then be transferred to the frontend via wireless network. On the other hand, we collect features extracted from the low level sensor data, and also the user feedback. Then we use a probabilistic graphical model to integrate the three kinds of information together to provide accurate recommendation.
Frontend
User feedback
Sensor signal features
feature extraction

7. Prototype interface Activities Listened completely Skipped Ranked
songs list
Skipped
Playback
Controls
automatic mode
manual mode

8. ACACF – Adaptive Context-Aware Content Filtering
Given the sensor feature f,
a song s is scored as:
Sensor-Context Model
Music-Context Model

9. Music-Context Model Initialization
Different people have agreement on suitable music for an activity.
Initialization. Prior beta(a, b) is initialized from music content analysis results.

10. Music-Context Model Adaptation by Approximate Inference
Approximate prior update:
User preference update:

11. Sensor-Context Model Selection
Six models are compared based on three criteria: (1) Energy consumption (2) Accuracy; (3) Incremental learning.
Energy consumption and incremental learning

12. Sensor-Context Model Selection (cont)
Accuracy of different models:

13. Sensor-Context Model Learning and Updating
Naïve Bayes:
Training and incremental training by MLE:

14. Accuracy of music content analysis
Retrieval performance

15. Recommendation Accuracy

16. Effectiveness of Adaptation

17. User needs study
With existing technologies, their short-term needs cannot be satisfied well.

18. Usability Study

19. Conclusion
The first context-aware mobile music recommendation system for daily activities
It satisfies users’ short-term needs better
A solution the cold-start problem
Unified probabilistic model

20. Future Work Let more people use it
Exploration/exploitation tradeoff using reinforcement learning
Incorporate collaborative filtering into the system

21. Thank You!

22. References
A. C. North, D. J. Hargreaves, and J. J. Hargreaves, “Uses of Music in Everyday Life,” Music Perception: An Interdisciplinary Journal, vol. 22, no. 1, 2004.
A. I. Schein, A. Popescul, L. H. Ungar, and D. M. Pennock, “Methods and metrics for cold-start recommendations,” in SIGIR, 2002.