1. A Case Study for Adaptive News Systems with Open User Model
NewsMe:
A Case Study for Adaptive News Systems with Open User Model
Preliminary Examination Paper 2007
Chirayu Wongchokprasitti
IS PhD Student
School of Information Sciences

2. NewsMe

3. NewsMe Overview Personalized News Access System
Feed the news that response to the user’s interest
82 RSS news feeds, 21 sources
8 News Topics
Ranking the news
Open User Model based system

4. NewsMe Interface 4 News Sections Recent News Recommended News
My Profile
News History

5. User Feedback Method Add a news item to Tracked News
Add a news item to Blacklist

6. User Model Manipulation
Update rating of news in user model

7. User Model Manipulation (Con’t)
List all history of viewed news
Update rating of news in user model

8. Learning User Models for News Access
The system uses a machine learning approach to build a simple model of each user’s interests.
A similarity-based method achieves the balance of learning and adapts quickly to change interests while avoiding brittleness.

9. Learning User Models for News Access (cont.)
The purpose of the user model
First, it should contain information about recently read events, so that stories which belong to the same thread can be identified.
To allow for identification of news that user already knows.
The k-nearest-neighbor algorithm (kNN) is used to achieve the desired functionality.
Convert news contents to tf-idf vectors (term-frequency/inverse-document-frequency).
Use the cosine similarity measure to quantify the similarity of two vectors.

10. Learning User Models for News Access (cont.)
Decay Function
Freshness of news content is our issue.
Freshness should decay exponentially day by day.
Freshness of news remains a half after fed 7 days.
is the initial freshness of news content.
is a decay instance, which its value is around

11. Study Design
20 Participants
Assign to be Information Analysts
2 News Topic: US and Business
2 Sessions, 3 stages per session
Retrieved News: Nov 28th – Dec 12th, 2006
Google Notebook extension (

12. Implicit VS Explicit Feedback
Implicit feedback
Assuming every news user read is a tracked news
Explicit feedback
Users add news items to their user model
Tracked news as Positive sample
Blacklist News as Negative sample

13. Performance hypotheses are:
H1: The open model system with user profile manipulation by users performs better than the open model system without them,
H1.1: The open model system with explicit feedback generates results with better performance, and,
H1.2: Users with explicit feedback system demonstrate higher task performance.

14. Hypotheses (Con’t) User Perspective hypotheses are:
H2: Users prefer the user profile manipulation features in the open model system,
H2.1: Users appreciate better in the system with explicit feedback, and,
H2.2: Users appreciate the ability to control their profiles.

15. Preliminary Results The Ground Truth System Performance Analysis
User Performance Analysis
User Feedback Analysis

16. The Ground Truth F-measure defines as follows:
Summary of news items in the study

17. System Performance Analysis

18. System Precision @ First Screen

19. System 60

20. System 100

21. News Items Manipulation vs. System Performance (Stage 2)

22. Tracked News Blacklist (Stage 2)

23. Tracked News History (Stage 2)

24. Blacklist Tracked News (Stage 2)

25. News Items Manipulation vs. System Performance (Stage 3)

26. Tracked News Blacklist (Stage 3)

27. Tracked News History (Stage 3)

28. Blacklist Tracked News (Stage 3)

29. User Performance Analysis

30. User Precision

31. User Avg. Rank of Selected Items

32. User Feedback Analysis
A two-way ANOVA was performed on the questionnaire data to examine significant differences in user answers by system and by stage.
On the question 3, subjects indicated they trusted in system’s ability to find useful information for the US topic versus the Business topic in overall (p-value = 0.017).
On the question 7, subjects indicated My Profile helps them to understand how the system finds useful news items for the US topic versus the Business topic in overall (p-value = 0.013).

33. Future Work
Open Model with explicit feedback did not outperform the baseline.
The experiment indicates that without caution, user model manipulation not only benefit the performance but lower the output.
Binary rating might not be a suitable way.
Fuzzy rating is a good way to study further.

34. Q & A