1. University Of Haifa Israel ‘Friends Group’ in Recommender Systems Dr. Yuval Dan-Gur

2. University Of Haifa Israel 2007Chais Yuval Dan-Gur 2 They are already here Recommender Systems They are already here IMDB Ski-EuropeMatchmaker Amazon

3. University Of Haifa Israel 2007Chais Yuval Dan-Gur 3 Black boxes no transparency (Herlocker et al., 2000). Exploration/Exploitation tradeoff range of recommendations Vs. match level (Balabanovic, 1998). Data sparseness and 'first rater‘ number of raters compared to the number of items (Terveen and Hill, 2001). Human taste non-linear and non-stable (Freedman, 1998; Pescovitz, 2000). How well do operational systems do? Recommender Systems How well do operational systems do?

4. University Of Haifa Israel 2007Chais Yuval Dan-Gur 4 Recommendation giving and taking are social, biased processes - also in an automated system. We reviewed possible influence and biases of several social and behavioral phenomena: Social Comparison Theory (Festinger, 1954). Attributing human qualities to computers (Nass and Moon, 2000). Self-Serving Hypothesis in HCI (Nass and Moon, 1998). Accepting advice from a system (Dijkstra, 1998, 1999; Dijkstra, Liebrand and Timminga, 1998; Murphy and Yetmar, 1996). Diffusion of responsibility (Darley and Latane, 1968). ELM - Elaboration Likelihood Model (Petty and Cacioppo, 1986). Subjective value of information (Rafaeli and Raban, 2003). Motivation for this research

5. University Of Haifa Israel 2007Chais Yuval Dan-Gur 5 Typical social aspects do not have an equivalent in recommender systems: The ability to choose recommendation providers. Querying for explanations is impossible in most systems (Herlocker et al., 2000; Preece, 1999; Terveen and Hill, 2001). Filtering recommendation producers based on the item under concern or the situated environment. Is recommendation a social process?

6. University Of Haifa Israel 2007Chais Yuval Dan-Gur 6 Human-computer dyad presents social rules and follows some established behavioral patterns (Moon and Nass, 1998; Nass and Moon, 2000; Wood and Taylor, 1991). We suggest that recommendation seeking is a natural social process that existed since the early days of tribal humanity (Cosley et al., 2003; Jungermann, 1999), and should be examined accordingly even when the process is automated. Should automated recommendation procedure be addressed as a social process?

7. University Of Haifa Israel 2007Chais Yuval Dan-Gur 7 ‘Friends group' describes a sub-group of ‘neighbors group’ that their characteristics are purposefully selected. ‘Friends group' differs from 'neighbors group‘: The user is involved in forming the recommending group rather than relying upon an automatic procedure. The user can choose the characteristics required for a recommendation provider to be included in the 'friends group'. Number of recommenders may only decrease, leading to recommendations that may be OBEJECTIVEY – poorer. The concept emerged mainly from the "Social Comparison Theory". ‘Friends Group’?

8. University Of Haifa Israel 2007Chais Yuval Dan-Gur 8 1.Will users prefer to assume wider control over the recommendation process or to accept it as a "computerized oracle"? 2.Does the attitude of the recommendation seeker towards an advising group obey social rules (specifically the "social comparison" process), even when the user is aware that recommendations are processed and generated by a computerized system? 3.What are the characteristics of the 'friends' selected by the recommendation seeker to participate in his/her advising group when given an option to choose? Research Questions

9. University Of Haifa Israel 2007Chais Yuval Dan-Gur 9 1.H1: Recommendation seekers will prefer to use controlled 'friends groups' over automatically, machine-generated 'neighbors groups'. 2.H2: Recommendations produced by user- controlled 'friends groups' will be more accepted and complied with by recommendation seekers than those produced by 'neighbors groups'. 3.H3: Recommendation seekers will choose personally-similar 'friends' for their advising group. Research Hypotheses

10. University Of Haifa Israel 2007Chais Yuval Dan-Gur 10 System available at http://qsia.org.http://qsia.org QSIA is a collaborative system for collection, management, sharing and assignment of knowledge items for learning. The system consists of various modules that allow the creation and editing of learning items, conducting online educational tasks and recommendation module that assists users in filtering relevant information. QSIA supports both user-controlled ‘Friends’ advising group and auto-formed ‘Neighbors’ group. Research Tool - QSIA

11. University Of Haifa Israel 2007Chais Yuval Dan-Gur 11 User's involvement in the formation of the advising group. Immediate usage of the "liked" recommended items in the same system. Applying recommender technology to knowledge items for distance learning - not "natural" for recommender systems. Research Tool - QSIA Research Tool - QSIA - continued

12. University Of Haifa Israel 2007Chais Yuval Dan-Gur 12 User has to choose Recommendation seekingSelf Browsing Friends User chooses characteristics of Friends Recommendation list F Acceptance of items Acceptance/Rejection Neighbors Recommendation list N Acceptance of items Acceptance/Rejection Interacting with QSIA - Conceptual Model H1H1 H3H3 H2H2

13. University Of Haifa Israel 2007Chais Yuval Dan-Gur 13 ‘Friends Group’ Characteristics in QSIA

14. University Of Haifa Israel 2007Chais Yuval Dan-Gur 14 Period of the field study - 2002 to 2004. Number of users (teachers and students) – approximately 3000, most of them students. Number of items - 10,000. Served item-requests - 31,000. Mainly by self- browsing. Item rankings – 3000, evaluated by around 300 users. Study groups – 183. Recommendations seeking data (either friends or neighbors), includes 895 requests (818 by students and 77 by teachers) generated by 108 active users. Research - Data

15. 2007ChaisYuval Dan-Gur15 Recom. Seeker Characteristics of recommendation seeker in QSIA: Group membership. Grade level. Status/role. Decision on Source of Recom.: Friends or Neighbors. SoR=F g SoR=N g User choice of Friends' characteristic s. Recom. engine with friends' algorithm. Recom. engine with neighbors' algorithm. Recom. Seeker Recom. list when SoR= N g Recom. list when SoR= F g Recom. list when SoR= N g Accepted Rejected Items used (in various forms and actions) by the user in QSIA system. Whole population of users, their profiles, and a database of users' evaluations of items. H 1 is examined here H 3 is examined here H 2 is examined here

16. University Of Haifa Israel 2007Chais Yuval Dan-Gur 16 Results - 1 The "Depth of Use" (DoU j ), a variable that represents the maximum number of times that the j th user had asked for recommendations.

17. University Of Haifa Israel 2007Chais Yuval Dan-Gur 17 Results - 2 Instances that consist of three or more data scores (DoU≥3)

18. University Of Haifa Israel 2007Chais Yuval Dan-Gur 18 Results - 3 Acceptance of a recommended item is counted whenever the recommendation seeker "applies the item" or "simulates an item". Rejection of a recommended item is counted whenever the recommendation seeker restricts the consecutive action to only "viewing the item".

19. University Of Haifa Israel 2007Chais Yuval Dan-Gur 19 Results - 4 same users both sources Acceptances and rejections for the same users who asked for recommendations from both sources Acceptance Ratios According to SoR SoR=N g SoR=F g 377264 Number of records 19 Number of users 0.30.29 Std. Dev. 56%70% Mean acceptance ratio 14% Mean difference 0.050 α (Wilcoxon, one tailed)

20. University Of Haifa Israel 2007Chais Yuval Dan-Gur 20 Results - 5 one source Users who asked for recommendations only from one source, either 'friends groups' or 'neighbors groups'

21. University Of Haifa Israel 2007Chais Yuval Dan-Gur 21 Results - 5 Results - 5 - continued We identified 36 items that have the following characteristics: These items were recommended to users by both 'friends groups' and 'neighbors groups'. Users acted upon these recommendations, either by rejection or by acceptance in both scenarios of SoR's. Our intention is to test the difference in acceptance and rejection ratios of the same items, when they were offered to users by 'friends groups' and 'neighbors groups', based on 394 usage records.

22. University Of Haifa Israel 2007Chais Yuval Dan-Gur 22 Results - 5 Results - 5 - continued

23. University Of Haifa Israel 2007Chais Yuval Dan-Gur 23 Results - 6 FRI (Frequently Recommended Items) = Items recommended approximately 5 times more frequent than the average recommendations number of an item.

24. University Of Haifa Israel 2007Chais Yuval Dan-Gur 24 Results - 7 Friends characteristic = Group

25. University Of Haifa Israel 2007Chais Yuval Dan-Gur 25 Results - 8 Friends characteristic = Role

26. University Of Haifa Israel 2007Chais Yuval Dan-Gur 26 First Hypothesis: Recommendation seekers will prefer to use controlled 'friends groups' over automatically, machine-generated 'neighbors groups‘ Supported: Users do develop a tendency to choose 'friends group' recommendations. The probability of this tendency increases in as more recommendations are sought. Also, "experienced" users choose 'friends groups' significantly more than "new" users. Main Findings -First Hypothesis

27. University Of Haifa Israel 2007Chais Yuval Dan-Gur 27 Second Hypothesis: Recommendations produced by user-controlled 'friends groups' will be more accepted and complied with by recommendation seekers than those produced by 'neighbors groups‘. Supported: We found a positive significant difference in the mean ratio of acceptance when we tested all users who had received and acted upon recommendations from both sources ('friends group' and 'neighbors group'). Main Findings -Second Hypothesis

28. University Of Haifa Israel 2007Chais Yuval Dan-Gur 28 There was a higher positive significant difference in the mean acceptance ratios (24%, α = 0.037) for users who received recommendations from only one source (either 'friends group' or 'neighbors group'). Also, when the same items were offered to users from both sources (N=36), the acceptance level was 6.5% higher when the recommendations were offered by 'friends groups' (P-value= 0.28). For the most frequently recommended items that were recommended by both 'friends group' and 'neighbors group', the acceptance ratio was 15.2% higher (N=4, α = 0.034) for the same items when they were recommended by 'friends groups'. Main Findings -Second Hypothesis Main Findings -Second Hypothesis - continued

29. University Of Haifa Israel 2007Chais Yuval Dan-Gur 29 Third Hypothesis: Recommendation seekers will choose personally-similar 'friends' for their advising group. Partially supported: There were many missing values in this part of our dataset: in almost half the records users made a group choice, in another quarter of the cases they made a role choice, and in only approximately 6% of the cases did users make a grade choice. Main Findings -Third Hypothesis

30. University Of Haifa Israel 2007Chais Yuval Dan-Gur 30 We analyzed the characteristics independently and found that in accordance with our hypothesis, users significantly prefer their own group over other groups (76.6%, α<0.0001). Role: students asked for teachers' recommendations 43% more than for students' recommendations (α<0.0001). We explained it by an alternative hypothesis stating that students choose teachers' recommendations because of their role authority and knowledge expertise (Wyeth and Watson, 1971). Main Findings -Third Hypothesis Main Findings -Third Hypothesis - continued

31. University Of Haifa Israel 2007Chais Yuval Dan-Gur 31 users increasingly sought recommendations from 'friends groups' Over time, users increasingly sought recommendations from 'friends groups' and the probability to do so increased with higher use of recommendations. acceptance level of recommendations was higher when they asked for 'friends groups' recommendations Users' acceptance level of recommendations was higher when they asked for 'friends groups' recommendations. In addition, the same items were more readily accepted when offered to the user by the 'friends group' than when offered by the 'neighbors group'. The difference in acceptance was higher for items that were recommended frequently. own group choice was the most important characteristic for users We had insufficient data for some of the planned statistical tests of the third hypothesis. Nevertheless, we concluded that own group choice was the most important characteristic for users to assign to their advising group members. So, what is new?

32. University Of Haifa Israel 2007Chais Yuval Dan-Gur 32 Absence of a comparable field study. We did not collect self-report of user motivations for their actions - we only collected data on the dependent variables and deduced the users' behavior. The participating populations, except in one case, were homogeneous: students and teachers of academic institutions. The characteristics of the advising group that were possible for the recommendation seeker to control were limited: group, grade level and role. QSIA - we did not trace similar systems as a benchmark for its unique characteristics. Lack of data - the algorithm of 'friends group' lowers the number of members of the "advising group" because of the characteristics' constraints, and thus, may produce recommendations of less objective quality. Weaknesses and Limitations

33. University Of Haifa Israel 2007Chais Yuval Dan-Gur 33 acceptance likelihood group users' involvement The research demonstrates that acceptance likelihood among users of social collaborative systems of the recommendations depends on the group that made the recommendations and on the users' involvement in the formation of that group. perceived, subjective quality The main new aspect of our findings is the relationship between the perceived, subjective quality of the recommendation, and the user's involvement in the formation of the advising group. Summary

34. University Of Haifa Israel Thank you! Questions?

35. University Of Haifa Israel 2007Chais Yuval Dan-Gur 35 Distribution of times items were recommended

36. 2007ChaisYuval Dan-Gur36 Summary of variables, data and analysis methods

37. 2007ChaisYuval Dan-Gur37 Longitudinal Analysis In all models, positive values of β i suggest an increase in the probability of asking for 'friends group' recommendations as the instance index rises. The logistic regression coefficients are also significant. All models have Goodness-of-Fit of more than 95% according to the accepted Pearson Chi-Square measure.

38. University Of Haifa Israel 2007Chais Yuval Dan-Gur 38 The core task of a recommender system is to recommend, in a personalized manner, interesting and valuable items and help users make appropriate choices from a large number of alternatives, without sufficient personal experience or awareness of the items' alternatives (Grasso et al., 2000; Oard and Kim, 1998; Resnick and Varian, 1997). What is their core task? Recommender Systems What is their core task?

39. University Of Haifa Israel 2007Chais Yuval Dan-Gur 39 Users rank items – either explicit or implicit. The recommendation seeker is matched with a ‘neighbors-group’: users who’s rankings are highly correlated with his/hers. The “weighted opinion” of the neighbors group is assumed to be a good prediction of the opinion of the recommendation seeker towards an unseen item. New items, new users and new rankings (of recommendations providers or recommendation seeker) can cause recommendations to vary. How do they work? Recommender Systems How do they work?

40. University Of Haifa Israel 2007Chais Yuval Dan-Gur 40 They introduce a collaborative aspect of to the search: The opinions of others in the search space become a central issue, replacing previous foundations such as the wisdom and excellence of algorithm developers (Rafaeli et al., 2005). Requirements from users are simplified and more task-oriented: not exact queries, but only rating and evaluating items to the degree that they are relevant for them and worth consuming. Why are they interesting? Recommender Systems Why are they interesting?

41. University Of Haifa Israel 2007Chais Yuval Dan-Gur 41 The requirements from the item domain are narrowed down to just being ratable. No machine-parsed items are necessary (Shardanand and Maes, 1995; Traill et al., 1997). This simplicity opens the domain to "taste products", whose relevance is completely subjective, sometimes biased and almost can not be evaluated by others (for example: experts) for distinct users. Why are they interesting? Recommender Systems Why are they interesting? - continued