1. Social Role-Aware Emotion Contagion in Image Social Networks
Yang Yang, Jia Jia, Boya Wu, and Jie Tang
Department of Computer Science and Technology
Tsinghua University
As the title suggests, we take image social networks as the basis of our study

2. Image social network (e. g
Image social network (e.g., Flickr) users post photos, which express their emotional statuses.
Taking Flickr as an example, users are allowed to publish photos.
Comparing with text, image is a more subjective media, which to some extent express users’ emotional status
The Spiderman, for instance, must be very happy when taking this photo.

3. Does Emotion contagion exist in image social networks?
Users are connected …
Does Emotion contagion exist in image social networks?
At the meanwhile, users are connected with each other in image social networks. Thus when Spiderman publishes the photo, it will be exposed to his neighbors and may influence them to publish photos to express different emotions, which in turn influence their own friends, and the influence may further spread.
This process is called as emotion contagion, which describes how users influence each other’s emotional statuses and how the influence propagates through the network.
A natural question arise here is: does emotion contagion exist in image social networks?
#Emotion contagion is a process in which a person or group influences the emotions or behavior of another person or group.
Emotion Contagion: The cascade of users’ emotional statuses influence each other

4. Social Roles of Users Bad boys League of heroes
Before we answer the question, we first take a look at the underlying social network where the emotional contagion may occur.
We see that the network actually consists of two different communities. One is the league of super heroes, and another one is full of bad boys.
There are two types of users worth to mention.
One is opinion leaders, who take the central position in each community. For example, Spiderman, who connects other heroes in the league, is definitely an opinion leader.
Opinion leaders: users taking central positions in communities

5. Will social roles influence emotion contagions?
Social Roles of Users
Bad boys
Will social roles influence emotion contagions?
League of heroes
Another kind of users we are interested in are structural hole spanners, who bridge otherwise disconnected communities, like Mr. Simpsons, who connects heroes and bad boys.
So we call these two kinds of users as playing different social roles and the second question we aim to answer in this work is, if the emotion contagion exists, will users with different social roles have different behavior patterns during emotion contagion, in other words, will social roles influence emotion contagions？
Structural hole spanners: users bridge otherwise disconnected communities

6. Predicting Users’ Emotional Status
Input: An image social network G=<V, M, E, R>, where V is a set of users, M is a set of images, E represents following relationships between users, and each element in R (v, m, t) denotes that user v publishes image m at time t.
We use a matrix Y to denote users’ emotional status, where yvt indicates v’s emotion at time t. yvt {happiness, surprise, anger, disgust, fear, sadness}
Task: Given G, Y, a time stamp t, our goal is to learn
Our ultimate goal in this work is to predict users’ emotional status. So we are given a image social network, which consists of users, images published by these users, following relations between users, and the publishing relations between users and images.
We use a matrix Y to denote users’ emotional status. Specifically, y_{vt} indicates user v’ emotion at time t. And we consider six emotions: happiness, surprise, anger, disgust, fear, and sadness.
Our task is, given the image social network and users’ emotions before time t, we aim to predict users’ emotion at time t
[1] Yang, Y.; Jia, J.; Zhang, S.; Wu, B.; Chen, Q.; Li, J.; Xing, C.; and Tang, J How do your friends on social media disclose your emotions? In AAAI’14, 2014.

7. Treat each individual independently
Related Work
Predicting users’ emotions by jointly modeling images and comments.
Yang, Y.; Jia, J.; Zhang, S.; Wu, B.; Chen, Q.; Li, J.; Xing, C.; and Tang, J How do your friends on social media disclose your emotions? In AAAI’14, 2014.
Predicting users’ emotions in mobile network by considering calling/messaging logs.
Tang, J.; Zhang, Y.; Sun, J.; Rao, J.; Yu, W.; Chen, Y.; and Fong, A. Quantitative study of individual emotional states in social networks. TAC’12, 2012.
Images drive event engagement (e.g., clicking “like” or adding comments) 100 times faster than text on Facebook.
Wang, X.; Jia, J.; Cai, L.; and Tang, J. Modeling emotion influence from images in social networks. IEEE TAFFECT COMPUT’15, 2015.
How to better predict users’ emotions by considering emotion contagions?
However, these methods treat each individual independently and ignore the correlations/influence among them.
Treat each individual independently

8. Three Qs to Answer
Q1: Does emotion contagion exist in image social networks?
Q2: Will social roles influence emotion contagion?
Q3: How to better predict the emotional status of users in social networks by considering emotion contagion?
We conduct several experiments on a Flickr dataset to answer these three questions one by one.

9. Q1: Existence Q1.1: When your friends are happy, will you be happy?
We answer the first question from two aspects. Taking the happiness emotion as an example, we study when my friends are happy, whether I tend to be happy.
The x-axis in the figure indicates the number of friends being happy, and the y-axis indicates the probability that I will be happy.
We see that when the number of a user’s friends being happy increases, the probability rows roughly linear, which implies that the emotional status of a user will be influenced by her friends.

10. Q1: Existence
Influence
Q1.2: When predicting a user’s emotional status, will her friends help?
Historical post logs +
Previous emotion
Image features
User v’s emotional status at time t
Predict
happiness, surprise, anger, disgust, fear, sadness
From another angle, we study when predicting a user’s emotions, whether the emotional status of her friends will help.
Here we compare two different methods, the first one only considers individual features, such as the users’ historical post log, her previous emotions, visual features of images she publishes, and use these features to feed to a machine learning classifier to predict a user’s emotional status.
The second method considers all features the first method includes, plus friends’ emotions as a long vector. +
Friends’ emotions

11. Emotion contagion does occur in image social networks
Q1: Existence
Influence
Q1.2: When predicting a user’s emotional status, will her friends help?
Historical post logs +
Previous emotion
Image features
Emotion contagion does occur in image social networks
For the results, we see that in most cases, friends’ emotions improve the performance, indicates the friends do help in the prediction.
However, exceptions happen in predicting emotions of disgust and fear, which suggests the way we model friend’s emotions here is not precise enough.
We will try to solve this issue latter, but for now, I think we are safe to claim that emotion contagion does occur in image social networks. +
Friends’ emotions

12. Still holds in emotion contagion?
Q2: Social Role
Opinion leaders: 20% of users with largest PageRank scores;
Structural hole spanners: 20% of users with lowest network constraint scores;
Others are remaining as ordinary users.
OL and SH are more influential than ordinary users in information diffusion [Yang’15].
Still holds in emotion contagion?
[1] Y. Yang, J. Tang, C. W.-k. Leung, Y. Sun, Q. Chen, J. Li, and Q. Yang. Rain: Social role-aware information diffusion. In AAAI’15, 2015.

13. Q2: Social Role Happy Fear Happy Fear
X: number of friends with different social roles.
Y: probability being a certain emotion.
Happy
Fear

14. positive emotion delights friends
Q2: Social Role
Happy
Fear
X: number of friends with different social roles.
Y: probability being a certain emotion.
Happy
positive emotion delights friends
Fear
positive emotion tends to delight friends, making them be more happy and less fear

15. Q2: Social Role Happy Fear Happy Fear
X: number of friends with different social roles.
Y: probability being a certain emotion.
Happy
Fear
when there are 1-2 users being negative emotion, their common friends tend to be less happy and more fear

16. “Emotional comfort” phenomena
Q2: Social Role
Happy
Fear
X: number of friends with different social roles.
Y: probability being a certain emotion.
Happy
Fear
“Emotional comfort” phenomena
however, when this number increases, the user will be more happy and less fear
“emotional comfort”: when a user is surrounded by a few friends with negative emotions, the user and her friends will comfort each other and get better mood.

17. Q2: Social Role Happy Fear Happy
X: number of friends with different social roles.
Y: probability being a certain emotion.
Happy
Opinion leaders are more influential on positive emotions
Fear
We also see that opinion leaders are most influential when they are happy, while
ordinary users have more influence when they have negative emotional status (e.g., fear).
Intuitively, opinion leaders and structural hole spanners are not necessarily as close to their friends as ordinary users.
Ordinary users are more influential on negative emotions

18. Q2: Social Role Happy Fear Happy
X: number of friends with different social roles.
Y: probability being a certain emotion.
Happy
Influence of opinion leaders and structural holes change faster than ordinary users.
Fear
Secondly, we observe that, as the number of infected friends grows, infection probabilities of opinion leaders and structural hole spanners change faster than ordinary users.

19. Q3: Model
P(Y|G): Conditional probability of users’ emotional status given input data

20. Q3: Model
P(Y|G)=πg(.) …
g(xvt, yvt): Correlation between v’s emotion and the image she posts at t.

21. h(yut-t’, yvt): Correlation between v’s emotion at time t and t-t’.
Q3: Model
P(Y|G)=π{g(.)h(.)} …
h(yut-t’, yvt): Correlation between v’s emotion at time t and t-t’.

22. Q3: Model
P(Y|G)=π{g(.)h(.)l(.)}
l(yut-1, yvt): How v’s emotion at t is influenced by her friend u’s emotion at t-1.
Social role sensitive parameter

23. Experimental Results Flickr dataset: 2,060,353 images, 1,255,478 users
ground truth obtained by user tags
Distribution of users’ emotional statuses on Flickr:
happiness: 46.2%
surprise: 9.7%
anger: 8.0%
disgust: 5.3%
fear:17.3%
sadness: 13.5%

24. Experimental Results Baselines
Methods do not consider emotion contagion:
SVM, Logistic Regression (LR),
Naïve Bayes (NB), Bayesian Network (BN),
Gaussian Radial Basis Function Neural Network (RBF).
Methods ignore social role information: CRF
Our model: Role-aware

25. Experimental Results
Evaluation Metrics:
Precision
Recall
F1 Measure

26. Experimental Results

27. (d) Structural hole spanners
(a) Ground truth
(b) Random users
We finally use a case study to demonstrate how different social roles behave in our prediction task. The figure shows the emotion distributions and the degree of happiness
of users from different countries. Specifically, each figure from left to right shows the real emotion distributions and happiness degrees of all users, randomly sampled
users, opinion leaders and structural hole spanners, respectively. We see that in most countries, the happiness degrees of structural hole spanners and opinion leaders’ are higher
than those of the real distribution. This can be explained by the fact that structural hole spanners and opinion leaders are public figures on social networks, who attach much importance to their spreading of positive emotions, while ordinary users tend to share their daily lives.
(c) Opinion leaders
(d) Structural hole spanners

28. Conclusion
We study the interplay between users’ social roles and emotion contagions by answering 3 questions.
Does emotion contagion exist?
How social roles influence emotion contagion?
How to better predict users’ emotional status?
We propose the social role-aware contagion model and validate it on a real social network.

29. Social Role-Aware Emotion Contagion in Image Social Networks
Thank you!
Social Role-Aware Emotion Contagion in Image Social Networks
Yang Yang, Jia Jia, Boya Wu, and Jie Tang
Department of Computer Science and Technology
Tsinghua University
Contact: