1. Proportion of Original Tweets
How Does That Make You Feel?
Detecting and Understanding Abuse in Context
Emily Grace, Ph.D., Dustin Arendt, Ph.D., Svitlana Volkova, Ph.D.
OBJECTIVE
APPROACH
EXPERIMENTAL RESULTS
Classification of affect presence or +/- polarity in replies
Regression of degree of affect expressed in replies
Predictive Tasks
LSTMs with pre-trained GloVe embeddings
AdaBoost (baseline)
Models
Number of replies
Inverse variance of affect assignments
Length of replies
Sample Weighting
Tweet content, behavior
Style, psycholinguistic markers
User meta-data
Predictive Signals
RQ2: What types of emotion are easier to predict?
Classification
Fear, joy, and anger are easier to predict than disgust, sadness and surprise
More extreme levels of sentiment and emotion are more easily distinguished
Regression
Joy is the easiest to predict
Predicting specific level of affect evoked is significantly harder than classifying into high/low emotion levels
Twitter Data: 200K tweet-reply pairs in English
RQ1
How to identify features of Twitter content that evoke certain sentiments and emotions
RQ2
How to detect cases where tweet content is not sufficient to predict reply sentiment
RQ3
How to understand behaviors that fall outside of normal social dynamics on social media
QUALITATIVE ANALYSIS
RQ2: When is the evoked affect not predictable?
RQ1 & RQ3: What types of tweets generate positive versus negative replies?
RQ1 & RQ3: What features are predictive of different affects?
Error Analysis
Label noise: incorrect sentiment assignments
Outside knowledge: references to external events
Media: images or other media posted with tweet
Conversational drift: replies introduce new topics
Abuse/unrelated-anger: unprovoked angry or abusive responses to innocuous tweet
More negative replies
More Positive replies
Proportion of Original Tweets
FUTURE WORK
Going beyond text content: use results from error analysis to identify and implement additional predictive features – such as the images or other media content e.g., URLs of the tweets
Abuse detection: leverage the model results for the real-time detection of abusive and harassing behavior on social media by flagging model prediction errors