Sentiment analysis tools IS/ICT Jan Král, David Krénar, Roman Pospěch

What are SAT? Why use them? 2 Analysis of the feelings (emotions, opinions, …) Get information to the natural language (user comments, product reviews, …) Read data and analyze the content: manually – time and money, not comparable over time SAT – less time, provide consistent, accurate and impartial results

Why use Facebook? 3 1.23 billion users per day (December 2016) Own analysis tool – Insights Track user demographics in numerical terms (the number of likes, shares, …) Help map audience more interested posts FB stores this only for 180 days (Heijmans, 2015)

Text Insights 4 SAT for the numerical and the textual data Using public data + FB Graph API Analyze given content create keyword or keyphrase for each topic

Example – „Hypertension Hub“ 5 Fig. 1: Tag cloud of weighted keyphrases based on full page content (left); page metrics for the time interval 08/2013–11/2013 (right) Source: https://www.researchgate.net/publication/276886479_Text_Insights_Natural_Language_Analytics_for_Understanding_Social_Media_Engagement

Twitter sentiment analysis tools 6 ~300 million active users The main content of this social network are short posts called tweets Purpose – tweet sentiment classification (positive, negative or neutral)

Benchmark analysis 7 Goal – accuracy of tweet classification in comparison with human judgment 15* stand-alone and 5 workbench tools 5 data sets which included tweets from 5 different topics

Stand-alone vs. Workbench tools 8 Stand-alone tools ready-to-use right after you open them Workbench tools require supervised learning-based model development on a labeled training set

Stand-alone tools benchmark results 9 Tool Average accuracy (%) SentiStrength* 67.49 Chatterbox 67.43 Sentiment140* 66.46 Textalytics 66.22 Intridea 63.31 Table 1: Five best stand-alone tools sorted by highest average accuracy. Source: Abbasi et al., 2015.

Workbench tools benchmark results 10 Tool Average accuracy (%) BPEF 71.38 Lightside 69.35 FRN 69.17 EWGA 68.12 RapidMiner 66.86 Table 1: Workbench tools sorted by highest average accuracy. Source: Abbasi et al., 2015.

Source Abbasi, A., Hassan, A., Dhar, M. (2015). Benchmarking Twitter Sentiment Analysis Tools. Retrieved February 27, 2017, from https://www.researchgate.net/publication/ 273000042

12 Google Cloud Platform Two solutions Both independent on platform (Facebook, Twitter, web pages…) Prediction API Natural Language API

Prediction API Generic machine learning service Can solve regression or classification problems Sentiment analysis = classify text string with provided labels („sad“, „happy“…) Steps Collect data Label collected data Upload data to Google Cloud Storage Train model Make predictions based on trained model

Natural Language API Google’s sentiment analysis solution Based on natural language processing Only English is supported

Natural Language API - example „My experience with Amazon was to BAD. I ordered JBL FLIP-3 PORTABLE WIRELESS SPEAKER on 25 March 17. And on 26 March I received the order but in the box I got 3 pieces of STONE instead of wireless speaker. And they said it was the mistake. Fake.“ Source: www.consumeraffairs.com

Thank you for your attention.