1. www.decideo.fr/bruley Extract from various presentations: Bing Liu, Aditya Joshi, Aster Data … Sentiment Analysis michel.bruley@teradata.com January 2012

2. www.decideo.fr/bruley Introduction n Two main types of textual information: Facts and Opinions n Most current text information processing methods work with factual information (e.g., web search, text mining) n Sentiment analysis or opinion mining, computational study of opinions (sentiments, emotions) expressed in text n Why opinion mining now? Mainly because of the Web huge volumes of opinionated text.

3. www.decideo.fr/bruley What is Sentiment Analysis? n Identify the orientation of opinion in a piece of text (blogs, user comments, review websites, community websites, …), in others words determine if a sentence or a document expresses positive, negative, neutral sentiment towards some object? The movie was fabulous! The movie stars Mr. X The movie was horrible! [ Factual ][ Sentimental ]

4. www.decideo.fr/bruley SA at different levels The movie was interesting and fabulous interesting The movie was very boring boring Word-level SA Sentence-level SA Document-level SA The police stopped corruption His last movie was great. police (subj.) stopped (verb) corruption (obj.) His last movie was Great and interesting. This one’s a dud.

5. www.decideo.fr/bruley What is an Opinion? n An opinion is a quintuple: (o j, f jk, so ijkl, h i, t l ) where –o j is a target object –f jk is a feature of the object o j –so ijkl is the sentiment value of the opinion of the opinion holder h i on feature f jk of object o j at time t l –h i is an opinion holder –t l is the time when the opinion is expressed

6. www.decideo.fr/bruley Objective: structure the unstructured n Objective: Given an opinionated document, –Discover all quintuples (o j, f jk, so ijkl, h i, t l ), i.e., mine the five corresponding pieces of information in each quintuple n With the quintuples, –Unstructured Text  Structured Data Traditional data and visualization tools can be used to slice, dice and visualize the results in all kinds of ways Enable qualitative and quantitative analysis n With all quintuples, all kinds of analyses become possible

7. www.decideo.fr/bruley SA is not Just ONE Problem n Track direct opinions: –document –sentence –feature level n Compare opinions: different types of comparisons n Detect opinion spam detection: fake reviews

8. www.decideo.fr/bruley Polarity Classifier n First eliminate objective sentences, then use remaining sentences to classify document polarity (reduce noise)

9. www.decideo.fr/bruley Level of Analysis We can inquire about sentiment at various linguistic levels: n Words – objective, positive, negative, neutral n Clauses – “going out of my mind” n Sentences – possibly multiple sentiments n Documents

10. www.decideo.fr/bruley Words n Adjectives –objective: red, metallic –positive: honest, important, mature, large, patient –negative: harmful, hypocritical, inefficient –subjective (but not positive or negative): curious, peculiar, odd, likely, probable n Verbs –positive: praise, love –negative: blame, criticize –subjective: predict n Nouns –positive: pleasure, enjoyment –negative: pain, criticism –subjective: prediction, feeling

11. www.decideo.fr/bruley Clauses n Might flip word sentiment –“not good at all” –“not all good” n Might express sentiment not in any word –“convinced my watch had stopped” –“got up and walked out”

12. www.decideo.fr/bruley Some Problems n Which features to use? Words (unigrams), Phrases/n-grams, Sentences n How to interpret features for sentiment detection? Bag of words (IR), Annotated lexicons (WordNet, SentiWordNet), Syntactic patterns, Paragraph structure n Must consider other features due to… –Subtlety of sentiment expression irony expression of sentiment using neutral words –Domain/context dependence words/phrases can mean different things in different contexts and domains –Effect of syntax on semantics

13. www.decideo.fr/bruley Some Applications Examples n Review classification: Is a review positive or negative toward the movie? n Product review mining: What features of the ThinkPad T43 do customers like/dislike? n Tracking sentiments toward topics over time: Is anger ratcheting up or cooling down? n Prediction (election outcomes, market trends): Will Obama or Republican candidate win? n Etcetera

14. www.decideo.fr/bruley Aster Data position for Text Analysis Data Acquisition Pre-Processing Mining Analytic Applications Perform processing required to transform and store text data and information (stemming, parsing, indexing, entity extraction, …) Gather text from relevant sources (web crawling, document scanning, news feeds, Twitter feeds, …) Apply data mining techniques to derive insights about stored information (statistical analysis, classification, natural language processing, …) Leverage insights from text mining to provide information that improves decisions and processes (sentiment analysis, document management, fraud analysis, e-discovery,...) Third-Party Tools Fit Aster Data Fit Aster Data Value: Massive scalability of text storage and processing, Functions for text processing, Flexibility to develop diverse custom analytics and incorporate third-party libraries