New Directions in Social Media Tom Reamy Chief Knowledge Architect KAPS Group

2 Agenda  Introduction – Social Media and Text Analytics  Deeper than Positive-Negative  Building a Foundation for Social Media – Adding intelligence to BI, CI, and Sentiment  New Dimensions for Social Media – New taxonomies – Cognitive Science of Emotion  Applications – Expertise Analysis, Behavior Prediction, Crowd Sourcing – Integration with Predictive Analytics, Social Analytics  Conclusions

3 KAPS Group: General  Knowledge Architecture Professional Services – Network of Consultants  Partners – SAS, SAP, IBM, FAST, Smart Logic, Concept Searching – Attensity, Clarabridge, Lexalytics,  Strategy – IM & KM - Text Analytics, Social Media, Integration  Services: – Taxonomy/Text Analytics development, consulting, customization – Text Analytics Fast Start – Audit, Evaluation, Pilot – Social Media: Text based applications – design & development  Clients: – Genentech, Novartis, Northwestern Mutual Life, Financial Times, Hyatt, Home Depot, Harvard Business Library, British Parliament, Battelle, Amdocs, FDA, GAO, etc.  Applied Theory – Faceted taxonomies, complexity theory, natural categories, emotion taxonomies Presentations, Articles, White Papers –

4 Introduction – Social Media & Text Analytics Beyond Simple Sentiment  Beyond Good and Evil (positive and negative) – Social Media is approaching next stage (growing up) – Where is the value? How get better results?  Importance of Context – around positive and negative words – Rhetorical reversals – “I was expecting to love it” – Issues of sarcasm, (“Really Great Product”), slanguage  Granularity of Application – Early Categorization – Politics or Sports  Limited value of Positive and Negative – Degrees of intensity, complexity of emotions and documents  Addition of focus on behaviors – why someone calls a support center – and likely outcomes

5 Introduction – Social Media & Text Analytics Beyond Simple Sentiment  Two basic approaches: – Statistical Signature of Bag of Words – Dictionary of positive & negative words  Beware automatic solutions – Accuracy, Depth  Essential – need full categorization and concept extraction to get full value from social media  Categorization - Adds intelligence to all other components – extraction, sentiment, and beyond  Categorization/extraction rules – not just topical or sentiment  Combination with advanced social media analysis  Opens up whole new worlds of applications

6 Text Analytics Foundation for Social Media Text Analytics Features  Noun Phrase Extraction – Catalogs with variants, rule based dynamic  Sentiment Analysis – Objects and phrases – statistics & rules – Positive and Negative  Auto-categorization – Training sets, Terms, Semantic Networks – Rules: Boolean - AND, OR, NOT – Advanced – DIST(#), ORDDIST#, PARAGRAPH, SENTENCE  Text Analytics as Foundation – Disambiguation - Identification of objects, events, context – Build rules based, not simply Bag of Individual Words

Case Study – Categorization & Sentiment 7

8

9

New Dimensions for Social Media  New Taxonomies – Appraisal – Appraisal Groups – Adjective and modifiers – “not very good” – Four types – Attitude, Orientation, Graduation, Polarity – Supports more subtle distinctions than positive or negative  Emotion taxonomies – Joy, Sadness, Fear, Anger, Surprise, Disgust – New Complex – pride, shame, embarrassment, love, awe – New situational/transient – confusion, concentration, skepticism  Beyond Keywords – Analysis of phrases, multiple contexts – conditionals, oblique – Analysis of conversations – dynamic of exchange, private language 10

New Applications in Social Media  Expertise Analysis – Experts think & write differently – process, chunks – Categorization rules for documents, authors, communities  Applications: – Business & Customer intelligence, Voice of the Customer – Deeper understanding of communities, customers – better models – Security, threat detection – behavior prediction, Are they experts? – Expertise location- Generate automatic expertise characterization  Behavior Prediction–TA and Predictive Analytics, Social Analytics  Crowd Sourcing – technical support to Wiki’s  Political – conservative and liberal minds/texts – Disgust, shame, cooperation, openness 11

12 New Applications in Social Media  Analysis of Conversations – Techniques: self-revelation, humor, sharing of secrets, establishment of informal agreements, private language – Detect relationships among speakers and changes over time – Strength of social ties, informal hierarchies  Combination with other techniques – Expertise Analysis – plus Influencers – Quality of communication (strength of social ties, extent of private language, amount and nature of epistemic emotions – confusion+)  Experiments - Pronoun Analysis – personality types  Essay Evaluation Software - Apply to expertise characterization – Model levels of chunking, procedure words over content

13 New Applications in Social Media Behavior Prediction – Telecom Customer Service  Problem – distinguish customers likely to cancel from mere threats  Analyze customer support notes  General issues – creative spelling, second hand reports  Develop categorization rules – First – distinguish cancellation calls – not simple – Second - distinguish cancel what – one line or all – Third – distinguish real threats

14 New Applications in Social Media Behavior Prediction – Telecom Customer Service  Basic Rule – (START_20, (AND, – (DIST_7,"[cancel]", "[cancel-what-cust]"), – (NOT,(DIST_10, "[cancel]", (OR, "[one-line]", "[restore]", “[if]”)))))  Examples: – customer called to say he will cancell his account if the does not stop receiving a call from the ad agency. – cci and is upset that he has the asl charge and wants it off or her is going to cancel his act – ask about the contract expiration date as she wanted to cxl teh acct  Combine sophisticated rules with sentiment statistical training and Predictive Analytics and behavior monitoring

15 New Applications: Wisdom of Crowds Crowd Sourcing Technical Support  Example – Android User Forum  Develop a taxonomy of products, features, problem areas  Develop Categorization Rules: – “I use the SDK method and it isn't to bad a all. I'll get some pics up later, I am still trying to get the time to update from fresh 1.0 to 1.1.” – Find product & feature – forum structure – Find problem areas in response, nearby text for solution  Automatic – simply expose lists of “solutions” – Search Based application  Human mediated – experts scan and clean up solutions

16 New Directions in Social Media Text Analytics, Text Mining, and Predictive Analytics  Two Systems of the Brain – Fast, System 1, Immediate patterns (TM) – Slow, System 2, Conceptual, reasoning (TA)  Text Analytics – pre-processing for TM – Discover additional structure in unstructured text – Behavior Prediction – adding depth in individual documents – New variables for Predictive Analytics, Social Media Analytics – New dimensions – 90% of information  Text Mining for TA– Semi-automated taxonomy development – Bottom Up- terms in documents – frequency, date, clustering – Improve speed and quality – semi-automatic

17 New Directions in Social Media Conclusions  Social Media Analysis requires a hybrid approach – Software, text analytics, human judgment – Contexts are essential  Text Analytics needs new techniques and structures – Smaller, more dynamic taxonomies – Focus – verbs, adjectives, broader contexts, activity  Value from Social Media Analysis requires new structures – Appraisal taxonomies, emotion taxonomies Plus – Better models of documents and authors – multi-dimensional  Result: – Enhanced sentiment analysis / social media applications – Develop whole range of new applications

Questions? Tom Reamy KAPS Group Upcoming: Text Analytics Summit – Boston - June Text Analytics World – Boston – October – Speakers?