This meme comes from South Park (S2E17 1998) This meme comes from South Park (S2E17 1998). In this episode, the children’s underwear are being stolen from them by gnomes for the purpose of “profit.” The process, as explained by the gnomes goes something like this: Step 1. Collect underpants. Step 2. ??? Step 3. PROFIT 1. Tweet 2. ??? 3. profit Sakari Nahi

Sakari Nahi Coding Execute Officer .NET 2001 Azure 2011 Azure MVP 2016 Kompozure.com sakari.nahi@kompozure.com @sakarinahi

Presentation The presentation is a walk-through of an Azure solution for analyzing, predicting and visualizing data from a host of tweets. Tools used are Cognitive Services, HDInsight, Machine Learning and Power BI. Even while we are using tweets as a data source in this presentation, I’m hoping that some of you get ideas on how and what tools to use in order to analyze and make predictions on real business data. Get tweets from a randomly selected presidential candidate Run Azure Cognitive Services for Sentiment and Key Phrases Merge, analyze and export data with HDInsight Find clusters for content, retweets and sentiment with Machine Learning Predict tweet retweets via Machine Learning Present visuals via Power BI

Randomly selected presidential candidate

Examples out of 29885 tweets Such a great honor to be the Republican Nominee for President of the United States. I will work hard and never let you down! AMERICA FIRST! Sorry losers and haters, but my I.Q. is one of the highest -and you all know it! Please don’t feel so stupid or insecure,it’s not your fault Windmills are the greatest threat in the US to both bald and golden eagles. Media claims fictional ‘global warming’ is worse I have never seen a thin person drinking Diet Coke. Thanks- many are saying I'm the best 140 character writer in the world. It's easy when it's fun.

sentiment & key phrases D’Flow Source: Tweets Cognitive Services: sentiment & key phrases HDInsight: query, merge and export Machine Learning: clustering & predicting Power BI report & LinqPad demo Stored in Blob Storage Azure Cognitive Services is used to analyze tweet contents for sentiment and key phrases A Hadoop cluster is used to query, merge and analyze data, until it is exported into Blob Storage as CSV Azure Machine Learning trains clustering and prediction models and then stores the results in an Azure SQL Database Power BI reads data from Azure SQL and then presents it, while LinqPad is used to query ML prediction web service

Source: Tweets Cognitive Services HDInsight Machine Learning Power BI & LinqPad Tweet Id: 741007091947556864 Text: How long did it take your staff of 823 people to think that up--and where are your 33,000 emails that you deleted? Date: 2016-06-09 16:40:32 Favorites: 291140 Retweets: 169943 [top tweet during the campaign]

Power BI Power BI is the Microsoft dashboard solution Source: Tweets Cognitive Services HDInsight Machine Learning Power BI & LinqPad Power BI Power BI is the Microsoft dashboard solution Has capabilities to create interactive, real-time reports that can be embedded into your web solutions There’s Power BI Desktop and Online

Source: Tweets Cognitive Services HDInsight Machine Learning Power BI & LinqPad Cognitive Services Easy-to-use intelligent services based on machine learning Service categories: Language: understanding voice commands, text analytics, language recognition, spell check, translator Vision: face, emotion, computer vision, moderator Speech: conversion, speaker recognition, translator Search: Bing search, autosuggest Knowledge: recommendations, academic

Text Analytics Query: Sentiment response: Key Phrases response: Source: Tweets Cognitive Services HDInsight Machine Learning Power BI & LinqPad Text Analytics Query: Sentiment response: { "documents": [ "language": "en", "id": "1", "score": "0.15" # 0neg – 1pos "text": "First document" }, ... Key Phrases response: "id": "100", "text": "Final document" } ] "keyPhrases": [”phrase 1”, ... ] Make calls in batches: max size 1MB / 1000 documents, throttling at 100 calls / minute

Source: Tweets Cognitive Services HDInsight Machine Learning Power BI & LinqPad Tweet Id: 741007091947556864 Text: How long did it take your staff of 823 people to think that up--and where are your 33,000 emails that you deleted? Date: 2016-06-09 16:40:32 Favorites: 291140 Retweets: 169943 [top tweet during the campaign] Sentiment: 0.6646883 [slightly positive] Phrases: staff people

HDInsight Microsoft’s SaaS-offering of Apache Hadoop Source: Tweets Cognitive Services HDInsight Machine Learning Power BI & LinqPad HDInsight Microsoft’s SaaS-offering of Apache Hadoop Managed distributed MapReduce cluster for your Big Data needs Cluster types: Hadoop, Spark, HBase, Storm Two languages supported: Hive (SQL-like) and Pig (dataflow MR) MapReduce = filter+sort (Map) & summarize (Reduce) Hadoop: original distributed MapReduce cluster (what we have here) Spark: Distributed, in-memory batch analysis that can be several orders of magnitude faster than traditional MapReduce solutions. Hbase: Distributed, non-relational (column-oriented, key-value) database Storm: Distributed, real-time ingestion and analysis of streaming data

Source: Tweets Cognitive Services HDInsight Machine Learning Power BI & LinqPad Machine Learning A predictive analytics suite with several ready algorithms Out of the models created you can publish web services from Machine Learning, for both predicting and retraining purposes

Thank you!