1. This meme comes from South Park (S2E17 1998)
This meme comes from South Park (S2E ). In this episode, the children’s underwear are being stolen from them by gnomes for the purpose of “profit.”
The process, as explained by the gnomes goes something like this:
Step 1. Collect underpants.
Step 2. ???
Step 3. PROFIT
1. Tweet 2. ??? 3. profit
Sakari Nahi

2. Sakari Nahi Coding Execute Officer .NET 2001 Azure 2011 Azure MVP 2016
Kompozure.com
@sakarinahi

3. Presentation
The presentation is a walk-through of an Azure solution for analyzing, predicting and visualizing data from a host of tweets. Tools used are Cognitive Services, HDInsight, Machine Learning and Power BI.
Even while we are using tweets as a data source in this presentation, I’m hoping that some of you get ideas on how and what tools to use in order to analyze and make predictions on real business data.
Get tweets from a randomly selected presidential candidate
Run Azure Cognitive Services for Sentiment and Key Phrases
Merge, analyze and export data with HDInsight
Find clusters for content, retweets and sentiment with Machine Learning
Predict tweet retweets via Machine Learning
Present visuals via Power BI

4. Randomly selected presidential candidate

5. Examples out of tweets
Such a great honor to be the Republican Nominee for President of the United States. I will work hard and never let you down! AMERICA FIRST! Sorry losers and haters, but my I.Q. is one of the highest -and you all know it! Please don’t feel so stupid or insecure,it’s not your fault Windmills are the greatest threat in the US to both bald and golden eagles. Media claims fictional ‘global warming’ is worse I have never seen a thin person drinking Diet Coke. Thanks- many are saying I'm the best 140 character writer in the world. It's easy when it's fun.

6. sentiment & key phrases
D’Flow
Source: Tweets
Cognitive Services:
sentiment & key phrases
HDInsight:
query, merge and export
Machine Learning:
clustering &
predicting
Power BI report &
LinqPad demo
Stored in Blob Storage
Azure Cognitive Services is used to analyze tweet contents for sentiment and key phrases
A Hadoop cluster is used to query, merge and analyze data, until it is exported into Blob Storage as CSV
Azure Machine Learning trains clustering and prediction models and then stores the results in an Azure SQL Database
Power BI reads data from Azure SQL and then presents it, while LinqPad is used to query ML prediction web service

7. Source: Tweets
Cognitive Services
HDInsight
Machine Learning
Power BI &
LinqPad
Tweet
Id:
Text: How long did it take your staff of 823 people to think that up--and where are your 33,000 s that you deleted?
Date: :40:32
Favorites:
Retweets: [top tweet during the campaign]

8. Power BI Power BI is the Microsoft dashboard solution
Source: Tweets
Cognitive Services
HDInsight
Machine Learning
Power BI &
LinqPad
Power BI
Power BI is the Microsoft dashboard solution
Has capabilities to create interactive, real-time reports that can be embedded into your web solutions
There’s Power BI Desktop and Online

9. Source: Tweets
Cognitive Services
HDInsight
Machine Learning
Power BI &
LinqPad
Cognitive Services
Easy-to-use intelligent services based on machine learning
Service categories:
Language: understanding voice commands, text analytics, language recognition, spell check, translator
Vision: face, emotion, computer vision, moderator
Speech: conversion, speaker recognition, translator
Search: Bing search, autosuggest
Knowledge: recommendations, academic

10. Text Analytics Query: Sentiment response: Key Phrases response:
Source: Tweets
Cognitive Services
HDInsight
Machine Learning
Power BI &
LinqPad
Text Analytics
Query:
Sentiment response: {
"documents": [
"language": "en",
"id": "1",
"score": "0.15" # 0neg – 1pos
"text": "First document" },
...
Key Phrases response:
"id": "100",
"text": "Final document" } ]
"keyPhrases": [”phrase 1”, ... ]
Make calls in batches:
max size 1MB / 1000 documents, throttling at 100 calls / minute

11. Source: Tweets
Cognitive Services
HDInsight
Machine Learning
Power BI &
LinqPad
Tweet
Id:
Text: How long did it take your staff of 823 people to think that up--and where are your 33,000 s that you deleted?
Date: :40:32
Favorites:
Retweets: [top tweet during the campaign]
Sentiment: [slightly positive]
Phrases: staff people

12. HDInsight Microsoft’s SaaS-offering of Apache Hadoop
Source: Tweets
Cognitive Services
HDInsight
Machine Learning
Power BI &
LinqPad
HDInsight
Microsoft’s SaaS-offering of Apache Hadoop
Managed distributed MapReduce cluster for your Big Data needs
Cluster types: Hadoop, Spark, HBase, Storm
Two languages supported: Hive (SQL-like) and Pig (dataflow MR)
MapReduce = filter+sort (Map) & summarize (Reduce)
Hadoop: original distributed MapReduce cluster (what we have here)
Spark: Distributed, in-memory batch analysis that can be several orders of magnitude faster than traditional MapReduce solutions.
Hbase: Distributed, non-relational (column-oriented, key-value) database
Storm: Distributed, real-time ingestion and analysis of streaming data

13. Source: Tweets
Cognitive Services
HDInsight
Machine Learning
Power BI &
LinqPad
Machine Learning
A predictive analytics suite with several ready algorithms
Out of the models created you can publish web services from Machine Learning, for both predicting and retraining purposes

15. Thank you!