Presentation is loading. Please wait.

Presentation is loading. Please wait.

Gaining Real-Time IoT Insights Using Azure Stream Analytics, Azure ML, and Power BI.

Similar presentations


Presentation on theme: "Gaining Real-Time IoT Insights Using Azure Stream Analytics, Azure ML, and Power BI."— Presentation transcript:

1

2 Gaining Real-Time IoT Insights Using Azure Stream Analytics, Azure ML, and Power BI

3

4 Demo How real time analytics changes the business dynamics in the healthcare industry

5

6

7 One-stop ICT solution by Fujitsu’s Food & Agriculture Cloud. Variety of innovative solutions and services for agribusiness. SalesSalesProductionProductionManagementManagement Biz Analysis Sales Delivery Accountin g Store & Analyze Sensing Data One-stop ICT Solutions and their Support Services Collect Environment Data Optimize Each Operations 4 ①②③④ 1. Production Management 3. Greenhouse Horticulture 2. Remote Sensing Network 4. Animal Husbandry

8

9 Cultivation Technology Akisai Plant Factory in Fukushima 1 st Innovation Semi-conductor Manufacturing ICT Mass production of Clean Lettuce with expertise ×× 2 nd Innovation powered by Microsoft Azure Excel product quality Excel Productivity Data consolidation Copyright 2015 FUJITSU LIMITED

10 ・ Improve production & reduce costs ・ Expand channels and business ・ Improve production & reduce costs ・ Expand channels and business Improve Business ・ Collaborate with local communities/businesses ・ To be an incubation center ・ Promote an advanced agriculture to the world ・ Collaborate with local communities/businesses ・ To be an incubation center ・ Promote an advanced agriculture to the world ・ Expand line-ups of low-Potassium vegetables ・ Pursue tastier, healthier products ⇒ Control quality/ingredients with optimized envs ⇒ Control quality/ingredients with optimized envs ・ Expand line-ups of low-Potassium vegetables ・ Pursue tastier, healthier products ⇒ Control quality/ingredients with optimized envs ⇒ Control quality/ingredients with optimized envs ・ A reference model with FJ solutions ・ World-class showcase ・ Visitors impressed with Fujitsu! ・ A reference model with FJ solutions ・ World-class showcase ・ Visitors impressed with Fujitsu! PoC of ICT Solutions Contribute to Tohoku Recovery Provide Foods with pleasure

11 Overall optimization of production and management at the Akisai Plant Factory in Fukushima Copyright 2015 FUJITSU LIMITED

12

13 M2M/IoTPlatform Stream Analytics Cloud gateways Hot Path Cold Path Management Daashborad Hybrid Teamsite System On-Premises (AIZU Factory) Machine Learning Teamsite Copyright 2015 FUJITSU LIMITED

14 1. IoT and Big Data are key enablers of business innovation –but complexity of architecture and E2E stack is a big challenge 2.Equally important success factors –Careful selection of IoT ecosystem –Effective co-innovation partnership 3.End users want to focus just on IoT Apps that enable business growth 4.Need agile IoT App development, deployment and business process integration –Enabled by Cloud-based IoT Platform-as-a-Service functionality 5.Co-innovation of Fujitsu, Microsoft & Customers support rapid Proof-of-Business –To accelerate the business innovation learning curve Copyright 2015 FUJITSU LIMITED

15

16 Project “Inception” Microsoft Technology Center and NEC Todd Van Nurden Chief Architect, Technical Solutions, MTC

17 Problem: How do we deliver ambient intelligence?

18 Customer Needs Assistance How it works

19 Inception Framework Azure Hadoop Kiosk Biometrics Services Telemetry Services Kinect Sensor Azure Event Hub ASA Interactions ASA Biometrics ASA Telemetry Interactions Biometrics Telemetry Hive Script Interactions Hive Table Biometrics Hive Table Telemetry Hive Table Excel Inception – Logical/Physical Interaction Services

20 Inception

21 Demo Inception Framework

22 Introducing the Inception Framework

23 Go Do’s

24

25

26

27 Real-time Analytics Intake millions of events per second (up to 1 GB/s) Low processing latency, auto adaptive (sub-second to seconds) Correlate between different streams, or with reference data Find patterns or lack of patterns in data in real-time Fully Managed Cloud Service No hardware acquisition and maintenance No platform/infrastructure deployment and maintenance Easily expand your business globally leveraging Azure regions

28 Mission Critical Reliability Guaranteed event delivery Guaranteed business continuity: Automatic and fast recovery Effective Audits Privacy and security properties of solutions are evident Azure integration for monitoring and ops alerting Easy To Scale Scale from small to large on demand

29 Rapid Development with SQL like language High-level: focus on stream analytics solution Concise: less code to maintain Fast test: Rapid development and debugging First-class support for event streams and reference data Built in temporal semantics Built-in temporal windowing and joining Simple policy configuration to manage out-of-order events and late arrivals

30 Infrastructure – Procure and setup Develop solution (code) for ingress, processing and egress Develop solutions to integrate with other components like ML, BI etc Develop solutions to manage resiliency, such as infrastructure failures Develop solutions and infrastructure for increasing scale with business growth Monitoring and troubleshooting of solution

31

32 IDCreatedAtUserNameTimeZoneTextLanguageTopic T20:45:30Joshua XEastern Time (US & Canada)Oh, joy! More Live updatesenXBox T20:45:31Cristabel Streaming Xbox One games..enXBox … “A news media website wants to increase site traffic by covering trending topics on social media.” To determine which topics are immediately relevant to customers, they need real-time analytics about the tweet volume and sentiment for each topic. TwitterStream

33 Filters SELECT UserName, TimeZone FROM InputStream WHERE Topic = 'XBox' Show me the user name and time zone of tweets on the topic XBox "Haroon”, “Eastern Time (US & Canada)” "XO", “London” “Zach Dotseth“, “London”, “Football”,(…) "Haroon”, “Eastern Time (US & Canada)” “XBox”,(…) "XO",”London”, “XBox“, (…)

34 Windowing Concepts Windows can be tumbling, hopping, or sliding Windows are fixed length Must be used in a GROUP BY clause Output event will have the timestamp of the end of the window t1t2t5t6t3t4 Time Window 1Window 2Window 3 Aggregate Function (Sum) 1814 Output Events

35 SELECT Topic, Count(*) AS TotalTweets FROM TwitterStream TIMESTAMP BY CreatedAt GROUP BY Topic, HoppingWindow(second, 10, 5) “Every 5 seconds give me the count of tweets over the last 10 seconds” A 10-second Hopping Window with a 5-second “Hop”

36 {“XO”, 4, “Win10”}{“Jo”, 0, “Surface”} {“Foo”,4, “Bing”} {“Dip”, 2, “XBox”} {“XO”, 0, “Win10”} {“Dip”, 0, “Xbox”} {“Jo”, 4, “Surface”} {“Foo”, 0, “Bing”} Twitter Stream: (same stream, further down the timeline) SELECT TS1.UserName, TS1.Topic FROM TwitterStream TS1 TIMESTAMP BY CreatedAt JOIN TwitterStream TS2 TIMESTAMP BY CreatedAt ON TS1.UserName = TS2.UserName AND TS1.Topic = TS2.Topic AND DateDiff(second, TS1, TS2) BETWEEN 1 AND 60 WHERE TS1.SentimentScore != TS2.SentimentScore “List all users and the topics on which they switched their sentiment within a minute“

37 “Show me if a topic is not tweeted for 10 seconds since it was last tweeted” SELECT TS1.CreatedAt, TS1.Topic, TS1.UserName FROM TwitterStream TS1 TIMESTAMP BY CreatedAt LEFT OUTER JOIN TwitterStream TS2 TIMESTAMP BY CreatedAt ON TS1.Topic = TS2.Topic AND DateDiff(second, TS1, TS2) BETWEEN 1 AND 10 WHERE TS2.Topic IS NULL {“XO”, 4, “Win10”}{“WAA”, 2, “Microsoft”} {“AB”, 0, “Bing} {“Dip”, 4, “Xbox”} {“Foo”, 0, “Win10”}{“Tim”, 2, “Microsoft”} {“AB”, 0, “Bing”} Twitter Stream: (same stream, further down the timeline)

38 Reference Data Seamless correlation of event streams with reference data Static or slowly-changing data stored in blobs CSV and JSON files in Azure Blobs; scanned for new snapshots on a settable cadence JOIN (INNER or LEFT OUTER) between streams and reference data sources Reference data appears like another input: SELECT myRefData.Name, myStream.Value FROM myStream JOIN myRefData ON myStream.myKey = myRefData.myKey

39 Partitioning allows for parallel execution over scaled-out resources SELECT Count(*) AS Count, Topic FROM TwitterStream PARTITION BY PartitionId GROUP BY TumblingWindow(minute, 3), Topic, PartitionId QueryResult 1 QueryResult 2 QueryResult 3 Event Hub

40 WITH Step1 AS ( SELECT Count(*) AS CountTweets, Topic FROM TwitterStream PARTITION BY PartitionId GROUP BY TumblingWindow(second, 3), Topic, PartitionId ), Step2 AS ( SELECT Avg(CountTweets) FROM Step1 GROUP BY TumblingWindow(minute, 3) ) SELECT * INTO Output1 FROM Step1 SELECT * INTO Output2 FROM Step2 SELECT * INTO Output3 FROM Step2 A query can have multiple steps to enable pipeline execution A step is a sub-query defined using WITH (“common table expression”) Can be used to develop complex queries more elegantly by creating a intermediary named result Creates unit of execution for scaling out when PARTITION BY is used Each step’s output can be sent to multiple output targets using INTO

41 Machine Learning SELECT text, sentiment(text) AS score FROM myStream

42 Twitter Sentiment Analysis Demo

43 Stream Analytics is priced on two variables: Volume of data processed Streaming units required to process the data stream MeterPrice (USD) Volume of Data Processed  Volume of data processed by the streaming job (in GB) $.001 per GB Streaming Unit  Blended measure of cores, memory, and bandwidth $0.031 per hour * Streaming unit is a unit of compute capacity with a maximum throughput of 1MB/s

44 Daily Azure Stream Analytics cost for 1 MB/sec of average processing Volume of Data Processed Cost - $0.001 /GB * GB = $0.08 per day, streaming max 1 MB/s non-stop Streaming Unit Cost - $.031 /hr * 24 hrs = $0.74 per day, for 1 MB/sec max. throughput Total cost - $ $0.08 = $0.82 per day -or- $24.60 per month

45 Business Overviewhttp://azure.microsoft.com/en-us/services/stream-analytics/http://azure.microsoft.com/en-us/services/stream-analytics/ Documentationhttp://azure.microsoft.com/en- us/documentation/services/stream-analytics/http://azure.microsoft.com/en- us/documentation/services/stream-analytics/ Sampleshttps://github.com/streamanalytics/sampleshttps://github.com/streamanalytics/samples ASA Bloghttp://blogs.msdn.com/b/streamanalytics/rss.aspxhttp://blogs.msdn.com/b/streamanalytics/rss.aspx Follow us on Twitterhttps://twitter.com/AzureStreaming (follow ASA Forumhttps://social.msdn.microsoft.com/Forums/en- US/home?forum=AzureStreamAnalyticshttps://social.msdn.microsoft.com/Forums/en- US/home?forum=AzureStreamAnalytics Vote for ideashttp://feedback.azure.com/forums/ azure-stream-analyticshttp://feedback.azure.com/forums/ azure-stream-analytics ASA

46

47


Download ppt "Gaining Real-Time IoT Insights Using Azure Stream Analytics, Azure ML, and Power BI."

Similar presentations


Ads by Google