We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byLeon Larke
Modified about 1 year ago
© 2014 IBM Corporation Information Management Smart Data Analysis for IoT (Internet of Things) Applications Kun-Lung Wu, Ph.D., Manager Data-Intensive Systems & Analytics Group (IBM T. J. Watson Research Center) InfoSphere Streams Language & Research (IBM SWG)
© 2014 IBM Corporation Information Management As IoT applications become more pervasive, there is a real-time big data explosion Almost anything can be equipped and connected to the Internet Internet of Things They can generate, in real-time, streams and streams of data Real-Time Big Data Explosion Real-time data analysis is an integral part of many IoT applications Everything
© 2014 IBM Corporation Information Management Examples of IoT Applications 3 Smart cities Traffic control, emergency management, etc Health care Aiding the elderly, ICU alert management, health monitoring via wearable devices, etc Agriculture & food Precision farming, cold chain management, etc Industrial applications Manufacturing process monitoring, engine monitoring, etc Environmental monitoring Water, Waste, Air Quality, etc Retail applications
© 2014 IBM Corporation Information Management What is different in IoT data? There are many extremes Use uncertain dataUse more types data Veracity Process and act on data more quickly in real time Variety Volume There are greater amounts of data Velocity
© 2014 IBM Corporation Information Management Traditional versus IoT Big Data Available Information Analyzed Information Analyze ALL Available Information Traditional ApproachIoT Big Data Approach Analyze Small Subsets of InformationAnalyze All Information Leverage more of the data being captured
© 2014 IBM Corporation Information Management Traditional versus IoT Big Data Traditional ApproachIoT Big Data Approach Carefully Cleanse Information Before Any Analysis Analyze Information As Is, Cleanse As Needed A Small Amount of Carefully Cleansed Information Analyzed Information A Very Large Amount of Messy Information Analyzed Information Reduce effort required to leverage data
© 2014 IBM Corporation Information Management Traditional versus IoT Big Data Traditional ApproachIoT Big Data Approach Analyze data AFTER it has been processed and landed in a Warehouse or Mart Analyze data IN MOTION as it is generated, in real-time Leverage data as it is captured
© 2014 IBM Corporation Information Management 8 RE- Standard assumptionsRe-think for IoT data analysis Clean and correct dataTake advantage of and tolerate uncertainty Transactional guaranteesGood enough Normalized, structured dataStore data in elemental form Explicit relationships keptRelationships found at query ACID propertiesRelaxed constraints Centrally managed storageLoosely distributed data Store-and-processProcess in motion Reliable hardwareBuilt with full expectation of failures Query, insert, delete with SQLQuery, operators, analytics at point of data Reference/context data on diskReference and context data in memory
© 2014 IBM Corporation Information Management From data at rest to data in motion Data in Data at 9
© 2014 IBM Corporation Information Management Millions of events per second Microsecond Latency Traditional / Non-traditional data sources Real time delivery Powerful Analytics Algorithmic Trading Telco Churn Prediction Smart Grid Cyber Security Government / Law enforcement ICU Monitoring Environment Monitoring Volume Terabytes per second Petabytes per day Variety All kinds of data All kinds of analytics Velocity Insights in microseconds IBM InfoSphere Streams Delivers Real-Time Analytics For Big Data In Motion Example Streaming Data Sources: Video, audio, networks, social media
© 2014 IBM Corporation Information Management Modify Filter / Sample Classify Fuse Annotate Big Data in Real Time with Stream Processing Score Windowed Aggregates Analyze
© 2014 IBM Corporation Information Management Easy to extend: Built in adaptors Users add capability with familiar C++ and Java InfoSphere Streams: For superior real time analytic processing Compile groups of operators into single processes: Efficient use of cores Distributed execution Very fast data exchange Can be automatic or tuned Scaled with push of a button Streams Processing Language (SPL) built for Streaming applications: Reusable operators Rapid application development Continuous “pipeline” processing Flexible and high performance transport: Very low latency High data rates Use the data that gives you a competitive advantage: Can handle virtually any data type Use data that is too expensive and time sensitive for traditional approaches Easy to manage: Automatic placement Extend applications incrementally without downtime Multi-user / multiple applications Dynamic analysis: Programmatically change topology at runtime Create new subscriptions Create new port properties 12
© 2014 IBM Corporation Information Management What Are People Doing With Streams? Stock market Impact of weather on securities prices Analyze market data at ultra-low latencies Fraud prevention Detecting multi-party fraud Real-time fraud prevention e-Science Space weather prediction Detection of transient events Synchrotron atomic research Transportation Intelligent traffic management Smart Grid & Energy Transactive control Phasor Monitoring Unit Natural Systems Wildfire management Water management Telephony CDR processing Social analysis Churn prediction Geomapping Other Manufacturing Text Analysis Who’s Talking to Whom? ERP for Commodities FPGA Acceleration Law Enforcement, Defense & Cyber-Security Real-time multimodal surveillance Situational awareness Cyber security detection Health & Life Sciences Neonatal ICU monitoring Epidemic early warning system Remote healthcare monitoring 13
© 2014 IBM Corporation Information Management 14 Asian telco reduces billing costs and improves customer satisfaction Problem: Call volume increased to the point that batch processing in a warehouse no longer worked 1) Too expensive, 2) too slow, and 3) no capacity left for BI Solution: Real-time mediation and analysis of 8B CDRs per day Data processing time reduced from 12 hrs to 1 sec Hardware cost reduced to 1/8 th Further enabled: Proactively addressing issues impacting customer satisfaction, real time offers based on usage
© 2014 IBM Corporation Information Management Harnessing the Largest Predictive Focus Group in the World Purpose –Understand public sentiment towards an event: movie trailers –Deeply understand the potential customer profile: gender, occupation, intent to watch –Alter marketing launch plans based on insight Background –1.1 Billion Tweets analyzed –5.7 Million blogs/forum posts –3.5 million messages –Also: Facebook, Google+, Tumblr, Flickr
© 2014 IBM Corporation Information Management 16 Performing real-time analytics using physiological data from neonatal babies Continuously correlates data from medical monitors to detect subtle changes and alert hospital staff sooner Early warning gives caregivers the ability to proactively deal with complications “Helps detect life threatening conditions up to 24 hours sooner” University of Ontario Institute of Technology (UOIT) Detects Neonatal Patient Symptoms Sooner
© 2014 IBM Corporation Information Management Challenges and opportunities Approach overload –Is there a convergence of approaches? –Is there a “write once, use any technology” approach across tool types Skills to apply techniques –Reduce the skill required? –More people who can be data scientists, developers, and business/domain savvy? Uncertain data –Confidence levels need to follow data and decisions New analytic algorithms –Real time learning and adaptation? –More automation Availability –What does it mean for in-memory systems? –How should disaster recovery work? Cloud –Security of Data –Data movement Data governance, security, and privacy What new problems can we solve?
© 2014 IBM Corporation Information Management To Learn more Resources –Streams: streamsDev –IBM Big Data: ibm.com/bigdata –IBMBigDataHub.com –BigDataUniversity.com –Books / analyst papers
© 2014 IBM Corporation Information Management Try Stream Processing http://Ibm.co/streamsqs 2 download options! 19
© 2014 IBM Corporation Information Management 20
© 2010 IBM Corporation IBM InfoSphere Streams Enabling a smarter planet Roger Rea InfoSphere Streams Product Manager Sept 15, 2010.
Click to add text TWA New Job Types with Tivoli Workload Scheduler for Applications 8.6 TWS Education.
+ Big Data IST210 Class Lecture. + Big Data Summary by EMC Corporation (http://www.emc.com) More videos that.
© 2010 IBM Corporation Smarter Systems for a Smarter Planet Presenter Name – Presenter Title MM/DD/Year.
ITU Forum on Internet of Things: A New Age of Smarter Living 18th January 2016 Singapore A New Age of Smarter Living IoT Meets Big Data Standardization.
© 2011 IBM Corporation Smarter Software for a Smarter Planet The Capabilities of IBM Software Borislav Borissov SWG Manager, IBM.
30 th ASEI National Convention Disruptive Innovations & Technologies September 19, 2015 Dearborn, MI IoT, Big Data and Emerging Technologies Kameshwar.
© 2012 TeraMedica, Inc. Big Data: Challenges and Opportunities for Healthcare Joe Paxton Healthcare and Life Sciences Sales Leader.
+ Logentries Is a Real-Time Log Analytics Service for Aggregating, Analyzing, and Alerting on Log Data from Microsoft Azure Apps and Systems MICROSOFT.
What we know or see What’s actually there Wikipedia : In information technology, big data is a collection of data sets so large and complex that it.
M2M Evolution Battle of the Platforms ThingWorx Platform Overview Demo Build Something Cool Q & A Sections.
PowerOneData’s GENII Leverages Cloud Platform to Deliver Affordable, Scalable, and Accessible Meter Data Management Software to Customers COMPANY PROFILE:
© 2013 Amadeus IT Group SA 265ced1609a17cf1a a2ad ae8 Amadeus Travel Intelligence ‘Monetising’ big data sets Pascal Clement, Head of Travel.
© 2013 IBM Corporation October 4, 2013 IT Analytics and Big Data IBM Solutions Paul Smith (Smitty) Service Management Architect.
Thriving in a Hybrid World Dean J. Marsh Vice President, Client Success IBM Analytic Solutions.
Gaining Unprecedented Visibility into Microsoft Dynamics CRM with Halo’s Pipeline Advisor, Powered by the Microsoft Azure Cloud Platform MICROSOFT AZURE.
©2011 SAP AG. All rights reserved.1 Confidential SAP Sybase IQ– Big Data Opportunity Tele Cheat Sheet Topic, Audience, Industry Topic/ Product: SAP Sybase.
Smart Grid Big Data: Automating Analysis of Distribution Systems Steve Pascoe Manager Business Development E&O - NISC.
Built on the Powerful Microsoft Azure Platform, Forensic Advantage Helps Public Safety and National Security Agencies Collect, Analyze, Report, and Distribute.
MAR Capability Overview Deck Protean Analytics.
LIMPOPO DEPARTMENT OF ECONOMIC DEVELOPMENT, ENVIRONMENT AND TOURISM The heartland of southern Africa – development is about people! 2015 ICT YOUTH CONFERENCE.
Replace with Application Image Metrix Insights Leverages Internet of Things, Big Data to Deliver Actionable Healthcare Intelligence via Azure and Cortana.
Enterprise Alert on Microsoft Azure Fully Automates Critical Incident Communication and Transforms It into an Intelligent, Reliable, and Mobile Experience.
Powered by the Microsoft Azure Platform, Truck Tin Helps Your Sales Consultants Improve Efficiency, Information Sharing, Client Relations MICROSOFT AZURE.
Check Point Virtual Gateway Protects Microsoft Azure Cloud Infrastructure with a Full Range of Security Defenses and Threat Prevention Solutions MICROSOFT.
VMob Mobile Marketing Platform Delivers Highly Targeted Marketing Directly into Shoppers’ Existing Smartphone Apps from the Microsoft Azure Cloud MICROSOFT.
OpenField Consolidates Stadium Data, Provides CRM and Analysis Functions for an Intelligent, End-to-End Solution COMPANY PROFILE : OPENFIELD Founded by.
Microsoft Azure and DataStax: Start Anywhere and Scale to Any Size in the Cloud, On- Premises, or Both with a Leading Distributed Database MICROSOFT AZURE.
Course : Study of Digital Convergence. Name : Srijana Acharya. Student ID : Date : 11/28/2014. Big Data Analytics and the Telco : How Telcos.
Saasabi’s Analytical Processing Engine in the Cloud Makes Business Intelligence Affordable for Everyone COMPANY PROFILE: Saasabi Saasabi is a BizSpark.
Built on the Powerful Microsoft Azure Platform, Phyzit Helps Doctors Reduce Readmissions Through a Transitional Care Management App MICROSOFT AZURE ISV.
Increasing Manufacturing Uptime Is Made Easier with RtTech’s Industrial Facilities Application RtDuet, Powered by the Microsoft Azure Cloud MICROSOFT AZURE.
© 2010 IBM Corporation Business Analytics software Business Analytics Editable Text Editable Text Editable Text.
Improve the Performance, Scalability, and Reliability of Applications in the Cloud with jetNEXUS Load Balancer for Microsoft Azure MICROSOFT AZURE ISV.
IBM Digital Communities Solutions Wireless Clouds Connecting Our Communities Riz Khaliq Business Area Leader Community Broadband, IBM Global Government.
Copyright © 2014, SAS Institute Inc. All rights reserved. SAS SOLUTIONS SAS OTTAWA PLATFORM USER SOCIETY NOV 20 TH 2014.
Big Data Javad Azimi May First of All… Sorry about the language Feel free to ask any question Please share similar experiences.
Connect Applications and Business Partners in Integration Cloud, the Reliable and Transparent Integration Environment Built on Microsoft Azure MICROSOFT.
Data-Centric Security and User Access Controls for Hadoop on Microsoft Azure MICROSOFT AZURE APP BUILDER PROFILE: BLUETALON BlueTalon provides data-centric.
András Benczúr Head, “Big Data – Momentum” Research Group Big Data Analytics Institute for Computer.
Mobilise Your Business in Days with Crimson Tide’s mpro5 Enterprise Solution on Microsoft Azure! MICROSOFT AZURE ISV PROFILE: CRIMSON TIDE Crimson Tide.
This presentation was scheduled to be delivered by Brian Mitchell, Lead Architect, Microsoft Big Data COE Follow him Contact him.
Internet of Things ( Ref: Slideshare ). Contents Introduction/Overview The Internet of Things Applications of IoT Challenges and Barriers in IoT Future.
Axis AI Solves Challenges of Complex Data Extraction and Document Classification through Advanced Natural Language Processing and Machine Learning MICROSOFT.
© 2012 IBM Corporation IBM Security Systems 1 © 2013 IBM Corporation 1 Ecommerce Antoine Harfouche.
Smart Grid Technologies Damon Dougherty – Industry Manager.
Power LogOn® Adds Card-Based, Multi- Factor Authentication to Microsoft Azure Logon, Plus Password Management for All Other Logons MICROSOFT AZURE ISV.
Accumulus Delivers Enterprise Class Subscription Billing and Automation Solutions for Gaming, Retail, and More on the Scalable Microsoft Azure Platform.
IT Panel - ISO General Assembly, Oslo 2010© 2010 IBM Corporation1 Building a Smarter Planet IT Panel – ISO General Assembly, Oslo 2010 Adam Jollans,
© 2013 IBM Corporation Complexity must be conquered (Data is more than “just” big) Laura Haas IBM Almaden Research Center 1.
© 2017 SlidePlayer.com Inc. All rights reserved.