We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byNicholas Mould
Modified about 1 year ago
© 2013 IBM Corporation October 4, 2013 IT Analytics and Big Data Making Your Life Easier Paul Smith (Smitty) Service Management Architect
© 2013 IBM Corporation 2 Agenda Data Overload Search, Predict, Optimize How can IT Analytics help?
© 2013 IBM Corporation 3 Software problem led to two days of downtime at the largest bank in Europe has tarnished their image as the most reliable banking website. 3 The Bottom Line: In Today’s World, the App can never go DOWN!!! Trading halted for half a day on the biggest US exchange for financial options following an outage caused by software problems. Not surprisingly, many angry customers poured out their wrath via social networking after the largest video streaming company had a software outage for more than 20 hours Airline canceled more than 700 flights and another 765 flights are delayed due to a software outage – Blamed ticketing partner while the real problem was on their end Every business has 5-10 critical business process and applications. Slowdown or outage have a direct impact on their profits, revenue, customers and brand equity A leading freight company lost $120 million in revenue because IT was unaware that critical warning messages were associated with their key freight delivery application. They were unable to deliver packages for an entire day due to downtime.
© 2013 IBM Corporation 4 Relevant Operations Data is Huge A Typical Enterprise of 5000 servers with 125 applications across 2 or 3 data centers generates in excess of 1.4 TB of data per day 9 Gb Storage Data per day: 175K fiber ports 175 fiber ports,10 metrics per port, collected every 5 minutes,.5KB per port 25K volumes, 10 metrics per volume,.5KB per volume 5KB*(65K ports and volumes)*12*24 = 9.3 GB/day 2Gb Network performance data for Data Center networks 180x64 port Switches and 4 Routers to manage physical network. Data flow of approximately 1TB unstructured data, and.4TB metric data per day, Scaled to 20K servers, approx 4TB unstructured, 1.6TB metric data Daily Metric Output: 250 Mb of event data from 125,000 Events 125Mb of endpoint mgmt data from 5K servers 12 Gb of performance data for 5000 servers 1 Gb of performance for 5000 Virtual Machine 8 Gb or Application middleware data Assumptions: 40% of servers running monitored middleware Average 60 metrics each, collected every 15 min Average PMDB insert 1000 bytes, 40 inserts/server 500 Mb Application transaction tracking data for 125 Applications 1 Tb Log file data per day 200 Mb average per server (some will be smaller, some larger) Example: WAS instances typically produce 400MB-750MB logs/day.35Tb Security data collected per day
© 2013 IBM Corporation 5 Too Much Data Overwhelms IT Managing this much data requires Innovation Too Little: Limit Data Acquisition and risk missing important data Too Much: Flood IT Operations and risk missing important data Just Right? Today, we use Tools, Best Practices, Process, and Experience to get just the right amount of data Just Right: Analytics Solutions to examine all data, learn what is important, and escalate critical problems to Operations staff in a timely way. Just Right: Analytics Solutions to get to the heart of the problem. Just Right: Analytics Solutions to provide actionable insights. Not enough Data leads to Disaster
© 2013 IBM Corporation 6 Predict Enable predictive and preventative operations and application management with next generation behavioral learning analyticsPredict Specialized Capabilities VisibilityVisibility ControlControl AutomationAutomation Plug & Play Architecture Integrated suite of capabilities leveraging existing Application Performance Management, Event Management and Monitoring Solutions Search Accelerate problem resolution through rapid analysis of structured and unstructured data.. Diagnose application and infrastructure issues with expert advice.Search Optimize Optimize resource deployments with what-if and best fit planning tools. Track capacity and performance of applicationsOptimize Enabling business transformation through IT Analytics Capabilities all your datawith ease SearchPredict to do more with less Optimize Operational Environment Network Systems Security Applications VoiceMainframe Storage Wireless Workloads Assets It’s not just performance optimization. We also have to optimize with license cost and sub-capacity pricing in mind.
© 2013 IBM Corporation 7 Enabling business transformation through IT Analytics Predictive Outage Avoidance Ensure availability of applications and services Use learning tools to augment custom best practices Leverage statistical methods to maximize predictive warning Use past maintenance to predict part failures Predict Faster Problem Resolution Find & correct problems faster with tools that determine actions required to resolve issues Identify problems quicker with insight to large unstructured repositories Isolate problems quicker by bringing relevant unstructured data into problem investigations Repair problems quicker with the right details quickly to hand. Resolve Optimized Performance Track, Optimize, and Predict capacity and performance needs over time Track capacity and performance of applications and services in classic and cloud environments Optimize resource deployment with what-if and best fit planning tools Increase utilization of existing assets Perform Improved Insight Enhance visibility into systems resource relationships while increasing customer satisfaction Determine what resources are interdependent to assess impact of failures Gain insight into what is important to your customer Decrease customer churn and acquisition costs while increasing customer retention and satisfaction Know Lower IT Administration Costs with Automated Analytics Escalate performance and capacity issues automatically, reducing manual analysis efforts Reduce manual customization using learning tools that automatically adjust to new normal Detect and present problems with a proposed resolution, to be able to do more with less Advice on Risk based automation to automate low risk tasks and escalate high risk fixes.
© 2013 IBM Corporation 8 IT Operational Analytics PredictiveInsightsLogAnalysis Performance DataUnstructured Data Identify problems quicker with insight to large unstructured repositories Isolate problems quicker by bringing relevant unstructured data into problem investigations Repair problems quicker with the right details quickly to hand. Avoid Outages and service degradation through early detection of abnormalities Improve insight though the analytical discover of metric relationships and trends Reduce root cause analysis by reducing time to isolate faulty components in complex infrastructure “by 2016, 20% of global 2000 enterprises will have IT operations analytics architectures in place...”- Gartner
© 2013 IBM Corporation 9 Predictive Insights - The Problem If no there is no ‘early detection’ before the outage, operations teams can only react while outage is already in effect and already losing money... Why aren’t operations teams preventative today? - Too much data to analyze manually - Existing analytic techniques, such as standard thresholds, are not up to the task - They cannot detect problems while they are emerging (before business impact) - Set threshold too high, insufficient warning before total failure. - Set threshold too low, too much noise, everything is ignored
© 2013 IBM Corporation 10 Multivariate Analytics Statistical models can discover mathematical relationships between metrics The extent this can be achieved depends on a number of factors, such as: range and type of data, availability of data, and stability of environment. Analytics falls back to a single metric if metrics are unrelated. Core Banking Application Core Banking Application z/OS ESB AIX Java / WAS RHEL Oracle Windows Application Internet Banking G G I I B B D D C C E E F F H H A A
© 2013 IBM Corporation 11 Example Scenario: Internet Banking Application Granger based analytics learns the mathematical relationship between metrics Web Response Time WRT Bad WRT Good User Requests Time Web Response Time Anomaly EventBusiness Impacted Early Warning Learns ‘Web Response Time’ has a normal causal relationship with ‘User Requests’ - WRT gets slower as user load gets higher. If this healthy historical relationship breaks down, say due to a memory leak, an anomaly is raised immediately The problem is detected even while WRT service is “good” Emerging problems can be detected even while service level are good in absolute term G G I I B B D D C C E E F F H H A A Internet Banking Web Response Time User Requests Leak Typical Static Threshold
© 2013 IBM Corporation 12 Value Of The Watson Granger-based Analytic Approach Learn normal operational behaviour across the infrastructure, including how metrics behave together. Maximize Advance Warning: Identifies metric relationship changes that signal a problem long before traditional thresholds Identify problems before you know to look for them Detect service impacts that are not identifiable by fixed thresholds alone. Assists with root cause analysis by indicating the most offending metrics. Reduces expensive and time consuming false alerts. Provides a more intelligent real-time assessment of data, able to detect problems as they are emerging
© 2013 IBM Corporation 13 Centralized, Distributed, Cloud, Resilient Architectures Increase Data Volume Log Analysis – The Problem Where do I start?? Everything is “green” It’s SLOW!! 404 ERROR Logs, Traces,.. Events Metrics Transactions Config [10/9/12 5:51:38:295 GMT+05:30] a servlet E com.ibm.ws.webcontainer.servl et.ServletWrapper service SRVE0068E: Core files Find the right needle in the haystack – QUICKLY!
© 2013 IBM Corporation1414 Advanced search and text analytics across large volumes of data Index, search and analyze application, middleware, and infrastructure data Quickly search and visualize application errors across thousands of log records Cross index search across logs and documentation Integrate log search with existing service management tooling to gain multiple perspectives on a specific instance of a problem Log Analysis – Key Capabilities Accelerate problem isolation, identification and repair LogAnalysis 14
© 2013 IBM Corporation Analytics in IT - Capacity Management Definition from ITIL V3 –ITIL Capacity Management aims to ensure that the capacity of IT services and the IT infrastructure is able to deliver the agreed service level targets in a cost effective and timely manner. –Capacity Management considers all resources required to deliver the IT service, and plans for short, medium and long term business requirements. Sub Processes –Component Capacity Management –Service Capacity Management –Business Capacity Management –Capacity Management Reporting 15
© 2013 IBM Corporation Helps consolidate and reduce costs –Reduces HW and labor costs –Reduces number of physical servers required to run workloads –Reduce number of required licenses Helps ensure application availability –Are any resources overloaded? When will physical resources reach their limits? –Have there been any significant changes in my environment between two weeks? –Ensure supply can meet demand –Ensure business policies are met Helps optimize resource utilization –Right size virtual machines –Identify trends for workload balancing Why Capacity Management is important 16
© 2013 IBM Corporation Use Analytics to Forecast You already have the data! Use analytics to: Forecast resource bottlenecks Estimate impact of planned business change Estimate impact of planned outage (ie maintenance) Discover risky components Discover hidden limits and potential unstable components Give input to performance test decisions Experiment with placement of workloads (cost, license, performance, etc) 17
© 2013 IBM Corporation Thank You! 18
© 2013 IBM Corporation October 4, 2013 IT Analytics and Big Data IBM Solutions Paul Smith (Smitty) Service Management Architect.
Accelerating Product and Service Innovation © 2013 IBM Corporation IBM Integrated Solution for System z Development (ISDz) Henk van der Wijk 23 Januari.
© 2007 IBM Corporation IBM Information Management Accelerate information on demand with dynamic warehousing April 2007.
© 2011 IBM Corporation Smarter Software for a Smarter Planet The Capabilities of IBM Software Borislav Borissov SWG Manager, IBM.
Business Process Management: The Third Wave The Next 50 Years of IT.
John Whittle Sales Specialist Case Study: Manage Transactions Across the Enterprise Featuring BMC Middleware Management.
IBM SPSS Solutions A SELECT INTERNATIONAL COMPANY.
StorCast Enterprise Storage Resource Management. What is Enterprise Storage Resource Management?
® IBM Software Group © IBM Corporation 應用程式基礎設備虛擬化 — WebSphere Virtual Enterprise Lilian Wang( 王漪萍 ) WebSphere Technical Sales Support IBM Taiwan Software.
Peter Plevka, BMC Software Managing IT and Your Business – Optimizing Mainframe Cost and Performance.
Satish Babu Best practice license models in the context of the Cloud Date: 22 October 2013 Track 2: Reduce the cost of ICT and accelerating service delivery.
©2011 Quest Software, Inc. All rights reserved.. Database Management Martin Rapetti Business Development Manager.
Smart Grid Big Data: Automating Analysis of Distribution Systems Steve Pascoe Manager Business Development E&O - NISC.
Managing the Oracle Application Server with Oracle Enterprise Manager 10g.
© 2009 IBM Corporation Maximize Cost Savings While Improving Visibility Into Lines of Business Wendy Tam, CDC Product Marketing Manager
= WEEKS, MONTHS, YEARS OF DELAYED APPLICATION VALUE MISSED REVENUE OPPORTUNITIES, INCREASED COST AND RISK DEV QA PACKAGE COMMERCIAL SOFTWARE CUSTOM APPLICATION.
1 © 2014 by McGraw-Hill Education. This is proprietary material solely for authorized instructor use. Not authorized for sale or distribution in any manner.
The Importance Of Transactions In The World Of Analytics Doug Aoyama Director, Product Marketing.
Blazent / ServiceNow Messaging Guide. Transforming data into actionable intelligence Improve business outcomes by contextualizing data to make informed.
Demonstrating IT Relevance to Business Aligning IT and Business Goals with On Demand Automation Solutions Robert LeBlanc General Manager Tivoli Software.
1© Copyright 2015 EMC Corporation. All rights reserved. FEDERATION ENTERPRISE HYBRID CLOUD OPERATION SERVICES FULL RANGE OF SERVICES TO ASSIST YOUR STAFF.
Highlights Builds on Splunk implementations – extending enterprise value to include mission-critical IBM mainframe data. Unified mainframe data source.
© 2009 IBM Corporation IBM Cloud Computing Tivoli Service Automation Manager V7.2 The Core of the Service Management System for Cloud Computing.
The Business Value of CA Solutions Ovidiu VALEANU Senior Consultant DNA Software – CA Regional Representative.
The VERSO Product Returns Portal Incorporates Office 365 Outlook and Excel Add-Ins to Create Seamless Workflow for All Participating Users OFFICE 365 APP.
Microsoft and Symantec More Than The Sum Of Our Parts.
Cisco Consulting Services for Application-Centric Cloud Your Company Needs Fast IT Cisco Application-Centric Cloud Can Help.
© 2010 IBM Corporation Business Analytics software Business Analytics Editable Text Editable Text Editable Text.
IBM Software Group Tivoli Software from IBM Storage Resource Management Webcast Tele-Rep Training Manual Prepared by: Wunderman Customer Dialogue Group.
Testing in the Cloud with Tosca Testsuite: A Comprehensive Test Management and Test Automation Suite Built on Microsoft Azure MICROSOFT AZURE ISV PROFILE:
+ Logentries Is a Real-Time Log Analytics Service for Aggregating, Analyzing, and Alerting on Log Data from Microsoft Azure Apps and Systems MICROSOFT.
ICS Area Managers Training 2010 ITIL V3 Overview April 1, 2010.
Axis AI Solves Challenges of Complex Data Extraction and Document Classification through Advanced Natural Language Processing and Machine Learning MICROSOFT.
Never Down? A strategy for Sakai high availability Rob Lowden Director, System Infrastructure 12 June 2007.
Easily Organize Common Templates, Phrases, Statements and More to Quickly Reuse Within Microsoft Office 365 Using Dolphin Compose WHAT WE OFFER Replace.
IBM Americas SMB © 2007 IBM Corporation 1 IBM Americas SMB Infrastructure Solutions Marketing 2007 IT Optimization: Getting more out of your IT with help.
Data Center Management Microsoft System Center. Objective: Drive Cost of Data Center Management 78% Maintenance 22% New Issue:Issue: 78% of IT budgets.
- Kanwalpreet Singh - Kanwalpreet Singh Performance Testing.
Time lag between discovering issue and resolving Difficult to find solutions and patches that can help resolve issue Service outages expensive and.
Practical IT Research that Drives Measurable Results 1Info-Tech Research Group Get Moving with Server Virtualization.
© 2014 Level 3 Communications, LLC. All Rights Reserved. Proprietary and Confidential. Simple, End-to-End Performance Management Application Performance.
© 2015 IBM Corporation IBM PureApplication Executive Symposium Diego Segre Vice President, Middleware, Break down the barriers to digital.
SYSchange for z/OS By Pristine Software April 2009 Thomas Phillips April 2009 SYSchange Pristine Software.
Copyright © 2015 John Wiley & Sons, Inc. All rights reserved. 6-1 FitzGerald ● Dennis ● Durcikova Prepared by Taylor M. Wells: College of Business Administration,
+ Big Data IST210 Class Lecture. + Big Data Summary by EMC Corporation (http://www.emc.com) More videos that.
Take Control of Your Contracts with Dolphin 365, a Companion Product to Microsoft Office 365 That Leverages Your Investment and Reduces Risk OFFICE 365.
1 Real-time End-to-End Transaction Visibility into Distributed and Mainframe Applications Steve Saville Mainframe Technical Account Manager – Compuware.
® IBM Software Group © IBM Corporation IBM Information Server Understand - Information Analyzer.
INNOVATE THROUGH MOTIVATION Mobile Computing & Your Business KEVIN KIRKPATRICK – OWNER, MSP INC LOGO.
Office 365 is cloud- based productivity, hosted by Microsoft. Business-class Gain large, 50GB mailboxes that can send messages up to 25MB in size,
© 2017 SlidePlayer.com Inc. All rights reserved.