Download presentation
Presentation is loading. Please wait.
Published byWillis Hawkins Modified over 7 years ago
0
Getting More Intelligence from Your Mainframe: A Look at z IT Operational Analytics Solutions
1
Disclaimer Statement IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
2
Agenda Overview of analytics Our portfolio Product capabilities Q&A
3
Industry trends Costs and outages are top of mind for clients
A recent survey of z Systems clients showed cost reduction and outage prevention as the top 2 factors where they want to focus operational efforts Interest in IT Operations Analytics is on the rise Market dynamics show growth in IT Operations Analytics (ITOA) and decline in IT Operations Monitoring (ITOM), shifting from pure ITOM to an ITOM+ITOA hybrid SaaS is a growing consumption model Cloud-based delivery is growing 4x faster than on-premise delivery 60% of z Systems clients foresee their company adopting some form of cloud-based tooling between now and 2018 1 IDC – International Data Company CAGR =- Compound Annual Growth Rate 1) Market need for Combined ITOA capabilities with end-to-end capability The first generation of modern IT operations analytics software focused on log analytics and search Predictive analytics, anomaly detection, and business impact analysis based on end user, infrastructure, and application performance data are increasingly being paired with analysis of machine-generated logs to enable comprehensive root cause analysis Outage avoidance and efficiency savings through predictive analytics and SaaS Source: Dec 2015 survey of 32 z Systems customers – contact Anna Bridgen
4
Our Portfolio
5
“Insight to action in half the time” Investigate and Automate
Predict: Pro-Active Outage Avoidance Predict Problems before occurrence Log anomaly detection IT Operational Analytics Solutions on z Systems “Insight to action in half the time” Optimization IBM Capacity Mgmt Analytics. NEW - IBM z Operational Insights (SaaS) Predict NEW - IBM zAware (Anomaly Detection) IBM Operations Analytics – Predictive Insights Investigate and Automate NEW - IBM Operations Analytics for z Systems NEW - IBM Common Data Provider for z/OS Capacity Management Software cost analysis Enterprise resource and workload optimization SME insights in cloud (SaaS) Log Analytics Domain insights and expert advice Alert notification and automation Unified data collection for logs and SMF Pro-Active Outage Avoidance Predict Problems before occurrence Log anomaly detection Main Point: Analytics is now a key part of what customers are looking to improve on. As we have seen, analytics can help increase business value and IT metrics. Analytics is about: 1. Predict problems and anomalies – Current product is OMEGAMON V5.1.1 with IBM zAware support and Netview which also includes IBM zAware support 2. Search log information and reduce mean time to root cause – The current product in this area is IBM Operation Analytics for z Systems 3. Optimize analytics for both Business and IT – Capacity Management Analytics (CMA) for z/OS, is a suite that includes SPSS, Cognos and TDSz. Main Point: Analytics is now a key focus for our customers. As we have discussed, Operations Analytics can help increase business value by ensuring system and application availability and reducing Mean Time to Repair (MTTR). Operations Analytics is about: Predict - Proactively surfacing problems using anomaly detection. The current solution is IBM zAware. IBM zAware surfaces anomalies by analyzing z/OS and zLinux system logs. OMEGAMON and NetView integrate with IBM zAware by monitoring the IBM zAware anomaly scores, correlating log analysis with performance monitoring and providing the option to generate events and trigger automation. Search - Search for information, including logs and metrics to enable a much more efficient environment for performing problem determination. The current solution in this area is IBM Operations Analytics for z Systems. IOA for z Systems integrates with ITM/OMEGAMON and Network Operations Insights. Optimize – Provides analytics for both Business and IT. Capacity Management Analytics (CMA) for z/OS, is a suite that includes SPSS, Cognos and TDSz. CMA enables customers to forecast capacity and more recently provides a feature for forecasting the 4 hour rolling average enabling customers to manage subcap pricing. Predict: Pro-Active Outage Avoidance Predict problems before they occur Search & Analyze: Quickly search and analyze large volumes of data from a single search bar Perform log and performance analysis while searching Correlate messages from multiple logs for end-to-end problem diagnosis Optimize: Improve performance across IT Infrastructure On Premise Hybrid Cloud Predict NEW IBM zAware (Anomaly Detection) IBM Operational Analytics – Predictive Insights Investigate and Automate NEW - IBM Operational Analytics for z/OS
6
IBM Operations Analytics for z Systems v3.1
New Reduce Operational Cost, Bring Analytics to your Data, Smarter IT Service Management with Rapid Time to Value New Problem Insights View Consolidated view across the system for root cause analysis z/OS Security Insight Pack See patterns of security incidents by user or resource Analyze critical operations data Includes zAware Software Appliance Advanced machine learning to detect abnormal system behavior Insurance industry client example Experienced an application outage that resulted in the team working around the clock for 29 hours After the issue was resolved, the logs were captured and sent to IBM lab for analysis using IBM Operations Analytics for z Systems Within minutes, the IBM team was able to focus in on the root cause of problem and find the relevant PTF to resolve the issue ibm.biz/ioazlivedemo New Problem Insights View: Dynamically assists with Root Cause Timeline of impacting messages across key subsystems Top Hot Messages Table that recommends Next Steps z/OS Security Audit Events Insight Pack Analyze critical operations data related to business workloads VSAM ESDS source Websphere App Server z/OS (SMF120 Type 9) Fill your data lake with z operations data using LogStash zAware deliveray as Software Appliance Main Point: Search and analysis is the primary focus for Log Analytics and IBM Operations Analytics – Log Analysis provides this capability. This tool will enable you to perform problem determination and resolution more quickly and will ultimately decrease Mean Time To Recovery (MTTR). The Log Analysis server runs on Linux on x Systems or Linux on z Systems. The server can consume logs from multiple sources (distributed and mainframe systems), enabling users to search and analyze log data from all components of your cross-platform workloads or from all the log sources in your enterprise if you so choose. Customers are already seeing value from Analytics – One of the key values with IBM Operations Analytics is the ability to create Insight Packs designed to analyze specific logs. The offering named IBM Operations Analytics for z Systems includes the Log Analysis server as well as z/OS Insight Packs that enable search and analysis for z/OS logs and performance metrics. The initial release of the z/OS support was provided in March, 2014 under the product names ‘IBM SmartCloud Analytics - Log Analysis z/OS - Insight Packs – SYSLOG V1.1’ and ‘IBM SmartCloud Analytics - Log Analysis z/OS - Insight Packs - IBM WebSphere® Application Server V1.1’. Subsequent releases were named with the SmartCloud brand until April, 2015 when Version 2 of the product was rebranded to IBM Operations Analytics for z Systems V2.1. IBM Operations Analytics for z Systems provides the following: • Ability to collect z/OS logs across the enterprise and stream the logs to the Log Analysis server for the server to index and analyze. • Ability to index, search, and analyze application, middleware, and infrastructure log data across System z enterprise. • Ability to quickly search and visualize errors across huge volumes of log records. • Advanced search and text analytics across large volumes of data. • Expert advice by linking search results to available best practices and recommended resolution documentations. • Near real-time streaming of z/OS logs. The z/OS support consists of the following components: • z/OS log forwarder that is installed on the required z/OS LPARs where the logs are to be collected and forwarded. • SMF data provider that is installed on the required z/OS LPARs where SMF performance metrics are to be collected and forwarded. • Insight Packs to provide the index, search, and domain insights capability for logs and performance metrics. Search is provided for all messages in the logs and you can choose to search one or more or all logs. The user can also specify a timeframe of the search to help narrow the focus to the time period when the error occurred. The Insight Pack surfaces patterns as the logs are searched, enabling the user to quickly focus on errors and drill down to the offending problem area. IBM Operations Analytics for z Systems provides out-of-the-box insights and application views for z/OS, WebSphere, DB2, CICS, IMS and MQ with the addition of Network Insights in V2.1. Also in V2.1, we have included initial support for consuming and analyzing performance metrics using our SMF Data Provider component. The user interface is customizable such that users can build their own application views and create and save environment-specific queries. The search language is text based and easy to use, and users can easily create and save simple or complex search strings with minimal typing. The tool is helpful to novice as well as experienced users. Online help, product documentation and product videos are easily accessed from the Getting Started page. 5698-AAP V2.1.0 IBM Operations Analytics for z Systems Large Insurance Company – Customer story 1 Quote: “This tool can really save a pile of diagnostic time! “ Customer experienced a problem that took 29 hours to debug. This process required time from both IBM (Level 2) and multiple employees from that company. The account team contacted the IBM development team and described an outage at the customer site. The development team received the Syslogs from the customer, fed them into Operations Analytics Server and immediately saw the high volume of error messages on the two LPARs (thousands of error messages were Severe errors). Most errors were in DB2 and MQ. The development team immediately noticed the high volume of some very specific messages (mostly DB2). The Log Analysis Application views graphically displayed the message peeks (as compared to normal message flows). ‘Needles’ (error messages) in the haystacks (LPARs) were immediately evident through visual representation of the message spikes. Ultimately, the problem was caused by a bad PTF that was applied as part of a z/OS maintenance window. The Expert Advice feature was used to pinpoint the relevant maintenance to fix the problem (based on the error messages that were generated). One member of the development team was able to pinpoint the problem using IBM Operations Analytics for z Systems in under 30 minutes … It went from 29 hours to 29 minutes. Moral of the story - IBM Operations Analytics for z Systems would have helped decrease the amount of time required for problem determination. The log analysis provided by IBM Operations Analytics for z Systems would have highlighted the high volume of error messages visually (in both the application views AND the insights (message pattern detection) to determine the scope of the problem (ie which systems are affected) and identify which additional components are affected (ie MQ, IMS, CICS, etc.). Once the focus was narrowed down to the problem area, the Expert Advice feature was used to perform a quick search of the IBM support site to identify a fix for the problem (PTF, technote, white paper, etc.). Another Insurance Company – Customer story 2 Quote: “This tool can quickly prove it is not my fault!” The DB2 support team within the customer shop often spends many hours isolating problems to discover it is not in fact a DB2 problem and needs to be routed to another group. In this specific case in point, there were serious MQ errors and the DB2 team spent hours isolating the problem as an MQ problem. With IBM Operations Analytics for z Systems, it was proven that the team could have gone directly to the source of the issue immediately. This would have saved them hours, and cumulatively days, of spinning unproductive cycles and they could have routed the issue to the internal MQ support team immediately. Large Bank – Customer Story 3 Quote: “Faster than a speeding Bullet! “ Customer is running a WAS-based On-line Banking Application in a couple of datacenters. Often when they receive a trouble ticket from their external customer (i.e. the user of their online banking application), they cannot determine which datacenter originated the error messages. With IBM Operations Analytics for z Systems’ ability to consolidate logs, they stated they could reduce their initial isolation time significantly (maybe 50%) Government Agency IT department - Customer story 4 Quote: “Talk about Time to Value! “ In a recent customer engagement, the client was able to download, install and configure the solution and had an operational environment in 2.5 hrs!
7
IBM zAware v3.1 New Cognitive infrastructure to detect anomalous systems behavior in near real-time I zAware host Linux on system z z/OS IBM zAware zAware monitored clients IBM zAware Web GUI to monitor results z/VM New Software Appliance Delivery Deploy and launch within minutes Enhanced Proactive Anomaly Detection Act before there is a service outage via notification New Historical View See the history of an anomalous message in order to accelerate problem resolution Integrated with IBM Operations Analytics Launch directly to logs in the context of an anomaly Insurance industry client example Experienced an application outage that resulted in the team working around the clock for 29 hours After the issue was resolved, the logs were captured and sent to IBM lab for analysis using IBM Operations Analytics for z Systems Within minutes, the IBM team was able to focus in on the root cause of problem and find the relevant PTF to resolve the issue Monitors z/OS and Linux on z images running natively or as a guest ibm.biz/ioazlivedemo
8
IBM Common Data Provider for zSystems 1.1:
Simple to install, configure and use Built in filtering to control data volumes Multiple data sources Flexible output options Write to any destination Streaming SMF and log data IBM Common Data Provider for zSystems 1.1: Real Time Access to Analytics New A single source for all operational data streamed to the analytics platform of choice Simple to install, configure and use Multiple data sources Flexible output options Write to any destination Streaming SMF and log data Built in filtering to control data volumes CDP provides consumable, near real time operational data Built to improve the ability to manage the growing complexity of data requests Tivoli Decision Support for z/OS customers can write their SMF data directly to IDAA Reduce Risk to you Business: Detect threats with your Security products using live streaming data Optimize Costs and Efficiencies: Feed all IT Operations data to analytical engines from a single product Prevent Impact to Your Operation: Proactive Analysis of data in near Real Time as an early warning CDP provides consumable, near real time operational data Web-based interface to easily configure sources, transforms and destinations Data gatherers on the HOST easily installed in minutes Data available both on and off platform in near real time or batch mode Built to improve the ability to manage the growing complexity of data requests: All standard IBM SMF records can be collected in readable, consumable CSV format Collect once – write many saves time and money Open standard makes analytical data available to IBM and non-IBM analytics platforms Tivoli Decision Support for z/OS customers can write their SMF data direct to IDAA Operational and storage savings Key performance metrics available in near real time Access to the IDAA high speed query engine First consumer: SMF Data streamed to IOAz for a complete view of the enterprise
9
Breaking new ground in the Cloud with IBM z Operational Insights (SaaS)
Expert cost & performance insights for z Systems in minutes. Just add operational data. IT Service Management software, delivered as SaaS, addresses key pain points: z Systems performance & running costs Pressure on staff availability & SME skills Time-to-value of traditional on-premise tooling Analytics on z operational data, with embedded IBM expertise ‘CICS Essentials’, with intent to expend across subsystems Try IBM zOI for yourself at ibm.biz/try-zoi Potential benefits quantified upfront Tailored recommended actions First-in-kind comparisons to other users Built on expertise from z performance SMEs Clean, modern web browser-based interface Easy value assessment – free SaaS trial Time to value = minutes – SaaS means no install, deployment or configuration, and rolling updates Easy value assessment – free SaaS trial available through IBM Marketplace, accessible in seconds Embedded IBM expertise from z performance SMEs – you are guided to key optimization areas Potential optimizations/savings quantified upfront – providing ROI for project justifications A tailored experience – dynamically updated recommended actions, based on data analysis First-in-kind comparisons – benchmark your z environment’s performance vs others, a novel feature A delightful user experience – a clean and modern web browser-based interface
10
Product Capabilities IBM Operations Analytics for z Systems
11
IBM Operations Analytics for z Systems V3.1
Problem Insights and Anomaly Identification … Understanding the unknown NEW in V3.1: Inclusion of IBM zAware v3.1 anomaly detection with Problem Insights Rapid analysis of vast amounts of data, and even more data types New z/OS Security Audit Events Insight Pack Use in conjunction with NEW Common Data Provider v1.1 Key Values: Built-in IBM expertise to predict issues instead of waiting for failure, for fewer outages Analyze & intelligently search ops data with anomaly detection, for faster root cause analysis and Mean Time to Recovery
12
Problem Insights Automatically surfaces important messages found in the log data. Provides easy to read problem summary and suggested actions for problem resolution. Displays Anomaly Interval scores from IBM zAware. Link to search for this message in this time period Count of this message over the timeframe Total number of Problem Insights found per Sysplex Interval Score column shows anomaly score for the last time the problem occurred on this system Click to show the suggested actions for this message
13
Suggested Actions The Suggested Actions are presented as a pop-up window. Suggested Actions to investigate/resolve the issue Link to the Knowledge Center for this message ID
14
IBM Support Portal based Expert Advice
Search for expert advice with the click of a button Launch from client or server side Launch to Tech Note All IBM support site documents that reference messages from search results
15
Product Capabilities IBM zAware
16
IBM zAware v3.1 Software Appliance (included in IOAz v3.1)
IBM zAware v Benefits from Anomaly Identification NEW in V3.1: Anomaly detection with z/OS and Linux for z message logs Improved consumability with zAware delivered as a software appliance feature with IBM Operations Analytics for z systems Proactive outage avoidance with Alerts for identified anomalies View the history of an anomalous message for faster problem resolution Launch in Context into logs in the context of the Anomaly for quick message id search Key Values: Built-in IBM expertise to provide proactive outage avoidance instead of waiting for failure to happen; thus fewer outages Improve problem determination intelligently using anomaly detection on IT operational data for better Mean Time to Recovery
17
Integration between IOAz and zAware
zAware interval anomaly scores link back into zAware Select message and Launch to IOA/Search
18
Product Capabilities IBM Common Data Provider for z Systems
19
IBM Common Data Provider
The Common Data Provider (CDP) was driven by customer requests to address the growing Operational Analytics requirement. The CDP provides: A single source for z/OS Operational Data in a flexible, consumable format both on and off platform Near real-time data feed of SMF data and log data Single interface that is easy to configure and use The product is OTC so data collection is NOT ingestion based Read once - write many Multiple destinations in different formats for different consumers Batch data collection also available for deep dive analysis or to control CPU consumption Documented protocols and formats for sending and consuming data are provided, enabling data ingestion to widely used Industry Analytics Platforms or Enterprise-specific solutions for access and analysis Vision and Purpose An interactive framework for combining multiple views of the same data to provide a deeper understanding of the Enterprise
20
Maximize Existing Investments
IBM DB2 Analytics Accelerator IBM Tivoli Decision Support for z/OS (TDSz) customers can leverage their existing reporting systems by loading TDSz data direct in to the IDAA Today: TDSz and IDAA Copy from DB2 to IDAA. Time consuming and expensive SMF Data DB2 TDSz IDAA IDAA/SQL queries batch copy CDP Solution: Direct load to IDAA with TDSz schema Direct load of TDSz SMF data to IDAA with TDSz schema No data lands in DB2 tables (reduced DB2 footprint) Saves CPU on copy and aggregation Allows you to keep more data over a longer period Ability to store and query timestamp and interval data for deep dive No need to change reporting systems Save time and money DB2 CMA TDSz Schema IDAA IDAA/SQL queries SMF Data CDPz IDAA LOAD Standard TDSz Reports The challenges: – maintenance, deployment, configuration takes time - aren't sure where to start - current roadmap to improving has a large number of steps How we help: - We host the service - Build in expertise - Reduce time to value
21
Common Data Provider Architecture
Data Flow z/OS Common Data Provider Consumers On Platform Configuration Data Gatherers Data Handling Fixed Data System Data Engine IDAA TDSz CMA Consumers Off Platform batch Multiple IBM Operations Analytics On Premise SMF Data Streamer Non IBM Analytics Platforms Log Data Log Forwarder Web UI Enterprise-Specific Operations Data Solutions Syslog DB2 WAS MQ VSAM z/OSMF Three main component types Data Gathers – flexible, customizable, efficient Data Streamer – controls data formats and destinations User Interface – simple intuitive configuration Configuration
22
Data Gatherers System Data Engine Log Forwarder
Based on 30+ years of engineering Designed to collect and process SMF data No DB2 prereq – installed and usable within hours Data can remain unprocessed or unpacked into a readable, consumable format Formats the data into multiple formats for ease of ingestion (eg CSV or DB2 LOAD format) Supports all standard IBM SMF types Supports multiple sources of SMF data – archive, logstream or direct from the new SMF Buffer api in near real time Has built-in filtering to control data types and volumes Log Forwarder Gathers a variety of log data and some VSAM file formats for Analytics Engines Additional log support planned through ongoing continuous delivery Custom log types can be added and any dataset can be streamed giving great flexibility
23
Streaming Live Data The Data Streamer controls the destination and format of the Operations Data Receives data from the gatherers Splits and Annotates the log data into individual messages for ingestion into analytic engines Transforms data messages into the right format for the destination platform (eg UTF-8 and other code pages) Transport mechanism is TCP/IP – available as TSL for additional security Data sent in json wrapper for ingestion by Logstash for storage and analysis Extendable to other platforms like ELK and SPLUNK Streams data both on and off platform ziiP enabled for cost savings (pure Java) The challenges: – maintenance, deployment, configuration takes time - aren't sure where to start - current roadmap to improving has a large number of steps How we help: - We host the service - Build in expertise - Reduce time to value
24
Web Configuration Tool
The data sources, transformation and destinations are managed and controlled through a simple user interface Plug-in for z/OSMF Menu driven to configure: Data streams and their sources Any transformation requirements Output format Destination (on or off platform) Security Push Based model Host-based policy controls subscribers and data sources Policy can be secured by RACF for total control of data and subscribers The challenges: – maintenance, deployment, configuration takes time - aren't sure where to start - current roadmap to improving has a large number of steps How we help: - We host the service - Build in expertise - Reduce time to value
25
Product Capabilities IBM Operational Insights for z Systems
26
zOI Home page The challenges:
– maintenance, deployment, configuration takes time - aren't sure where to start - current roadmap to improving has a large number of steps How we help: - We host the service - Build in expertise - Reduce time to value
27
zOI CICS Java Offload The challenges:
– maintenance, deployment, configuration takes time - aren't sure where to start - current roadmap to improving has a large number of steps How we help: - We host the service - Build in expertise - Reduce time to value
28
zOI Benchmarking The challenges:
– maintenance, deployment, configuration takes time - aren't sure where to start - current roadmap to improving has a large number of steps How we help: - We host the service - Build in expertise - Reduce time to value
29
CICS Threadsafe The challenges:
– maintenance, deployment, configuration takes time - aren't sure where to start - current roadmap to improving has a large number of steps How we help: - We host the service - Build in expertise - Reduce time to value CICS OTE is an architecture introduced first in CICS V1.3 that was introduced for the following purposes: -To allow CICS to make better use of the mainframe With OTE, CICS can run more processes in parallel, increasing the throughput of work through the system and resulting in more work being done in the same amount of time. To benefit from the power of OTE, you must ensure that your applications are threadsafe. Having threadsafe applications ensures that, if the mainframe has many processors, and many processes are running in parallel, the threadsafe application runs correctly, and the right result is achieved. CICS ensures that its code runs correctly, but customers must ensure that their COBOL code, for example, implementing payroll, accounts, and ledger, runs correctly. If an application is threadsafe, it can be defined to CICS with a CONCURRENCY keyword so that it uses OTE. If an application is not threadsafe, CICS runs it without using OTE. Applications that cannot use OTE must run on the main CICS task control block (TCB), the quasi-reentrant (QR) TCB. Applications that use OTE can run on a CICS open TCB. A CICS system has only one QR TCB, and the CICS dispatcher shares use of the QR TCB between all the tasks. However, a single CICS system can have many open TCBs. Using OTE effectively keeps an application running on an open TCB for as long as possible and minimizes the number of times it must switch back to the QR TCB. The result is processor savings and improved throughput because the open TCBs can run in parallel and take advantage of the multiprocessor mainframe.
30
CICS Abend Analysis The challenges:
– maintenance, deployment, configuration takes time - aren't sure where to start - current roadmap to improving has a large number of steps How we help: - We host the service - Build in expertise - Reduce time to value
31
CICS CPU Region Constraints
The challenges: – maintenance, deployment, configuration takes time - aren't sure where to start - current roadmap to improving has a large number of steps How we help: - We host the service - Build in expertise - Reduce time to value
32
Thank You
33
z IT Operational Analytics Announcements
Predictive analytics for proactive outage prevention NEW: IBM Operations Analytics for z Systems v3.1 The new release which now includes IBM zAware v3.1 as a software appliance NEW: Inclusion of IBM zAware v3.1 can now provide IOAz program insights with anomaly detection for faster problem determination and better proactive outage avoidance paired with comprehensive root cause analysis that provides recommended actions. Operational efficiency, delivered in the cloud NEW: IBM z Operational Insights (z SaaS offering) Cloud-based analytics with embedded expertise from performance experts Just upload data to populate reports crafted by IBM performance experts Get estimated savings, expert recommendations for actions, and comparisons to other users IT Analytics your way: Smarter data handling NEW: IBM Common Data Provider for z/OS v1.1 A single source for z/OS Operational Data in a flexible, consumable format both on- and off-platform Can supply data to IBM analytics solutions, as well as other analytics platform targets Can supply faster DB2AA and IDAA loading of valuable z Systems data
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.