IBM Operations Analytics for z Systems Transforming Data into Insights The Next Generation of IT Service Management.

IBM Operations Analytics for z Systems Transforming Data into Insights The Next Generation of IT Service Management

Note to the presenter … This deck contains SEVERAL slides
Note to the presenter … This deck contains SEVERAL slides. It is intended to be modified by YOU to include the slides that YOU need for YOUR specific customer presentation. If you are just looking for the high level overview of IOAz (‘short’ deck) OR the 1-pager slide, please retrieve those from the sales kit:

Agenda Why IT Analytics?
Overview of IBM Operations Analytics for z Systems Functional capability What’s New in 2015 Architecture Out-of-the-box Value Customize to meet your needs Integration with Service Management tooling Additional Detail Bring Your own Data – Example using HMC log IOAz V2.2 Details CICS insights Network insights Security insights Log Forwarder improvements

Solution Branding IBM Operations Analytics for z Systems
This solution was previously branded as IBM SmartCloud Analytics - Log Analysis. The support to search and analyze z/OS logs was initially provided in March, 2014 under the following product names: IBM SmartCloud Analytics - Log Analysis z/OS - Insight Packs – SYSLOG V1.1 IBM SmartCloud Analytics - Log Analysis z/OS - Insight Packs - IBM WebSphere® Application Server V1.1 Subsequent releases were named with the SmartCloud brand until April 2015, when Version 2 of the product was rebranded to IBM Operations Analytics for z Systems Initial release under the new name: IBM Operations Analytics for z Systems v2.1 (GA on April 24, 2015) Current release: IBM Operations Analytics for z Systems v2.2 (GA on October 16, 2015) Note that the distributed version of the product is now named IBM Operations Analytics – Log Analysis

Smarter Infrastructure
Rapid growth of data from latest technologies can be supported seamlessly on z Systems z Systems scaling model and security to manage and optimize both Social, Mobile, Analytics Smarter Infrastructure Systems of Record Systems of Engagement Main Point: The IT industry is changing, but System z continues to be a key part of it. Much of this Cloud, Mobile and Social innovation is starting to be enabled by what is called “systems of engagement” that leverage ubiquitous cloud computing models, pervasive tooling and mobile access to bridge traditional IT “Systems of Record” to drive interactions closer to the customers and leverage relationships that are enabled by this shift. The amount data being generated by both SOR and SOE are growing rapidly. The opportunity to capture markets through optimized customer interaction is driving rapid innovation and iteration in the cloud leveraged by these new systems. At the same time infusion of intelligence in physical assets such as automobile, building systems, electrical utilities and traffic control systems, require models that can more easily scale to collect data and deliver content. Systems of Record are characterized by being what we think of as System z today, transactional, database, Command and Control. Systems of Record will be key in providing the data, security and availability needed for the new 24/7 requirements that come from Systems of Engagement. Systems of engagement are the new technologies, and System z can support them just as well. Linux on System z is a great platform that provides the security, availability and reliability of zEnterprise and supports Linux workloads. Both components are needed to successfully implement new business requirements driving by Big Data. Business Transactions Quality of Service Command & Control Facts and data “source of truth” z/OS Systems Mobile and Social Dynamic Interactions and Collaboration Insight, trends, analytics 5 5 5

Analytics for System z addresses rapid growth of data and next generation technology
Much greater amount of critical IT operational data (SMF, log, journal) than distributed-only environments. Focus on problem determination and time to resolution while placing premium on availability of services and applications. 100x to 1000x explosion in data flooding existing tools. New runtimes, programming languages needing complex instrumentation. By 2016, 40% of Global 2000 enterprises will have IT operations analytics architecture in place, up from < 1% in 2014, looking to integrate across their enterprise to reduce outages (Gartner). 90% of the Fortune 1000 companies are running z and have ‘Systems of Record’ dependencies for transactional processing and data serving applications. Main Point: As technology improves and data increases, there is a requirement be able to predict, search and optimize this new/additional data to gain insights from it that have not existed in the past. 6 6

Is managing IT today like sipping from a fire hose?
New Technologies like cloud, mobile and big data already challenging current Enterprise tools Too long to isolate, diagnose problems in applications and infrastructure. Complex application workloads span multiple platforms Increasing amounts of IT data: Performance metrics, events, infrastructure logs, application logs, configuration files, traces Existing IT tools need additional data analysis capabilities to manage of Systems of Engagement 100x to 1000x explosion in data flooding existing tools. New runtimes, programming languages needing complex instrumentation. Reactive analytics misses critical information leading to outages Need to move to a more proactive model Analysing ALL information better for predicting problems. Is managing IT today like sipping from a fire hose?

IBM is focused on managing end-to-end analytics for
improved performance and workload management Predict: Pro-Active Outage Avoidance Predict problems before they occur Search & Analyze: Quickly search and analyze large volumes of data from a single search bar Perform log and performance analysis while searching Correlate messages from multiple logs for end-to-end problem diagnosis Optimize: Improve performance across IT Infrastructure IBM Analytics solutions for z Systems Proactive Outage Avoidance Faster Problem Resolution Optimized Performance Main Point: Analytics is now a key focus for our customers. As we have discussed, Operations Analytics can help increase business value by ensuring system and application availability and reducing Mean Time to Repair (MTTR). Operations Analytics is about: Predict - Proactively surfacing problems using anomaly detection. The current solution is IBM zAware. IBM zAware surfaces anomalies by analyzing z/OS and zLinux system logs. OMEGAMON and NetView integrate with IBM zAware by monitoring the IBM zAware anomaly scores, correlating log analysis with performance monitoring and providing the option to generate events and trigger automation. Search - Search for information, including logs and metrics to enable a much more efficient environment for performing problem determination. The current solution in this area is IBM Operations Analytics for z Systems. IOA for z Systems integrates with ITM/OMEGAMON and Network Operations Insights. Optimize – Provides analytics for both Business and IT. Capacity Management Analytics (CMA) for z/OS, is a suite that includes SPSS, Cognos and TDSz. CMA enables customers to forecast capacity and more recently provides a feature for forecasting the 4 hour rolling average enabling customers to manage subcap pricing. Predict Search & Analyze Optimize IBM zAware IBM Capacity Management Analytics (CMA) IBM Operations Analytics for z Systems 8

IBM Operations Analytics for z Systems
Accelerate problem isolation and identification … Reduce mean time to repair Analyze various types of data (logs, metrics, events, trouble tickets) from multiple sources (mainframe and distributed) Locate problems from system, configuration, software logs and performance metrics using rapid index search and pattern analysis Isolate issues across various domains including OS, Middleware, applications, etc. Leverage Expert Advice via links to support documentation and operations notes to resolve problems quickly Visualize search results with analytic tools to rapidly determine root cause Out-of-the-box analysis and insights for z/OS, WebSphere, DB2, CICS, IMS, MQ, Network, Security as well as distributed systems Enable early error detection and broaden scope of automation with event notifications Fully customizable to meet your needs Network insights Security insights Event notification Hadoop support Analysis of performance metrics (SMF real time Data Provider) Integration with existing Service Management tooling (Automation, Monitoring, Event and Incident Management) Role-based access control Multi-time zone support in 2015 Main Point: Search and analysis is the primary focus for Log Analytics and IBM Operations Analytics – Log Analysis provides this capability. This tool will enable you to perform problem determination and resolution more quickly and will ultimately decrease Mean Time To Recovery (MTTR). The Log Analysis server runs on Linux on x Systems or Linux on z Systems. The server can consume logs from multiple sources (distributed and mainframe systems), enabling users to search and analyze log data from all components of your cross-platform workloads or from all the log sources in your enterprise if you so choose. Customers are already seeing value from Analytics – One of the key values with IBM Operations Analytics is the ability to create Insight Packs designed to analyze specific logs. The offering named IBM Operations Analytics for z Systems includes the Log Analysis server as well as z/OS Insight Packs that enable search and analysis for z/OS logs and performance metrics. The initial release of the z/OS support was provided in March, 2014 under the product names ‘IBM SmartCloud Analytics - Log Analysis z/OS - Insight Packs – SYSLOG V1.1’ and ‘IBM SmartCloud Analytics - Log Analysis z/OS - Insight Packs - IBM WebSphere® Application Server V1.1’. Subsequent releases were named with the SmartCloud brand until April, 2015 when Version 2 of the product was rebranded to IBM Operations Analytics for z Systems V2.1. IBM Operations Analytics for z Systems provides the following: • Ability to collect z/OS logs across the enterprise and stream the logs to the Log Analysis server for the server to index and analyze. • Ability to index, search, and analyze application, middleware, and infrastructure log data across System z enterprise. • Ability to quickly search and visualize errors across huge volumes of log records. • Advanced search and text analytics across large volumes of data. • Expert advice by linking search results to available best practices and recommended resolution documentations. • Near real-time streaming of z/OS logs. The z/OS support consists of the following components: • z/OS log forwarder that is installed on the required z/OS LPARs where the logs are to be collected and forwarded. • SMF data provider that is installed on the required z/OS LPARs where SMF performance metrics are to be collected and forwarded. • Insight Packs to provide the index, search, and domain insights capability for logs and performance metrics. Search is provided for all messages in the logs and you can choose to search one or more or all logs. The user can also specify a timeframe of the search to help narrow the focus to the time period when the error occurred. The Insight Pack surfaces patterns as the logs are searched, enabling the user to quickly focus on errors and drill down to the offending problem area. IBM Operations Analytics for z Systems provides out-of-the-box insights and application views for z/OS, WebSphere, DB2, CICS, IMS and MQ with the addition of Network Insights in V2.1. Also in V2.1, we have included initial support for consuming and analyzing performance metrics using our SMF Data Provider component. The user interface is customizable such that users can build their own application views and create and save environment-specific queries. The search language is text based and easy to use, and users can easily create and save simple or complex search strings with minimal typing. The tool is helpful to novice as well as experienced users. Online help, product documentation and product videos are easily accessed from the Getting Started page. 5698-AAP V2.1.0 IBM Operations Analytics for z Systems Large Insurance Company – Customer story 1 Quote: “This tool can really save a pile of diagnostic time! “ Customer experienced a problem that took 29 hours to debug. This process required time from both IBM (Level 2) and multiple employees from that company. The account team contacted the IBM development team and described an outage at the customer site. The development team received the Syslogs from the customer, fed them into Operations Analytics Server and immediately saw the high volume of error messages on the two LPARs (thousands of error messages were Severe errors). Most errors were in DB2 and MQ. The development team immediately noticed the high volume of some very specific messages (mostly DB2). The Log Analysis Application views graphically displayed the message peeks (as compared to normal message flows). ‘Needles’ (error messages) in the haystacks (LPARs) were immediately evident through visual representation of the message spikes. Ultimately, the problem was caused by a bad PTF that was applied as part of a z/OS maintenance window. The Expert Advice feature was used to pinpoint the relevant maintenance to fix the problem (based on the error messages that were generated). One member of the development team was able to pinpoint the problem using IBM Operations Analytics for z Systems in under 30 minutes … It went from 29 hours to 29 minutes. Moral of the story - IBM Operations Analytics for z Systems would have helped decrease the amount of time required for problem determination. The log analysis provided by IBM Operations Analytics for z Systems would have highlighted the high volume of error messages visually (in both the application views AND the insights (message pattern detection) to determine the scope of the problem (ie which systems are affected) and identify which additional components are affected (ie MQ, IMS, CICS, etc.). Once the focus was narrowed down to the problem area, the Expert Advice feature was used to perform a quick search of the IBM support site to identify a fix for the problem (PTF, technote, white paper, etc.). Another Insurance Company – Customer story 2 Quote: “This tool can quickly prove it is not my fault!” The DB2 support team within the customer shop often spends many hours isolating problems to discover it is not in fact a DB2 problem and needs to be routed to another group. In this specific case in point, there were serious MQ errors and the DB2 team spent hours isolating the problem as an MQ problem. With IBM Operations Analytics for z Systems, it was proven that the team could have gone directly to the source of the issue immediately. This would have saved them hours, and cumulatively days, of spinning unproductive cycles and they could have routed the issue to the internal MQ support team immediately. Large Bank – Customer Story 3 Quote: “Faster than a speeding Bullet! “ Customer is running a WAS-based On-line Banking Application in a couple of datacenters. Often when they receive a trouble ticket from their external customer (i.e. the user of their online banking application), they cannot determine which datacenter originated the error messages. With IBM Operations Analytics for z Systems’ ability to consolidate logs, they stated they could reduce their initial isolation time significantly (maybe 50%) Government Agency IT department - Customer story 4 Quote: “Talk about Time to Value! “ In a recent customer engagement, the client was able to download, install and configure the solution and had an operational environment in 2.5 hrs! SEARCH ANALYZE Launch to Support Doc RESOLVE INTEGRATE 9 9

… IBM Operations Analytics Architecture and Flows … … Mainframe z/OS
NetView Message Provider 1010 IBM Operations Analytics Architecture and Flows Mainframe z/OS SMF Data Real-time Data Provider SMF WAS SYSPRINT WAS SYSOUT Operations Analytics Server Alert Actions z/OS Syslog Applications Search Index z/OS Log Forwarder CICS MSGUSR CICS EYULOG Log USS Log Files Joblogs Annotators Alerts Current/ Archive Tier Hadoop Tier Generic Receiver NetView Message Gatherer NetView Netlog Insight Pack (z/OS) Script … Indexers Other Logs Distributed Systems Insight Packs EIF z/Linux File Agent Logstash Log or WAS SYSPRINT If you’re presenting to a customer that only cares about consuming mainframe data, then you should use this slide. There is another slide in backup that provides a more complete picture because it includes data coming from OMNIbus and distributed systems as well as z/OS. Note that Syslogd falls under USS Log Files. Distributed systems logs, insight packs, toolkits, etc. are documented here: Hadoop (frozen tier) and alerting is included in the 1Q, 2015 version of the IOA server. … SNMP WAS SYSOUT DB2 DB2 App The IBM Operations Analytics server is installed on z System (or x System) running Linux (64 bit) z/OS Insight Packs are installed on the IBM Operations Analytics server z/OS Log Forwarder / SMF Data Provider installed on each z/OS LPAR where you want to provide Search and Analysis Syslog Web Access Log … Other Logs

Simple search interface EASY to customize
Log data is analysed and insights are surfaced as you search Find problems you didn’t know existed Save My Search Timeframe Enter search string Search specific logs or ALL logs Quick Searches, Analysis, Annotations, Patterns, Expert Advice, Dashboards will populate the Navigation tree

Easy to use – Quick Search
Domain-specific ‘Quick Searches’ available out-of-the-box or create and save your own Provided with every z/OS Insight Pack Provided by subject matter experts, support teams and customers Immediate value out of the box Easy to modify or create and save your own

Dashboards, Information Links and Expert Advice
Visualize the data with Dashboards Quick links to additional information and support documents. Provided with every Insight Pack Expert Advice to access white papers, tech notes, APARs, etc. for faster problem resolution Dashboard views created by subject matter experts, support teams and customers Immediate value out of the box Easy to modify or create and save your own

Search for expert advice with the click of a button
Quickly and easily access IBM Support Portal based Expert Advice from Log Analysis Search for expert advice with the click of a button All IBM support site documents that reference messages from search results Launch to Tech Note

Analyze logs as you Search
Insights are surfaced automatically as you search. Patterns are surfaced based on the log type. Provided with every Insight Pack Logs are analysed automatically Log data is categorized by hostname, data source, message type, message source, etc. Patterns/Insights are surfaced to help you focus on the source of the problem. For example, log analysis automatically surfaces java exceptions in application logs. Perform searches and analyse multiple logs, organized per the needs of your enterprise. Create your own Insight Pack for any text logs with time stamps

Sample dashboard View your log and metric data however you like
Presenter name here.ppt BA Cognos 10 Template 4/15/2018 Sample dashboard View your log and metric data however you like Doesn’t need to be stuck w textual, can do visuals/graphs

Sample dashboard View your log and metric data however you like
Out-of-the-box dashboards (Example – Display message counts and java exceptions) OR Build Your Own Dashboard with the click of the mouse

Visualizing the Data Search and Analyze SMF Data (New in 2015)
Analyze your SMF data AND your log data for a complete view of the enterprise. CPU utilization, Working Set Size, Paging & IO Rates

Create your own – Queries, Dashboards, Feeds
Out-of-the-Box capabilities provide immediate value. Additionally, IOA can easily be tailored to your specific needs. Perform simple free-form searches using the standard set of search keywords and operators Build complex queries with range searches and DateMath functions To learn more, consult Online Help available from the Learn More → Search Bar → Search query syntax menu: BYOD – Bring your own Data – The z/OS Log Forwarder can be configured to forward your text logs to enable Search, Analysis, Dashboards and Expert advice. BYOIP – Build your own Insight Pack BYOV – Build your own Views (Graphs, Charts and Dashboards)

Customer Experiences Large Insurance Company (29 hours down to 29 minutes) Experienced an application outage that resulted in the team working around the clock for 29 hours. Multiple customers and IBM support staff poured through logs and traces to determine the root cause of the issue. After the issue was resolved, the logs were captured and sent to IBM lab for analysis using IBM Operations Analytics for z Systems. Within minutes, the IBM team was able to focus in on the root cause of the problem and to find the relevant PTF to resolve the issue through the integrated expert advice. State Agency (up and running in 2.5 hours) Were able to download, install, configure and use IBM Operations Analytics for z Systems to search their logs in 2.5 hours. Numerous Customers (improve visibility and find problems you weren’t aware of) Errors lurking in logs that are never examined because they don’t necessarily cause SLA or performance problems. For example, IBM Operations Analytics for z Systems found Over 4,000 invalid login attempts in a three day period that had otherwise gone unnoticed. MQ channel errors causing MQ errors in logs from distributed systems – not being monitored SQL errors in multiple logs

2121 New capabilities in 4Q, 2015 General capabilities (delivered via IBM Operations Analytics – Log Analysis and included with IOAz) Additional real-time alerting actions: SNMP Traps, EIF Events Role-based access control Support for multiple time zones and time intervals Service Desk Extension: Incident and service request analytics z/OS capabilities (included in the z/OS Insight Pack) Additional CICS insights from SMF 110 and EYULOG Additional network insights from NetView netlog Security insights Pattern-based configuration for z/OS Log Forwarder job log data gatherer Additional out-of-the-box searches for DB2 and MQ Translation of z/OS Insight Packs (English + 10 languages) and documentation M ain Point: Analytics is now a key part of what customers are looking to improve on. As we have seen, analytics can help increase business value and IT metrics. A nalytics is about: 1 . Predict problems and anomalies – Current product is OMEGAMON V5.1.1 with IBM zAware support and NetView which also includes IBM zAware 2 . Search for information, including logs – The current product in this area is SmartCloud Analytics – Log Analysis 3 . Optimize analytics for both Business and IT – Capacity Management Analytics (CMA) for z/OS, is a suite that includes SPSS, Cognos and TDSz. I BM SmartCloud Analytics - Predictive Insights R educe outages and increase service performance with predictive problem detection I BM® SmartCloud® Analytics – Predictive Insights can provide early problem detection to predict application or middleware problems before they impact service. The software helps you avoid application outages and increase service performance. I BM SmartCloud Analytics – Predictive Insights helps you: A void outages to increase application availability and reduce service degradation. P erform faster root cause analysis to isolate problems sooner. R educe operational costs without the need for complex service models or specialized skills. 21

Alerting actions: SNMP Traps, EIF Events
IOA now enables you to generate SNMP Traps and EIF Events. This is in addition to existing notifications (text, , etc.) Benefit: Utilize your existing event management tooling to track, highlight, enrich, correlate and act upon conditions that are identified in their operational data by IBM Operations Analytics for z Systems through the use of SNMP Traps, Informs or EIF events. Broaden your scope of automation. Use NetView or other automation tools to take automatic action on any messages or other operational data as long as that data is consumed by IBM Operations Analytics for z Systems. This expands your current automation capabilities to automate on ANY data source that is fed into IBM Operations Analytics. Personas supported: Alice (Subject Matter Novice) Jim (Subject Matter Expert) Zach (Senior Systems Programmer)

Role-based access control and audit
Benefit: Role-based access control and auditing capabilities enable customers to maintain compliance with their data segregation and access control requirements. It is of special interest for service provider environments in which segregation of data is of particular importance. Personas supported: Alice (Subject Matter Novice) Eric (Application Developer) Jim (Subject Matter Expert) Zach (Senior Systems Programmer)

Support for multiple time zones and time intervals
Benefit: All users connected to a single IOA Log Analytics server, regardless of their location, are able to view search results and graphs in their local time zone or in a different time zone of their choice. This new capability is particularly helpful for teams that are distributed across multiple time zones. Applications can now specify more than a single occurrence of a relative time interval. Instead of specifying “Last Day”, applications can specify “Last 3 Days” for example. Personas supported: Alice (Subject Matter Novice) Eric (Application Developer) Jim (Subject Matter Expert) Zach (Senior Systems Programmer)

Integration with Service Management Solutions
IOAz integrates with Monitoring, Automation and Event Management Automation NetView / SA (or other Automation tooling) Receive and enrich, action or forward Events from ANY log source (not just Syslog) Event Management Netcool Operations Insights (NOI) Launch to IOAz to analyze logs and metrics (IOA is included with NOI) Search and analyze Events Receive, correlate, enrich and action Events from IOAz (NOI or other Event Management System) Incident Management IBM Service Desk (or other incident management / trouble ticketing solutions) Generate Events to create Trouble Tickets Analyze Trouble Tickets Monitoring OMEGAMON Launch in context to IBM Operations Analytics from OMEGAMON and ITM workspaces OMEGAMON Insight Pack to analyze ITM logs (RKLVLOG) Service Management Unite (included with Performance Management and Service Management Suites) Launch in context to analyze logs and SMF data in context of performance problem diagnosis

Event Management and Automation
Using IOAz to broaden the scope of Event Management and Automation

Be Proactive! Enhance your Visibility & Automation Capabilities
IOAz can generate notifications for messages from any log in your enterprise. Event processing Generate Events from ANY log message(s) or other data in IOA Notifications can be in the form of: Text message SNMP Trap EIF Event Increase scope of log monitoring and automation Improve event correlation Be Proactive!

Getting the most out of IOA notification capabilities
IBM Operations Analytics provides the ability to generate events based on messages, combination of messages over time, number of occurrences, etc. Notifications can be generated from any data source: Messages from Mainframe and Distributed Logs SMF data Events Other Examples include: Send an or text message whenever a specific message(s) is written to a log … For example, message IRRB069I (RACF is being shut down) Generate a SNMP Trap or EIF event when there are more then 500 failed logon attempts in a 30 minute period

Send notifications in many forms …
Index alert action (ie send events back into IOA so they can be searched) You can use the index alert action template to index any triggered alerts. / Text alert action You can use the template to send an when a condition is met. s can easily be sent as text messages by most carriers EIF alert action You can use the EIF template to send an EIF formatted event when a condition is met. SNMP Trap alert action You can use the SNMP Trap template to send an SNMP Trap when a condition is met. Script alert action You can use the Script template to execute a custom script when a condition is met. Write to Log alert action You can use the Write to Log template to write an entry to a log file of your choice when a condition is met.

Sending Events to any Event Receiver
Management System (NOI, OMNIbus or other Event Management tool) Alert Actions Index Event Receiver Data Source 1 IOA Server Log Ingestion Pipeline Alert Runtime Data Source 2 Automation (NetView/SA or other automation tool) EIF Data Source N Event Receiver SNMP IOA can generate standard SNMP Traps and/or EIF Events that can be received and processed by ANY Event Receiver. Any Event Processor Script Event Receiver 30 30

Event Configuration is Simple
From IOAz Specify the message or messages to trigger the Notification Choose the event criteria (message IDs, number of occurrences, time period, etc.) Specify address of Event Receiver hostname/port or address From your automation tool Create an automation statement(s) to: Enrich the Event Forward the Event Automate to correct the problem Other Increase the scope of automation to include ANY log message Most z/OS automation tools are limited to z/OS Syslog and Console messages From your Event Management tool Enrich the Event Correlate with other Events and Log Messages Automate to correct the problem Create Trouble Tickets Other Scenario: MQ environment spanning z/OS and Distributed systems. MQ channel goes down. MQ message is written to distributed system log. IOAz triggers an event from the message in the distributed log Event is sent to z/OS automation tool (ie NetView / SA) Automation restarts the MQ channel. Failure is resolved quickly, avoiding an actual problem. Correlate z/OS Events with Events from distributed systems to resolve problems end-to-end

Event driven automation scenarios
3232 Event driven automation scenarios There are many scenarios where events can drive automation. Prior to IOAz, these scenarios were limited to events being driven from Syslog, because most z/OS automation tools only monitor the z/OS Syslog. Since IOAz has access to many more logs than Syslog, we now have the ability to drive automation from messages coming from other logs and even other platforms. We have included just a few examples in the subsequent slides. The possibilities are endless. Benefit: The subject matter expert can now access messages from ANY log in the enterprise Events coming from IOA can be consumed by ANY Event receiver to automate, enrich, correlate or forward Events or generate trouble tickets Events can be generated in SNMP or EIF format. As a result, the events can be consumed by any Event Receiver (Event Management or Automation tool). Since IBM Operation Analytics for z systems can generate events from ANY message it consumes and NetView can act as an event receiver, NetView can now automate on ANY log message (not just messages from Syslog). This scenario will work with any automation tool that can drive automation from events. This feature will enable customers the ability to ‘TAKE ACTION’ on any messages being consumed by IBM Operation Analytics for z Systems. Optimized Performance 32

Sample scenario for MQ WebSphere MQ channel stopped abnormally MQ server runs on Windows with a MQ channel defined to MQ running on z/OS. MQ server detects that the MQ channel to z/OS is not active and writes error messages to the Windows MQ AMQError log. Subsequent MQ communications fail. Without IBM Operations Analytics for z Systems: The ‘Channel down’ message is never proactively observed and the support team(s) struggle for hours to debug the problem and finally re-initiate the Channel. With IBM Operations Analytics for z Systems: IBM Operations Analytics for z Systems detects the problem through MQ error messages written to the Windows MQ AMQError log. IBM Operations Analytics for z Systems generates an SNMP Trap (or EIF event) and forwards it to NetView (or other automation solution). Automation is driven from this event and resolves the problem by issuing a command to restart the MQ channel. Customer Scenario (prior to using IOAz) MQ outage caused several hours of downtown and application failures. Multiple SMEs worked on the issue. MQ issues are often hard to debug. Environment (with IOAz) IOA server (running on System x or System z) receiving data from multiple sources MQ server running on Windows server Log File Agent (LFA) sending log data from Windows server into IOA server NetView is running on z/OS and is driving Event and Message automation (Note that this could be ANY automation tool that can act as an Event receiver) Scenario Overview (with IOAz) MQ channel defined to z/OS system and MQ server on Windows stops abnormally. MQ server generates ‘channel down’ message (AMQ9999). LFA sends AMQ9999 message to IBM Operations Analytics server IBM Operation Analytics sends SNMP trap (or EIF event) to NetView NetView issues command response to restart MQ channel Outage avoided with IOAz!

Sample scenario for DB2 DDF DB2 DDF applications timed out DB2 runs on z/OS; IBM Operations Analytics for z Systems collects DB2MSTR address space log. The customer applies bulk maintenance for z/OS and DB2 over the weekend. After application of maintenance, DB2 DDF applications experience time-outs. Without IBM Operations Analytics for z Systems: Because maintenance occurs on a Saturday, operators do not catch the resulting problem until later. The DBA is notified on Saturday evening, a PMR is opened against IBM DB2, and diagnostics are started with the DB2 and TCP/IP L2 teams. By Monday morning, none of the agents can run transactions. DB2 and z/OS maintenance have to be backed out. With IBM Operations Analytics for z Systems: IBM Operation Analytics is able to detect the time-out problem immediately after the maintenance is applied. Operators are notified immediately and are able to determine the root cause of the issue. End users do not experience downtime when they come into work on Monday morning. Customer Scenario (prior to using IOAz) Customer applied z/OS and DB2 maintenance during weekend maintenance window. After the maintenance was applied, DB2 DDF applications started to fail due to ‘time-outs’. DBA was finally notified on Saturday evening, after several hours of failures. DB2 and TCP/IP level 2 teams tried to debug the problem. By Monday morning, all transactions were failing. DB2 and z/OS maintenance had to be backed out. Environment (with IOAz) IOA server (running on System x or System z) receiving data from multiple sources DB2 is running on z/OS z/OS Log Forwarder sending DB2MSTR address space log data into IOA server NetView is running on z/OS and is driving Event and Message automation (Note that this could be ANY automation tool that can act as an Event receiver) Scenario Overview (with IOAz) DB2 errors written to DB2MSTR address space log after maintenance is applied z/OS Log Forwarder sends messages from DB2MSTR address space log to IBM Operations Analytics server IBM Operation Analytics receives DSNL511I, IXL043I and other DB2 failure messages and sends SNMP trap (or EIF event) to NetView NetView issues commands to collect additional data and forwards the Event to the Event Management system so a trouble ticket can be created for the SME Issue reported immediately with IOAz. Maintenance backed out. Problem avoided!

Log Analysis and Event Management in Netcool Operations Insight (IOA is included in the box with NOI) Search and analyze events, logs and metrics using IOA and Netcool Operations Insight. Easily identify ‘related’ Events that may be candidates for suppression Identify “difficult to spot” seasonal events that often result in regular periodic problems Easily identify which events occur in clusters Leverage visualizations that help you quickly isolate more sever and significant problems. Also provides opportunities for event reduction thus improving operational efficiency. 35

Log Analysis – Streamline Incident Management
Incident Management The traditional incident management process usually begins with one or more trouble tickets being opened for an incident (for example, slow response time for a specific application). The first step is to engage the application support team and associated Subject Matter Experts for each of the application components (WebSphere, CICS, DB2, etc.). Each SME examine data from their specific subsystem and we usually experience a phenomenon commonly referred to as ‘ticket hopping’. During the ticket hopping phase, the trouble ticket will be reassigned multiple times before it lands on the correct SME’s lap. Over the lifetime of the incident, there is very little collaboration with respect to data and there’s usually a fair amount of ‘finger pointing’. In the post mortem session, we usually conclude that the ‘time to resolution’ is very high and so is the number of people involved in the process of diagnosing the problem. With IBM Operations Analytics for z Systems: IBM Operation Analytics will provide a unified view of the data, enabling the application support team to quickly focus on the problem component. The ability to search and analyze the data helps to quickly identify the problem area and the expert advice feature assists in finding the solution or workaround. If an SME is needed for a specific component, you can transfer the ticket to that SME with the data that was surfaced by IOAz. Post mortem reveals that time to resolution is significantly decreased by as much as 50% with less involvement by the SME community. To be more proactive and improve mean time to recovery even more, the team can incorporate the use of IOA notifications to immediately notify (Text, , SNMP Trap or EIF Event) that a problem is occurring. Early detection will significantly decrease time to resolution and automation can be triggered to resolve the issue before the problem affects the end user. I would like to introduce to you couple of solutions which demonstrate the use cases of IT Operations Analytics. Firstly, we will talk about Log Analysis Solution. If we take the example of a traditional incident lifecycle, we see that users report issues to service desk or monitoring tools generate events. Operations team (L1 support) assigns the incident to a resolver group. Subsequently the first resolver group engages other teams to drive incident troubleshooting and resolution. This is a time taking process as each of the teams perform troubleshooting in silos and do not have a unified view Log Analysis Solution ingests system and sub-system logs from infrastructure and application components to provide unified time sequenced view of logs with the ability quickly search thru massive amount of data for specific issues. Log analysis enables the team to identify when and where the error happened. This drives swift engagement of the right resolver team/s in parallel. The key differentiator is reduction in time to isolate and resolve problems. 36

Integration with Performance Monitoring
OMEGAMON + IBM Operations Analytics – Launch in Context from TEP The One Two – Punch: Combine two very powerful tools to ensure performance and high availability of your enterprise. Perform log analysis in context of OMEGAMON workspaces – This approach enables OMEGAMON users to perform in-context log analysis while doing problem determination From your OMEGAMON workspace, use the IOA search bar to search logs (using LPAR or Sysplex as the default context) Easy to implement - Configure TEP to display the IOA search bar Launch IOA from OMEGAMON performance monitoring workspaces to search logs in context You need to install the following maintenance to enable the TEP launch-in-context to Operations Analytics for z Systems Required changes to distributed components: ITM TEPS: Provisional fix TIV-ITM-FP0004-IV67740 Obtain FP5 fix by subscribing to: Required changes to z/OS components: PARMGEN: FMID HKCI310, Interim Feature APAR OA46184 (PTF UA76016) Obtain fix: ITM 630 z/OS TEMA update FMID HKDS630, APAR OA46976 (PTF UA76202, , available 2/28/15) Obtain fix: OMEGAMON XE for WebSphere MQ Monitoring: FMID HKMQ730, APAR OA46839 (PTF UA76091, available 2/28/15) Obtain fix: OMEGAMON XE for WebSphere Message Broker Monitoring: FMID HKQI730, APAR OA46840 (PTF UA76092, available 2/28/15) Obtain fix: OMEGAMON XE for Storage: FMID HKS3530 APAR OA46871 Subscribe and obtain fix: 37

Search and Analyze Operational Data in Context
Select a row first. In this example, a row specifies a Queue Manager. Specify a search string and timeframe to analyze operational data from the appropriate system(s)

Analysis of Operational Data
Launch into IBM Operations Analytics to analyze logs and other operational data to gain additional perspective and insights and help diagnose root cause. IBM Operations Analytics analyzes log, metric and event data and surfaces insights Built on industry expertise Expert Advice for faster time to resolution Expand analysis to include additional data sources (from mainframe and distributed systems)

Integration with existing Service Management solutions
(in a nutshell) IBM zAware POWerful tools integrate to ensure performance and high availability of your enterprise. Surface anomalies Automation & Problem Determination NetView Performance Monitoring ITM/OMEGAMON Event Management OMNIbus/NOI Incident Management Control Desk Alert, enrich, correlate and automate End of presentation. Service Management Unite Search and analyze logs, metrics, events and incident reports Launch from ITM, OMEGAMON, Service Management Unite & NOI 40 IBM Operations Analytics 40

Send us your logs! Or Take IOAz for a Test Drive
Request a product demo using logs from your own test, development or production environments IBM will load your logs into an IBM Operations Analytics server, then demo the results back to you A secure, dedicated drop box will be assigned to you You will be sent detail upload instructions via Any file uploaded will be automatically moved to a dedicated IBM Operations Analytics environment within 24 hours All log data will be purged from the IBM Operations Analytics environment within 48 hours after the demo event To request your hosted demo, visit: Or Take IOAz for a Test Drive A guided demo is provided online at:

IOA for z Systems Early Access and Beta Program https://ibm.biz/BdEkZV
Announcing the IBM Operations Analytics for z Systems Early Access and Beta Program! In 2015, we built on the strong foundation established over recent months as we develop and implement our product roadmap. We are looking for customers and business partners worldwide who would like to help influence our roadmap and test new capabilities. The program is open-ended; interested participants may join at any time and stay on as long as they wish. That said, it is our desire to establish a set of “customer sponsor” relationships that will become instrumental in shaping the future of our offering. To see the full program announcement, and to learn how to sign up, please visit us in our developerWorks community at:

Additional IBM Operations Analytics Reference Material
Analytics Overview Video IOA for z Systems videos: Overview: Domain Insights: Installation and Configuration: IOA for z Systems Documentation Knowledge Center: IOA – Log Analysis (server) Documentation Service Management Connect Knowledge Center

More Detail and drill down to the next level
The remainder of the deck includes details for the following: Bring your own Data Feeding additional logs into IOA Options Example using the Hardware Management Console log IOAz 2.2 Enhanced CICS insights Enhanced Network insights Security insights Log Forwarder improvements

Bring Your Own Data

Enhance your Visibility – Avoid ‘Blind Spots’
In addition to the out-of-the-box Insights for z/OS, DB2, IMS, CICS, MQ, Network and Security, IOAz can also be customized to meet the needs of your enterprise. BYOD – Bring your own data Enable Search and surface Insights for any text log (messages must have time stamps) Create your own Saved Searches Create your own dashboards, graphs and charts Gain access to any log data in the enterprise to debug end-to-end applications, generate notifications and surface events Increase scope of log monitoring and automation Remove ‘blind spots’ Learn how simple and easy it is to customize IBM Operations Analytics for z Systems. During this session, using HMC logs as an example, we will show you how to add new data sources, create new insights and analyze log and metric data that YOU want to analyze.

IBM Operations Analytics for z Systems
IOAz provides a number of defined data sources to help you ingest your data (out-of-the-box Insight Pack) z/OS – Syslog (including CICS, IMS, DB2, MQ, Security, Network, etc.), CICS MSGUSR and EYULOG, WebSphere sysout and sysprint, USS syslogd, SMF, NetView Distributed Systems – MQ, DB2, Javacore, WebSphere, Service Desk, Microsoft SQL, Active Directory, and many more. See the following URL for the latest list of Insight Packs available for IOA-LA: But what if you want to add your own custom data types ? (BYOD) Custom Application Logs Generated Report Files Statistical Records Job Log data Others ?

Example: HMC Log as a data source
Customers often ask us if IOAz can consume logs from applications that they have written or even just some other log in the enterprise that is not currently supported by an existing Insight Pack. The answer to this question is ‘YES!’. As long as it is a text log and the messages have a time stamp, they can be consumed by the IOA server. Anyone can write an Insight Pack … And there are several options, depending on what you want to do with the data … In all cases, it is Quick and Easy. We have received several requests to provide an Insight Pack for the HMC. Retrieve the HMC log We are using the HMC Log Tool (HLT) from the TechDocs library This tool allows you to retrieve log information using the zEnterprise Web Services APIs and generates reports on the desired information. Data is written to a file on USS (IOAz can consume USS files) Feed the log data into IOAz for analysis and audit purposes. Multiple options here depending on what you want to do with the data. All are simple.

What do you want to do with the data?
IOAz can consume and analyze any text log. There are 3 options to enable this depending on what you want to do with the log data Search with Default Annotations (takes 10 minutes to configure) Configure the z/OS Log Forwarder (data source type = ‘other’) to send the log data to IOAz. The IOAz generic receiver will index and annotate the data to make it available for Search. You can create and save quick searches, graphs and dashboards. Annotate and analyze a CSV style log file (takes 20 minutes) The DSV toolkit can be used to process any log file that follows a “Delimiter Separated Value” format. Simply edit a text file describing the DSV layout. Each separated value column becomes an annotated field in IOA Annotate and analyze any text or log file (1-2 hours) Allows for complete control on how files are read into IOA and parsed. This approach allows for more sophisticated pattern searching and correlation. Can be written in Java, AQL, Python, etc. There is no programming required for the simple search capabilities. The DSV toolkit is handy if you have a comma separated file and each line has a timestamp. Again, there is no programming required.

Example: HMC Insight Pack
02/04/ :54:13.300: 2007: User SooAcsadmin has acknowledged viewing l 02/04/ :31:59.860: 1100: The system clock has changed. 02/05/ :57:42.790: 734: Remote support call generated on USYS is bei 02/05/ :57:45.900: 674: Remote support call generated on USYS comple 02/11/ :08:01.680: 734: Remote support call generated on USYS is bei 02/11/ :08:02.810: 674: Remote support call generated on USYS comple 02/11/ :08:12.770: 734: Remote support call generated on USYS is bei 02/11/ :08:13.790: 674: Remote support call generated on USYS comple 02/12/ :14:18.810: 734: Remote support call generated on USYS is bei 02/12/ :14:39.470: 674: Remote support call generated on USYS comple 02/12/ :27:19.550: 1100: The system clock has changed. 02/12/ :32:16.650: 734: Remote support call generated on USYS is bei 02/12/ :32:19.350: 674: Remote support call generated on USYS comple 02/12/ :12:20.230: 722: An upgrade to EC level N98841 was performed. Above is an excerpt from an HMC log. The log contains time stamps, event IDs and text. In the following slides, we will show the difference between generic processing (Search with Default Annotations) and processing using the HMC Insight pack. The HMC API was used to retrieve the log data and the logs were stored in the USS file system. The z/OS Log Forwarder was configured to send the log to IOAz.

HMC Insight Pack – Search (using Generic Receiver)
Generic receiver discovered patterns in the logs. Graphical view of message volumes. It take approximately 10 minutes to configure the log forwarder to send a log to IOA. The generic receiver enables search and analysis, graphical views, save searches, expert advice, etc. But in this case, it discovers too many generic patterns. We decided to write a quick insight pack to surface better insights. This view is created by simply sending the HMC data through the generic receiver. The generic receiver identifies patterns and keywords in the message texts and annotates those automatically. Search capability generates helpful search results.

Creating a Java based Insight Pack
Eclipse plugin available to assist in creating insight packs. The plugin generates the required insight pack structure and supporting files. All that is required is to write 2 pieces of java code and define the record structure of the resulting lines for IOA Splitter.java – Receives blocks of log data and breaks it up into distinct lines of data. Annotator.java – Receives each line of data and identifies the individual fields to be annotated for IOA. Finally define what the name of the data source will be and map the splitter and annotator code using the supplied plugin. Run an ant build using the provided ant build sample It takes an approximately 1-2 hours to write an Insight Pack. Time is significantly reduced after writing your first one. Java you can get to a much deeper level of detail and control. And since we wanted to do more with the data than just search and annotate we chose to use a Java implementation of the insight pack. It took 2 hours.

Example: HMC Insight Pack in Java
The 734 is the Event ID which would be useful to annotate on 02/04/ :54:13.300: 2007: User SooAcsadmin has acknowledged viewing l 02/04/ :31:59.860: 1100: The system clock has changed. 02/05/ :57:42.790: 734: Remote support call generated on USYS is bei 02/05/ :57:45.900: 674: Remote support call generated on USYS comple 02/11/ :08:01.680: 734: Remote support call generated on USYS is bei 02/11/ :08:02.810: 674: Remote support call generated on USYS comple 02/11/ :08:12.770: 734: Remote support call generated on USYS is bei 02/11/ :08:13.790: 674: Remote support call generated on USYS comple 02/12/ :14:18.810: 734: Remote support call generated on USYS is bei 02/12/ :14:39.470: 674: Remote support call generated on USYS comple 02/12/ :27:19.550: 1100: The system clock has changed. 02/12/ :32:16.650: 734: Remote support call generated on USYS is bei 02/12/ :32:19.350: 674: Remote support call generated on USYS comple 02/12/ :12:20.230: 722: An upgrade to EC level N98841 was performed. Annotated fields show up in the Search Patterns Fields are assigned to a key-value pair and can show up in the grid view This was created through a Java insight pack. The Insight Pack code has 2 functions, identify distinct and complete lines of the log and then identify fields in each line. The log file should be examined to determine what data would make searching and analyzing the log easiest, and that data should be targeted for annotation. In this case the fields are the timestamp, the EventID and the MessageText. An Insight Pack can surface important messages, text strings, text patterns, etc. This is just a simple example.

Create custom quick searches
Once the data is in IOA, it’s easy to create quick searches by simply running a search and saving it. Save button lets you add any search to the saved search menu

Creating custom graphs – Remote Support Calls
To create a graph of data, simply run a query, select column(s) to graph and then click the graph button and save it # of times HMC did a call home to get an update .. Just an example You can graph / chart and build dashboards for any data in the log You can also generate events or send notification to / text This example is an example of several months of data … one log gathered per day. but you can control the frequency with a chron job or other The graph button allows you to graph 1 or more data columns.

Creating custom dashboards
Multiple graphs can be saved to single dashboard Distribution of Event IDs over time Login failures Critical data backups Remote Support Calls

Reference Material – Building Insight Packs
IBM Operations Analytics – Log Analysis resources web site An example of creating a Java based insight pack

IOAz V2.2 More detail

Key enhancement: Enhanced CICS insights

Enhanced CICS Insights
6161 Enhanced CICS Insights In IOAz V2.2, the subject matter expert and/or application developer can gain insight and quickly isolate and diagnose CICS problems from a combination of CICS performance metrics and log data. CICS Monitoring Exceptions and Global Transaction Statistics are now available to aid in diagnosis. IOA notifications enable proactive monitoring for early detection and automation. IOAz consumes, analyses and provides insights using the following CICS data: CICS message data gathered from the following logs SYSLOG MSGUSR EYULOG SMF 30 – CICS Jobs CPU utilization IO Rate Paging Rate Working Set CICS SMF 110 data CICS Monitoring Exceptions CICS Global Transaction statistics (per APPLID) Personas supported: Alice (Subject Matter Novice) Eric (Application Developer) Jim (Subject Matter Expert) Customer benefit: Customer can use SMF and log data across multiple CICS regions and systems to help isolate and debug CICS issues. Scenarios addressed: Jim, the Subject Matter Expert, can view a search dashboard to view tasks exceeding maximum threshold per CICS Region over time Alice, the Subject Matter Novice, can view a search dashboard to view transaction rate per CICS Region over time Eric, the Application Developer, can view a search dashboard to view wait for storage events and storage manager messages per CICS Region over time.

Enhanced CICS Insights New Quick Searches for CICS
6262 Enhanced CICS Insights New Quick Searches for CICS IOAz V2.2 includes the following ‘Quick Searches’ (in addition to the current set of predefined searches for SMF30) Display all CICS Exceptions Display CICS Wait on Storage Exceptions Display CICS Policy Exception Display CICS Transaction Interval Summary Display CICS End of Day Summary- last week Display CICS Transactions summaries where Tasks = MAXTASKS. New CICS Dashboards CICS Regions Dashboard Interactive dashboard that integrates SMF and log data to diagnose: Wait on Storage events per region over last day Exceptions by Resource ID over last day Short on Storage per region over last day Tasks at Maximum Threshold over last day Storage Violations per region over last day CICS Regions Transaction Dashboard Static dashboard showing: Transactions – top 5 regions over last day Transactions – Max and Average per region over last day

6363 CICS Demo Scenario Eric the application developer is investigating an issue with his application running slowly. Prior to IOAz, Eric would have had to browse through each log looking for information on the state of CICS in general and his application specifically. This might involve the SYSLOG for the system and MSGUSR and EYULOG for each CICS region. If his application spanned multiple systems, he would have to log on to each system first and again manually browse each SYSYLOG and MSGUSR and EYULOG for each CICS region. He would be required to log on to yet a separate application to view SMF data. Once he has identified all the data sources, he will then need to manually correlate the data across the multiple sources to diagnose the problem.

CICS Demo Scenario, continued
6464 CICS Demo Scenario, continued With IOAz, Eric can logon to a single user interface and access all of the relevant logs in the enterprise and also reference SMF data to determine if an associated performance problem or CICS monitoring exception has occurred. First he checks the CICS Jobs dash board and does see a spike in CPU, I/O Rate and Working Set size. He looks at the Transaction dashboards and sees a corresponding spike in the transactions. He then looks at the Regions dashboard and does see some exceptions related to storage issues. Double clicking on the exceptions (insights automatically surfaced on the left side of the user interface) he can see most are related to the progname ICC$HEL Then he checks the integrated IOA log search. This allows him to quickly search the SYSLOG, MSGUSR, EYULOG and SMF. In here he can search specifically for his application name to see where problems might lie, in this case ICC$HEL To be more proactive, Eric can configure a notification when a message, combination of messages or additional data indicates a problem. Notifications can be in the form of a text message, , SNMP Trap or EIF Event and can be used to automatically resolve an issue, gather additional data or generate a trouble ticket.

Key enhancement: Enhanced network insights

Enhanced Network Insights
6666 Enhanced Network Insights Network insights were first introduced in IOAz V2.1 to enable the search and analysis of z/OS network data provided by VTAM, TCPIP and syslogd message logs. In IOAz V2.2, you can now ingest, search and analyze NetView message data; specifically the NetView messages that are written to the NetView netlog. Customers can quickly and easily identify issues that are surfaced only through NetView netlog without actually going to NetView or searching the NetView Canzlog. In addition to network messages, the NetView Netlog contains other messages; for example, messages related to NetView automation failures. Since IOAz can consume logs from multiple systems, you can perform a single search across all logs to determine if a problem is occurring on more than one system. IOA notifications enable proactive monitoring for early detection of network or automation-related issues. Personas supported: Alice (Subject Matter Novice) Eric (Application Developer) Jim (Subject Matter Expert) Customer benefit: Customer can use SMF and log data across multiple CICS regions and systems to help isolate and debug CICS issues. Scenarios addressed: Jim, the Subject Matter Expert, can view a search dashboard to view tasks exceeding maximum threshold per CICS Region over time Alice, the Subject Matter Novice, can view a search dashboard to view transaction rate per CICS Region over time Eric, the Application Developer, can view a search dashboard to view wait for storage events and storage manager messages per CICS Region over time.

Enhanced Network Insights
6767 Enhanced Network Insights New Quick Searches for NetView Netlog IOAz V2.2 includes the following ‘Quick Searches’ for NetView Netlog (in addition to the current set of predefined searches for Network insights provided in IOAz V2.1) All NetView for z/OS Messages All NetView for z/OS Action, Decision or Error messages NetView for z/OS messages that indicate automation table violations NetView for z/OS messages that indicate command authorization table violations NetView for z/OS messages that indicate resource limits or storage thresholds NetView for z/OS messages that indicate insufficient access authority or security environment violations New NetView Netlog Dashboards NetView Message Counts - Top 5 per hour over Last Day NetView Message Type Counts - Top 5 per hour over Last Day Total NetView Message Counts per hour over Last Day NetView Messages by Hostname - Top 5 per hour over Last Day NetView Message Types by Hostname - Top 5 over Last Day Total NetView Messages by Hostname per hour over Last Day Saved searches and dashboards are provided to show examples of what types of data can be searched. Annotated fields include the system name, NetView for z/OS domain, NetView operator ID and NetView HDRMTYPE as well as message ID/prefix/type/text. NetView for z/OS messages can be used to drive alerts which can generate s, run scripts, log messages or include an alert index.

Insights from NetView netlog
User Scenario 1 Alice as the Subject Matter Novice is notified of a problem with the NetView automation table. A new automation statement has been added to the automation table to perform an automation action on an autotask when a specific message is processed. Even though the message is processed, the automation action did not occur. Instead of signing on to NetView, Alice uses IBM Operations Analytics for z Systems and issues a Saved Search for NetView Automation to look for possible errors. The search results find a NetView message: DWO032E AUTOMATION ACTION action COULD NOT BE ROUTED TO TASK(S) task. Alice determines from this message that the problem is not with the actual automation table statement but instead is with the autotask where the automation action is to be performed. Next she can search for the specific autotask in the NetView messages to ensure the autotask is configured properly and started. Alice can search and analyze ALL of the NetView logs at one time by issuing a single search command from IOAz. Alice can create automation to check for these messages in the future and then issue the command to restart the autotask if it is not already started.

User Scenario 2 Jim as the Subject Matter Expert needs to determine if there were any attempts by NetView operators to issue unauthorized commands over the last week. Even though a NetView operator is allowed to sign on to NetView, an operator can be assigned access to a specific set of commands. Instead of going to each NetView domain and looking for related security messages in the NetView Canzlog, Jim uses IBM Operations Analytics for z Systems and issues a Saved Search for NetView Security to look for unauthorized command attempts. Jim can search and analyze ALL of the NetView logs at one time by issuing a single search command from IOAz. The search results show Jim that there were multiple attempts to issue unauthorized commands on over the last week. BNH232E 'userid' IS NOT AUTHORIZED TO ISSUE COMMAND 'command’ BNH233E THE COMMAND 'command' IS PROTECTED BY COMMAND IDENTIFIER 'commandid' IN 'auth_method’ DSI213I ACCESS TO 'object' IS NOT AUTHORIZED

Search logs from multiple systems or choose to see results from specific systems No typing necessary. Just click on one of the IBM-provided quick searches or a saved search Search results Analysis reveals patterns which can be combined to build more complex searches; simply by selecting them.

Key enhancement: Security insights

7272 Security Insights IOAz V2.2 introduces security insights so the subject matter expert, security administrator and/or application developer can quickly identify RACF failures and security issues. RACF failures often contribute to failed applications (for example, failure in read/write operations due to missing or incorrect RACF authorization). RACF security issues such as invalid authority and a significant number of invalid logon attempts can be a sign of a security breach. IOA notifications enable proactive monitoring for early detection of RACF security issues. In this initial iteration of Security Insights, we are utilizing RACF Security messages in the SYSLOG and USS syslogd Personas supported: Alice (Subject Matter Novice) Eric (Application Developer) Jim (Subject Matter Expert) Customer benefit: Customers can quickly and easily identify RACF based security issues that are surfaced through system logs. Scenarios addressed: As the Subject Matter Expert for security, Jim would like to be able to identify all instances of someone attempting to access resources without proper authorization. As the Subject Matter Novice for security, Alice would like to be able to see when there are significant spikes in invalid logon attempts. As the Subject Matter Expert for security, Jim would like to be able to browse all invalid authority messages

Security Insights New Quick Searches for Security-related issues
7373 Security Insights New Quick Searches for Security-related issues Display all RACF Messages Display all RACF Action Error or Warning messages Display all Insufficient Access errors User attempted to access something but does not have the proper access authority Display all Insufficient Authority errors User attempted to execute a program but does not have the proper authority to do so Display all Invalid Password messages Any variation of the ICH/IRR messages dealing with invalid passwords. New Security Dashboards Security Message Counts - Top 5 per hour over Last Day Security Message Type Counts - Top 5 per hour over Last Day Total Security Message Counts per hour over Last Day Security Messages by Hostname - Top 5 per hour over Last Day Security Message Types by Hostname - Top 5 over Last Day Total Security Messages by Hostname per hour over Last Day Quick Searches and dashboards are based on message with IHA or IRR prefixes. All RACF messages will get the standard SYSLOG header annotations including SystemName, JobName RACF Messages can be used to drive alerts which can generate s, run scripts, log messages or include in alert index. Intrusion Detection is included with the z/OS Network insight pack

Key enhancement: Pattern-based z/OS Log Forwarder data source configuration

Log collection configuration for complex environments ... made easy
Significant reduction in time to configure log collection and analysis from large environments Log Forwarder configuration has been enhanced to save time, be more dynamic and less error-prone. The System programmer is responsible for configuration of the z/OS Log Forwarder(s). Depending on the number of log files being sent to the IOA server, the Log Forwarder configuration can be time-consuming. Today, it is a manual task and it can be error-prone. Log Forwarder configuration has been enhanced in IOA V2.2 to support a wildcard and discovery feature to save time and decrease the possibility of configuration errors.

Zach the System Programmer has a large number of WebSphere Application servers, or a large number of CICS regions, and he wants to collect data from all of them. Prior to IOAz, Zach would have had to configure the z/OS Log Forwarder for each and every job name. This can be a time-consuming process, even when using the supplied Configuration Assistant. If you have 50 job names to configure, and each takes just two minutes, it will take an hour and 40 minutes to configure all 50. This can be error-prone given the manual nature of the task – mistyping a job name means that job logs are not being ingested. If a new WebSphere Application Server or CICS region is added after the z/OS Log Forwarder is started, the z/OS Log Forwarder must be restarted to pick up the new job log gatherer definition.

With IOAz, Log collection configuration is much easier. Zack can now create a single job log gatherer definition with one or more wildcard characters in the Job Name field. This definition serves as a template for all jobs that match the job name pattern. A handful of definitions can now cover many job names. Instead of an hour and 40 minutes to create definitions for 50 job names, it may take a minute or two. Valid wildcard characters are: * which represents any sequence of zero or more characters ? which represents any single character The z/OS Log Forwarder will start a data gatherer internally for each job name on JES spool that matches the wildcard job name value. The z/OS Log Forwarder continues searching for job names that match the pattern even after initialization has been completed. If a new job name appears it will attempt to start a job log gatherer for it.

IBM Operations Analytics for z Systems Transforming Data into Insights The Next Generation of IT Service Management.

Similar presentations

Presentation on theme: "IBM Operations Analytics for z Systems Transforming Data into Insights The Next Generation of IT Service Management."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

IBM Operations Analytics for z Systems Transforming Data into Insights The Next Generation of IT Service Management.

Similar presentations

Presentation on theme: "IBM Operations Analytics for z Systems Transforming Data into Insights The Next Generation of IT Service Management."— Presentation transcript:

Similar presentations

About project

Feedback