IBM iDoctor for i Overview

IBM iDoctor for i Overview
Dawn May

IBM iDoctor for i Tools and services for performance investigation
Job Watcher Disk Watcher PEX Analyzer Collection Services Investigator Heap Analyzer

iDoctor for i the presentation flow Introduction
Overview and a brief history Where these tools fit The iDoctor GUI Unique Features Components Collection Services Investigator Disk Watcher Job Watcher PEX-Analyzer

iDoctor for i - Introduction
Overview iDoctor is suite of dynamic tools developed by the Rochester Support Center used to collect, investigate and analyze performance data on System i. The tools are used to monitor overall system health at a high "overview" level or to drill down to the performance details within job(s), disk unit(s) and/or programs over data collected during performance situations. Frequently used by IBM, along with customers and consultants to help solve complex performance issues quickly. Feedback from iDoctor users continues to help shape the enhancements incorporated into new iDoctor components and features.

PRODUCTIVITY iDoctor for i - Introduction Goals
Broaden the user base for Performance Investigation enable Operators, Programmers, IS Management as well as Performance Specialists, Consultants Simplify and automate processes Provide quick, immediate access to collected data Provide more analysis options Reduce the dependency on PEX traces PRODUCTIVITY

a historical progression - the tools developed and used by service IBM iDoctor for IBM i PEX Analyzer - Job Watcher - Collection Services Investigator - Disk Watcher - Heap Analyzer iDoctor for iSeries PEX Analyzer - Job Watcher - Heap Analysis Tools Data Viewer Data Grapher Simplified Collection Enhanced Analysis PEX Service Analysis Tools Programs - Commands - Queries Instrumentation TRACE STATS PROFILE SMTRACE TPST SAM I M P I

Level of detail by performance tool Function Performance Tool Capacity planning PM for Power Systems High-level system/job monitoring iDoctor – Collection Services Investigator IBM i Performance Tools (PT1) Management Central Monitors Medium-level system/job monitoring iDoctor – Job Watcher WRKSYSACT IBM i Job Watcher (PT1) / STRJW Low-level system/job tracing, stats, profiling iDoctor – PEX Analyzer IBM i Performance EXplorer / PRTPEXRPT Disk stats/tracing iDoctor – Disk Watcher IBM i Disk Watcher (PT1) / STRDW “Classic” JVM Heap Analysis iDoctor – Heap Analyzer DMPJVM

iDoctor for i – The iDoctor GUI
iDoctor vs. 6.1 Performance Tools LPP (Performance Data Investigator) iDoctor 6.1 Performance Tools Windows Client iDoctor GUI Fee Frequent updates Prior release support ( ) Web Browser Systems Director Navigator Fee/Free Updated once per release Requires conversion Both GUIs use the same OS commands and produce the same raw data. Different approaches applied to summarization, analysis techniques

Overview Windows client offers flexibility and functionality not yet in web version All components offer a similar user experience The GUI provides access for iDoctor components installed on servers running IBM i V5R3 or higher. Requirements: System i access for Windows License keys for Job Watcher (includes DW, CSI) and PEX Analyzer component usage.

Unique Features Update History Automatic checking for new PTFs / client updates SQL Editor Monitors Graph compare mode Time range toggling Alternate views Dynamic graph legend Super Collections Situational Analysis

Update History Window Keeps you informed of the latest updates and enhancements

Client update notification The iDoctor client provides automatic checks for new builds Checking performed at startup and can to be turned off in Preferences. PC firewall must allow iDoctor.exe access to the Internet.

Automatic PTF notification The iDoctor client checks for new Job Watcher and PEX PTFs New PTF numbers identified on the iDoctor website are sent to the PC. PC firewall must allow iDoctor.exe access to the Internet.

SQL Editor Available above every graph/table in iDoctor Edit SQL/rerun reports as desired

Monitors Allows 24x7 collection of Job Watcher, PEX Analyzer or Disk Watcher data. Retain only the desired amount of historical data. When a problem occurs you will (hopefully) have the data you need to solve it.

Graphing multiple collections Dashed lines indicate a time break/different collection.

Graph compare mode Compare ASPs, disk units, jobs and more.

Time range graphs/toggle Allows for either broad or detailed views of time based graphs Time range interval toggle (clock icon) provides customization of interval size Bar selection allows for easier drill down into desired time periods Graph compare mode allows different time groupings to be analyzed. These features apply to all time-based graphs in iDoctor.

Time range graphs toggle example

Alternate views Allows quick comparisons between different graphs Utilizes the report data already stored in memory to give instant display Button shown below enables this option

iDoctor for i – The iDoctor GUI Alternate Views Example

Dynamic graph legend Graph legend allows drag and drop or context menu usage to customize the data shown on the graph

Super Collections Provides the capability to capture multiple types of collections simultaneously. Job Watcher is always collected in a Super Collection while CS, PEX and/or Disk Watcher are optional.

Situational Analysis Identifies any potential problem jobs found in the collection. Built on the vast knowledge of IBM performance experts and past experiences. Optional part of the Job Watcher summary. Different background colors/flyovers on the overview graphs indicate jobs of interest for occurring time periods.

Situational Analysis

Synchronized Table View for Graphs Provides easy access to the “real data” behind the graph with synchronized scrolling and selection.

iDoctor for i – Collection Services Investigator
Overview General Information Using Collection Services Investigator For wait analysis For system monitoring

General Information GUI client provides dozens of graphs for the high level job based data collected by Collection Services. Data is typically collected 24x7 at 15 minute intervals. Metrics shown include waits, I/Os, CPU, IFS and more. Works against the Collection Services database files CRTPFRDTA must be ran to produce the database files (GUI option for that also available). Typically 1 collection = 1 24 hour period. Included with a Job Watcher license.

Using Collection Services Investigator Graph Types Available Collection wide by time interval By default these offer drill down into Rankings graphs for the selected time period. Rankings Thread, job, job user, generic job, current user, pool, priority, subsystem. By default these offer drill down into the next graph type for the selected objects. Selected “objects” by time interval Objects = thread, job, generic job, ASP, disk pool, etc.

Using Collection Services Investigator Wait Analysis to Locate Bottlenecks Provides support to graph waits and CPU utilization with drill down to jobs. At V5R3/V5R4 correctly displays the wait bucket graphs based on the in use mapping for that collection. At V6R1, CS bucket mapping matches the Job Watcher mapping. Monitor System Performance Use the high level graphs to monitor performance based on the desired metrics CPU, I/Os, transactions, etc.

Collection Overview showing object lock contention

Collection Overview “Waits Analysis” Example
iDoctor for i – Collection Services Investigator Collection Overview “Waits Analysis” Example This graph is showing time spent in journal, object lock contention.

Drill Down Example Showing All 3 Graph Types
iDoctor for i – Collection Services Investigator Drill Down Example Showing All 3 Graph Types

CPU consumption by thread example
iDoctor for i – Collection Services Investigator CPU consumption by thread example

Physical disk I/O totals example
iDoctor for i – Collection Services Investigator Physical disk I/O totals example

iDoctor for i – Disk Watcher
Overview Graphical user interface over Disk Watcher data drill-down capabilities statistical and trace data Included with the iDoctor Job Watcher component Includes a trace summary function to provide graphing with drill down of I/O rates, average response times, I/O time buckets and more.

What is Disk Watcher? OS function offering the ability to trace and summarize I/O activity utilizing a wrapping flight recorder. Provides near real-time diagnosis of disk performance issues. Collects operational details used to determine the hardware, job, program, and object associated with each I/O. Provides the data needed to analyze disk performance problems through the use of I/O statistical reports and views. Surfaces data beyond that provided by such tools as: WRKDSKSTS WRKSYSSTS WRKSYSACT V5R3M0, V5R3M5, V5R4M0, V5R4M5 available via PTF V6R1M0 available as part of the base OS

With Disk Watcher you can …
Evaluate the performance of I/O operations to: Internal disks External disks (LUNs) Multi-path Evaluate the performance of I/O queuing Determine how performance may be improved by re-spreading data across units Determine the optimal placement of devices, IOAs, or buses Trace every I/O performed Collect statistical summary data AND/OR

Disk Watcher - Goals Reduce disk analysis complexity
Provide quick, immediate access to collected data Near ‘real time’ analysis Provide for “after the fact” analysis of the collected data Reduce the dependency on PEX traces Provide conditional data collection To limit the size of a collection

Disk Watcher – Collection Options
Trace TDE resolution Program resolution Object resolution Conditional data collection Stats (summarization) Fixed interval time or dynamic

Disk Watcher – A user can define:
What to collect Disk unit, ASP, Pool, I/O type Resolution level (TDE, program, object) How long to collect Time, size, intervals Conditional data collection Service time, deferred queue time, response time

Disk Watcher – Wrapping Flight Recorder
Dynamically start dumping data when the flight recorder is 80% full Or select a specific time interval Dumping should lead new entries. If new entries overtake the dumping or if the selected time interval is too long then we wrap the flight recorder and have missed data.

Launching iDoctor Disk Watcher

Disk Watcher – Select and Launch

Disk Watcher - Start

Disk Watcher - Start Wizard will guide you

Disk Watcher – Collection Options
Basic Collection name and location Collection type (Summary/Trace/Both) Variable/dynamic interval length Advanced Data availability ASP threshold overrides (system/user) Optional hardware information Disk Selection Storage pools (ASPs) Disk units Memory pools Termination Disk space Time Intervals

Disk Watcher – Graphing

Disk Watcher – Graph Selections
Trace graph choices Statistical graph choices

I/O counts and times

Operation/second with response time by disk unit

Disk Watcher – Report Selections
Graphs can be used to drill down for additional report information

Disk Watcher – Data files
Related data files are easily accessed Custom queries and graphing

iDoctor for i Job Watcher
summary Job Watcher provides real-time, non-intrusive, detailed and summarized views of job/thread/task performance data - some of which, until now, were unavailable on a System i server it's the step to take to help avoid a system wide trace or to help ensure that a trace will yield useful data it's a super WRKSYSACT that displays both "running" and "waiting" components for a job includes the Collection Services Investigator and Disk Watcher analysis tools

iDoctor for i PEX Analyzer
summary PEX Analyzer simplifies the collection and enhances the analysis of PROFILE, STATS and Trace data provides the details necessary for the low-level analysis of processor utilization, DASD operations, file space usage, waits, file opens and much more it's the system performance "X-ray"

iDoctor for i - Website http://www-912.ibm.com/i_dir/iDoctor.nsf
Documentation Downloads FAQ Consulting Services Education

The Redbook was published in early 2005 but still can be very useful in understanding the fundamentals of Job Watcher.

PEX Stats Redbook

transaction response time
iDoctor for i the tandem used to expose run/wait components PEX Analyzer PROFILE - program instructions STATS - program calls TRACE - timed events Job Watcher snapshot current status snapshot deltas Job/Thread/Task Run/Wait Components batch run time running waiting CPUq DASD blocked idle transaction response time

iDoctor for i JOBxxx some PEX Analyzer and Job Watcher comparisons
PEX (Performance Explorer)‏ - Trace (Method: record all events)‏ (PRO: exact timing, complete event detail)‏ (CON: takes time and resource to dump and analyze)‏ - STATS (Method: summarize all calls)‏ (PRO: details the time spent in program modules)‏ (CON: overhead adds/alters code path length of what your investigating JOBxxx interval delta snapshot snapshot Job Watcher (Method: record snapshots, compute interval deltas)‏ (PRO: quick, non-intrusive and relatively cheap to collect data)‏ (CON: a bit sketchy, could miss jobs initiating/terminating within intervals)‏

iDoctor for i the challenge
why Job Watcher was developed the challenge trace data is preferred because of the detail but a trace takes time to dump and analyze and busy, larger, faster systems can generate even more trace data within shorter trace windows and often, until a trace is dumped and analyzed, you don't know if the trace captured what you needed

iDoctor for i Job Watcher as a trace alternative
why Job Watcher was developed... Job Watcher as a trace alternative its rapid snapshot, interval analysis, and data harvesting options can reduce the need for trace as a first step - or ensure that a trace underway has captured worthwhile data it's a non-intrusive collector (like WRKSYSACT) its trigger wizard allows the setup of "and/or" rules based on the data collected which can be used to initiate a call to a user program

iDoctor for i Data Harvested
the Job Watcher snapshot Data Harvested most of the counters captured by WRKSYSACT, WRKACTJOB, and Collection Services for Jobs, including threads and tasks the program stack (50 deep, 1000 starting in V5R3), activation group data, active SQL data, wait buckets, Java statistics the current wait, its duration, the object waited on, the holder if available

iDoctor for i Same Interval Analysis as Collection Services
the Job Watcher snapshot interval Same Interval Analysis as Collection Services cumulative counter deltas Plus wait status at snapshot Job Watcher factors the current wait duration into the interval statistics intervals as small as 100 msecs are possible and may provide enough information to avoid a trace

iDoctor for i the Job Watcher snapshot interval and the current wait snapshot #1 snapshot #2 snapshot interval other waits JOBxxxx cpu current wait

iDoctor for i current wait examples current wait snapshot #1
Eye Wait Wait Wait ENUM Catcher Bucket Description Object Holder 158 SRd 10 DASD Read Request XYZfile na 7 QTQ 27 TreeQueue XYZqueue na 100 Rex 11 Seize Read exclusive XYZindex XYZjob

iDoctor for i OK if idle wait (like Key/Think time)‏
current wait can span snapshot intervals snapshot #1 snapshot #2 current wait OK if idle wait (like Key/Think time)‏ But if a long DASD request or a block? (and the current wait has not ended yet)‏

JW – the snapshot approach
Job 1000 Job 0001 t i m e JW snapshot analysis begins (satisfied rule can call user program)‏ (Interval statistics written to DB)‏ data is available for viewing/analysis

JW – the run/wait summary for a snapshot interval
Job running waiting … … … wait buckets CPUq Syn DASD Lock CPU the system maintains counts and accumulated time for all jobs/threads/tasks

JW – the last current wait at snapshot time
~1000 instrumented wait points ~200 unique wait identifiers (ENUMs)‏ mapped to ~30 wait buckets Job the unique wait identifier (ENUM)‏ the wait time (so far)‏ the object waited on the holder of the object (when available) Long ”bad waits” can be spotted right here (blocks, contention)‏

Collected Data - Intervalized, Summarized, Averaged, Expanded
avg Tnx rsp utilization 40% - 30% - 20% - 10% - t i m e utilization 100 90% - 80% - 70% - 60% - 50% - 40% - 30% - 20% - 10% - t i m e avg Tnx rsp

iDoctor for i long intervals
interval duration - the pros and cons of long intervals snapshot #1 snapshot #2 snapshot interval other waits JOBxxxx cpu current wait long intervals less recording overhead and space required depend on averages, rates and percentages and the changes in same (bursts may not be apparent) good for strategic planning - capacity and operations - for identifying when peak loading occurs

iDoctor for i short intervals
interval duration - the pros and cons of short intervals snapshot snapshot snapshot snapshot other waits other waits JOBxxxx cpu cpu current wait 1 2 3 4 reported current waits short intervals more recording overhead and file space required bursts become more apparent more frequent current wait and stack info (avoid traces)‏ good for performance investigation

iDoctor for i Starting a watch
the "watch" Starting a watch select active jobs / threads / tasks (even job queue entries) from a GUI list set interval duration, watch duration, watch name, library and description, and what details to collect optionally set a trigger to control when the watch should start or what program to call if specified conditions are met during a watch (a trigger wizard is provided.)‏

iDoctor for i the "watch" IBM i 6.1 include the Job Watcher Performance Data Collector Commands to define Job Watcher collections and to start and end Job Watcher are now included with 6.1 ADDJWDFN, RMVJWDFN STRJW, ENDJW Job Watcher data can be collected without the iDoctor product Job Watcher GUI included with the IBM i Performance Tools LPP (5761-PT1, Option 3)‏

iDoctor for i Summary of all intervals (% of time by component)‏
viewing the "watch" - during the watch or after a completed watch Summary of all intervals (% of time by component)‏ The Run/Wait Signature (% of time by component by interval)‏ DASD waits block waits JOBxxxx cpu idle wait JOBxxxx

Drill for details about the watch
iDoctor for i viewing the "watch" - during the watch or after a completed watch Drill for details about the watch DASD waits block waits JOBxxxx cpu idle wait current current interval current wait wait cpu interval usecs wait usecs % interval % interval 1 2 3 4 5 (select fields, sort columns, query records and more)‏

Drill for details about an interval
iDoctor for i – Job Watcher viewing the “collection" - during or after completion Drill for details about an interval JOBxxxx Program Stack Run/Wait Component Counts and Times Current Wait, Wait Object, Holder Basic Statistics, Transitions, etc. SQL Activation Group Comm Java

iDoctor for i ...PEX Analyzer events could show (if collected)‏
PEX Analyzer surfaces even more detail ...PEX Analyzer events could show (if collected)‏ every physical DASD request, its size and duration, the IOP and unit, and the object name, type and address file opens and closes MI entry and exit and more if required

iDoctor for i

iDoctor for i %Time Representation Job Signature Tnx Signature
more about Run/Wait Components %Time Representation Job Signature Tnx Signature running waiting CPU CPUq DASD Blocks COMM Other Idle CPU CPUq DASD Blocks COMM Other

iDoctor for i Job Watcher snapshots and snapshot intervals
both Job Watcher and PEX Analyzer expose Run/Wait components Job Watcher snapshots and snapshot intervals groups waits into wait buckets accurate total times and counts non-intrusive to the Job's code path length quick "real time" and post collection investigation possible PEX Analyzer time stamped task switch type events each individual wait exact timing of every individual wait adds a little to the Job's code path length must dump and analyze the trace data

iDoctor for i ...PEX Analyzer events could show (if collected)‏
PEX Analyzer surfaces even more detail ...PEX Analyzer events could show (if collected)‏ every physical DASD request, its size and duration, the IOP and unit, and the object name, type and address file opens and closes MI entry and exit and more if required

Trademarks and Disclaimers
8 IBM Corporation All rights reserved. References in this document to IBM products or services do not imply that IBM intends to make them available in every country. Trademarks of International Business Machines Corporation in the United States, other countries, or both can be found on the World Wide Web at Adobe, Acrobat, PostScript and all Adobe-based trademarks are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, other countries, or both. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. UNIX is a registered trademark of The Open Group in the United States and other countries. Cell Broadband Engine and Cell/B.E. are trademarks of Sony Computer Entertainment, Inc., in the United States, other countries, or both and are used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. The customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-IBM products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Prices are suggested U.S. list prices and are subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

IBM iDoctor for i Overview

Similar presentations

Presentation on theme: "IBM iDoctor for i Overview"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

IBM iDoctor for i Overview

Similar presentations

Presentation on theme: "IBM iDoctor for i Overview"— Presentation transcript:

Similar presentations

About project

Feedback