Presentation is loading. Please wait.

Presentation is loading. Please wait.

SCMG’11 Igor Trubin, PhD SEDS-Lite: Using Open Source Tools (R, BIRT and MySQL) to Report and Analyze Performance Data.

Similar presentations


Presentation on theme: "SCMG’11 Igor Trubin, PhD SEDS-Lite: Using Open Source Tools (R, BIRT and MySQL) to Report and Analyze Performance Data."— Presentation transcript:

1 SCMG’11 Igor Trubin, PhD SEDS-Lite: Using Open Source Tools (R, BIRT and MySQL) to Report and Analyze Performance Data

2 SCMG’11 2 A genda oSEDS-Lite project: Open source based implementation of Statistical Exception Detection System - SEDS-Lite Introduction SEDS-Lite Introduction oOpen Source and “free” tools for Capacity Management – review oControl Charts as a good test of using different tools for Capacity management - Power of Control Charts and IT-Chart Concept (Part 1) Power of Control Charts and IT-Chart Concept (Part 1) oR-system to built Control Charts ( System Management by Exception: Power of Control Charts Mar 25, 2009) System Management by Exception: Power of Control Charts oHow To Build IT-Control Chart - Use the Excel Pivot Table!How To Build IT-Control Chart - Use the Excel Pivot Table! oBIRT based Control Chart BIRT based Control Chart oBIRT Data Cubes Usage for Performance Data Analysis BIRT Data Cubes Usage for Performance Data Analysis oBuilding IT-Control Chart by BIRT against Data from the MySQL Database Building IT-Control Chart by BIRT against Data from the MySQL Database oEV-Control Chart (to build using BIRT)EV-Control Chart oSimplified Exception Detection using BIRT: UCL=LCL : How many standard deviations do we use for Control Charting? Use ZERO! UCL=LCL : How many standard deviations do we use for Control Charting? Use ZERO!

3 SCMG’11 SEDS-Lite Open source based implementation of Statistical Exception Detection System - SEDS-Lite IntroductionSEDS-Lite Introduction

4 SCMG’11 4 Open Source and “free” tools for Capacity Management Previous reviews: oMeasureIT - Issue Open Source Capacity & Performance Management Tools for Windows & Unix Systems ( Brian Johnson )MeasureIT - Issue Open Source Capacity & Performance oCapacity Planning and Performance Monitoring with Free Tools (for mspcmg 2011! - Adrian Cockcroft)Capacity Planning and Performance Monitoring with Free Tools Adrian Cockcroft The missed point is BI and Generic DB open source tools usage for Performance Analysis and Reporting oBIRT (Business Intelligence and Reporting Tools)BIRT oMySQLMySQL oNoSQLNoSQL

5 SCMG’11 5 Control Charts as a good test of using different tools for Capacity management What is the Control Chart? o Power of Control Charts and IT-Chart Concept (CMG’09 -’10) Power of Control Charts and IT-Chart Concept

6 SCMG’11 6 R-system to built Control Charts The input data is Unix File Space Utilization: R-script R-script (published on my blog):published on my blog (FYI: qcc: An R package for quality control charting :qcc: An R package for quality control charting project.org/web/packages/qcc/index.html project.org/web/packages/qcc/index.html ) oEXAMPLE3: Monthly Profile R download: CMG’09 workshop “ Power of Control Charts “, SCMG - Mar 25, 2009 Power of Control Charts

7 SCMG’11 7 R-system to built Control Charts CMG’09 workshop “ Power of Control Charts “, SCMG - Mar 25, 2009 Power of Control Charts

8 SCMG’11 BIRT based Control Chart What is BIRT – see BIRT Report Designer Tutorial DemoBIRT Report Designer Tutorial Demo There is a tool that generates control charts against performance data using BIRT:a tool The IT-Control Chart would give much more value for analyzing time stamped historical data. Is that possible to build using BIRT?IT-Control Chart 8

9 SCMG’11 9 The case where the input data source is CSV file: BIRT based IT-Control Chart: Live demo But how to calculate that UCL and LCL numbers!?

10 SCMG’11 10 Weekhour calculation: How to calculate that UCL and LCL numbers!? Use EXCEL Pivot Table Pivot Table creation (IT-Chart Builder)IT-Chart Builder Pivot Chart built

11 SCMG’11 11 Data behind IT Control chart is a Cube: How to calculate that UCL and LCL numbers!? BIRT Data Cubes Usage for Performance Data Analysis Input data is raw stamped measures The result (in the form of BIRT report designer preview) is on the picture at left

12 SCMG’11 12 (1) The Reference set or base-line is based on the input raw data with some filtering and computed columns (weekday and weekhour) How to calculate that UCL and LCL numbers!? BIRT Data Cubes Usage for Performance Data Analysis (2) the Actual data set is the same but having the different filter: (raw[“date”] Greater “ ”)

13 SCMG’11 13 (3) To combine both data sets for comparing base-line vs. actual, the “Data Set1” is built as a “Joint Data Set” by the following BIRT Query builder: How to calculate that UCL and LCL numbers!? BIRT Data Cubes Usage for Performance Data Analysis (4) Then the Data Cube was built in the BIRT Data Cube Builder with the structure shown on the following screen: (5) The next step is building report starting with Cross Table (which is picked as an object from BIRT Report designer “Pallete”): The picture above shows also what fields are chosen from Cube to Cross table.

14 SCMG’11 14 (6) The final step is dropping “Chart” object from “Palette” and adding UCL calculation using Expression Builder for additional Value (Y) Series: How to calculate that UCL and LCL numbers!? BIRT Data Cubes Usage for Performance Data Analysis

15 SCMG’11 15 To see the result one needs just to run the report or to use a "preview' tab on the report designer window: BIRT Data Cubes Usage for Performance Data Analysis

16 SCMG’11 16 How to calculate that UCL and LCL numbers!? SQL script for MySQL Database (upload) (1) That raw data (the same) need to be uploaded to some table (CPUutil) in the MySQL schema (ServerMetric) by using the following script (sqlScriptToUploadCSVforSEDS.sql):

17 SCMG’11 17 (2) Then the output (result) data (ActualVsHistoric table) is built using the following script (sqlScriptToControlChartforSEDS.sql): How to calculate that UCL and LCL numbers!? SQL script for MySQL Database (Calculate)

18 SCMG’11 18 (3) In the BIRT, the connection to MySQL database is established (to MySQLti with schema ServerMetrics to table ActualVsHistorical): Building IT-Control Chart by BIRT against Data from the MySQL Database

19 SCMG’11 19 (4) the chart is developed the same way like that was done in BIRT based Control Chart post (slide 14):BIRT based Control Chart Building IT-Control Chart by BIRT against Data from the MySQL Database

20 SCMG’11 20 (5) In BIRT you can specify report parameters, that could be then a part of any constants including for filtering (to change a baseline or to provide server or metric names). Building IT-Control Chart by BIRT against Data from the MySQL Database (6) Finally the report should be run to get the following result, which is almost identical with the one built for BIRT based Control Chart post (slide 15)BIRT based Control Chart Lastly: R-system can be used to run the same SQL script against MySQL by using Package 'RMySQL‘!!!! Package 'RMySQL‘

21 SCMG’11 21 EV meta-metric introduced in 2001 as a measure of anomaly severity. ( The Exception Value Concept to Measure Magnitude of Systems Behavior Anomalies) The Exception Value Concept to Measure Magnitude of Systems Behavior Anomalies EV-Control Chart (to build using BIRT) It is the difference (integral) between actual data and control limits. In CMG’08 paper “Exception Based Modeling and Forecasting” That metric was plotted using Excel to explain how it could be used for a new trend starting point recognition: “Exception Based Modeling and Forecasting

22 SCMG’11 22 Plotting that meta-metric and/or two their components (EV+ and EV-) over time gives a valuable picture of system behavior. EV-Control Chart (to build using BIRT) The BIRT example of the EV-chart against the same sample data: EV calculation

23 SCMG’11 23 EV-Control Chart (to build using BIRT) The EXCEL example of the EV-chart against the same sample data:

24 SCMG’11 24 It is based on real script developed and successfully tested against real data for BIRT tool to get exceptional servers list: BIRT tool SEDS-Lite: Exception Detection (EV-based) using BIRT Below is example of DB2-like SQL script to implement that. Lite version: (UCL=LCL)

25 SCMG’11 25 References Global and Application Levels Exception Detection System, Based on MASF Technique Global and Application Levels Exception Detection System, Based on MASF Technique Igor Trubin: “ Global and Application Levels Exception Detection System, Based on MASF Technique ”, Proceedings of the Computer Measurement Group, (http://www.cmg.org/measureit/shared/trubin_02.pdf) Global and Application Levels Exception Detection System, Based on MASF Technique (http://www.cmg.org/measureit/shared/trubin_02.pdf) Disk Subsystem Capacity Management Based on Business Drivers I/O Performance Metrics and MASF Disk Subsystem Capacity Management Based on Business Drivers I/O Performance Metrics and MASF Linwood Merritt, Igor Trubin: “ Disk Subsystem Capacity Management Based on Business Drivers I/O Performance Metrics and MASF ”, Proceedings of the Computer Measurement Group, (http://regions.cmg.org/regions/ncacmg/downloads/june162004_session3.doc) Disk Subsystem Capacity Management Based on Business Drivers I/O Performance Metrics and MASF (http://regions.cmg.org/regions/ncacmg/downloads/june162004_session3.doc) Mainframe Global and Workload Level Statistical Exception Detection System, Based on MASF Mainframe Global and Workload Level Statistical Exception Detection System, Based on MASF Linwood Merritt, Igor Trubin: : “ Mainframe Global and Workload Level Statistical Exception Detection System, Based on MASF ”, Proceedings of the Computer Measurement Group, (http://www.cmg.org/membersonly/2004/papers/4179.pdf) Mainframe Global and Workload Level Statistical Exception Detection System, Based on MASFhttp://www.cmg.org/membersonly/2004/papers/4179.pdf) Capturing Workload Pathology by Statistical Exception Detection System Capturing Workload Pathology by Statistical Exception Detection System Igor Trubin: “ Capturing Workload Pathology by Statistical Exception Detection System ”, Proceedings of the Computer Measurement Group, (http://www.cmg.org/membersonly/2005/papers/5016.pdf) Capturing Workload Pathology by Statistical Exception Detection System (http://www.cmg.org/membersonly/2005/papers/5016.pdf) System Management by Exception, Part 6 System Management by Exception, Part 6 Igor Trubin: “ System Management by Exception, Part 6 ”, Proceedings of the Computer Measurement Group, (http://www.cmg.org/membersonly/2006/papers/6120.pdf) System Management by Exception, Part 6 (http://www.cmg.org/membersonly/2006/papers/6120.pdf) System Management by Exception, Part Final System Management by Exception, Part Final Igor Trubin: “ System Management by Exception, Part Final ”, Proceedings of the Computer Measurement Group, 2007.( ) System Management by Exception, Part Final Exception Based Modeling and Forecasting Exception Based Modeling and Forecasting Igor Trubin: “ Exception Based Modeling and Forecasting ”, Proceedings of the Computer Measurement Group, ( ) Exception Based Modeling and Forecasting IT-Control Chart IT-Control Chart Igor Trubin: “ IT-Control Chart ”, Proceedings of the Computer Measurement Group, (http://itrubin.blogspot.com/2010/07/my-new-cmg10-paper-it-control-charts.html ) IT-Control Charthttp://itrubin.blogspot.com/2010/07/my-new-cmg10-paper-it-control-charts.html

26 SCMG’11 26 SCMG’11 Questions? Igor Trubin, PhD Everything is ready to build SEDS-Lite: Using Open Source Tools (R, BIRT and MySQL)


Download ppt "SCMG’11 Igor Trubin, PhD SEDS-Lite: Using Open Source Tools (R, BIRT and MySQL) to Report and Analyze Performance Data."

Similar presentations


Ads by Google