Debbie Sheetz Sr. Staff Consultant

Top Ten Topics in Performance Reporting/Modeling for Virtualization using BMC Performance Assurance
Debbie Sheetz Sr. Staff Consultant BMC Software, Customer Support/Waltham, MA

Presentation Overview
Overview of Customer-Selected Virtualization Topics AIX Partitions SMT effect on Performance reporting Performance reporting Server consolidation to AIX partitions VMware ESX Server Server consolidation to VMware HP Partitions Solaris Container (Zone) Modeling techniques and results reporting © Copyright 3/25/2017 BMC Software, Inc.

© Copyright 3/25/2017 BMC Software, Inc.
Why virtualization? IT needs to improve the ratio between service delivery and cost More cost-effective hardware can be purchased (lower power) Higher concentrations of hardware resources = lower cost Hardware cost/transaction Manpower required for management Software license cost Square foot/HVAC cost A popular choice is mainframe-like Distributed Systems hardware with Partitioning technology Allows dissimilar workloads to coexist, but to be managed separately Allows for higher resource utilization, e.g. same hardware can be shared by work which has non-overlapping peaks © Copyright 3/25/2017 BMC Software, Inc.

Different Virtualization Strategies from each Vendor
Several virtualization strategies are popular IBM partitions VMware virtual machines HP partitions Solaris containers Currently the most popular choice among BPA customers is IBM partitioning Many companies have production AIX 5.3 partitioned environments, and the number of POWER5 CPUs managed this way is growing This presentation assumes some familiarity with IBM’s AIX Partitioning, HP partitioning, VMware, and Solaris Containers This presentation assumes basic familiarity with the BMC Performance Assurance components: Analyzer, Predictor, Visualizer, Manager © Copyright 3/25/2017 BMC Software, Inc.

Tools shown in this presentation
This presentation shows the combination of BMC Performance Assurance for Virtual Servers (Release 7.4 of Analyzer/Predictor and Visualizer ) and extensions distributed by BMC Customer Support as a result of field experience with these products Visualizer extensions available at ftp://ftp.bmc.com/pub/visualizer/vgd/ Most of the custom metrics can be obtained from “out-of-the-box” graphics, with different metric labels, metric groupings, and/or metric units Presentation surveying different Virtualization Strategies available from Customer Support (Do CPUs Count? written for UKCMG 2007) Specific Virtualization techniques white papers available from BMC Customer Support (FAQs for AIX Partitions, FAQs for VMware) Performance Assurance Suite for Virtual Servers, product documentation ( Not shown in this presentation (but part of BPA Virtual Servers) Perceiver Investigate 7.4 © Copyright 3/25/2017 BMC Software, Inc.

Topic: How SMT is reported
What is SMT? IBM’s Simultaneous Multi-Threading provides resources to dispatch two threads simultaneously to one physical processor, i.e. core (pretty useful doc at This is NOT dual-core which provides two independent physical processors Available first with AIX 5.2 on POWER5 processors Partitioning coexists with SMT, but does not depend on it Diagram from IBM redbook Advanced POWER Virtualization on IBM System p5 © Copyright 3/25/2017 BMC Software, Inc.

How SMT is reported What is SMT (continued)? How does it look when it’s reported? Each physical processor is reported as two logical processors by AIX AIX CPU accounting (i.e. PURR) ensures that CPU usage metrics are correct The sum of the two logical processors’ CPU utilization = utilization of one physical processor This is NOT like Windows Hyper-Threading which doesn’t have equivalent CPU metrics Approaches for reporting SMT SMT-aware reporting Logical/physical processor relationship is exploited Recommended BPA solution is at least 7.3 Performance Assurance console and AIX 7.3 collector for AIX 5.2/5.3 BMC Performance Manager (formerly PATROL) support first available with AIX collector Non-SMT aware reporting CPU utilization (i.e. % of capacity in use) is reported as ½ of what it really is Performance Assurance 7.2 © Copyright 3/25/2017 BMC Software, Inc.

How SMT is reported How is SMT presented? Logical processors are aggregated to physical processors for Analyzer reports Predictor Visualizer (and Perceiver using Visualizer datasource) Logical processor view for Native UNIX tools (e.g. lcpu, lbusy, etc. means Logical processor) Raw measurement data per processor Investigate (and Perceiver using UDR datasource) Analyzer Interval (“browse data”) view Computer CPU Utilization report (Perceiver CPU Utilization – All Processors chart) Details “by metric and metric group” available in FAQs for AIX Partitions white paper The aggregated (physical processor) view ensures consistency for CPU reporting, regardless of whether or not SMT is enabled Can compare across AIX systems, other UNIX systems © Copyright 3/25/2017 BMC Software, Inc.

Topic: LPAR performance reporting challenges
What is unique about performance reporting for LPARs (partitions)? This is not “business as usual”, i.e. CPU capacity only dependent on a fixed number of available CPUs. Instead, CPU capacity now depends on three levels of processor allocation (1) Total physical processors (2) Processors assigned to DLPARs vs. Shared Pool AND (3) Each DLPAR requires adequate processors (3) Each SPLPAR requires adequate Virtual processors AND Entitlement processors AND Shared POOL processors And all of this while workloads are changing Traditional CPU capacity approaches cannot capture these simultaneous complexities © Copyright 3/25/2017 BMC Software, Inc.

Topic: How to report CPU for a Physical Frame
How is CPU reported for the physical system from the CDB? Simultaneously shows % of capacity used in 3 categories, per physical system: total physical system use average SPLPAR use of the entitlement processor allocation, and average SPLPAR use of the virtual processor allocation Note: This is a “custom” Visualizer graphic; also in Perceiver AIX Physical System Overview © Copyright 3/25/2017 BMC Software, Inc.

How to report CPU for a Physical Frame
How is CPU reported for the physical system from the CDB? Supplement previous view for the physical system with graphics for individual partitions Top 10 Virtual Used and Top 10 Entitlement Used (next slide) Gives total picture for long-term trending and looks for individual trouble spots that need immediate attention Note: This is a “custom” Visualizer graphic; also in Perceiver AIX Physical System Overview © Copyright 3/25/2017 BMC Software, Inc.

How is CPU reported for the physical system from the CDB? Supplement previous view for the physical system with graphics for individual partitions Top 10 Virtual Used (previous slide) and Top 10 Entitlement Used Gives total picture for long-term trending and looks for individual trouble spots that need immediate attention Note: This is a “custom” Visualizer graphic also in Perceiver AIX Physical System Overview © Copyright 3/25/2017 BMC Software, Inc.

How is CPU reported for the physical system from the CDB? Physical System CPU usage can be shown for all DLPARs and all SPLPARs Can select either Processors_Used (shown) or % Physical_Used (next slide) Can also show per DLPAR usage or per SPLPAR usage (shown next slide) First available with 7.4/4.2.02 Note: This is a “custom” Visualizer graphic © Copyright 3/25/2017 BMC Software, Inc.

How is CPU reported for the physical system from the CDB? Physical System CPU usage reflects total Shared Pool usage, even when all SPLPARs have not been instrumented (first available with 7.4/4.2.02) ROP = Rest of Pool, represents CPU usage of SPLPARs without collectors Can select either Processors_Used (shown previous slide) or % Physical_Used Note: This is a “custom” Visualizer graphic; also in Perceiver AIX Partition Overview © Copyright 3/25/2017 BMC Software, Inc.

Topic: How to report CPU for LPARs
How is CPU reported for LPARs from the CDB? Can also drill down on one SPLPAR, all three metrics simultaneously, or can view metrics one at a time … Note: This is a “custom” Visualizer graphic; also in Perceiver AIX Partition Capacity View © Copyright 3/25/2017 BMC Software, Inc.

How to report CPU for LPARs
How is CPU reported for LPARs from the CDB? Shows individual SPLPARs use of the physical system, e.g. total never exceeding 25% of capacity % Proc SMT label indicates that this is per physical processor (not per logical processor) © Copyright 3/25/2017 BMC Software, Inc.

How is CPU reported for LPARs from the CDB? Shows individual SPLPARs use of the virtual processor allocation, e.g. never exceeding 40% of capacity % LProc label indicates that this is per logical (i.e. virtual processor) © Copyright 3/25/2017 BMC Software, Inc.

How is CPU reported for LPARs from the CDB? Shows individual SPLPAR’s use of the entitlement processor allocation, e.g. sometimes exceeding 100% of capacity, which is possible with an uncapped SPLPAR © Copyright 3/25/2017 BMC Software, Inc.

Topic: How LPARs are represented for Modeling/Sizing
How is CPU reported for LPARs for modeling/sizing from the CDB? For capacity planning, viewing partition use expressed as SPECints, particularly across physical systems, is the perfect way to see exactly how much is being consumed and what size physical system will be needed to support any particular combination of LPARs. Note: This is a “custom” Visualizer graphic © Copyright 3/25/2017 BMC Software, Inc.

Topic: ‘What-if’ of non-partitioned to partitioned configuration: Strategy Is modeling or sizing required for the CPU? High level CPU utilization projection required No response time projections required No growth or overhead projections required Mid level System growth projections required Overhead projections required Low level Workload response time projection required System/workload growth projections required Use Sizing Use Modeling Use Modeling Specify workloads © Copyright 3/25/2017 BMC Software, Inc.

Topic: ‘What-if’ of non-partitioned to partitioned configuration: Sizing Sizing Technique Determine required CPU utilization objective Specify desired CPU utilization threshold (.e.g. 80%) Can reduce that further to account for System growth General “headroom” Determine SPECints available on proposed CPU configuration(s) Example shown on the next slide Multiply CPU utilization objective by the available SPECints Compare the available SPECints to the CDB graph showing the required SPECints for the proposed combined systems © Copyright 3/25/2017 BMC Software, Inc.

‘What-if’ of non-partitioned to partitioned configuration: Sizing
Using the Predictor GUI to determine available SPECInts Create a new Physical System Specify the desired rating basis Filter by desired hardware vendor Performance rating allows you to choose appropriately powered CPUs Select the desired CPU © Copyright 3/25/2017 BMC Software, Inc.

Using the Predictor GUI to determine available SPECInts (continued) Select the desired number of processors View the resulting total SPECints available © Copyright 3/25/2017 BMC Software, Inc.

Will the proposed CPU accommodate this grouping of systems? YES The selected CPU has 34.4 SPECInts available (from previous slide) Using a CPU objective of 80%, that would be .80 * 34.4 = 27.5 Measured maximum usage is about 27 SPECints, so that just fits in 27.5 Note: This is a “custom” Visualizer graphic © Copyright 3/25/2017 BMC Software, Inc.

Topic: ‘What-if’ of adding standalone AIX systems onto existing partitioned configuration: Sizing Will the current Physical Frame accommodate these additional systems? The selected Physical Frame has 280 SPECints available Construct a Visualizer “group” containing the existing LPARs on the frame, plus the proposed standalone AIX systems, then … Note: This is a “custom” Visualizer graphic © Copyright 3/25/2017 BMC Software, Inc.

‘What-if’ of adding standalone AIX systems onto existing partitioned configuration: Sizing Will the current Physical Frame accommodate these additional systems? YES The selected Physical Frame has 280 SPECInts available (from previous slide) Using a CPU objective of 80%, that would be .80 * 280 = 224 SPECint capacity is available Measured maximum usage is about 150 SPECints, so that easily fits in 224 Note: This is a “custom” Visualizer graphic © Copyright 3/25/2017 BMC Software, Inc.

‘What-if’ of adding standalone AIX systems onto existing partitioned configuration: Sizing Sizing can be done for metrics in addition to the CPU CPU sizing by SPECint (technique shown previous slides) I/O in MB/sec Disk Space Used in GB (and Disk Space Configured) Memory Used in GB, non-FileSystem Memory Used (and Memory Configured) Network traffic in Mb/sec Note: This is a “custom” Visualizer graphic © Copyright 3/25/2017 BMC Software, Inc.

Topic: ‘What-if’ changes to current partitioned configuration: Modeling Modeling process steps Build baseline model for current partitioned system Use ‘Best Practices’ implementation What-ifs can include Revise CPU configuration of the partitioned system Revise configuration of individual DLPARs and SPLPARs Move additional work onto the physical system Move entire computers and/or Import desired Workload(s) from systems not in this model Revise CPU utilization objectives System growth (use Planner facilities) Evaluate the model and observe messages/reports for warnings that a CPU objective has been exceeded © Copyright 3/25/2017 BMC Software, Inc.

Topic: Implementation ‘Best Practices’ for AIX Reporting and Modeling
Confirm that the baseline analysis contains all the important LPARs and that they have been properly instrumented: Instrument all important LPARs with a (or later) collector This ensures proper identification of SMT as well as access to the new metrics required to properly report on LPAR resource consumption Ensure that at least one LPAR per physical system is collecting the HMC configuration data page 81, LPARs may be processed in separate Manager domains/policies, but must be populated to the same Visualizer database If you choose to split LPARs from the same physical system across multiple domains/policies, item (2) requires that a least one LPAR per physical system per domain/policy is collecting the HMC configuration Missing LPARs can compromise having complete results! © Copyright 3/25/2017 BMC Software, Inc.

Implementation ‘Best Practices’ for AIX Reporting and Modeling
Every properly instrumented LPAR will appear under the appropriate physical system name, designated as DLPAR or SPLPAR (CDB graphics) Physical system name with properly recognized LPARs indented below With 7.4/ DLPAR and SPLPAR groupings are shown Note: This is a “custom” Visualizer graphic © Copyright 3/25/2017 BMC Software, Inc.

Topic: Implementation ‘Best Practices’ for AIX Sizing/Modeling
(2) Confirm that the CPU model is properly identified for both the physical system (Analyze - Physical System report) and the logical systems (Analyze - Hardware Summary report) Should be -SMT model for most partitioned systems Always requires User Translation Table entry: Name="IBM, ", VendorID=IBM,Rate=1650, FreqTolerance=1 See FAQ for AIX Partitions paper or contact BMC Customer Support for details © Copyright 3/25/2017 BMC Software, Inc.

Topic: General ‘Best Practices’ for Reporting/Sizing/Modeling
Check periodically for product updates console patches ftp://ftp.bmc.com/pub/perform/resolutions/ pdf applicable collector patches ftp://ftp.bmc.com/pub/perform/resolutions/ pdf hardware table Periodically review articles in the Knowledge Base ( For most AIX environments, the DISREGARD-DISK Analyzer override should be used. This ensures that only physical disks are reported and logical disks are not (ftp://ftp.bmc.com/pub/perform/resolutions/ pdf) For all AIX environments, ensure that the MAINTAIN DISK I/O HISTORY has been set properly (ftp://ftp.bmc.com/pub/perform/resolutions/ pdf). This is automatic with 7.4 collector. For all environments, setting CUT-DISK in Manager (or in Analyzer for interactive processing) improves the usability of the model and Visualizer results by eliminating information about disks with minimal activity (ftp://ftp.bmc.com/pub/perform/resolutions/ pdf) Sign up for proactive notification so that any important problems and their solutions are automatically sent to you © Copyright 3/25/2017 BMC Software, Inc.

Topic: ‘What-if’ of consolidating to VMware host configuration: Strategy Techniques for sizing/modeling VMware follow the same overall strategy as for AIX partitions (see slides Strategy for ‘What-if’ of non-partitioned to partitioned) Sizing techniques are identical (see slides Sizing technique using Visualizer) Modeling techniques are somewhat different because the modeling construct is different for a VMware host Documentation available (page 48, 7.4 Windows console offers new time-saving features for this modeling Will be shown in the next slides © Copyright 3/25/2017 BMC Software, Inc.

Topic: ‘What-if’ of consolidating to VMware host configuration: Modeling Modeling process steps (GUI samples shown next slides) Build baseline model for work to be moved Recommend “zzz” workload only Set Baseline ON Optionally Export desired Workload(s) from systems for transfer to another model Specify proposed CPU configuration of the VMware Host system Use New VMware Host System under Computers Specify VMware overhead utilization Specify required CPU utilization objective Define new Virtual Systems representing each of the proposed systems on the host Move work onto the new VMware host system Optionally Import desired Workload(s) from systems not in this model Move existing Workload(s) onto specific Virtual System Model System growth (use Scenario Planner) Evaluate the model and observe messages/reports for warnings that a CPU objective has been exceeded Additional information (e.g. FAQs for VMware white paper) is available © Copyright 3/25/2017 BMC Software, Inc.

Modeling Techniques Catalog: Moving Workloads (Export)
If the workload is not in the desired model, it can be transferred from one model to another Begin with the model which contains the workload to be moved in its baseline configuration Be sure the model has been previously baselined For each workload to be moved Export (Windows console) or Save to Library (UNIX console) Now use the target ‘what-if’ model For each workload to add Import (Windows console) or Add from Library (UNIX console) Specify the target server GUI examples shown next slides © Copyright 3/25/2017 BMC Software, Inc.

Modeling Techniques Catalog: Moving Workloads (Export)
Workload Export (Windows console shown) © Copyright 3/25/2017 BMC Software, Inc.

Modeling Techniques Catalog: VMware Host System Define new VMware Host System (under Computers) (first in 7.3) Specify CPU, Memory, System Services overhead, CPU Utilization Objectives © Copyright 3/25/2017 BMC Software, Inc.

Modeling Techniques Catalog: VMware Host System Configure one or more New Virtual Systems for the VMware Host Specify Operating system, virtual processors, Memory, CPU shares NO NEED TO FILL THIS IN © Copyright 3/25/2017 BMC Software, Inc.

Modeling Techniques Catalog: Moving Workloads onto a Virtual System Windows 7.4 console has new “Move Workloads” feature Select Computer Types of Virtual Systems 2. Select one or all workloads 3. Select the virtual system 4. Add 1. Get Workloads from another model (optional) © Copyright 3/25/2017 BMC Software, Inc.

Modeling Techniques Catalog: VMware Modeling results
Predict-VMware Summary report displays complete results CPU utilization, CPU response time effects (total wait time and VM wait time) © Copyright 3/25/2017 BMC Software, Inc.

Topic: VMware performance reporting challenges
What is unique about performance reporting for VMware? Similar to other partitioned Virtualization architectures, CPU capacity depends on two levels of available CPU resources Each Virtual System requires adequate (1) Virtual processors AND (2) VMware host processors While workloads are changing Performance measurements from the VMware host are generally reliable ESX Server 3.01: Use 7.3 patch or 7.4 collector ESX Server 2.5.3: Use 7.3 production (or later) collector Ongoing research into issues caused by taking a Virtual system offline, stopping the bgsagent, etc. Performance measurements from the virtual systems themselves are not generally reliable Many guest OS do not have virtualized performance statistics Collector overhead is generated for EACH guest © Copyright 3/25/2017 BMC Software, Inc.

Topic: HP partition performance reporting challenges
What is unique about performance reporting for HP partitions? Similar to other partitioned Virtualization architectures, CPU capacity depends on three levels of available CPU resources (1) Total physical processors (2) Processors assigned to each nPar (2) Each standalone nPar requires adequate processors (3) Each vPar within an nPar requires adequate Virtual processors AND nPar processors While workloads are changing © Copyright 3/25/2017 BMC Software, Inc.

Topic: How to report CPU for a Physical Frame
How is CPU reported for the physical system from the CDB? Shows total % of physical system capacity used Shows % of physical system capacity used by partition © Copyright 3/25/2017 BMC Software, Inc.

Topic: How to report CPU for each Partition
How is CPU allocation for each partition reported from the CDB? Shows total number of processors accessed per nPar or per nPar/vPar (100% = 1 processor) Dynamic shifting of processors between vPars within an nPar causes processor allocation to vary by interval © Copyright 3/25/2017 BMC Software, Inc.

Topic: Implementation ‘Best Practices’ for HP Reporting and Modeling
Confirm that the baseline analysis contains all the important partitions and that they have been properly instrumented: Instrument all important partitions with a (or later) collector This ensures access to the new metrics required to properly report on partition resource consumption Ensure that at least one partition per nPar has a collector (required to obtain complete partition configuration data) Partitions may be processed in separate Manager domains/policies, but must be populated to the same Visualizer database Missing partitions will compromise having complete results! © Copyright 3/25/2017 BMC Software, Inc.

Implementation ‘Best Practices’ for HP Reporting and Modeling
Each instrumented partition will appear under the appropriate physical system name, designated as NPAR or NPARVPAR (CDB graphics) Physical system name with properly recognized partitions indented one level below © Copyright 3/25/2017 BMC Software, Inc.

Topic: Implementation ‘Best Practices’ for HP Sizing/Modeling
(2) Confirm that the CPU model is properly identified for both the physical system (Analyze - Physical System report) and the logical systems (Analyze - Hardware Summary report) Follow “General ‘Best Practices’” (shown earlier for AIX) © Copyright 3/25/2017 BMC Software, Inc.

Topic: Solaris Container Modeling - Instrumentation
What is unique about performance reporting for Solaris Containers? Similar to other partitioned Virtualization architectures, CPU capacity depends on two levels of available CPU resources Each Zone requires adequate (1) Virtual processors AND (2) Solaris hardware domain (DSD) processors Performance measurements are obtained from a single collector running in the Global Zone Use 7.3 patch or 7.4 Solaris collector Specify how you want to view the work within the hardware domain: by zone, by project, or by “regular” workload using Analyzer (use 7.4 console) For Windows, do Properties, select Advanced Properties, Build workload by For UNIX, use File -> Properties, Build workloads by The DSD is represented by a Computer, each Zone by a Workload © Copyright 3/25/2017 BMC Software, Inc.

Topic: Solaris Container Modeling – Reporting CPU Usage by Zone
How to report CPU usage by zone (CDB graphics) Shows total % of DSD CPU used Shows % of DSD used by Zone © Copyright 3/25/2017 BMC Software, Inc.

Topic: Solaris Container Modeling – Specifying Projected Growth
How to model increasing load using Scenario Planner Specify overall % growth rate (shown, 25%, then 100%) Optionally, can specify different growth for each zone, use Plan Workload Override option (not shown) Run Plan, activating Generate Visualizer File option, and populate into Visualizer modeling DB © Copyright 3/25/2017 BMC Software, Inc.

Topic: Solaris Container Modeling – Reporting Projected CPU Usage by Zone How to report projected CPU usage by zone (CDB graphics) Shows total % of DSD CPU used for baseline each growth period Also shows % of DSD used by Zone Note: Baseline model interval was peak prime shift period (20% of available capacity) © Copyright 3/25/2017 BMC Software, Inc.

Summary Follow implementation ‘Best Practices’ Use Support website Knowledge articles Open a case with Customer Support for customized assistance with implementation Allow extra time for the first implementation for a particular type of Virtualization Apply tools specifically designed to support Virtualization Generally require at least 7.3 collector and 7.3 console Use CDB for a complete view of CPU Usage (historical and future) Measurements from physical systems (AIX and HP) Three graphs per physical system gives a complete reporting solution More detailed graphs can be used for specific analysis projects Management only needs to see “% Capacity Used” for the physical system – everything else is a “tuning” problem! Modeling results for virtual systems View physical systems and individual partitions View workload performance results within the partitions (HP and AIX only) Predictor Scenario Planner results can be shown, too © Copyright 3/25/2017 BMC Software, Inc.

Summary When using Predictor Use the appropriate modeling construct for each type of Virtualization (e.g. Physical System, VMware Host System, Computer) Use the corresponding specific modeling report for each type of Virtualization (e.g. Physical System Summary, VMware Summary, Computer Workload) Can use CDB graphics for Sizing when Predictor Modeling isn’t required Use “custom” Visualizer graphics to show CPU, I/O, Memory, Disk Space, and Network results © Copyright 3/25/2017 BMC Software, Inc.

Debbie Sheetz Sr. Staff Consultant

Similar presentations

Presentation on theme: "Debbie Sheetz Sr. Staff Consultant"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Debbie Sheetz Sr. Staff Consultant

Similar presentations

Presentation on theme: "Debbie Sheetz Sr. Staff Consultant"— Presentation transcript:

Similar presentations

About project

Feedback