1. IBM Power Systems: Tools and Strategies for Capacity Planning
Lab Services Capacity Planning Tool with IBM Workload Estimator
Michael Quaranta – Senior Managing Consultant IBM Systems Lab Services
September 2015

2. History and Motivation for the Capacity Planning Tool
Summer 2013: A project by the Power L3 Performance Team to analyze statistical CPU consumption across several sets of customer data discover under-utilization of Power systems. Analysis via the “R” language.
Fall 2013: IBM Systems Lab Services takes ownership of the tool, making it more general purpose, and expanding input formats for flexibility.
2014: A large customer becomes a beta user of the tool, tool is further developed and refined analyzing hundreds of frames and thousands of partitions. Hundreds of cores were reclaimed, system utilization improved to nearly 70% and consequently the customer was able to postpone additional capital expense to deploy new workloads.
Today: The Capacity Planning Tool (CPT) has been available as a PowerCare offering as well as for use in pre- and post-sales engagements. Thirty five customer engagements have been performed to date.

3. Poll:
My company has a team dedicated to enterprise capacity management

4. Poll: My Power Capacity Management Strategy is:
1. Set it and forget it
2. Annual/semi-annual/quarterly review and update
3. On-demand updates (user requests e.g.)
4. Nightly/Weekly DLPAR cron job

5. Poll: My Power Capacity Management Monitoring Strategy is:
1. None, but I think we probably have NMON files somewhere
2. Freeware e.g. LPAR2RRD
3. Enterprise class and cross platform (ITM/BMC etc.)
4. Other

6. Capacity Planning Tool: Overview

7. Introduction
The Capacity Planning Tool has been designed to take historical CPU consumption data for logical partitions (LPARs) on IBM Power™ systems and fit them to a more appropriate sizing, based on a combination of IBM best practices, and customer business requirements.

8. Inputs
The Capacity Planning Tool (CPT) is was designed to be flexible, and handle a number of different input formats:
NMON
ITM
BMC
LPAR2RRD
HMC LPARUTIL
HMC PCM
VMSTAT
CUSTOM

9. Capacity Planning Tool – High Level
For review with application teams
NMON
HMC lparutil
MSExcel reports
CPT CSV
CPT Analytic Engine
ITM
Perl pre-processor
common format
CSV file
Workload Estimator CSV
BMC
various input formats
supported via Perl “plugin”
WLE output for server consolidation study
© Copyright IBM Corporation 2014

10. High level diagram: general reporting tool
CSV summary
file
High level diagram: general reporting tool
Aligned CSV file
cpt.r
script
CSV file
PNG plots

11. High level diagram: MS Excel Output
cpt.r csv output
makexls.pl
<frame>.xls
file
High level diagram: MS Excel Output
PNG files
PNG files
PNG files

12. High level diagram: Workload Estimator Flow
cpt.r csv output
cpt.r
WLE
CSV file
High level diagram: Workload Estimator Flow

13. (right?) Sizing Partitions
Are we looking for spare change under the couch pillows?
We need to study how many Power cores are physically
consumed by a partition
• On average?
• Or possibly better yet, statistically
– Look more closely at shared pool consumption

14. Averaging – a closer look
Averaging loses peaks.
Infrequent sampling loses peaks.
Aren’t peaks how we want to size our virtual processors?
“The best practice for LPAR entitlement would be to match the entitlement capacity to average utilization and let the peak addressed by additional uncapped capacity.”
- POWER7 Virtualization Best Practice Guide

15. Physical CPU Consumption (a look under the hood)
lslparutil -m Server-9117-MMC-SN105C627 -s s
-r lpar --filter lpar_names=tk_client4 -n 5
-F time,lpar_name,time_cycles,capped_cycles,uncapped_cycles
08/19/ :00:01,tk_client4, , ,
08/19/ :01:01,tk_client4, , ,
08/19/ :02:01,tk_client4, , ,
08/19/ :03:01,tk_client4, , ,
08/19/ :04:01,tk_client4, , ,
PHYSC =
( ) + ( )
( )

16. Observations All we have to work with are a bunch of counters
– No instantaneous physical consumption measurement
– So all measurements are averages over sample intervals
• There are a lot of 10ms dispatch cycles in a 30 second interval

17. Averaging – In Graphic Detail
LPAR with EC=1.0

18. Taking a Dip into the Pool
Using our previous example, where entitlement of 1 core never
appears to be exceeded…
we still have non-zero uncapped cycles(???)
> 0

19. Sampling Rates and Retention

20. CPT – sampling and duration:
Prefer frequent sampling
– Minutes or seconds – not hours or days
Want a duration that spans significant times in your business
• Calculate consumption by percentile
– Look at the frequency physical consumption exceeds certain values

21. Poll: My CPU core consumption sampling rate is?
My CPU core consumption retention is?

22. Poll:
My customers are comfortable with a virtualization and a shared processor pool

23. CPT Strategy - Percentiles
Size partitions to some percentile of physc – e.g. 80th Percentile
- i.e. partition uses shared pool resources 20% of the time
“A percentile is a measure used in statistics indicating the value below which
a given percentage of observations in a group of observations fall. ”
Are you (and your customers) comfortable with sharing 20% of the time?

24. Failover Capacity

25. Poll: 1. We want failover capacity within entitlement
2. We allow failover capacity to use shared pool (degraded performance)

26. CPT and Clusters How do we handle clusters?
– We don’t want to drastically reduce the size of a standby partition that’s currently inactive
– We want to consider “failover capacity”
Active / Active vs. Active / Standby
– Active / Standby – size standby node to proposed active node size
– Active / Active – needs different approach e.g. size all nodes to
(sum of physical consumption)
(number nodes – 1)

27. Failover and Power Enterprise Pools
A quick plug for Power Enterprise Pools –
now you don’t need to have failover capacity standing by
you can use Power Enterprise Pools to move capacity from primary nodes frame
to standby nodes frame and then DLPAR.

28. CPT Tool: Demo

29. CPT Tool: System Requirements
The CPT tool suite is written using scripts, in Perl and R languages.
The CPT tool suite was developed and has been running well on the Windows™ platform using:
 Strawberry Perl (
 R version 3 for Windows (
In order to have a more “unix-like” experience, the tools are best run in the MobaXterm environment (
Alternatively, Perl and R could be run on Linux (pre-compiled binaries exist for R and Perl), Mac OSX, or AIX (R would have to be compiled on an AIX system)

30. CPT: Demo w/ WLE