1. August 01, 2008 Performance Modeling John Meisenbacher, MasterCard Worldwide

2. August 01, 20082 If You Have a Transaction Processing System… I can create a simple accurate CPU performance model… ….that can help you make your system more efficient… …in one week… … with the data you already collect… …and so can you !

3. August 01, 20083 What is a Performance Model? It provides the quantitative information you need to: – Manage the performance of your system; and – Understand the impact of: – software changes; – hardware / configuration changes; and – actual / projected transaction load changes.

4. August 01, 20084 The CPU Performance Model What is the best model? – The best model is only as complex as it needs to be to provide information that is sufficiently accurate to help you make the right decision.

5. August 01, 20085 Performance Management Goals Support capacity planning Identify performance changes with changing hardware, software and transactions. Isolate performance issues within the system. An accurate production model should: – Be easy to use / understand; and – Provide low cost and timely information.

6. August 01, 20086 About Transaction Processing… There are basic features of most transaction systems in the real world. The rate of transactions per second (TPS), varies: – During the day, there are peak hours – During the week, there are peak days – During the year, there are peak months / seasons Transaction processing systems are typically sized for the peak hour of the peak day of the next peak season.

7. August 01, 20087 Daily Transaction Cycles

8. August 01, 20088 So, You’re Asking Yourself… Q. Is my hardware (CPU) properly sized to support my peak transaction period in my next peak season? A. Possible ways to answer this question… – Use a commercial capacity planning tool. – Run a performance test in the lab. – Use a CPU model validated in production.

9. August 01, 20089 Commercial Capacity Planning Tools Commercial capacity planning tools typically focus on: – system size, – acute system issues, and – hardware projections. Potential challenges: – These tools typically do not easily integrate with transaction rates, and therefore cannot model transaction systems. – CCP tools may be good for monitoring and alerts, but unless transaction loads are factored in, this approach cannot give an precise answer.

10. August 01, 200810 Daily CPU Usage Pattern

11. August 01, 200811 Build a Performance Test Environment This approach requires: Building and maintaining a hardware test environment that is as close as possible to production. Loading production software and building a database that is as close as possible to production. Developing a way to send production-like transactions through the system. Maxing out the system to see when it breaks. Hoping your test setup is close enough to production to matter (or calibrating it with a production CPU model).

12. August 01, 200812 A CPU Model of the System A CPU model of the transaction processing systems integrates three key workstreams: 1. Business – the transactions processed; 2. Capacity Planning – the hardware systems used to process the transactions; and 3. Software Engineering – the software systems and databases that process the transactions.

13. August 01, 200813 About Transaction Rates… Most online real world transactions rise and fall during the day, week, month, and year. The CPU method takes advantage of these cycles. Multiple data points are used for correlation analysis. The approach can be used in production, possibly eliminating the need for a performance test environment.

14. August 01, 200814 Transaction Processing and CPU Usage Sample

15. August 01, 200815 Data Reported in Time Increments Report data around the peak period. CPU usage in some time increment, generally between 10 and 20 minutes. (15 minutes is common) – Shorter time periods provide more data and variation, but may not improve accuracy. – Longer time periods require less data and hide variation- reducing accuracy and precision. Transactions are typically logged by time. Multiple days can be easily merged into one dataset.

16. August 01, 200816 Create the Model Display results graphically The model from the previous graph is simply defined: – CPU utilization = CPU per TPS * TPS + overhead – y = a*x + b “System capacity” is the TPS rate that drives the system to your “safe” limit, such as 80% CPU utilization. – TPS Limit = (“Safe” CPU limit – overhead ) / CPU per TPS Most systems are linear, but also can be exponential or logarithmic.

17. August 01, 200817 Graph of Correlation Shapes

18. August 01, 200818 Correlate TPS and CPU

19. August 01, 200819 Transaction Processing Systems The software-intensive system typically includes processes related to transactions: – The actual transaction processing – Data management for look-ups and logging – Communication processing – Format conversion – Also may include other processes, such as system monitoring, that are independent of transaction rates.

20. August 01, 200820 CPU Divided by Type of Processing

21. August 01, 200821 Use the Model to Manage the System Factor the results into your planning – Use the data to help identify any efficiencies for the upcoming peak season The change, in this real-world case, was made to transaction replication in the database. The release was rolled out into product and a new model created.

22. August 01, 200822 System After Application Performance Modification

23. August 01, 200823 Manage Performance within a Release The precision of the model can be used to measure changes in the system. Data from the previous day can be collected and quickly analyzed to determine if changes to the system have improved or degraded performance.

24. August 01, 200824 Variation of System Capacity Monitoring Scripts added And removed Other scripts optimized OK after release OK during peak period

25. August 01, 200825 Work with System Experts Key to success: support the business through cooperation between capacity planning and software experts. Creating the model, generally fairly easy, is only the first step. – Work with developers or vendors to successfully manage and adjust system performance. Support the model’s quantitative information with experience and insight

26. August 01, 200826 Trust Your Experience and Intuition Use the model along with your understanding of hardware and software systems to direct your analysis and decision. Collect more production data to evaluate and refine these decisions with the goals of: – saving money for your company, and – delivering quality results to your customers.

27. Thank you.