1. © 2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice Software Performance Engineering Oldrich Kepka, PCoE SOA Performance Architect Created: Mar 2008

2. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 2 Agenda About PCoE Group Introduction to SPE SPE applied to Software Development Lifecycle −Performance Risk Assessment and Performance Objectives −Architecture, Design Guidelines and Code Review −Performance Modeling −Performance Testing −Deployment Guidelines Q&A

3. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 3 Performance Center of Excellence HP Group −responsible for implementing full SPE lifecycle. −responsible for knowledge about performance collecting Currently 5 teams −AD −AM −PPM −SOA −Opsware −?

4. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 4 Introduction Key Components −Performance Team −Development −Product −SPE Methodology −Tools −Knowledge Key Objectives −Increase business revenue by ensuring the system can process transactions within the requisite timeframe. −Eliminate late system deployment due to performance issues −Eliminate avoidable system rework due to performance issues −Avoid additional and unnecessary hardware acquisition costs −Reduce increased software maintenance costs due to performance problems in production Proactive instead of Reactive

5. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 5 Software Performance Engineering Building Blocks, Roles & Life Cycle Engineering for Performance Performance Objectives (Response Time, Throughput, Resource Utilization, Workload) Measuring, Testing, Tuning Measuring: Response Time, Throughput, Resource Utilization, Workload Testing: Load Testing, Stress Testing, Capacity Testing Tuning: Network, System, Platform, Application Roles Managers Performance Engineers Architects Developers Testers Administrators Life Cycle Requirements Design Develop Test Deploy Maintain Performance Modeling (Scenarios, Objectives, Workloads, Requirements, Budgets, Metrics) Architecture, Design Guidelines & Code Review (Principles, Practices, Patterns & Anti Patterns, Coupling & Cohesion, Communication, Concurrency, Resource Mgmt, Caching, State Mgmt, Data Structures, Algorithms) Performance Risks Assessment (Project wise, Technical wise) Deployment Guidelines (Benchmarking, Capacity Planning, Configuration Tuning)

6. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 6 Software Performance Engineering Overlay on Software Life Cycle Software Development Life Cycle Requirements Design DevelopTest Deploy & Maintain Performance Risks & Objectives Performance Modeling Architecture, Design Guidelines & Code Review Measuring, Testing, Tuning Deployment Guidelines

7. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 7 SDLC Phase: Requirements, Design Goals −Identify high level risks during requirements/design phases, mitigate or manage them −Start discussion on topic of performance early −Reduce problem space Mostly subjective evaluation in these categories: −Business Criticality −Execution Frequency −New Architecture elements or frameworks −Scheduling or Batch Nature Risk Assessment Ranking −Low −Medium −High Performance Risks & Objectives Performance Risks Analysis

8. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 8 HP SOA Systinet Example Performance Risks & Objectives Performance Risks Analysis cont. RequirementDescriptionStatusRiskDetail F01New Repository APIImplementingHighlink F02Single EAR Depl.DoneLowlink F03J2EE Based SecurityDoneLowlink F04, F102Atom-based RESTImplementingMediumlink

9. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 9 SDLC Phase: Requirements Pre-requisites −Performance Risks Analysis −Performance Requirements Goals −Define measurable performance objectives that are aligned with customer needs and agreed upon all stakeholders Two approaches −Absolute goals Analyze customer environments (Workload Characterization)Workload Characterization Determine the right workload for the testing environment. −Workload consists of: number of users, concurrent active users, data volumes, key scenarios and transaction volumes Define objectives for: Response Time, Throughput, Resource Utilization Suitable for new projects −Relative goals (based on historical data) Group existing operations to sets: −Based on response time : fast, medium, slow, asynchronous −Based on resource utilization Then assign each operation desired group The main advantage is that we need not deal with target environment, which is sometimes impossible. Performance Risks & Objectives Performance Objectives

10. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 10 Absolute objectives example from BAC 7.5 (just in terms of TRT) Note: These numbers usually expressed in 90% quantile. Relative objectives example from HP SOA Systinet 2 (just in terms of TRT) −Old operations −New or updated operations Software Performance Engineering Performance Objectives (cont.) ModuleBusiness ProcessOperationOK [s]Minor [s]Critical [s] MyBACBrows_My_BACSelect_Page204060 Maximize_Portlet61630 Maximize_Portlet_Back61630 ModuleBusiness ProcessOperationTRT AllLogin_LogoutLoginMedium LogutMedium Policy ManagerBrowse_BPClick_SearchSlow ModuleBusiness ProcessOperationTRT AllFultext_searchSimple_Fultext_SearchSlow

11. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 11 SDLC Phase: Design Pre-requisites −Performance Objectives −Performance Risk Analysis Goals −Ensure application key design choices takes performance principles under consideration Practices −All System Architects, Software Designers and Performance Engineers participate −System Architects and Software Designers takes performance into account when creating designs −Performance Engineer reviews their work and participate on architecture meeting −Prototyping Architecture, Design Guidelines & Code Review Performance Design Guidelines

12. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 12 SDLC Phase: Develop Pre-requisites −Performance Objectives −Performance Risk Analysis Goals −Ensure that code is written so that it uses resources effectively. Practices −Usually done when some sensible unit of code is finished. −Done by Performance Engineer in co-operation with Software Designer. −Output – set of defects Architecture, Design Guidelines & Code Review Code Review

13. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 13 SDLC Phase: Design, Develop Goals −Educate development about performance aspects of technologies used. −Force development to think about things important for performance. −Learn development from their own mistakes. −Get feedback form development. Practices −Bi-weekly meetings Frameworks and libraries used Code Review, Testing etc. results Discussion Architecture, Design Guidelines & Code Review Development Education

14. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 14 SDLC Phase: Design, Develop, Test Pre-requisites −Performance Objectives, Architecture Design, Performance Risk Analysis, Workload Analysis Goals −Characterize and understand the system’s behavior so that effective decisions may be made. −Orchestrate design, develop and test phases to ensure that performance goals are met. Process 1.Identify key scenarios 2.Identify workloads 3.Identify budget Budget are your constraints – execution time, resource utilization For example peak CPU utilization 75% and memory consumption 4GB 4.Identify performance objectives 5.Identify processing steps Itemize your scenarios 6.Allocate budget Assign execution times to steps −If you don’t know how, divide it equally. Later on you will correct your assignment. Assign resource utilization 7.Evaluate – modify your design, reevaluate requirements, change budged allocation 8.Validate – prototypes and performance measuring 9.Repeat 4. 5. 6. until goals are met. Performance Modeling

15. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 15 SDLC Phase: Develop, Test Goals −Identify how well application performs in relation to performance objectives Key Types of Performance Testing −Performance Test Load Test −Behavior under normal and peak load. −Endurance Test (Longevity Test) – MTBF and MTTF Stress Test −Behavior when pushed beyond normal or peak load −Synchronization issues, race conditions and memory leaks −Spike Test – behavior under repeated short peaks Capacity Test −To plan future system growth. −To identify scaling strategy. Additional Concepts −Unit Test −Component Test −Smoke Test Measuring, Testing & Tuning Performance Testing

16. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 16 SDLC Phase: Test, Deploy, Maintain Goals −Identify how the application performance stands in relation to defined performance goals −Helps to identify the bottlenecks that affect application performance Practices −Measure System and Platform metrics −Measure Network −Profiling −Analyze log files −Instrument the application Measuring, Testing & Tuning Performance Measuring

17. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 17 SDLC Phase: Develop, Test, Deploy, Maintain Goals −identification and systematic elimination of bottlenecks to improve performance Inputs −Good product and environment knowledge. Categories for Tuning −Application. Includes the application configuration parameters. −Platform. Includes the host operating system, together with Web and database servers −System. Includes the hardware resources on a server. These resources include CPU, memory, disk I/O, and network I/O. Process 1.Run test and measure relevant metrics 2.Result analysis (JVM behavior, thread pools, connection pools, DB wait events etc.) 3.Propose parameter modification based on results 4.Modify test environment accordingly 5.Run test again 6.Compare results (iterate) Measuring, Testing & Tuning Performance Tuning

18. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 18 SDLC Phase: Test, Deploy Goals −Assess the relative performance of an application in particular environment. −Equip field personnel with valuable information to facilitate product sales Details −Base for customer environment validation. −Characterize HW requirements −Proving capabilities in comparing to competitors −Promoting sales PCoE teams provide benchmarking kits for their customers. Deployment Guidelines Performance Benchmarking

19. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 19 SDLC Phase: Test, Deploy, Maintain Goals −Planning for growth and forecasting peak usage periods in order to meet system and application capacity requirements Practices −Transaction cost analysis Compile a user profile Execute discrete tests Measure the cost of each operation Calculate the cost of an average user profile Calculate site capacity Verify site capacity −Predictive analysis Collect performance data Query the existing historical data Analyze the historical performance data Predict the future requirements Deployment Guidelines Capacity Planning

20. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 20 HP LoadRunner HP Analysis HP Diagnostics YourKit Idea 7.0 Oracle AWR reports Oracle Remote Diagnostic Agent Our own tools Software Performance Engineering Tools

21. PCoE Site - http://rndportal.mercury.co.il/sites/PCoE/default.aspxhttp://rndportal.mercury.co.il/sites/PCoE/default.aspx 21 Software Performance Engineering Q&A