Eris Performance & Load Testing

Performance & Load Testing Basics Part 1

Performance & Load Testing Basics Introduction to Performance Testing Introduction to Performance Testing Difference between Performance, Load and Stress Testing Difference between Performance, Load and Stress Testing Why Performance Testing? Why Performance Testing? When is it required? When is it required? What should be tested? What should be tested? Performance Testing Process Performance Testing Process Load Test configuration for a web system Load Test configuration for a web system Practice Questions Practice Questions

Introduction to Performance Testing  Performance testing is the process of determining the speed or effectiveness of a computer, network, software program or device.  Before going into the details, we should understand the factors that governs Performance testing: Throughput Throughput Response Time Response Time Tuning Tuning Benchmarking Benchmarking

Throughput  Capability of a product to handle multiple transactions in a give period.  Throughput represents the number of requests/business transactions processed by the product in a specified time duration.  As the number of concurrent users increase, the throughput increases almost linearly with the number of requests. As there is very little congestion within the Application Server system queues.

Response Time  It is equally important to find out how much time each of the transactions took to complete.  Response time is defined as the delay between the point of request and the first response from the product.  The response time increases proportionally to the user load.

Tuning  Tuning is the procedure by which product performance is enhanced by setting different values to the parameters of the product, operating system and other components.  Tuning improves the product performance without having to touch the source code of the product.

Benchmarking  A very well-improved performance of a product makes no business sense if that performance does not match up to the competitive products.  A careful analysis is needed to chalk out the list of transactions to be compared across products so that an apple-apple comparison becomes possible.

Performance Testing- Definition  The testing to evaluate the response time (speed), throughput and utilization of system to execute its required functions in comparison with different versions of the same product or a different competitive product is called Performance Testing.  Performance testing is done to derive benchmark numbers for the system.  Heavy load is not applied to the system  Tuning is performed until the system under test achieves the expected levels of performance.

Difference between Performance, Load and Stress Testing Load Testing  Process of exercising the system under test by feeding it the largest tasks it can operate with.  Constantly increasing the load on the system via automated tools to simulate real time scenario with virtual users. Examples:  Testing a word processor by editing a very large document.  For Web Application load is defined in terms of concurrent users or HTTP connections.

Difference between Performance, Load and Stress Testing Stress Testing  Trying to break the system under test by overwhelming its resources or by taking resources away from it.  Purpose is to make sure that the system fails and recovers gracefully. Example:  Double the baseline number for concurrent users/HTTP connections.  Randomly shut down and restart ports on the network switches/routers that connects servers.

Why Performance Testing  Identifies problems early on before they become costly to resolve.  Reduces development cycles.  Produces better quality, more scalable code.  Prevents revenue and credibility loss due to poor Web site performance.  Enables intelligent planning for future expansion.  To ensure that the system meets performance expectations such as response time, throughput etc. under given levels of load.  Expose bugs that do not surface in cursory testing, such as memory management bugs, memory leaks, buffer overflows, etc.

When is it required? Design Phase: Pages containing lots of images and multimedia for reasonable wait times. Heavy loads are less important than knowing which types of content cause slowdowns. Development Phase: To check results of individual pages and processes, looking for breaking points, unnecessary code and bottlenecks. Deployment Phase: To identify the minimum hardware and software requirements for the application.

What should be tested?  High frequency transactions: The most frequently used transactions have the potential to impact the performance of all of the other transactions if they are not efficient.  Mission Critical transactions: The more important transactions that facilitate the core objectives of the system should be included, as failure under load of these transactions has, by definition, the greatest impact.  Read Transactions: At least one READ ONLY transaction should be included, so that performance of such transactions can be differentiated from other more complex transactions.  Update Transactions: At least one update transaction should be included so that performance of such transactions can be differentiated from other transactions.

Performance Testing Process

 Determine the performance testing objectives  Describe the application to test using a application model 1. Describe the Hardware environment 2. Create a Benchmark (Agenda) to be recorded in Phase 2. A.Define what tasks each user will perform B.Define (or estimate) the percentage of users per task. 1.Planning

Record Record the defined testing activities that will be used as a foundation for your load test scripts. Record the defined testing activities that will be used as a foundation for your load test scripts. One activity per task or multiple activities depending on user task definition One activity per task or multiple activities depending on user task definitionModify Modify load test scripts defined by recorder to reflect more realistic Load test simulations. Modify load test scripts defined by recorder to reflect more realistic Load test simulations. Defining the project, users Defining the project, users Randomize parameters (Data, times, environment) Randomize parameters (Data, times, environment) Randomize user activities that occur during the load test Randomize user activities that occur during the load test 3.Modify 2.Record 1.Planning

 Monitoring the scenario: We monitor scenario execution using the various online runtime monitors.  Monitoring the scenario: We monitor scenario execution using the various online runtime monitors.  Analysing test results: During scenario execution, the tool records the performance of the application under different loads. We use the graphs and reports to analyse the application’s performance. 6. Analyze 5.Monitor

Load Test configuration for a web system

Load Testing Tool Part 2

Test Tool Modules  Recorder  Scenario Generator  Scenario Parameter Editor  Test Case Creator  Runnig Tests  Analyzing Results

Recorder Recorder runs on the project server Recorder runs on the project server It serves the client requests It serves the client requests Starts or stops recording at any time Starts or stops recording at any time Records any user interaction with the server Records any user interaction with the server Resets recording Resets recording Renders screen capture to client application. Renders screen capture to client application. It listens the end-user operations on the application and saves it as a test scenario It listens the end-user operations on the application and saves it as a test scenario

Scenario Generator

Wraps any application Wraps any application The end user actions on the embedded application are listened. The end user actions on the embedded application are listened. Starts,stops and clears recording. Starts,stops and clears recording. Captures all the user events Captures all the user events Saves the operation as a new scenario Saves the operation as a new scenario

Scenario Parameter Editor

Lists all the scenarios which are recorded by the scenario recorder Lists all the scenarios which are recorded by the scenario recorder Displays scenario parameters which are posted by the end user to server during the interaction with the application. Displays scenario parameters which are posted by the end user to server during the interaction with the application. Any parameter value may be modified with the new one, in addition to this, parameter names maybe also edited, considering the transactions at the back end system, they may require different input values in order to prevent constraints, therefore multiple value entry for a single parameter is available, in each execution parameter value in the list is used to perform the transactions successfully. Any parameter value may be modified with the new one, in addition to this, parameter names maybe also edited, considering the transactions at the back end system, they may require different input values in order to prevent constraints, therefore multiple value entry for a single parameter is available, in each execution parameter value in the list is used to perform the transactions successfully. Any modified scenario can be saved as a new scenario Any modified scenario can be saved as a new scenario

Parameter Value Entry

Parameter Name Modification Any parameter name may be modified, considering the functional tests it is important to send values with different parameter names. Any parameter name may be modified, considering the functional tests it is important to send values with different parameter names.

Test Case Creator A test case may be created by the combination of scenarios A test case may be created by the combination of scenarios You can define many scenarios to be executed in a single test case. You can define many scenarios to be executed in a single test case.

Testers Load Generation System Under Test Analysis? All of you, click the GO button again Load Generator Web server Database server Coordinator

Defines the scenario execution strategy Defines the scenario execution strategy A scenario can be executed with n users with n time repetation and n mili seconds sleep time between the executions. A scenario can be executed with n users with n time repetation and n mili seconds sleep time between the executions. Scenarios are defined to work together to form a test case. Scenarios are defined to work together to form a test case. Considering the different scenario execution definition and combination, many test case can be created to be executed. Considering the different scenario execution definition and combination, many test case can be created to be executed.

Running Tests

A test case is selected from the pre defined list. A test case is selected from the pre defined list. Test case may be viewed and modified at any time. Test case may be viewed and modified at any time. When run button clicked, all the users created and executes the test case which contains scenarios. When run button clicked, all the users created and executes the test case which contains scenarios. At any time, the test execution may be stopped or suspended. At any time, the test execution may be stopped or suspended. Test progress is illustrated on the GUI. Test progress is illustrated on the GUI. Regarding the execution status, scenario and users have different color. Regarding the execution status, scenario and users have different color. All the users are created at the same time and performs the scenarios. All the users are created at the same time and performs the scenarios. The execution of the scenarios and user behaviours may be viewed real time. The execution of the scenarios and user behaviours may be viewed real time.

Analyzing Results

Load GenerationSystem Under Test Load Test Controller Web server Database server Vuser host Analysis Controller

While the test is being executed the results are displayed at runtime. While the test is being executed the results are displayed at runtime. The original and virtual scenario response time are displayed on the screen. The original and virtual scenario response time are displayed on the screen. Scenario execution start and end time information is displayed along with average response time, the status field for the scenario changes while the test is in progress, it may be working,suspended,stopped or failed. Scenario execution start and end time information is displayed along with average response time, the status field for the scenario changes while the test is in progress, it may be working,suspended,stopped or failed. Scenario execution times are compared with a graphic at the right corner of the screen, the system gives information about the response time ratios, in addition to this, during the test the change in the ratios may be observed. Scenario execution times are compared with a graphic at the right corner of the screen, the system gives information about the response time ratios, in addition to this, during the test the change in the ratios may be observed. In the second graphic at the right frame, the cut point response times are displayed for each scenario In the second graphic at the right frame, the cut point response times are displayed for each scenario

For each scenario all the user execution detail is displayed. For each scenario all the user execution detail is displayed. For each users,the execution start and end times, average response time and status information is displayed considering the scenario. For each users,the execution start and end times, average response time and status information is displayed considering the scenario. When the scenario icon is clicked, the user details belonging to that scenario is dipayed at the bottom. When the scenario icon is clicked, the user details belonging to that scenario is dipayed at the bottom. At the right frame belowest graphic displayes the user response times for each scenario, the change in the response time for each user can be viewed in this graphic. At the right frame belowest graphic displayes the user response times for each scenario, the change in the response time for each user can be viewed in this graphic. For each user two action button is available, the first one displayes the user interaction steps with the server during the execution of the scenario, the second one compares the execution steps with the original scenario. For each user two action button is available, the first one displayes the user interaction steps with the server during the execution of the scenario, the second one compares the execution steps with the original scenario.

User scenario execution

The user scenario execution is diplayed as a graphic. The user scenario execution is diplayed as a graphic. The graphic displayes the user interaction with the server to perform a single scenario. The graphic displayes the user interaction with the server to perform a single scenario. The right axis displayes the execution steps while the left one prints the execution time for each step in miliseconds. The right axis displayes the execution steps while the left one prints the execution time for each step in miliseconds. The yellow lines display the execution time which passes in the server. The yellow lines display the execution time which passes in the server. The green lines display the execution time which passes in the client side, in other words,it is data rendering time, if the network is slow or the data amount is huge then the green line execution times will be higher accordingly. The green lines display the execution time which passes in the client side, in other words,it is data rendering time, if the network is slow or the data amount is huge then the green line execution times will be higher accordingly. All the graphics can be printed out. All the graphics can be printed out.

The second graphic at the below displayes the original end user scenario execution time considering all the steps. The second graphic at the below displayes the original end user scenario execution time considering all the steps. The execution time for each step is displayed in miliseconds. The execution time for each step is displayed in miliseconds. For both graphics if the value mark points are clicked the data exchange with the server and other details are displayed. For both graphics if the value mark points are clicked the data exchange with the server and other details are displayed. All the graphics can be downloaded with the jpeg,png,svg and pdf file formats. All the graphics can be downloaded with the jpeg,png,svg and pdf file formats. All the graphics can be printed out. All the graphics can be printed out.

Scenario Execution Step Detail

Each user scenario execution consists of many steps,for a single step all the posted and received data is diplayed. Each user scenario execution consists of many steps,for a single step all the posted and received data is diplayed. The step response time, client posted datas like request, form and headers are displayed. The step response time, client posted datas like request, form and headers are displayed. The server response data in html format are displayed at the bottom frame. The server response data in html format are displayed at the bottom frame. Server response time,response type,content length and client data render time may be observed. Server response time,response type,content length and client data render time may be observed. All the steps can be iterated. All the steps can be iterated. In each step, all the data exchange details are displayed. In each step, all the data exchange details are displayed. If there is an error in the response html, the error message is also displayed with an indication. If there is an error in the response html, the error message is also displayed with an indication. Execution step details are available for both original and virtual users. Execution step details are available for both original and virtual users.

Settings Application server url information maybe defined Application server url information maybe defined Considering the server response html, any error structure may be defined to decide whether the execution produced an error or not. Considering the server response html, any error structure may be defined to decide whether the execution produced an error or not. considering the scenario execution status, this may help to decide on the failure cases. considering the scenario execution status, this may help to decide on the failure cases.

Thank You