In software engineering, performance testing is in general testing performed to determine how a system performs in terms of responsiveness and stability under a particular workload.software engineeringtestingsystem
Load testing: A load test is usually conducted to understand the behavior of the system under a specific expected load. Stress testing: is normally used to understand the upper limits of capacity within the system. Endurance testing (soak testing): is usually done to a determine if the system can sustain the continuous expected load. During endurance tests, memory utilization is monitored to detect potential leaks Spike testing: Spike testing is done by suddenly increasing the number of, or load generated by, users by a very large amount and observing the behavior of the system. The goal is to determine whether performance will suffer, the system will fail, or it will be able to handle dramatic changes in load.
Load testing is the process of putting demand on a system or device and measuring its response. Load testing is performed to determine a system’s behavior under both normal and anticipated peak load conditions It helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation
HP LoadRunner is an automated performance and testing product from Hewlett-Packard for examining system behavior and performance, while generating actual load. HP LoadRunner can simulate hundreds or thousands of concurrent users to put the application through the rigors of real-life user loads, while collecting information from key infrastructure components (Web servers, database servers etc.) The results can then be analyzed in detail, to explore the reasons for particular behavior. Example Consider the client-side application for an automated teller machine (ATM).
Web Server DB Server Application Server DB Server Vuser host Controller Automation overcomes Resource Limitation Replace Tester with “Virtual Users” Runs many vusers on few machine Controller manages the vusers Meaningful results with analysis tools Repeat test with scripted actions Monitoring & Analysis Tool System Under Test Yahoo!!! Load Generation
Plan Load Test Create Vuser Scripts Create Scenarios Run Scenarios Analyze Results Tune System Based on Analysis Phase 1Phase 2Phase 3Phase 4Phase 5 LoadRunner Controller & Analysis LoadRunner VUGen
Virtual User: Load Runner emulates the environment in which thousands of users works with a client server system concurrently. Load runner creates or replaces human users with virtual users (Vuser). Virtual Scripts: The actions performed by the human users are recorded in the form of script. The script generated by the load runner is Vuser script. The script when replayed emulates the real user performing the business actions. The scripting language used is C, C++, Java, TSL. Actions: Actions are the set of transactions performed in an application to accomplish business tasks Each Virtual user scripts will always have 3 default actions. 1. Vuser_init (Used for Logon to the application ) 2. Actions ( Used For Business action that needs to be recorded) 3. Vuser_end (Used for Log off from the application )
The Virtual User Generator, also known as VuGen, enables you to develop Vuser scripts for a variety of application types and communication protocols. VuGen not only records the Vuser scripts, but also runs them. To verify that the script runs correctly, you run it in stand- alone mode. When your scripts runs Correctly, you incorporate it into a LoadTR
Record the Script Use for Load Scenario Replay and verify Enhance the script Configure the time settings Replay and verify
VuGen does not record the activities performed by the client on the application. VuGen creates the script by recording the activity between the client and the server. VuGen Client running an application Server
Lets Record a Script 1. Login it to Mercury Tool 2. Book a Flight 3. View Itenary 4. Sign-Out
Why Add LoadRunner Transaction ? Only means measuring performance Help to Identify performance problems Breaks Complex problems into simpler Measure high business risk transactions Allows performance comparison between different load tests What LoadRunner Transactions Measure ? It measure the system performance resulting from one or more user actions Transaction 1 Measure the login actions Transaction 2 Measure everything that happens after the login Transaction 3 Measure the save order action
Playback is to ensure that required business process has been recorded properly Logon (Recording optional) User Actions (Business Processes) This section may be iterated (repeated) during one test run Logoff (Recording Optional) Vuser_Init.c Action 1.c, Action2.c, etc (e.g. create order, ship order) Vuser_end.c
Step 1: Change setting in mercury Tours Step 2: Re-record the script developed earlier in Training Login in mercury Tours Book a Flight Very Itenary Logout
The Script emulating the same business process that was working earlier fails now ??? WHY ????
Many applications use dynamic values that change each time you use the application. Some service assign a unique session ID for every new session. When you try to replay a recorded session the application creates a new session ID that differ from the recorded session ID and the script fails
CLIENTSERVER While Scripting Give Session ID Session ID ABC sent ABC HARDCODED Start New session With ID ABC New Session Started
CLIENTSERVER While Replaying Give Session ID Session ID XYZ sent HARDCODED Session ID ABC Start New session With ID ABC Session Failed. XYZ expected
CLIENTSERVER Solution Give Session ID Session ID ZZZ sent SESSION ID PARSED Start New session With ID ZZZ Session Started.
Parameterize the Session ID Use the parameter value further communication instead of hard-coded value. This is nothing but Correlation !!!
Correlation can be done in 2 ways 1) Manual 2) Automatic
I have recorded and saved two scripts 1. Demo_11 2. Demo_12 Which emulate the same business process 1. Login in mercury Tours 2. Book a Flight 3. Very Itenary 4. Logout
Steps Find the dynamic value to capture Find the server’s response, containing the dynamic value Capture the dynamic value in a parameter Replace every occurrence of dynamic value with parameter Check Changes
In built function used for parsing Syntax Web_reg_save_param(“Parameter Name”, “LB”, “RB”, LB= Left Boundary RB= Right Boundary
Steps Replay the Script Select values to correlate Verify
In our sample script, while booking flight we choose seating preference as Aisle. In real life scenario, user will choose different preferences To make our test as close as possible to a real life scenario we need parameterization
It provide the ability to test your script with different values Reduce the size of script
Steps require for parameterization Replacing the constant value in the Vuser script with parameters Setting the properties and data source for the parameters so different values can selected at run-time
File Type of Parameter Group Name Iteration Number Load Generate d Name Random Number Date/Time Unique Number Vuser ID XML
Group Name replaces the parameter with the name of the Vuser Group. You specify the name of the Vuser Group when you create a scenario. When you run a script from VuGen, the Group name is always None. To set properties for the Group Name parameter type: Select one of the available formats or create a new one. You select a format to specify the length of the parameter string. For details, see Customizing Parameter Formats. Click Close to accept the settings and close the Parameter Properties dialog box.
Iteration Number Iteration Number replaces the parameter with the current iteration number. Load Generator Name Load Generator Name replaces the parameter with the name of the Vuser script's load generator. The load generator is the computer on which the Vuser is running. Random Number Random Number replaces the parameter with a random number. You set a range of numbers by specifying minimum and maximum values. You can use the Random Number parameter type to sample your system's behavior within a possible range of values. For example, to run a query for 50 employees, where employee ID numbers range from 1 through 1000, create 50 Vusers and set the minimum to 1 and maximum to Each Vuser receives a random number, from within the range of 1 to 1000.
Unique Number Unique Number replaces the parameter with a unique number. When you create a Unique Number type parameter, you specify a start number and a block size. The block size indicates the size of the block of numbers assigned to each Vuser. Each Vuser begins at the bottom of its range and increments the parameter value for each iteration. For example, if you set the Start number at 1 with a block of 500, the first Vuser uses the value 1 and the next Vuser uses the value 501, in their first iterations. The number of digits in the unique number string together with the block size determine the number of iterations and Vusers. For example, if you are limited to five digits using a block size of 500, only 100,000 numbers (0-99,999) are available. It is therefore possible to run only 200 Vusers, with each Vuser running 500 iterations. Vuser ID Vuser ID replaces the parameter with the ID number assigned to the Vuser by the Controller during a scenario run. When you run a script from VuGen, the Vuser ID is always -1. Note: This parameter type applies primarily to LoadRunner users.
Many a time while surfing the internet contents are not downloaded and displayed completely. Display of data at client machine is integral part of the business process. To ensure that all business processes have completed end to end, while the server handles concurrent requests, we need content checks. Load runner provide two types of content checks 1. Text check 2. Image check These check point can be added during or after recording a script. Checkpoint added in a script stores the expected value, compares the actual value on the web page to the expected value during play back and report the comparison result status as either PASS or FAIL.
Lets create a text check to verify the text Flight Departing appears on client screen. Flight departing
We use the HP LoadRunner Controller to manage and maintain the scenarios. Using the Controller, we control all the Vusers in a scenario from a single Workstation. Load Generator. When we execute a scenario, the Controller distributes each Vuser in the scenario to a load generator. The load generator is the machine that executes the Vuser script, enabling the Vuser to emulate the actions of a human user.
Using HP LoadRunner, We divide our application performance testing requirements into scenarios. A scenario defines the events that occur during each testing session. Thus, for example, a scenario defines and controls the number of users to emulate, the actions that they perform, and the machines on which they run their emulations
Before scenario execution Create scenario Sets up run-time configuration During Scenario Execution Runs many Vusers simultaneously Control each Vuser (initialize, run, pause, stop) Displays execution status for each Vuser Displays message from each Vuser Monitors server resources After scenario execution Collect and organizes performance data Launches the Analysis tools
The Run tab displays the following default online graphs: 1. Running Vusers - Whole Scenario graph displays the number of Vusers running at a given time. 2. Transaction Response Time - Whole Scenario graph shows the amount of time it takes for each transaction to be completed. 3. Hits per Second - Whole Scenario graph displays the number of hits (HTTP requests) made to the Web server by Vusers during each second of the scenario run. 4. Windows Resources graph displays the Windows resources measured during a scenario.
Throughput Graph The Throughput graph shows the amount of data (measured in bytes) that the Vusers receive from the server at any given second. Elapsed Time: It is the total time taken since the request is sent and the result obtained.
Vuser scripts include functions that measure and record system performance during load-testing sessions. During a scenario run, you can monitor the network and server resources. Following a scenario run, you can view performance analysis data in reports and graphs.
Analysis contains three primary windows: Session Explorer Properties window Graph Viewing Area Graph Legend
What is a rendezvous point ? You insert rendezvous points into Vuser scripts to emulate heavy user load on the server. Rendezvous points instruct Vusers to wait during test execution for multiple Vusers to arrive at a certain point, so that they may simultaneously perform a task. For example, to emulate peak load on the bank server, you can insert a rendezvous point instructing 100 Vusers to deposit cash into their accounts at the same time
How can you add rendezvous points in load runner VUGen? Before the rendezvous point added in the test script. One has to identify the appropriate step, where the rendezvous point needs to be added. Rendezvous points in VUGen can be added in two ways: 1. Add the rendezvous point from Tree View A.Select the step in tree view B.Select the Menu "Insert -> Rendezvous“ C.Give the name of the rendezvous point 2. Add the rendezvous point from the test script a.Select the step in script view b.Select the Menu "Insert -> Rendezvous“ c.Give the name of the rendezvous point
LoadRunner supports multithread environments. The primary advantage of a multithread environment is the ability to run more Vusers per machine. To enable multithreading, select the Run Vuser as a Thread check box. To disable multithreading and run a Vuser as a complete process, select the Run Vuser as a Process check box.
VuGen provides the facility to use multithreading. This enables more Vusers to be run per generator. If the Vuser is run as a process, the same driver program is loaded into memory for each Vuser, thus taking up a large amount of memory. This limits the number of Vusers that can be run on a single generator. If the Vuser is run as a thread, only one instance of the driver program is loaded into memory for the given number of Vusers (say 100). Each thread shares the memory of the parent driver program, thus enabling more Vusers to be run per generator.