3 Software performance testing In software engineering, performance testing is in general testing performed to determine how a system performs in terms of responsiveness and stability under a particular workload.
4 Performance testing types Load testing: A load test is usually conducted to understand the behavior of the system under a specific expected load.Stress testing: is normally used to understand the upper limits of capacity within the system.Endurance testing (soak testing): is usually done to a determine if the system can sustain the continuous expected load. During endurance tests, memory utilization is monitored to detect potential leaksSpike testing: Spike testing is done by suddenly increasing the number of, or load generated by, users by a very large amount and observing the behavior of the system. The goal is to determine whether performance will suffer, the system will fail, or it will be able to handle dramatic changes in load.
5 Load testingLoad testing is the process of putting demand on a system or device and measuring its response.Load testing is performed to determine a system’s behavior under both normal and anticipated peak load conditionsIt helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation
6 Introduction to LoadRunner HP LoadRunner is an automated performance and testing product from Hewlett-Packard for examining system behavior and performance, while generating actual load.HP LoadRunner can simulate hundreds or thousands of concurrent users to put the application through the rigors of real-life user loads, while collecting information from key infrastructure components (Web servers, database servers etc.)The results can then be analyzed in detail, to explore the reasons for particular behavior.ExampleConsider the client-side application for an automated teller machine (ATM).
7 LoadRunner Architecture Automation overcomes Resource LimitationReplace Tester with “Virtual Users”Runs many vusers on few machineController manages the vusersMeaningful results with analysis toolsRepeat test with scripted actionsMonitoring & Analysis ToolControllerWeb ServerDB ServerVuser hostYahoo!!!Application ServerDB ServerSystem Under TestLoad Generation
9 Tune System Based on Analysis LoadRunner Basic FlowPlan Load TestCreate Vuser ScriptsCreate ScenariosRun ScenariosAnalyze ResultsPhase 1Phase 2Phase 3Phase 4Phase 5Tune System Based on AnalysisLoadRunner VUGenLoadRunner Controller & Analysis
10 Load Runner- Definitions Virtual User:Load Runner emulates the environment in which thousands of users works with a client server system concurrently. Load runner creates or replaces human users with virtual users (Vuser).Virtual Scripts:The actions performed by the human users are recorded in the form of script. The script generated by the load runner is Vuser script. The script when replayed emulates the real user performing the business actions. The scripting language used is C, C++, Java, TSL.Actions:Actions are the set of transactions performed in an application to accomplish business tasksEach Virtual user scripts will always have 3 default actions.Vuser_init (Used for Logon to the application )Actions ( Used For Business action that needs to be recorded)Vuser_end (Used for Log off from the application )
11 Introducing VUGenThe Virtual User Generator, also known as VuGen, enables you to develop Vuser scripts for a variety of application types and communication protocols. VuGen not only records the Vuser scripts, but also runs them. To verify that the script runs correctly, you run it in stand- alone mode. When your scripts runs Correctly, you incorporate it into a LoadTR
12 Script Development process Record the ScriptReplay and verifyEnhance the scriptReplay and verifyConfigure the time settingsUse for Load Scenario
13 How recording is done in LR? VuGenClient running an applicationServerVuGen does not record the activities performed by the client on the application. VuGen creates the script by recording the activity between the client and the server.
14 VuGen Hands On Lets Record a Script Login it to Mercury Tool Book a FlightView ItenarySign-Out
19 Transaction Why Add LoadRunner Transaction ? Only means measuring performanceHelp to Identify performance problemsBreaks Complex problems into simplerMeasure high business risk transactionsAllows performance comparison between different load testsWhat LoadRunner Transactions Measure ?It measure the system performance resulting from one or more user actionsTransaction 1Measure the login actionsTransaction 2Measure everything that happens after the loginTransaction 3Measure the save order action
20 Virtual User Playback Model Playback is to ensure that required business process has been recorded properlyLogon(Recording optional)User Actions(Business Processes)This section may be iterated (repeated) during one test runLogoff(Recording Optional)Vuser_Init.cAction 1.c, Action2.c, etc(e.g. create order, ship order)Vuser_end.c
22 Common Play back Problem Step 1: Change setting in mercury ToursStep 2: Re-record the script developed earlier in TrainingLogin in mercury ToursBook a FlightVery ItenaryLogout
23 SCRIPT FAILS!!!!The Script emulating the same business process that was working earlier fails now ???WHY ????
24 Reason For Script Failure Many applications use dynamic values that change each time you use the application.Some service assign a unique session ID for every new session.When you try to replay a recorded session the application creates a new session ID that differ from the recorded session ID and the script fails
25 Start New session With ID ABC Client-Server CommunicationWhile ScriptingABC HARDCODEDGive Session IDCLIENTSERVERSession ID ABC sentStart New session With ID ABCNew Session Started
26 Client-Server Communication While ReplayingHARDCODED Session ID ABCGive Session IDCLIENTSERVERSession ID XYZ sentStart New session With ID ABCSession Failed. XYZ expected
27 Client-Server Communication SolutionSESSION ID PARSEDGive Session IDCLIENTSERVERSession ID ZZZ sentStart New session With ID ZZZSession Started.
28 Solution Parameterize the Session ID Use the parameter value further communication instead of hard-coded value.This is nothing but Correlation !!!
29 Types of Correlation Techniques Correlation can be done in 2 waysManualAutomatic
31 Vuser Comparision I have recorded and saved two scripts Demo_11 Which emulate the same business processLogin in mercury ToursBook a FlightVery ItenaryLogout
32 Vuser comparison Steps Find the dynamic value to captureFind the server’s response, containing the dynamic valueCapture the dynamic value in a parameterReplace every occurrence of dynamic value with parameterCheck Changes
33 Web_reg_save_param In built function used for parsing Syntax Web_reg_save_param(“Parameter Name”, “LB”, “RB”,LB= Left BoundaryRB= Right Boundary
34 Automatic Correlation StepsReplay the ScriptSelect values to correlateVerify
36 Need for Parameterization In our sample script, while booking flight we choose seating preference as Aisle.In real life scenario , user will choose different preferencesTo make our test as close as possible to a real life scenario we need parameterization
37 Parameterization helps in It provide the ability to test your script with different valuesReduce the size of script
38 Steps in parameterization Steps require for parameterizationReplacing the constant value in the Vuser script with parametersSetting the properties and data source for the parameters so different values can selected at run-time
39 VuGen Parameters Vuser ID XML Unique Number Date/Time Random Number Type of ParameterFileLoad Generated NameGroup NameIteration Number
41 Group NameGroup Name replaces the parameter with the name of the Vuser Group. You specify the name of the Vuser Group when you create a scenario. When you run a script from VuGen, the Group name is always None.To set properties for the Group Name parameter type:Select one of the available formats or create a new one. You select a format to specify the length of the parameter string. For details, see Customizing Parameter Formats.Click Close to accept the settings and close the Parameter Properties dialog box.
42 VuGen Parameters cont..Iteration Number Iteration Number replaces the parameter with the current iteration number. Load Generator Name Load Generator Name replaces the parameter with the name of the Vuser script's load generator. The load generator is the computer on which the Vuser is running. Random Number Random Number replaces the parameter with a random number. You set a range of numbers by specifying minimum and maximum values. You can use the Random Number parameter type to sample your system's behavior within a possible range of values. For example, to run a query for 50 employees, where employee ID numbers range from 1 through 1000, create 50 Vusers and set the minimum to 1 and maximum to Each Vuser receives a random number, from within the range of 1 to 1000.
43 VuGen Parameters cont.. Unique Number Unique Number replaces the parameter with a unique number.When you create a Unique Number type parameter, you specify a start number and a block size. The block size indicates the size of the block of numbers assigned to each Vuser. Each Vuser begins at the bottom of its range and increments the parameter value for each iteration. For example, if you set the Start number at 1 with a block of 500, the first Vuser uses the value 1 and the next Vuser uses the value 501, in their first iterations.The number of digits in the unique number string together with the block size determine the number of iterations and Vusers. For example, if you are limited to five digits using a block size of 500, only 100,000 numbers (0-99,999) are available. It is therefore possible to run only 200 Vusers, with each Vuser running 500 iterations.Vuser IDVuser ID replaces the parameter with the ID number assigned to the Vuser by the Controller during a scenario run. When you run a script from VuGen, the Vuser ID is always -1.Note: This parameter type applies primarily to LoadRunner users.
45 Verification Checkpoints Many a time while surfing the internet contents are not downloaded and displayed completely.Display of data at client machine is integral part of the business process.To ensure that all business processes have completed end to end, while the server handles concurrent requests, we need content checks.Load runner provide two types of content checksText checkImage checkThese check point can be added during or after recording a script.Checkpoint added in a script stores the expected value, compares the actual value on the web page to the expected value during play back and report the comparison result status as either PASS or FAIL.
46 Text CheckLets create a text check to verify the text Flight Departing appears on client screen.Flight departing
47 Controller Load Generator. We use the HP LoadRunner Controller to manage and maintain the scenarios.Using the Controller, we control all the Vusers in a scenario from a single Workstation.Load Generator.When we execute a scenario, the Controller distributes each Vuser in the scenario to a load generator.The load generator is the machine that executes the Vuser script, enabling the Vuser to emulate the actions of a human user.
48 ScenarioUsing HP LoadRunner, We divide our application performance testing requirements into scenarios. A scenario defines the events that occur during each testing session. Thus, for example, a scenario defines and controls the number of users to emulate, the actions that they perform, and the machines on which they run their emulations
49 What Controller does ? Before scenario execution Create scenarioSets up run-time configurationDuring Scenario ExecutionRuns many Vusers simultaneouslyControl each Vuser (initialize, run, pause, stop)Displays execution status for each VuserDisplays message from each VuserMonitors server resourcesAfter scenario executionCollect and organizes performance dataLaunches the Analysis tools
52 Performance graphsThe Run tab displays the following default online graphs:Running Vusers - Whole Scenario graph displays the number of Vusers running at a given time.Transaction Response Time - Whole Scenario graph shows the amount of time it takes for each transaction to be completed.Hits per Second - Whole Scenario graph displays the number of hits (HTTP requests) made to the Web server by Vusers during each second of the scenario run.Windows Resources graph displays the Windows resources measured during a scenario.
53 Other InformationThroughput Graph The Throughput graph shows the amount of data (measured in bytes) that the Vusers receive from the server at any given second. Elapsed Time: It is the total time taken since the request is sent and the result obtained.
54 Performance analysisVuser scripts include functions that measure and record system performance during load-testing sessions.During a scenario run, you can monitor the network and server resources.Following a scenario run, you can view performance analysis data in reports and graphs.
58 Rendezvous Point What is a rendezvous point ? You insert rendezvous points into Vuser scripts to emulate heavy user load on the server.Rendezvous points instruct Vusers to wait during test execution for multiple Vusers to arrive at a certain point, so that they may simultaneously perform a task.For example, to emulate peak load on the bank server, you can insert a rendezvous point instructing 100 Vusers to deposit cash into their accounts at the same time
59 How to add rendezvous points? How can you add rendezvous points in load runner VUGen?Before the rendezvous point added in the test script. One has to identify the appropriate step, where the rendezvous point needs to be added.Rendezvous points in VUGen can be added in two ways:Add the rendezvous point from Tree ViewSelect the step in tree viewSelect the Menu "Insert -> Rendezvous“Give the name of the rendezvous pointAdd the rendezvous point from the test scriptSelect the step in script view
60 Multithreading LoadRunner supports multithread environments. The primary advantage of a multithread environment is the ability to runmore Vusers per machine.To enable multithreading, select the Run Vuser as a Thread check box.To disable multithreading and run a Vuser as a complete process, select the Run Vuser as a Process check box.
61 Vuser as thread or Process VuGen provides the facility to use multithreading. This enables more Vusers to be run per generator.If the Vuser is run as a process, the same driver program is loaded into memory for each Vuser, thus taking up a large amount of memory. This limits the number of Vusers that can be run on a single generator.If the Vuser is run as a thread, only one instance of the driver program is loaded into memory for the given number of Vusers (say 100). Each thread shares the memory of the parent driver program, thus enabling more Vusers to be run per generator.