1. CS5123 Software Validation and Quality Assurance
Lecture 6
Performance Testing

2. Performance Testing Load Testing Soak Testing Stress Testing
Spike Testing 2

3. Input loads 3 Size of inputs Number of inputs provided
An extremely long SQL query
An extremely large html file for a browser
Number of inputs provided
Number of students supported in a school management system
Number of web pages opened in a browser
Frequency of inputs provided
Number of SQL queries made per second
Number of Http requests made per second 3

4. Performance Measures
Input Lag
Response Time
Throughput 4

5. Performance Measures 5

6. Load Testing
Provide input under the maximal designed load of software and observe behavior
Purpose:
See whether the software works normally
Find potential bottlenecks of performance: Profiling
Instrument each major component (e.g., method) to see how much time / memory is spent on it
Sampling is sometimes used to reduce overhead 6

7. Load Testing 7 Test steps Determine the content of inputs
Usually can be a large amount of identical or similar inputs
The input can be simple or very complex (to check the performance of software when handling complex input)
Determine the frequency of input feeding
Determine how long the input feeding lasts
Design Load
Input Load 7
time

8. Input feeding
A multi thread program to feed inputs randomly in a given period of time
Sometimes require multiple machines to feed inputs
Usually only consider valid inputs 8

9. Stress Testing
Provide input OVER the maximal designed load of software and observe behavior
The software is expected to fail
Purpose:
Observe when (how much load) the software is going to fail
Observe the how the failure looks like: crash? CPU or memory used up? Can be recovered or not?
Observe whether the system can partially work when failure happens 9

10. Stress Testing
Illustration
Design Load
Input Load
time 10

11. Soak Testing
Provide heavy input load (slightly under designed maximal load) for a long time
Purpose:
Testing for how long time the software can work normally under heavy input load
Usually memory and disk oriented
Observe the memory / disk usage trend (abnormal increase in the usage) 11

12. Soak Testing
Illustration
Design Load
Input Load
time 12

13. Spike Testing
Provide extremely heavy input load (OVER designed maximal load) for a very short time
Purpose:
Test how the software can handle a input load burst
Probable expected behavior:
Temporarily refuse inputs that cannot be handled
Provide some temporary services for the inputs to wait until the burst ends 13

14. Spike Testing
Illustration
Design Load
Input Load
time 14

15. Performance Diagnosis
Find out why performance problems happen
Figure out how to optimize software to achieve higher performance 15

16. Profilers 16

17. Memory Profilers 17

18. Performance Testing: Review
Load Testing
Stress Testing
Soak Testing
Spike Testing 18

19. Demo
Usage of JMeter to perform load testing for web application and databases 19

20. Review: Performance Testing
Different Types of Performance Tesing
Load Testing
Soak Testing
Stress Testing
Spike Testing
Profilers
CPU profilers
Memory profilers 20