Black Box Software Testing Fall 2004 PART 13 -- STRESS TESTING by Cem Kaner, J.D., Ph.D. Professor of Software Engineering Florida Institute of Technology and James Bach Principal, Satisfice Inc. Copyright (c) Cem Kaner & James Bach, 2000-2004 This work is licensed under the Creative Commons Attribution-ShareAlike License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA. These notes are partially based on research that was supported by NSF Grant EIA-0113539 ITR/SY+PE: "Improving the Education of Software Testers." Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Stress Testing: Readings Schneier, Cryptogram Whittaker & Jorgenson, How to Break Software

Stress Testing Tag line “Overwhelm the product.” Fundamental question or goal Learn about the capabilities and weaknesses of the product by driving it through failure and beyond. What does failure at extremes tell us about changes needed in the program’s handling of normal cases? Paradigmatic case(s) Buffer overflow bugs High volumes of data, device connections, long transaction chains Low memory conditions, device failures, viruses, other crises. Strengths Expose weaknesses that will arise in the field. Expose security risks. Perhaps good for assessing performance, reliability, or efficiency. Blind spots Weaknesses that are not made more visible by stress. Tandem: db operation, take out a disk drive; one day rocked the system until it fell over and then checked for errors “Earthquakes happen…” [“Guy pulled the wings off software”] Used in security testing Extrapolate from the extreme cases to see if there are lessons to apply to normal cases. Looking for things the programmer missed.

Stress testing Look for functions or sub-systems of the product that may be vulnerable to failure due to challenging input or constrained resources. Identify input or resources related to those functions or sub-systems. Select or generate challenging data and platform configurations to test with: e.g., large or complex data structures, high loads, long test runs, many test cases, limited memory, etc. Force the program to fail, watch how it fails, report the vulnerability