Presentation on theme: "Chapter 11: Testing The dynamic verification of the behavior of a program on a finite set of test cases, suitable selected from the usually infinite execution."— Presentation transcript:
Chapter 11: Testing The dynamic verification of the behavior of a program on a finite set of test cases, suitable selected from the usually infinite execution domain, against the expected behavior
Testing Fundamentals Testing is the one step in software engineering process that could be viewed as destructive rather than constructive. A successful test is one that breaks the software. A successful test is one that uncovers an as yet undiscovered defect. Testing can not show the absence of defects, it can only show that software defects are present. For most software exhaustive testing is not possible.
Testing Unit testing White-box Code walkthroughs and inspections Integration testing Bottom-up Top-down Sandwich Big Bang Functional testing (black-box) Performance testing Acceptance testing Installation testing
System Testing Process Functional testing: does the integrated system perform as promised by the requirements specification? Performance testing: are the non- functional requirements met?
Functional Testing Test cases derived from requirements specification document Black box testing Independent testers Test both valid and invalid input and the success of the test is determined by the produced output Equivalence partitioning Boundary values
Equivalence partitioning Reduces the number of test cases to a necessary minimum and select the right test cases to cover all possible scenarios Example: method accepts an int value for month (1..12) as a parameter
Pop quiz How does system testing differ from unit and integration testing? How does unit testing differ from integration testing?
In a method, a variable did not get initialized properly. Which type of testing would most likely expose this defect? A. Unit testing B. Integration testing C. Functional testing D. Performance testing E. Acceptance testing F. Installation testing
A gas pump system is supposed to allow the user to choose whether or not a receipt is printed, but the print function has not been implemented. Which type of testing is most likely to expose this defect? A. Unit testing B. Integration testing C. Functional testing D. Performance testing E. Acceptance testing F. Installation testing
A configuration file used by the reporting subsystem is not placed in the correct directory in the customer's environment. Which type of testing is most likely to expose this defect? A. Unit testing B. Integration testing C. Function testing D. Performance testing E. Acceptance testing F. Installation testing
The customer is unhappy with the number of screens that must be traversed before getting to the parts list screen, a screen accessed frequently when using the system. Which type of testing is most likely to expose this defect? A. Unit testing B. Integration testing C. Function testing D. Performance testing E. Acceptance testing F. Installation testing
Short answer question What is the difference between verification and validation testing?
Test Documentation Test plan: describes system and plan for testing all functions and characteristics Test case specification: details each test and defines criteria for evaluating each feature Test incident report: results of each test Test report summary: lists all failures from the tests that need to be investigated
Test Plan (part of 480 QA plan) Define the subsystems to be tested (and not tested) Describe the process to follow for unit, integration, system, and acceptance testing List the test cases Plan the test schedule
Regression Testing Identifies new faults that may have been introduced as current ones are being corrected Verifies that a new version or release still performs the same functions in the same manner as an older version or release
Quality Assurance concerns Software reliability: operating without failure under given condition for a given time interval Software availability: operating successfully according to specification at a given point in time Software maintainability: for a given condition of use, a maintenance activity can be carried out within stated time interval, procedures and resources
Testing Safety-Critical Systems Recognize that testing cannot remove all faults or risks Assume that every mistake users can make will be made Do not assume that low-probability, high- impact events will not happen Emphasize requirements definition, testing, code and specification reviews, and configuration control Cleanroom testing
Different Levels of Failure Severity Catastrophic: causes death or system loss Critical: causes severe injury or major system damage Marginal: causes minor injury or minor system damage Minor: causes no injury or system damage
One-minute quiz What is meant by regression testing? Is regression testing used for verification or validation?