1. Lecture 8: Testing, Verification and Validation
Dr Valentina Plekhanova
University of Sunderland, UK

2. Software Testing Fundamentals: Some Testing Terminology
Faults - a mistake in the code that causes the software to not behave as expected (causes).
Failures - the act of a product not behaving as expected - the manifestation of a fault (symptoms).
Lecture 8
Valentina Plekhanova

3. Some Testing Terminology
Validation - establishing the fitness of a software product for its use - "are we building the right product?"
Verification - establishing the correspondence between the software and its specification - "are we building the product right?"
Lecture 8
Valentina Plekhanova

4. Some Testing Terminology
Test case - the collection of inputs, predicted results and execution conditions for a single test.
Pass/fail criteria - decision rules used to determine whether a product passes or fails a given test.
Test suite - a collection of test cases necessary to "adequately" test a product.
Lecture 8
Valentina Plekhanova

5. Some Testing Terminology
Test plan - a document describing the scope, approach, resources and schedule of intended testing activity - identifies features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning.
Lecture 8
Valentina Plekhanova

6. Some Testing Terminology
Bug report - a method of transmitting the occurrence of a discrepancy between actual and expected output to someone who cares for "follow-up" - also known as discrepancy report, defect report, problem report, etc.
Lecture 8
Valentina Plekhanova

7. What is Software Testing?
Process of executing a computer program and comparing the actual behaviour with the expected behaviour.
The comparison is intended to detect deviations (if any) between the actual behaviour and the expected behaviour.
Testing serves as a barrier to allowing low quality products from reaching the customer.
Lecture 8
Valentina Plekhanova

8. Testing Objectives
Testing is a process of executing a program with the intent of finding an error.
A benefit of testing is that it demonstrates that the software appears to be working as stated in the specifications.
Lecture 8
Valentina Plekhanova

9. Testing Objectives
The data collected through testing can also provide an indication of the software's reliability and quality.
But, testing cannot show the absence of defect -- it can only show that software defects are present.
Functional Requirements and Non-functional Requirements must be tested.
Lecture 8
Valentina Plekhanova

10. Software Testing is Not Debugging
Testing establishes the existence of defects
Debugging is concerned with locating and correcting these defects.
Lecture 8
Valentina Plekhanova

11. software testing discovers unexpected behaviour;
defect classification identifies the situation as a software error, a design error, a specification error, a testing error, etc. - if available, a "work-around" is identified;
if a software error, then debugging is performed which tracks down the cause of the error, and attempts to fix it.
Lecture 8
Valentina Plekhanova

12. The Testing Process Unit Testing Sub-System Testing System Testing
Acceptance Testing or Alpha Testing
Beta Testing
Lecture 8
Valentina Plekhanova

13. The Testing Process: Major Stages
Unit Testing: Individual components are tested to ensure that they operate correctly.
 Module Testing: A module is a collection of dependent components such as an objects, procedures, and functions. A module encapsulates related components so can be tested without other system modules.
Lecture 8
Valentina Plekhanova

14. The Testing Process
Sub-System Testing: This phase involves testing collections of modules, which have been integrated into sub-system. The sub-system test process should concentrate on the detection of interface errors by rigorously exercising these interfaces.
Lecture 8
Valentina Plekhanova

15. The Testing Process: System Testing
System Testing: The testing process is concerned with finding errors, which normally result from unanticipated interactions between sub-systems and components. It is also concerned with validating that the system meets its functional and non-functional requirements.
Lecture 8
Valentina Plekhanova

16. The Testing Process: Acceptance Testing
Acceptance Testing: Acceptance testing often reveals errors and omissions in the system requirements definition [Sommerville].
Lecture 8
Valentina Plekhanova

17. The Testing Process
Acceptance testing is sometimes called alpha testing.
When a system is to be marked as a software product, a testing process called beta testing is often used.
Lecture 8
Valentina Plekhanova

18. The Testing Process: Beta Testing
Beta testing involves delivering a system to a number of potential customers who agree to use that system and to report problems to the system developers.
Lecture 8
Valentina Plekhanova

19. The Testing Process: Regression Testing
Errors in program components may be identified at the later stage of the testing process. Repairing program defects may introduce new defects so testing should be repeated after the system is modified. This is sometimes called regression testing [Sommerville].
Lecture 8
Valentina Plekhanova

20. White Box Testing [Sommerville]
White box testing is an approach to testing where the tests are derived from knowledge of the software structure.
Test cases can be derived that
guarantee that all independent paths within a module have been exercised at least once;
exercise all logical decisions on their true and false sides;
execute all loops at their boundaries and within their operational bounds;
exercise internal data structures to ensure their validity.
Lecture 8
Valentina Plekhanova

21. Black Box Testing
Black box testing attempts to derive sets of inputs that will fully exercise all the functional requirements of a system.
It is not an alternative to white box testing.
Lecture 8
Valentina Plekhanova

22. Black Box Testing incorrect or missing functions; interface errors;
errors in data structures or external database access;
performance errors;
initialisation and termination errors.
Lecture 8
Valentina Plekhanova

23. White Box & Black Box Testing
White box testing should be performed early in the testing process, while black box testing tends to be applied during later stages.
Lecture 8
Valentina Plekhanova

24. Tests are designed to answer the following questions:
How is the function's validity tested?
What classes of input will make good test cases?
Is the system particularly sensitive to certain input values?
How are the boundaries of a data class isolated?
What data rates and data volume can the system tolerate?
What effect will specific combinations of data have on system operation?
Lecture 8
Valentina Plekhanova