SOFTWARE TESTING AND QUALITY ASSURANCE

Software Testing

What is SW Testing?  Sommerville (Sommerville, 2004) defines testing as: “Software testing involves running an implementation of the software with test data. You examine the outputs of the software and its operational behavior to check that it is performing as required. Testing is a dynamic technique of verification and validation”

What Are We Testing? I. System Testing II. Unit Testing

5 I. System Testing  System Functional Test  Test entire system against the functional requirements.  System Performance Test  Test the non-functional requirements of the system. For example, Response times, load testing etc.  System Acceptance Test  Set of tests that the software must pass before it is accepted by the client.

6 Trivial Example - System Testing  Test the login mechanism using correct and incorrect login.  Test the search facility using queries against known source to check that the search mechanism is actually documents.  Test system presentation facility to check that information about documents is displayed properly.  Test the mechanism to request permission for downloading.  Test the response indicating that the download document is available.

7 II. Unit Testing module to be tested test cases results softwareengineer

Test Cases  Test case consists of……  Set of test inputs, of if the program is non-terminating, a sequence of test inputs.  Expected results when the inputs are executed; and  Execution conditions or execution environment in which the inputs are to be executed. These steps generally remain same from unit testing to system testing.

9 Test Case Selection  Coverage criterion;  Equivalence Partitioning  Boundary-Value Analysis  Coverage-Based Testing Control-flow Data-flow  Expected behavior of every test input to be generated. (Test Oracles)  Testing environment.

10 Test Oracles  Determines whether or not the program has passed or failed the test case.  A test oracle is  A program  A process  A body of data In many cases - directly form the requirements. For example, a test case assessing performance - performance threshold. Difficult to automate or to assess their quality

11 Test Execution  Test inputs on the ‘program-under-test’  Record the actual behavior. Generally can be automated to an extend !!!!

12 Test Evaluation  Compare the actual behavior with the expected behavior. Generally can be automated to an extend !!!!

13 Test Reporting  Report the outcome of the testing.  Developers  Project Mangers etc. Generally can be automated to an extend !!!!

Types of Testing Testing Black-Box Testing White-Box Testing

15 Black-Box Testing requirements events input output

16 Black-Box Testing  Test cases are derived from formal specification of the system.  Test case selection can be done without any reference to the program design or code.  Only tests the functionality and features of the program.  Not the internal operation.

17 Black-Box Testing  Advantages  Test case selection is done before the implementation of a program.  Help in getting the design and coding correct with respect to the specification.

18 White-Box Testing

19 White-Box Testing  Test cases are derived from the internal design specification or actual code for the program.  Advantages  Tests the internal details of the code;  Checks all paths that a program can execute.  Limitations  Wait until after designing and coding the program under test in order to select test cases.

20 Key Points  A system typically undergoes a range of testing types.  Each type of testing is aimed at detecting different kinds of failures.  Testing boils down to the selection and execution of test cases.  Black-box and white-box testing are complementary approaches.

Quality Assurance (QA)

What is QA?  Software quality is a complex and subjective mixture of several attributes or factors and there is no universal definition or a unique metric to quantify software quality  Software quality is measured by analyzing the various attributes that are significant to a certain domain or application

QA Goal  The goal of quality assurance is to improve software quality and to determine the degree to which the actual behavior of the software is consistent with the intended behavior or quality of this software.

Quality Factors and Criteria Table 17.1: McCall’s quality factors [10].

When to Start QA