1. Testing in the Small (aka Unit Testing, Class Testing)

2. Two fundamental approaches: Black box –Based on specification –Inner structure of test object is not considered White box –Based on specification –Inner structure of test object is the basis of test case selection

3. 3 Essential BB Strategies Use Case Testing Equivalence Class Testing Boundary Value Analysis

4. Use Case Testing Use the use cases to test the system

5. Equivalence Relations: Reflexive Symmetric Transitive

6. Equivalence classes: Basic Strategy Partition the input into equivalence classes –This is the tricky part Test a subset from each class

7. Basic strategy: Example:fact(n): if n =200 error if 0<=n<=20, exact value if 20<n<200, approximate value within.1% What classes can you see?

8. Simple Example: Suppose you are building an airline reservation system. A traveler can be a child, an adult, or a senior. The price depends on the type of traveler. The seat reservation does not depend on the type of traveler. How many test cases are required for the reservation component and the billing component?

9. Finding equivalence classes: Identify restrictions for inputs and outputs in the specification If there is a continuous numerical domain, create one valid and two or three invalid classes (above, below, and NaN) If a number of values is required, create one valid and two invalid classes If a set of values is specified where each may be treated differently, create a class for each element of the set and one more for elements outside the set If there is a condition, create two classes, one satisfying and one not satisfying the condition Organize them in a table

10. Boundary Values: Programs that fail at interior elements of a class usually fail at the boundaries too. (This says that not all elements of the “equivalence class” are equivalent.)

11. Hints If a domain is a restricted set, check the boundaries. e.g., D=[1,10], test 0, 1, 10, 11 –It may be possible to test the boundaries of outputs, also. For ordered sets, check the first and last elements For complex data structures, the empty list, full lists, the zero array, and the null pointer should be tested Extremely large data sets should be tested Check for off-by-one errors

12. Other BB Strategies Control Flow Testing Data Flow Testing Transaction Flow Testing Domain Testing Syntax Testing Finite State Testing

13. White box testing

14. Definitions 1 Test set T –Finite set of test cases –T is successful if every test in T produces the specified result Test selection criterion C –A set of test sets. –We specify some condition that each of the test sets in C must satisfy, such as “has at least one positive and one negative number” –We say test set T satisfies C if T  C C is consistent if –for any two test sets satisfying C, T 1  C and T 2  C, T 1 is successful if and only if T 2 is –All the test sets satisfying a criterion expose the same errors.

15. Definitions 2 Criterion C is complete if –whenever P is incorrect, some T  C exposes the error –  T  C | (  t=(r,d)  T | S (r,d)   (P(r, d))) –Note that not every T need expose the error C 1 is finer than C 2 if –  T 1  C 1  T 2  C 2  T 2  T 1 –For every test set satisfying C 1 there is a smaller (subset) test set satisfying C 2 –C 1 requires more test cases

16. Exercise C 1 is more reliable than C 2 if whenever P is incorrect, there is no test set T 2 in C 2 that causes P to fail but some T 1 in C 1 does not. Prove that finer does not imply more reliable.

17. The problem with testing There’s no way to know in general if a test criterion is consistent.

18. Criteria Statement coverage Edge coverage Condition coverage Multiple-condition coverage

19. Example 2 A B D E C A is read(x, y, z) B is if ((x>10) && (y<20)) D is if (z >10) E is repeat until g = 5 Show me different test sets that satisfy the 4 criteria

20. Usefulness Effectiveness of black box is similar to white box, but the mistakes found are different. (Hetzel 1976, Myers 1978) Examples of errors not detected by white box testing: –Lock up PC if keyboard used during disk IO (neglected to turn off interrupts in pre ’84 versions of DOS) –Monitor destroyed if both color and monochrome monitors used to play PC games