1. Random Testing

2. Partition Testing
All the testing techniques we have described can be viewed as a special case of partition testing.
The partition testing partitions the input domain into a set of (possibly overlapped) subdomains, one or more input values are selected from each subdomain as test cases.

3. Structural Testing is Partition Testing
Control flow testing
Data flow testing

4. Functional Testing is Partition Testing
Domain testing
Finite state machine testing

5. Mutation Testing is Partition Testing
Each subdomain consists of the input values that distinguish the behaviors of a mutant and the program under test.
All input values that cannot distinguish any mutant are collected into a subdomain.

6. Statistical Testing is Partition Testing
Statistical testing is a proportional partition testing.
The number of input values selected from a subdomain is proportional to the usage probability of that subdomain.

7. Random Testing
Random testing is an extreme case of the partition testing.
There is only one subdomain in the random testing.
Input values in the domain are selected using uniform distribution.

8. Random Testing vs. Partition Testing
All testing techniques other than random testing are knowledge-based.
Intuitively, the effectiveness of partition testing should be much better than random testing.
Experimental results show that the effectiveness of random testing is very close to that of partition testing.

9. Analytical Assessment
Consider a program P with domain D of size d.
An input point that produces incorrect outputs is called a failure-causing input.
Assume that P has m failure-causing inputs and d >> m.
Let n be the number of test cases selected.
Let  denote the failure rate, i.e., the probability that a failure-causing input will be selected as a test case.

10. Random Testing
Assume that every element of D is equally likely to be selected as a test case and that  = m / d.
Let Pr denote the probability of finding at least one failure-causing input in n randomly selected test cases.
Then, Pr = 1 – (1 – )n.

11. Partition Testing
Assume that the domain D is divided into k subdomains, D1, D2, …, Dk, of size d1, d2, …, dk, and failure rates 1, 2, …, k, respectively.
Let mi denote the number of failure-causing inputs in Di.
Assume that every element of Di is equally likely to be selected as a test case and that i = mi / di.

12. Partition Testing
Let Pp denote the probability of finding at least one failure-causing input using partition testing with ni test cases selected randomly from each Di.
Then, Pp = 1 – ki=1 (1 – i)ni.

13. Observation 1
Partition testing can be better, worse, or the same as random testing depending on how the partition is performed.

14. Examples
Let d = 100, m = 8, and n = 2. Then Pr = 1 – (1 – 8/100)2 = 0.15.
Let k = 2, d1 = d2 = 50, m1 = m2 = 4, and n1 = n2 = 1. Then Pp = 1 – (1 – 4/50)2 = 0.15 = Pr.
Let k = 2, d1 = 92, d2 = 8, m1 = 0, m2 = 8, and n1 = n2 = 1. Then Pp = 1 – (1 – 0)(1 – 1) = 1 > 0.15 = Pr.
Let k = 2, d1 = 1, d2 = 99, m1 = 0, m2 = 8, and n1 = n2 = 1. Then Pp = 1 – (1 – 0)(1 – 8/99) = 0.08 < 0.15 = Pr.

15. Observations 2 and 3
Pp is maximized if one subdomain contains only failure-causing inputs. Such subdomains are called homogeneous (or revealing) subdomains.
Pp is minimized if n1 = n2 = … = nk = 1, d1 = d2 = … = dk-1 = 1, dk = d – k + 1, m1 = m2 = … = mk-1 = 0, and mk = the number of all failure-causing inputs.

16. Observation 4
If d1 = d2 = … = dk, and n1 = n2 = … = nk, then Pp  Pr .
If, in addition, m1 = m2 = … = mk, then Pp = Pr .
Let k = 2, d1 = d2 = 50, n1 = n2 = 1, m1 = 0, and m2 = 8. Then Pp = 1 – (1 – 0)(1 – 8/50) = 0.16 > 0.15 = Pr.
Let k = 2, d1 = d2 = 50, n1 = n2 = 1, and m1 = m2 = 4. Then Pp = 1 – (1 – 4/50)2 = 0.15 = Pr.

17. Observation 5
Let pi denote the probability that a randomly chosen test case is from subdomain Di.
If d1 = d2 = … = dk, and n1 = n2 = … = nk, but pi  di / d for some 1  i  k, then partition testing can be better, worse, or the same as random testing.

18. An Example
Let k = 2, d1 = d2 = 60, n1 = n2 = 1, m1 = 4, m2 = 0, p1 = 0.9, and p2 = 0.1. Then  = 0.9 (4/60) (0) = 0.06, Pr = 1 – (1 – 0.06)2 = 0.12, Pp = 1 – (1 – 4/60)(1 – 0) = 0.07.

19. Examples
Assume that d = 100, k = 10, d1 = … = d10 = 10, n1 = … = n10 = 1.
Let m = 1. Then Pr = 1 – (0.99)10 = 0.096, Pp (max) = 0.1.
Let m = 2. Then Pr = 1 – (0.98)10 = 0.183, Pp (max) = 0.2.
Let m = 3. Then Pr = 1 – (0.97)10 = 0.263, Pp (max) = 0.3.
Let m = 4. Then Pr = 1 – (0.96)10 = 0.335, Pp (max) = 0.4.

20. Observation 6
If d1 = d2 = … = dk, then partition testing can be better, worse, or the same as random testing.
Assume d = 100, k = 2, d1 = d2 = 50, n = 10, and m = 4. Then Pr = 1 – (1 – 0.04)10 = 0.34.
Assume m1 = 4, m2 = 0, n1 = 1, n2 = 9. Then Pp = 1 – (1 – 4/50) = 0.08.

21. Observation 7 Assume that d1 = … = dk, and m1 = … = mk. Then Pp = Pr.
Assume that 1 = … = k. Then  = i and Pp = Pr.

22. Observation 7
Let D be partitioned into k subdomains and assume that n1 = … = nk = c test cases are selected from each subdomain. Then partition testing can be better, worse, or the same as random testing.
Let k = 2, d1 = 1, d2 = 99, m1 = 0, m2 = 8, and n1 = n2 = 1. Then Pp = 1 – (1 – 0)(1 – 8/99) = 0.08 < 0.15 = Pr.

23. Empirical Assessment: Failure Rate
Class failure rate 
Count
Pp (max)
d = K  = n .

24. Empirical Assessment: High Failure Rate
Probability of
high failure rate Tp Tr Ratio
25 classes.
high failure rate: 98%~100%, low failure rate: 0%~4.9%.

25. Empirical Assessment: Lower Bound of High Failure Rate
Lower Bound Tp Tr Ratio
25 classes.
high failure rate(2%).
low failure rate(98%): 0%~4.9%.

26. Empirical Assessment: Upper Bound of Low Failure Rate
Upper Bound Tp Tr Ratio
25 classes.
high failure rate(2%): 98%~100%.
low failure rate(98%).

27. Empirical Assessment: Bounds of High Failure Rate and Low Failure Rate
Failure rate bounds
High Low Tp Tr Ratio
25 classes.
high failure rate(2%).
low failure rate(98%).

28. Empirical Assessment: 14 Programs
Strategy Mean No. of Test Cases Bugs Found (%)
Random Testing
Branch Testing
All-use Testing

29. Empirical Assessment: 7 Programs
Strategy Mean No. of Test Cases Bugs Found (%)
Random Testing
Branch Testing
All-use Testing