1. 1841f06detprob3 Testing Basics Detection probability

2. 2841f06detprob3 Methodology to evaluate testing criterion u Pick sample program, sample faults, sample operational profile, two criteria. u For each criterion, randomly generate test suites that satisfy the criterion. u Calculate the percentage of the suites that detect each of the faults. u The criterion that has the higher percentage is better.

3. 3841f06detprob3 Triangle Example cin >> a >> b >> c ; type = “scalene”; if (a == b || a == c || b == c) type= “isosceles”; if (a == b && a == c) type = “equilateral”; if (a >= b+c || b >= a+c || c >= a+b) type=“not a triangle”; if (a <= 0 || b <= 0 || c <= 0) type=“bad input”; cout<< type;

4. 4841f06detprob3 Control Flow Graph Operational profile 3,3,3abcdegiequi 3,3,4abcegiisos 3,3,5abcegiisos 3,3,6abcefginot 3,4,3abcegiisos 3,4,4abcegiisos 3,4,5acegiscal 3,4,6acegiscal All inputs are equally likely

5. 5841f06detprob3 What are the failure probability for each color (separately)? cin >> a >> b >> c ; type = “scalene”; if (a == b || a == c && b == c) type= “isosceles”; if (a == b || a == c) type = “equilateral”; if (a >= b+c || b >= a+c || c > a+b) type=“not a triangle”; if (a <= 0 || b <= 0 || c <= 0) type=“bad input”; cout<< type; Blue GreenRed

6. 6841f06detprob3 TTYP – probability of detection u What is the probability of detection with one randomly chosen test case? u What is the probability of detection with two randomly chosen test cases?

8. 8841f06detprob3 TTYP – per path u What is the probability of detection with one randomly chosen test case per path? u What is the probability of detection with an equal number of randomly chosen test cases?

9. 9841f06detprob3 TTYP – smaller subdomains u What might be better smaller subdomains?

10. 10841f06detprob3 TTYP - subdomains u Are paths the best subdomains? u Would a functional decomposition be better? u Should we re-define the term subdomain?

11. 11841f06detprob3 fault 1 fault 2 fault 3 fault 4 criterion 10.080.030.040.85 criterion 20.070.10.050.91 criterion 30.060.120.080.88 criterion 40.050.40.060.89 Comparing Criteria

12. 12841f06detprob3 Solving this choice u What are the assumptions? u Analogy with dice?

13. 13841f06detprob3 TTYP – smaller subdomains u What might be better smaller subdomains? u Would MCC (multiple condition coverage) be better subdomains

14. 14841f06detprob3 TTYP2 – C0 and C1 coverage u How do we deal with C0 and C1 coverage since they are not subdomain testing methodologies?

15. 15841f06detprob3 TTYP3 u How could you estimate the det prob of C0 or C1 testing?

16. 16841f06detprob3 Marble Problem u Assume that there is a bag of marbles from which marbles are drawn with replacement. u What is the maximum likelihood estimate of p (the probability of drawing a purple) marble if you draw exactly n purple marbles in a row? u ? P such that (p) n (1-p) is max or u ? P such that (p) n =.5

17. 17841f06detprob3 Evaluating Testing Methods by Delivered Reliability Frankl, Hamlet, Littlewood, Strigini IEEE TOSE Aug98 For Tuesday, Sep 4– study through section 2.3