Detection probability Testing Basics Detection probability

L5asg – detprob for subdomain tests What is the probability of detection with one randomly chosen test case per path? What is the probability of detection with an equal number of randomly chosen test cases?

Control Flow Graph Operational profile 3,3,3 abcdegi equi 3,3,4 abcegi isos 3,3,5 abcegi isos 3,3,6 abcefgi not 3,4,3 abcegi isos 3,4,4 abcegi isos 3,4,5 acegi scal 3,4,6 acegi scal All inputs are equally likely

What are the failure probability for each color (separately)? cin >> a >> b >> c ; type = “scalene”; if (a == b || a == c && b == c) type= “isosceles”; if (a == b || a == c) type = “equilateral”; if (a >= b+c || b >= a+c || c > a+b) type=“not a triangle”; if (a <= 0 || b <= 0 || c <= 0) type=“bad input”; cout<< type; Blue Green Red

TTYP – smaller subdomains What might be better smaller subdomains? Would MCC (multiple condition coverage) be better subdomains

TTYP2 – C0 and C1 coverage How do we deal with C0 and C1 coverage since they are not subdomain testing methodologies?

Evaluating Testing Methods by Delivered Reliability Frankl, Hamlet, Littlewood, Strigini IEEE TOSE Aug98

Testing Debug Operational

Fault Detection Probability Probability of a testing methodology finding a fault (if it existed)

Partition vs Random

Tests, Specifications, meets Test or test case single value of program input functional program - one input produces an output Specification - S set of input-output pairs Program meets specification iff for all x in spec, actual output matches spec output

Q: probability distribution Q - probability distribution over input domain Q:D -> [0,1] and S Q(t) = 1

Q : Failure Probability Q - failure probability for a randomly drawn point is S Q*d Where d(t) = 1 if f and 0 if s and f-phi(failure) and s-sigma(success) How does this relate to our notation?

Reliability R(N) = (1- Q)N

Assumptions of initial model

Terms q d

3.2 SFR, w/o subdomains d = StinF V(t) P(Q=0) = 1-(1-d)T P(Q=q) = (1-d)T E(Q) = 0* P(Q=0) +q* P(Q=q) = q(1-d)T

Thurs, Sep 6 Read next section of article