1. Sudipto Ghosh CS 406 Fall 99 November 16, 1999
Software Testing
Sudipto Ghosh
CS 406 Fall 99
November 16, 1999

2. Learning Objectives Data flow-based testing Test adequacy assessment
11/16/99
CS 406 Testing

3. Problems with path testing
With each predicate, there is a combinatorial explosion in the number of paths.
Potentially infinite number of paths because of loops.
Not all paths a feasible, i.e., there may not be inputs for which the path is executed.
Suppose that somehow, we satisfied this criterion. Would we find all the faults?
11/16/99
CS 406 Testing

4. More problems A path is tested only if it is present!!
Consider the example:
double logBase19 (double x) {
return ln(x)/ln(19); }
Missing condition is:
if(x > 0)
11/16/99
CS 406 Testing

5. More problems
It is possible to test every path without detecting the fault in the product.
Function to test the equality of 3 integers:
(Fallacious) assumption:
If the average of the 3 numbers is equal to the first, then they are equal.
boolean areEqual(int x,int y,int z){
if ((x+y+z)/3 == x) return TRUE;
else return FALSE; }
Exercise: Think of test cases.
11/16/99
CS 406 Testing

6. Reducing the number of paths to be covered
Find something between branch and path coverage.
Select a set of paths that ensure branch coverage and try some other paths that help reveal errors.
Consider two paths same if they differ only in their sub-paths that are caused due to loops.
Restrict test cases to linear code sequences
Identify set of points L, from which control flow may jump; includes entry, exit, branch statements
Paths begin and end at elements in L.
11/16/99
CS 406 Testing

7. Data Flow-Based Testing
Select paths to be executed based on data flow analysis
Information about where
variables are defined
variables are used
Idea is to make sure that the definitions of variables and their subsequent use is tested
11/16/99
CS 406 Testing

8. Variables in statements
Variables occur in statements as
Definition – def
Variables on the left side of a statement ( c )
c := 10;
Variable is used in a computation – c-use
Variables on the right-hand side of a statement ( c )
temp := c + 5;
Variables used in a write statement
Variable is used in a predicate – p-use
if( c!= MAXLENGTH ) {
11/16/99
CS 406 Testing

9. An example (xy) scanf(x, y); if(y < 0) pow = 0 – y; else pow = y;
z = 1.0;
while(pow != 0)
{ z = z * x; pow = pow – 1; }
if ( y < 0 )
z = 1.0/z;
printf(z);
11/16/99
CS 406 Testing

10. Def/use graph of example1
def = {x, y}: c-use = {} y y
def = {pow}:
c-use = {y} 3 2
def = {pow}:
c-use = {y} 4
def = {z}: c-use = {} 5
def = {}: c-use = {}
pow
pow
def = {z, pow}:
c-use = {x, z, pow} 6 7
def = {}: c-use = {} y y
def = {z}:
c-use = {z} 9
def = {}:
c-use = {z} 8
11/16/99
CS 406 Testing

11. More on def/use graphs
With each node, we associate the c-uses of a variable
With each edge, we associate the p-uses of a variable
11/16/99
CS 406 Testing

12. Def-clear paths Def-clear path with respect to a variable x:
Path from node i to node j, such that there is no def of x in the nodes in the path from i to j. Nodes i and j may have a def.
Find out examples from the previous graph.
Path from node i to edge (j, k), such that there is no def of x in the nodes in the path.
11/16/99
CS 406 Testing

13. Global c-use A c-use of a variable is considered global if:
there is not def of x within the block preceding the c-use
Example?
11/16/99
CS 406 Testing

14. Global def
A def is global if it can be used outside the block in which it is defined.
A def of a variable x, in a node i, is a global def, if:
it is the last def of x in the block (node) i
a def-clear path exists from i to another node which has a global c-use of x
Example?
11/16/99
CS 406 Testing

15. Set def(i)
Set of variables for which there is a global def in the node i.
Examples:
11/16/99
CS 406 Testing

16. Set c-use(i)
Set of variables for which there is a global def in the node i.
Examples:
11/16/99
CS 406 Testing

17. Set p-use(i, j)
For an edge (i, j), the set p-use(i, j), is the set of variables for which there is a p-use for the edge (i, j).
Examples:
11/16/99
CS 406 Testing

18. Set dcu(x, i)
dcu(x, i) represents all those nodes in which the global c-use of x uses the value assigned by the def of x in i.
Suppose a variable x is in def(i) of node i.
Set of nodes, such that :
Each node has x in its c-use
There is a def-clear path from i to j
Examples:
11/16/99
CS 406 Testing

19. Set dpu(x, i)
dpu(x, i) represents all those edges in which the p-use of x uses the value assigned by the def of x in i.
Suppose a variable x is in def(i) of node i.
Set of edges, such that :
each edge has x in its p-use
There is a def-clear path from i to (j, k)
11/16/99
CS 406 Testing

20. Test case selection criteria
Let G be the def-use graph for a program
Let P be a set of all the complete paths of G that were executed during the test.
Examples:
11/16/99
CS 406 Testing

21. All-defs criterion
Make sure that the definitions of all variables are tested.
For every def of every variable, one of its uses (either p-use or c-use) must be included in a path.
For every node i in G and every x in def(i), P includes a def-clear path w.r.t. x to some member of dcu(x, i)
11/16/99
CS 406 Testing

22. All-p-uses criterion All p-uses of all definitions should be tested.
For every x  def (i), P includes a def-clear path w.r.t. x from i to all members of dpu(x, i).
Note that a c-use may not be tested
Can require
all p-uses, some c-uses criterion
11/16/99
CS 406 Testing

23. All c-uses criterion All c-uses of all definitions should be tested.
For every x  def (i), P includes a def-clear path w.r.t. x from i to all members of dcu(x, i).
Note that a p-use may not be tested
Can require
all c-uses, some p-uses criterion
11/16/99
CS 406 Testing

24. All-uses criterion
All p-uses and all c-uses of a definition must be exercised.
The set P must include,
for every node i and every x  def(i)
a def-clear path w.r.t. x from i to all elements of dcu(x, i) and to all elements of dpu(x, i).
11/16/99
CS 406 Testing

25. Number of test cases
Can be shown theoretically that the limit of the size of test set is up to quadratic in the number of two-way decision statements in the program.
Empirical studies have shown that the actual number of test cases that satisfy a criterion is quite small and increases linearly with the number of two-way decisions.
11/16/99
CS 406 Testing

26. Inclusion relationship between criteria
all-paths (path coverage)
all-uses
all-c-uses/
some-p-uses
all-p-uses/
some-c-uses
all-defs
all-p-uses
all-edges (branch coverage)
all-nodes (statement coverage)
11/16/99
CS 406 Testing

27. Implication of inclusion
Suppose C1 includes C2 (C1  C2), I.e test cases satisfying C1 also satisfy C2
Does it mean that C1 is always better than C2?
Statistically, the set of test cases satisfying C1 will be better than those satisfying C2.
Testing done using these criteria performs better than randomly selected test cases.
11/16/99
CS 406 Testing

28. Test assessment Develop Question: a test set T
a collection of test inputs
Question:
How good is T?
Test assessment is the measurement of the goodness of T.
Test assessment is carried out based on one or more criteria.
11/16/99
CS 406 Testing

29. Test adequacy assessment
These criteria are known as test adequacy criteria.
Test assessment is also known as test adequacy assessment.
11/16/99
CS 406 Testing

30. Test adequacy assessment (contd)
Test adequacy assessment provides the following information:
A metric, also known as adequacy score, or coverage, usually between 0 and 1.
A list of all the weaknesses found in T, which when removed, will raise the score to 1.
The weaknesses depend on the criteria used for assessment.
11/16/99
CS 406 Testing

31. Test adequacy assessment (contd)
Once the coverage has been computed, and the weaknesses identified, one can improve T.
Improvement of T is done by examining one or more weaknesses and constructing new test requirements designed to overcome the weaknesses.
11/16/99
CS 406 Testing

32. Test adequacy assessment (contd)
This is continued until all weaknesses are overcome, i.e., the adequacy criterion is satisfied (coverage = 1)
In some instances, it may not be possible to satisfy the adequacy criteria for one or more of the following reasons:
Lack of sufficient resources (people, time)
Weaknesses that cannot be removed because they are infeasible
The cost of removing the weakness is not justified
11/16/99
CS 406 Testing

33. Test adequacy assessment (contd)
While improving T by removing its weaknesses, one usually tests the program more thoroughly than it has been tested so far.
This additional testing is likely to result in the discovery of remaining errors.
Hence, we say that test assessment and improvement helps in the improvement of software reliability.
11/16/99
CS 406 Testing

34. Test adequacy assessment - summary procedure
Develop T
Select an adequacy criterion C
Measure adequacy criterion of T w.r.t C
Is T adequate?
Yes No
Yes
Improve T
More testing is warranted? No
DONE!!
11/16/99
CS 406 Testing