1. Chapter 8: Path Testing
Csci 565

2. Objectives Program Graphs Decision-to-decision path (DD-Paths)
Test Coverage Metrics
Basis Path Testing
Observation on McCabe Basis Path Method
Essential Complexity

3. Structural Coverage Analysis
The specification based testing can be used to verify the intended functionality
The techniques cannot be used with respect to unintended functionality
Structural testing (white box) is complementary solution
Shows reachability

4. Program Graph Resilient properties of PG Program Graph (PG)
A directed graph G= (V, E)
Where
V = a set of nodes corresponding to program statements in an imperative programming languages (e.g., C)
E = a set of edges corresponding to flow of control
Resilient properties of PG
can be nested, which is the equivalent of a subroutine call
can be sequenced, which is the equivalent of sequential execution of two instructions

5. Program Graphs for elements of Programming langauges

6. ignore

7. Number of paths 100 trillion if we iterate 20 times the loop having five paths inside the loop: …51 :
Takes one billion years to test this program.

10. f t t f f f f

11. Test Selection Criteria?
Test Selection coverage Criteria
Help to measure the adequacy of a test suite
E.g., if we use statement coverage to test 40% of the code, it means 60% of code was never executed
Help to decide when to STOP testing
E.g., use other criteria to cover 85%
Coverage?
refers to the extent by which a given verification activity has satisfied its objectives
a measure, not a method or a test
expressed as the percentage of an activity that is accomplished.
The amount of code structure that has been exercised can be measured by different criteria. Several of
these structural coverage criteria are discussed briefly in the next section.

12. Types of Structural Coverages
Typically structural coverage criteria are divided into two types:
Data flow
Control flow
Data flow criteria?
measure the flow of data between variable assignments and references to the variables
Control flow criteria?
measure the flow of control between statements and sequences of statements

13. E.F. Millers’s Coverage Metrics

14. Overview of the families of test selection criteria
Families of test selection criteria include
Structural model coverage criteria
All statements (C0)
All DD-path (C1) (chain)
All conditions (predicates) to each outcome (T orF) (C1p)
All paths (C)
C  C1p  C0
Multiple condition coverage (CMCC)
A test set achieves CMCC if it exercises all possible combinations of condition outcomes in each decision
For example, for N conditions, it requires 2N tests (truth table)
Dependent pairs Cd (Define/Usage paths)

15. Example of MCC: a compound condition of three variables of a, b, c.

16. Statement testing (C0)
Let T be a test suite for a Program P, then T satisfies the statement criterion for P, iff, for each statement S of P, there exists at least one test case in T that causes the execution of S.
In terms of Flow graph model of program P, it is the same as visiting each single node on some execution path exercised by a test case in T
C0 = number of executed statements/ total number of statements

17. Statement testing: Small Java Program

18. Statement testing: Flow Graph
C0: path ace
Test case:
A=2, B=0, X=3

19. C1p: Every predicate
Let T be a test suite for a program P. T satisfies the decision adequacy criterion for P, iff,
for each branch B (or predicate P), there exists at least one test case in T that causes execution of B.
This is the same as stating that every edge in the flow graph model of program P belongs to some execution path exercised by a test case in T
C1p = number of executed branches/ total number of branches
T satisfies the branch adequacy criterion if C1p =1

20. predicate Testing: Flow Graph
C1: p1: acd and p2: abe
T1: A=3, B=0, X=1
T2: A=2, B=1, X=1
P1:a,c,d

21. predicateTesting: Flow Graph
C1: p1: acd
T1: A=3, B=0, X=1

22. predicate Testing: Flow Graph
C1: p2: abe
T2: A=2, B=1, X=1

23. Predicate Testing: Flow Graph
Four conditions:
A>1, B=0, and A=2, X>1
Need test cases to force when
1) A>1, and A<=1
2) B=0, and B<>1
3) A=2, and A<> 2
4) X>1, and X<=1

24. C2 metric : Simple Loop Coverage + loop coverage
C2 metric demands the coverage of C1 and loop testing
Simple loop testing
Requires that every loop decision to be tested for two possible outcomes:
One to force traversing the loop body
The other one to exist

25. Loop boundary adequacy criterion (LC)
A test suite T for a program P satisfies the LC criterion iff for each loop l in P:
There is at least one execution in which control reaches the loop, and then the loop control condition evaluates to False the first time it is evaluated (zero time)
There is at least on e execution in which control reaches the loop and then the body of the loop is executed exactly once before control leaves the loop (one time)
There is at least one execution in which the body of the loop is repeated more than once (many times)

26. Dd Metric: C1 coverage + every define/usage
Refers to the most common dependency in DD-paths known as define/usages (covered in chapter 9: dataflow testing)
The dependency issues closely related to infeasible path problem associated with the control flow testing
E.g., C and H, and D and H pair in figure 8.5
The H branch can be traversed s when C = true
The H branch can not be traversed when D= false
The path is syntactically feasible but semantically is impossible

27. CMCC: Multiple Condition Coverage
A more complete extension that includes both the basic condition and branch adequacy criteria is called the multiple condition coverage CMCC
CMCC requires a test case for each possible evaluation of compound conditions.
For N basic conditions, we need 2N combinations of N basic conditions
Short-circuit evaluation is effective in reducing the above number to a more manageable number.

28. CMCC: Multiple Condition Coverage
Four conditions:
A>1, B=0, and A=2, X>1
Need test cases to force when
1) A>1, and A<=1
2) B=0, and B<>1
3) A=2, and A<> 2
4) X>1, and X<=1
Test cases:
T1: A=2, B=0, X=4
T2: A=1, B=1, X=1

29. Truth table for (A>1 AND B =0)
conditions T1 T2 T3 T4 A
true
false B
A>1 AND B=0
ACTIONS
X=X/A x -

30. Truth table for (A=2 or X>1)
conditions T5 T6 T7 T8 A
true
false X
A=2 OR x>1
ACTIONS
X=X+1 x -

31. Test Data and Test cases for CMCC

32. General Form for Compound Condition
Finally, multiple (compound) condition coverage requires test cases that ensure each possible combination of inputs to a decision is executed at least once; that is, multiple condition coverage requires exhaustive testing of
the input combinations to a decision. In theory, multiple condition coverage is the most desirable structural coverage measure; but, it is impractical for many cases. For a decision with n inputs, multiple condition coverage requires 2n tests.1

33. Example 2: Compound Condition

34. Modified Condition/Decision Coverage (MC/DC)
Requires that each basic condition be shown to independently affect the outcome of each decision
a condition has independent effect when that condition alone determines the outcome of the decision (AKA unique-cause approach)
For each basic condition C,
select two test cases such that the truth values of all evaluated conditions except C are the same,
Also,the compound condition as a whole evaluates to True for one of those test cases and False for the other
MC/DC is mandated by RTCA/Do-178B
The MC/DC criterion enhances the condition/decision coverage criterion by requiring that each condition be shown to independently affect the outcome of the decision. The independence requirement
ensures that the effect of each condition is tested relative to the other conditions. However, achieving MC/DC requires more thoughtful selection of the test cases, and,
in general, a minimum of n+1 test cases for a decision with n inputs. For the example (A or B), test cases (TF), (FT), and (FF) provide MC/DC. For decisions with a large number of inputs, MC/DC requires
considerably more test cases than any of the coverage measures discussed above.
The requirement to show the independent effect of each condition within a decision makes MC/DC unique among coverage criteria. Without any constraining definitions, determining whether a condition
has independent effect might seem rather simple: a condition has independent effect when that condition alone determines the outcome of the decision. At first glance, this simple definition seems to be
consistent with the intent of MC/DC. Whether the simple notion is truly consistent with DO-178B, however, requires knowing the full DO-178B definition13 of MC/DC plus the definitions for condition
and decision (ref. 2).
Condition–A Boolean expression containing no Boolean operators.
Decision–A Boolean expression composed of conditions and zero or more Boolean operators. A
decision without a Boolean operator is a condition. If a condition appears more than once in a
decision, each occurrence is a distinct condition.
Modified Condition/Decision Coverage–Every point of entry and exit in the program has been
invoked at least once, every condition in a decision in the program has taken all possible
outcomes at least once, every decision in the program has taken all possible outcomes at least
once, and each condition in a decision has been shown to independently affect that decision’s
outcome. A condition is shown to independently affect a decision’s outcome by varying just that
condition while holding fixed all other possible conditions.

35. Terminologies: Condition, Decision
A Boolean expression containing no Boolean operators.
E.g., A>B
Decision (compound Boolean expressions)?
A Boolean expression composed of conditions and zero or more Boolean operators. (AND, OR)
(A>B OR (C=B AND D=H))
A decision without a Boolean operator is a condition.
Also, If a condition appears more than once in a decision, each occurrence is a distinct condition.

36. Understanding Logical operator

37. Minimum testing to meet MC/DC for AND

38. Minimum testing to meet MC/DC for OR

39. Minimum testing to meet MC/DC for OR

40. MC/DC for Z= (A or B) AND (C OR D)
Test case 2 is coupled by test case 10 for A
Test cases from 1-16

41. Path testing C (or P)
A test suite T for a program P satisfies the path adequacy criterion iff
for each path pi in P, there exists at least one test case in T that causes the execution of pi
C = # of executed paths/total number of paths
C cannot be achieved because the number of paths in a program having loop is unbounded, so this criterion cannot be satisfied for any but the most trivial programs.

42. Linearly independent paths
Number of linearly independent paths: V(G)=e-n+2p=10-7+2(1)=5 where e is number of edges; n is number of nodes, and p is number of programs

43. ex1= p2+p3-p1
ex1= (1,0,1,2,0,0,0,0,1,0) +(1,0,0,0,1,0,0,1,0,1)-(1,0,0,1,0,0,0,0,1,0)
ex1 = (1,0,1,1,1,0,0,1,0,1)=A,B,C,B,E,F,G
ex2=2p2-p1
ex2=(2,0,2,4,0,0,0,0,2,0) – (1,0,0,1,0,0,0,0,1,0)= (1,0,2,3,0,0,0,0,1,0)=A,B,C,B,C,B,C,G

44. McCabe’s Algorithmic Procedure (Baseline method)
McCable’s Algorithmic Procedure can be used to identify a set of basis paths
Select a path with highest number of decision nodes
Retrace each decision in baseline path (i.e., normal path that can be arbitrary path)
Flip each decision to create a new path
E.g.
p1: A,B,C,B,E,F,G (baseline, flip A)
P2: A,D,E,F,G (flip p1 at D)
P3:A,D,F,G (take D)
P4:A,B,E,F,G (flip p1 at B)
P5:A,B,C,G (flip p1 at C)
-The basis paths is different from the one in table 2
-A basis path is not required to be unique

45. f t t f f f f

46. Example: Applying McCabe Procedure

47. Observations on McCabe’s Basis Path
Infeasible paths

48. Essential Complexity V(G)=1wellStr(P)
V(G) >=10 requires substantial testing
Use the characteristics of code to select appropriate criteria

51. Region 1: specified behavior that implemented by feasible paths
Region 2, 6: Topologically feasible paths
Region 3: feasible paths that are not specified ( extra functionalities)
Region 4: Specified behavior that almost implemented (infeasible paths)
Region 5: Specified behavior that are not implemented yet (cannot use path testing here)
Region 7: Unspecified, infeasible, yet topologically feasible paths (no problem can occur)

52. More on White-box testing
The purpose of structural coverage analysis with the associated structural coverage analysis resolution is to complement requirements-based testing as follows:
Provide evidence that the code structure was verified to the degree required for the applicable software level;
Provide a means to support demonstration of absence of unintended functions;
no information about whether the code is doing what it is supposed to be doing as specified in the requirements
Establish the thoroughness of requirements-based testing.
When drafting DO-178B/ED-12B, it was realized that requirements-based testing cannot
completely provide this kind of evidence with respect to unintended functions. Code that is
implemented without being linked to requirements may not be exercised by requirements-based
tests. Such code could result in unintended functions. Therefore, something additional should be
done since unintended functions could affect safety. A technically feasible solution was found in
structural coverage analysis.
The rationale is that if requirements-based testing proves that all intended functions are properly
implemented, and if structural coverage analysis demonstrates that all existing code is reachable
and adequately tested, then these two together provide a greater level of confidence that there are no
unintended functions

53. References
Material in this presentation is taken from the following resources:
Paul C. Jorgensen. Software Testing: A Craftsman’s Approach, 3rd Edition. Auerbach Publication, 2008
Mauro Pezze and Michal Young. Software Testing And Analysis. Wiley 2008
Kelly j. Hayhurst et al. A Practical Tutoring on Modified Condition/Decision Coverage NASA/TM , May 2001.