1. Mary Jean Harrold Georgia Institute of Technology
Testing Techniques
Mary Jean Harrold
Georgia Institute of Technology

2. Outline Introduction Black-box coverage measures
equivalence partitioning
boundary-value analysis
White-box coverage measures
control flow
condition
data-flow
fault-based
Integration techniques
Regression testing techniques

3. Verification and Validation (V&V)
Verification answers the question: Are we building the product right? Thus, it refers to the testing we do to assure that that the product resulting from one phase is a correct translation from the product resulting from the previous phase. For example, we attempt to verify that the implementation is consistent with the design.
Validation answers the question: Are we building the right product? Thus, it refers to the testing we do to assure that the resulting software is traceable to customer requirements.

4. Software Development Phases and Testing
Requirements Analysis Phase:
Design Phase:
Implementation Phase:
Integration Phase:
Maintenance Phase:
Develop test plan and system tests; perform technical review
Develop integration tests; perform technical review
Develop and run unit tests; perform technical review
Run integration tests
Run system tests
Run regression tests

5. Software Development Phases and Testing (Graphical View)
Requirements
Analysis
System
High-Level
Design
Integration
Low-Level
Design
Unit
Coding
Unit
Delivery
Acceptance
Maintenance
Regression

6. Some Important Facts
Fact 1: It’s cheaper to find and fix faults at the unit level -- most faults are incorrect computations within subroutines
Fact 2: It’s good to do as much testing early as you can -- you can’t test quality in, you have to build it in

7. Testing Terminology P is a program, S is the specification for P
P is correct for d in D if P(d) = S(d)
A d in D is called a test case
A finite subset of D is called a test suite or test set P D d R S

8. Testing Terminology* Failure: an erroneous result
incorrect outputs/response for given inputs/stimuli
fails to meet real-time constraints
Error: incorrect concept
may cause failures if not corrected
Fault: the cause of one or more failures
discovered after release
*Additional definitions in “Testing Definitions” handout

9. Testing Terminology
IMPORTANT: Fault doesn’t imply failure - a program can be coincidentally correct in that it executes the fault but does not fail
SQUARE(z) // finds square of z
y = 2 * z
print y
Fault: line “y = 2 * z” should be “y = z * z”
Failure: for input y = 2
Coincidentally correct: for other inputs

10. Coverage Criteria
Test Selection (Coverage) Criterion C: a rule for selecting d in D to place in T
We want a C that gives test suites that guarantee correctness
We settle for a C that gives test suites that improve confidence
Limitations of coverage criteria: achieving coverage doesn’t show correctness
Types of criteria:
black-box: specification, state
white-box: control flow, condition, data-flow, fault-based

11. Coverage Criteria Coverage criteria C is used in two ways:
1. To measure the quality of a test suite T (i.e., help us evaluate T) the adequacy score is the percentage of “coverable” items identified using C that are covered by T
2. To guide the selection of an C-adequate test suite T (i.e., tell us when to stop testing): the adequacy requires that some percentage of “coverable” items are covered by tests in T

12. Black-Box Testing
Selection of test suite is based on some aspect (such as requirements, specification, function) other than the code
We won’t discuss further

13. White-Box Testing
Selection of test suite is based on some aspect of the code
We’ll consider several examples
control flow
statement
branch
basis path
path
condition
simple
multiple
loop
dataflow
all-uses
all-du-paths
fault based
mutation

14. Some Terminology
Test Requirements: those aspects of the program that must be covered according to the coverage criterion
Test Specification: constraints on the input that will cause a test requirement to be satisfied
Test Case (Test Suite): a set of inputs that satisfies a test specification

15. Sum: Example Program 1. read i 2. read j 3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
endif
i= i + 1
read j
endwhile
9. print sum

16. White-Box Testing Statement Coverage For Sum
test requirements: statements in the program
test specifications: constraints on inputs that cause statements to be executed
test cases: inputs that satisfy the constraints in the test specifications
For Sum
test requirements: statements 1-9
test specifications: (0<i<=10 and j>0 at least once)
test suite: i = 10, j1 = 4, j2 = 10; expect sum = 4

17. White-Box Testing Branch Coverage For branch coverage, use
test requirements: branches in program
test specifications: constraints on inputs that cause branches to be executed
test cases: inputs that satisfy the constraints in the test specifications
For branch coverage, use
control-flow graph (CFG): nodes represent statements (or sequences of statements) in the program; edges represent flow of control between statements

18. Sum: Control-Flow Graph
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
endif
i= i + 1
read j
endwhile
9. print sum
1,2,3 T 4 5 T F 6 F 9
7,8
Some CFG edges are labeled (e.g., (4,5) and (5,7))
Some CFG edges are backedges (e.g., (7-8,4))

19. White-Box Testing Branch Coverage For Sum
test requirements: 4T, 4F, 5T, 5F
test specifications: for example,
for 4T (0<i<=10, j’s can be any integer)
for 5F
(0<i<=10, at least one of the (used) j’s <=0)
test suite: i = 10, j1 = 4, j2 = 10; expect sum = 4
i = 10, j1 = -3, j2 =5; expect sum = 0

20. White-Box Testing Basis-Path Coverage For basis-path coverage, need
test requirements: a set of basis paths in the program
test specifications: constraints on inputs that cause these basis paths to be executed
test cases: inputs that satisfy the constraints in the test specifications
For basis-path coverage, need
cyclomatic complexity: defines the maximum number of independent paths in the program.

21. Cyclomatic Complexity; Basis Paths
Cyclomatic complexity (CC) can be computed in three ways:
1. e - n + 2p, where e is the number of edges in the CFG, n is the number of nodes in the CFG, p is the number of connected components in the CFG
2. number of predicates + 1 for structured programs
3. number of regions in the program
CC is an upper bound on the number of basis paths

22. Sum: Cyclomatic Complexity and Basis Paths
1,2,3
CC (by three methods)
1. e = 7, n = 6, p = 1
* 1 = 3
2. 2 predicates (4 and 5)
2 + 1 = 3
3. 3 regions:
one inside 5,6,7-8
one inside 4,5,7-8
one outside graph T 4 5 T F 6 F 9
7,8

23. Sum: Cyclomatic Complexity and Basis Paths
1,2,3 4 5 6
7,8 9 T F
Basis Paths
there can be many sets but no set can be bigger than CC
Two example sets are:
, 4, 9
1-2-3, 4, 5, 6, 7-8, 4, 9
1-2-3, 4, 5, 7-8, 4, 9
, 4, 5, 6, 7-8, 4, 9

24. Sum2: Example Program 1,2 3 4,5 6 11 7,8 9 10 T F T F T F main( ) {
3. while (i <= 5) {
scanf(“%d”, &j);
If (j < 0)
continue;
sum = sum + j;
if (sum > 10)
break;
10. i = i + 1; }
11.printf(“sum is %d”, sum); }
1,2 T 3
4,5 F T 6 F 11
7,8 T F 9 10

25. White-Box Testing Basis Path Coverage For Sum
test requirements: a set of basis paths
test specifications: for example,
(you complete)
test suite: (you complete)

26. White-Box Testing Condition Coverage
test requirements: each condition must be T, F under test
test specifications: constraints on inputs that cause each condition to be true, each condition to be false
test cases: inputs that satisfy the constraints in the test specifications

27. Sum: Example Program For Sum test requirements:
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
endif
i= i + 1
read j
endwhile
9. print sum
For Sum
test requirements:
in 4, I>0 must be T, F
in f4, I<=10 must be T, F
5 must be T, F
test specifications:
(you complete)
test suite: (you complete)

28. White-Box Testing Multiple Condition Coverage
test requirements: each condition possible combination of truth values must be assigned
test specifications: constraints on inputs that cause the requirements to hold
test cases: inputs that satisfy the constraints in the test specifications

29. Sum: Example Program For Sum test requirements:
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
endif
i= i + 1
read j
endwhile
9. print sum
For Sum
test requirements:
in 4, conditions must be
TT, TF, FT, FF
in 5, conditions must be
T, F
test specifications:
(you complete)
test suite:

30. White-Box Testing Loop Coverage For Sum
test requirements: skip loop; once through loop; twice through loop; max through loop
test specifications: constraints on inputs that cause loop iterations to be executed
test cases: inputs that satisfy the constraints in the test specifications
For Sum
(you complete)

31. White-Box Testing Data-flow Coverage: All Uses
test requirements: sets of definition-use pairs (du-pairs); with respect to variable v,
a definition of v is a reference to v where the value of v is changed (e.g., assignment to v, input a value of v)
a use v is a reference to v where the value of v is fetched by not changed (e.g., v on right-hand side of assignment, v is output)
a definition-clear subpath for a definition d of v and a use u of v is a subpath in the CFG between d and u on which v is not redefined
test specifications
test cases

32. Definition-Clear Paths; All-Uses Coverage
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
endif
i= i + 1
read j
endwhile
9. print sum
1,2,3
defs of i, j, sum 4 5
uses of i
use of j 6
uses of sum, j
def of sum
use of sum 9
use of i
defs of i, j
7,8
Def-use pairs for i: (1, (4,5)), (1, (4,9)), (1, 7), (7, (4,5)), (7, (4,9))

33. Definition-Clear Paths; All-Uses Coverage
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
1,2,3
defs sum 4 5
use of sum
def of sum 6
use of sum 9 6a
use of sum
7,8

34. White-Box Testing Mutation Analysis/Testing Major Premise:
The quality of a test set is related to the ability of that test set to differentiate the program being tested from a set of marginally different alternative programs
Differentiating Programs:
A test case differentiates two programs if it causes the two programs to produce different results

35. Mutate: make a small syntactic change
White-Box Testing
Mutation Analysis/Testing -- Terminology
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
Mutate: make a small syntactic change

36. Mutate: make a small syntactic change
White-Box Testing
Mutation Analysis/Testing -- Terminology
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
Mutate: make a small syntactic change

37. Mutate: make a small syntactic change
White-Box Testing
Mutation Analysis/Testing -- Terminology
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum - j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
Mutate: make a small syntactic change

38. Mutate: make a small syntactic change
White-Box Testing
Mutation Analysis/Testing -- Terminology
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum * j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
Mutate: make a small syntactic change

39. Mutate: make a small syntactic change
White-Box Testing
Mutation Analysis/Testing -- Terminology
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + i
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
Mutate: make a small syntactic change

40. Mutate: make a small syntactic change
White-Box Testing
Mutation Analysis/Testing -- Terminology
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < 10) do
if (j >0)
sum = sum + i
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
Mutate: make a small syntactic change

41. Mutate: make a small syntactic change
White-Box Testing
Mutation Analysis/Testing -- Terminology
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < 10) do
if (j >0)
sum = sum + i
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
Mutate: make a small syntactic change

42. White-Box Testing Mutation Analysis/Testing -- Terminology
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i > 10) do
if (j >0)
sum = sum + i
6a print sum
endif
i= i + 1
read j
endwhile
9. print sum
Mutate: make a small syntactic change
Mutation: the changed statement
Mutant: program with a mutated statement

43. Sum: Example Mutations
1. read i
2. read j
3. sum = 0
4. while (i > 0) and (i < = 10) do
if (j >0)
sum = sum + j
endif
i= i + 1
read j
endwhile
9. print sum
1. SVR -- scalar variable replacement: replace each occurrence of variable v with all other variables in scope
e.g., change 1 to
read j
read sum
2. AOR -- arithmetic operator replacement: replace each occurrence of an arithmetic operator with all possibilities
e.g., change 6 to
sum = sum -j, etc.

44. Sum3: Example Program We now have three functions: Main, Sum, and Add
int Sum (int I, j)
int sum
1. sum = 0
2. while (i > 0) and (i < = 10) do
if (j >0)
sum = Add(sum, j)
endif
i= Add(I, 1)
read j
endwhile
9. return sum
We now have three functions: Main, Sum, and Add
How do we develop and test a system with many modules?
Stubs need to simulate modules used by current module
Drivers need to simulate modules that use current module

45. System Integration, Build Plan, Integration Testing
Development or integration strategies
Decide the order in which components of the system will be developed

46. Order of Integration: USES Relation
For any two distinct modules Mi and Mj, we say that Mi USES Mj if and only if correct execution of Mj is necessary for Mi to complete the task described in its specification.
If Mi USES Mj, we also say that Mi is a client of Mj because Mi requires the services that Mj provides

47. Integration: Uses Graph
Play Game
Collect Cards
Determine Winner
Play Hand
Deal
Announce Winner
Compare Hands
Get Hand Value

48. Common build strategies
Big Bang
All coding precedes all integration
Bottom Up
Start at low-level utility modules
Top Down
Start at high-level control modules
Incremental/Sandwich
Integrate control modules top down and utility modules bottom up

49. Big Bang Integration Collect Cards Determine Winner Play Hand Deal
Play Game
Announce Winner
Compare Hands
Get Hand Value

50. Big Bang Integration Advantages Disadvantages
No need to write stubs or drivers
Disadvantages
Difficult to identify units causing errors
Critical modules receive no extra testing
Major design flaws are discovered very late
No flexibility in scheduling

51. Bottom Up Integration Implement a module Implement a test driver
often a quick prototype
Execute tests
Replace test driver with its implementation
Implement a test driver for new module
Repeat until all modules are integrated

52. Bottom Up Integration Driver Announce Winner Compare Hands
Get Hand Value

53. Bottom Up Integration Determine Winner Announce Winner Compare Hands
Get Hand Value

54. Bottom Up Integration Driver Determine Winner Announce Winner
Compare Hands
Get Hand Value

55. Bottom Up Integration Driver Collect Cards Play Hand Deal
Determine Winner
Announce Winner
Compare Hands
Get Hand Value

56. Bottom Up Integration Play Game Collect Cards Play Hand Deal
Determine Winner
Announce Winner
Compare Hands
Get Hand Value

57. Bottom Up Integration Advantages no need to write stubs
low level utilities are well tested
high-level (often simple) drivers are not as well tested
Disadvantages
not all low level utilities are important
emphasis on low level functionality
major design flaws are discovered late

58. Top Down Integration Implement highest level component
Create stubs for called component
Test the component with the stubs
Implement and test stubs one by one
use stubs for any called components
Repeat until all stubs are implemented

59. Top Down Integration
Play Game
Stub

60. Top Down Integration Play Game Collect Cards Determine Winner
Play Hand
Deal
Stub

61. Top Down Integration Play Game Collect Cards Determine Winner
Play Hand
Deal
Announce Winner
Compare Hands
Get Hand Value

62. Top Down Integration Advantages no need to write drivers
high-level drivers are well tested
unimportant low-level utilities are not as well tested
Disadvantages
important low-level utilities are not tested as well
simple high-level drivers may not need the extra testing

63. Incremental Top Down Integration
Identify most important path through the uses graph
implement and test the highest module in the path
creating drivers and stubs as necessary
select the next module on that path
repeat until the path is done

64. Incremental Top Down Integration (cont.)
Select next most important path and follow the same process
Repeat until all paths have been implemented
Paths do not have to start at the root or end with leaves

65. Incremental Integration
Play Game
Determine Winner
Compare Hands
Collect Cards
Play Hand
Deal
Announce Winner
Get Hand Value

66. Incremental Top Down Integration
Advantages
test the most important functionality first
easy to isolate errors
find major design flaws early
have an incomplete but “working” system early
good for morale, visibility, early assessment
minimizes the need for drivers and stubs

67. Developing An Incremental Build Plan
Identify most important paths through the uses graph
most used functionality; centrality
complex component that plays a key role
Save “frills” for later paths
Produce something that works early
add features incrementally for easier testing and debugging

68. Components of Test Plan
1. Scope of testing
2. Test plan
A. Test phases and builds
B. Schedule
C. Overhead software
D. Environment and resources
3. Test procedure n (description of test for build n)
A. Order of integration
B. Unit tests for modules in build
C. Test environment
D. Test case data
E. Expected results for build n
4. Actual test results
5. References
6. Appendices

69. Software Maintenance Process
Identify
changes
to P
Modify P
to get P’
Select T’ in T
for P’
P’ correct
for T’?
Find faults
in P’ F F
Create T’’
for untested
parts of P’ T
Create T’’’
T + T’’ T
P’ correct
for T’’?

70. Selective Retest P’ correct for T’? P’ correct for T’’? Identify
changes
to P
Modify P
to get P’
Select T’ in T
for P’
P’ correct
for T’?
Find faults
in P’ F F T
Create T’’
for untested
parts of P’ T
P’ correct
for T’’?
Create T’’’
T + T’’

71. Selective Retest
Test
Suite T
Program P
P’ Version
of P ?

72. Selective Retest Regression test selection problem
Q1: Which tests do not need to be rerun?
Q2: Which tests need to be run unchanged?
Q3: Which tests must be run modified?
Q4: How can we prioritize selected tests?
Test suite management problem
Q5: Which tests are obsolete?
Q6: Which tests are redundant?
Q7: How can we minimize test suites?
Coverage identification problem
Q8: What new tests are needed?

73. Regression Test Selection: Q1, Q2
Suite T
Program P
P’ Version
of P ?

74. Regression Test Selection: Q1, Q2
Program P
P’ Version
of P

75. Regression Test Selection: Q1, Q2
Program P T
P’ Version
of P

76. Regression Test Selection: Q1, Q2
Program P T
P’ Version
of P T’

77. Sample Program entry exit S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 F T
Procedure AVG
S1 count = 0
S2 fread(fptr, n)
S3 while (not EOF) do
S if (n < 0)
S return (error)
else
S nums[count] = n
S count ++
endif
S fread(fptr, n)
endwhile
S9 avg = mean(nums,count)
S10 return(avg)

78. Execution Tracing Test Input Output t1 empty file 0 entry S1 S2 S3 S9
exit F S1 S2
Test Input Output
t empty file F S3 T S4 T F S5 S6 S7 S8 S9
S10
exit

79. Execution Tracing Test Input Output t1 empty file 0 t2 -1 error entry
exit

80. Execution Tracing Test Input Output t1 empty file 0 entry S1 S2 S3 S9
exit F S1 S2
Test Input Output
t empty file F S3 T S4 T F S5 S6 S7 S8 S9
S10
exit

81. Execution Tracing Test Input Output t1 empty file 0 t2 -1 error
entry S1 S2
Test Input Output
t empty file
t error t F S3 T S4 T F S5 S6 S7 S8 S9
S10
exit

82. Test History Information
entry S1
{t1, t2, t3} S2
Test Input Output
t empty file
t error t
{t1,t3} S3
{t2, t3} S4
{t2}
{t3} S5 S6 S7 S8 S9
S10
exit

83. Regression Test Selection Algorithm
entry
exit T F
Procedure AVG
S1 count = 0
S2 fread(fptr, n)
S3 while (not EOF) do
S if (n < 0)
S return (error)
else
S nums[count] = n
S count ++
endif
S fread(fptr, n)
endwhile
S9 avg = mean(nums,count)
S10 return(avg)

84. Modified Version of AVG
Procedure AVG
S1’ count = 0
S2’ fread(fptr, n)
S3’ while (not EOF) do
S4’ if (n <= 0)
S5a print (“bad input”)
S5’ return (error)
else
S6’ nums[count] = n
endif
S8’ fread(fptr, n)
endwhile
S9’ avg = mean(nums,count)
S10’ return(avg)
Procedure AVG
S1 count = 0
S2 fread(fptr, n)
S3 while (not EOF) do
S if (n < 0)
S return (error)
else
S nums[count] = n
S count ++
endif
S fread(fptr, n)
endwhile
S9 avg = mean(nums,count)
S10 return(avg)

85. CFG Walk {t1, t2, t3} {t1, t3} {t2, t3} {t2} {t3} entry S1 entry’ S1’
S5a
S6’
S5’ S7 S8
S8’ S9
S9’
S10
exit
S10’
exit’

86. CFG Walk {t1, t2, t3} {t1, t3} {t2, t3} {t2} {t3} entry S1 entry’ S1’
S5a
S6’
S5’ S7 S8
S8’ S9
S9’
S10
exit
S10’
exit’

87. CFG Walk {t1, t2, t3} {t1, t3} {t2, t3} {t2} {t3} entry entry’ S1 S1’
S5a
S6’
S5’ S7 S8
S8’ S9
S9’
S10
exit
S10’
exit’

88. CFG Walk {t1, t2, t3} {t1, t3} {t2, t3} {t2} {t3} entry entry’ S1 S1’
S5a
S6’
S5’ S7 S8
S8’ S9
S9’
S10
exit
S10’
exit’

89. CFG Walk {t1, t2, t3} {t1, t3} {t2, t3} {t2} {t3} entry entry’ S1 S1’
S5a
S6’
S5’ S7 S8
S8’ S9
S9’
S10
exit
S10’
exit’

90. CFG Walk {t1, t2, t3} {t1, t3} {t2, t3} {t2} {t3} entry entry’ S1 S1’
S5a
S6’
S5’ S7 S8
S8’ S9
S9’
S10
exit
S10’
exit’

91. CFG Walk {t1, t2, t3} {t1, t3} {t2, t3} {t2} {t3} entry entry’ S1 S1’
S5a
S6’
S5’ S7 S8
S8’ S9
S9’
S10
exit
S10’
exit’

92. CFG Walk {t1, t2, t3} {t1, t3} {t2, t3} {t2} {t3} entry entry’ S1 S1’
S5a
S6’
S5’ S7 S8
S8’ S9
S9’
S10
exit
S10’
exit’

93. CFG Walk {t1, t2, t3} {t1, t3} {t2, t3} {t2} {t3} entry entry’ S1 S1’
S5a
S6’
S5’ S7 S8
S8’ S9
S9’
S10
exit
S10’
exit’

94. CFG Walk {t1, t2, t3} {t1, t3} {t2, t3} {t2} {t3} entry S1 entry’ S1’
S5a
S6’
S5’ S7 S8
S8’ S9
S9’
S10
exit
S10’
exit’

95. Prototype Implementation
Program P
Test Suite T
DejaVu
Test Suite T’
Modified
Program P’

96. C Study Seven C programs (300LOC), Siemens Labs, 7-42 versions, tests
Internet Empire C program (50KLOC), 5 versions, no tests
Microsoft NT Calc C program (2KLOC), 9 versions, 388 tests

97. Study of C Programs
Seven C programs (300LOC), Siemens Labs, 7-42 versions, tests
Microsoft NT Calc C program (2KLOC), 9 versions, 388 tests
European Space Agency C program for satellite antennae adjustment (11KLOC), 33 versions, 10,000 tests
Internet Empire C program (50KLOC), 5 versions, no tests

98. Test Selection for C Programs