White Box Testing Arun Lakhotia University of Southwestern Louisiana P.O. Box Lafayette, LA 70504, USA

References: 1) Jalote, section 7.3 2) Roger Pressman, Software Engineering: A Practitioner’s Approach.

Testing Scenario 1 n If we know the function that the product has been designed to perform, tests can show that each function is fully operational. n This approach called Black-Box Testing.

Testing Scenario 2 n If we know the internal workings of a product, tests can show that internal operations perform according to specification, and all components have been exercised. n This approach is called White-Box (Structural) Testing.

Question n Why do we need both? n White box testing wants to force some desired coverage of different structures -- coverage based testing.

Coverage Types 1) Statement Coverage 2) Branch Coverage 3) Path Testing

Statement Coverage n The goal is to execute every statement at least once. n Is this a good or bad idea? int abs( int x ) { if ( x >= 0 ) x = 0 - x; return x; } n If we execute this with {x=0} the criteria will be satisfied - every statement will be executed. But the error will not be detected. Why?

Branch Coverage n Each branch is executed once. Each decision is evaluated to T and F at least once. n This will usually imply statement coverage. n Does this criteria “find” the fault in the abs() function?

Branch Coverage (cont’d) n Branch coverage has problems with multiple (compound) conditions within a decision (&&, ||) n Consider the following: /* x is valid if 0<=x<=100 */ int check( int x ) { if( x>=0 && x<=200 ) return TRUE; else return FALSE; } n If we test this module with x  {-5,5}, then the error will not be revealed.

Path Testing n A path is a sequence of branches executed when a program is executed from start to finish. n Also called a logical path. n The problem with path testing is that even small programs can have very large numbers of paths, especially if they contain loops.

Path Testing (cont’d) n Consider the following flowchart:

Path Testing (cont’d) n There are 100 trillion possible paths to traverse! This would take over 3,000 years. n Obviously, we need to limit the number of paths to execute...

Basis Path Testing n Involves the construction of a flow graph and using the graph to determine a basis set of execution paths for the program. n McCabe’s Cyclomatic Complexity n C.C. = # paths in the basis set.

Constructing a Flow Graph n “primitiv es” sequen ce if-else FT while- do

Flow Graphs (cont’d) more “primitives” do- until case (switch)

Flow Graphs (advanced) if( a || b ) x( ); else y( ); x a yx b if( a && b ) x( ); else y( ); x a yx b T T F F

Example: n Procedure Sort 1) while records remain do read record 2) if record.field1 = 0 3) process record; store record; increment counter; 4) else if record.field2 = 0; 5) reset counter; 6) else process record; 7) endif endif 8) enddo 9) end

Procedure Sort Flow Graph

Cyclomatic Complexity n Once we’ve found the flow graph (G), we can compute the cyclomatic complexity, V(G). n It is simply the number of enclosed regions in the graph plus 1: V(G) = # Regions + 1 V(G) = = 4 n This means that we need 4 independent paths through the graph.

Independent Paths n By independent paths, we mean that no path is a combination of the others in the set. n When constructing them, each “new” path should include some node not previously used. n A good guidelne is to start with a simple path, and change each condition.

Example n Path 1: 1-9 n Path 2: n Path 3: n Path 4: n Note: is not independent because it simply combines paths 2 and 4. n If we choose data to force execution of these 4 basis paths, then we will be executed and every condition will be evaluated on TRUE and FALSE sides.

Loop Testing n Path testing may only force the execution of loops with one TRUE and one FALSE value. n Since loops are the cornerstone of many algorithms, we should derive specific tests to test loop constructs.

Loop Classes n Simple n Nested n Concatenated n Unstructured (FORTRAN, COBOL)

Testing Simple Loops n Assume N is the maximum number of allowable passes through the loop: 1. Skip the loop entirely 2. One pass through the loop 3. Two passes through the loop 4. m passes through the loop where m < N 5. N - 1, N + 1, N passes.

Testing Nested Loops 1. Start at innermost loop. 2. Perform simple loop test for innermost loop. 3. Work outward, conducting tests for the next loop, keeping outer loops at minimum values and other nested loops to “typical” values. 4. Continue until all loops have been tested.

Testing Concatenated Loops n Independend vs. dependent n See text pg. 399

Testing Unstructured Loops n rewrite them

Test Case Generation n Manual generation is tedious. n Some automated methods: u Random generation until desired coverage is reached -- inefficient. u Tools to tell the tester what paths have been covered. Then tester can derive additional test cases to increase coverage.

Instrumentation n Programs are instrumented with probes. n Probes keep track of which branches have been executed. n A preprocessor inserts the probes at strategic locations. n The program is executed on the test data. n A postprocessor analyzes the results. n btool, bmake, bsummary, breport