IMSE Week 18 White Box or Structural Testing Reading:Sommerville (4th edition) ch 22 orPressman (4th edition) ch 16

White Box or Structural Testing l also known as glass box testing, each name implying that the internal structure of the program is visible, not hidden inside a black box l tester makes use of the implementation, analysing the source code to derive test cases l complements black box testing, white box test cases are added to the black box ones l provides a more comprehensive level of testing

Paths l program is inspected to determine the paths through it l a path is a way of getting from the start to the end of a program l test cases are selected which force execution through all/most/some of these paths l flow charts may be used to picture the paths through the program, showing sequences, selections and iterations l flow graphs are sometimes used, similar to flow charts but simpler l both are very useful for path testing, although not essential - simply inspecting the source code may be equally effective

An Example - the Pseudocode repeat input value if value is less than zero display “value is negative” else if value is greater than zero display “value is positive” else display “value is zero” 7 times

The Example as a Flowchart

The Example as a Flowgraph

No. of Paths l how many different ways are there of getting from the start to the end of our example program? l one way to find out is to –first number the boxes, diamonds and circles on the diagram –second, list their order of execution for each path through the program or module l for example, path 1 = 1,2,4,7,1,2,4,7,.....1,2,4,7,8

Selective Path Testing l in all non-trivial programs it is impossible to test all the paths through the program, especially when loops occur l instead do not test all paths, only take selected ones l aim to execute all statements at least once l all conditional statements must therefore be tested for both true and false cases l note that even though every statement may have been executed, a bug may not have been detected if it is only caused by a particular path combination

Basis Path Testing l method of selecting a comprehensive set of paths for testing l the paths selected using this method are all the independent paths i.e. those with some unique part l step 1 - determine how many independent paths there are l no. of independent paths = the cyclomatic complexity or V(G) of the module l cyclomatic complexity = no. of simple decisions + 1 or no. of enclosed areas + 1 l in our example above, V(G) =

Basis Path Testing (contd) l step 2 - now that we know how many independent paths there are, list them: l step 3 - derive test cases to exercise these paths l all conditions will be tested for both true and false l all loops will be executed at least once

Cyclomatic Complexity l useful software metric, measures the complexity of a module l the higher the V(G) the higher the probability of errors l a max V(G) of 10 is suggested for 1 module, above which adequate testing is extremely difficult l originated by Thomas McCabe who is also responsible for basis path testing l even though we know how many test cases are needed to execute every statement, identifying which test cases are needed may still be difficult l tools exist (e.g. dynamic program analysers) which keep track of those sections of a program that have been executed

White Box Testing of Loops minimum conditions to be tested for simple loops: l skip the loop entirely l only one pass through the loop l two passes through the loop if n is the maximum no. of allowable passes: l m passes through the loop, where m < n l (n - 1), n, and (n + 1) passes through the loop

White Box Testing of Data l do not concentrate solely on control flow, basis path testing and cyclomatic complexity in white box testing l data complexity should also be a major concern l e.g. a 7 element array containing the days of the week l the source code may access the array via a loop indexing the elements: loop exit when Days (i) = A; i = i + 1; end loop l this has a V(G) of 1, but needs at least 7 test cases

White Box versus Black Box l question: which strategy (if any) is better, white box or black box testing?

Advantages of Black Box Testing black box is better in that it discovers: l missing functions l functions which don’t correspond to the spec l weaknesses in the spec (e.g. “any integer” rather than to 32767) l interface errors between modules l performance weaknesses l initialisation errors l termination errors

Advantages of White Box Testing white box is better than black box in that: l it goes into a level of detail far beyond just looking at the output from a program, and is therefore clearly better than black box testing on individual small modules l it discovers extra non-specified functions that black box wouldn’t know to look for - such extras can only be discovered by inspecting the code (e.g. non-documented illegal options supplied by a disreputable programmer)

Conclusion l the list of advantages for black box testing looks more impressive but the first white box advantage concerning level of detail makes white box methods indispensable l but this level of detail has disadvantages, and black box becomes more relevant as modules are aggregated and become larger and larger l then we can’t think of path coverage and must be satisfied with simply determining that the product seems to do what we intended l answer: both are essential, they complement each other