Software Testing Techniques

Software Testing Testing is the process of exercising a program with the specific intent of finding errors prior to delivery to the end user.

Software Development Process Tangible Product Abstract Concept Testing Tries hard to demolish the software

Developer must discard the preconceived idea that his software is always correct.

Software Testing Objectives Uncovering different cases of errors Needing minimum effort and time Showing that software functions work according to specification It cannot show absence of errors or defects. It can only show that errors are present

Software Testing Principles Customer thinks - most serious defect is not meeting his requirements All tests can be planned and designed before any coding begins All tests should be traceable to customer requirements Tests should be planned long before testing begins The “pareto” principle applies to software testing Exhaustive testing is not possible To be most effective, testing should be conducted by a third party 80 % of errors originate from 20% of programs Too many paths in a normal -size program

Who Tests the Software? developer independent tester Understands the system Must learn about the system, but, will test "gently" but, will attempt to break it and, is driven by "delivery" and, is driven by quality

What is a GOOD Test? Has a high probability of finding an error It must not be redundant. Every test must have a different purpose It should neither be too simple, nor too complex.

What Testing Shows errors requirements conformance performance an indication of quality

Testability Operability—it operates cleanly Observability—the results of each test case are readily observed Controlability—the degree to which testing can be automated and optimized Decomposability—testing can be targeted Simplicity—reduce complex architecture and logic to simplify tests Stability—few changes are requested during testing Understandability—of the design

Test Case Design Design of test cases is as challenging as designing the software itself A rich variety of test case design methods have evolved Methods provide a systematic approach o testing Methods provide a mechanism for that ensures the completeness of tests and high likelihood of finding an error

Software Testing white-box black-box methods methods Methods Strategies

White Box Testing Close examination of procedural detail Logical paths through the software are tested by providing test cases that exercise specific sets of conditions and/or loops Status of the program is examined at various points to determine if the actual status matches the expected status

Does White Box Testing lead to 100% correct program? The answer is: No It is not possible to exhaustively test every program path because the number paths is simply too large

Exhaustive Testing loop < 20 X 14 There are 10 possible paths! If we execute one test per millisecond, it would take 3,170 years to test this program!!

Selective Testing Selected path loop < 20 X

Black Box Testing Know the specific function that the program is designed to perform Test that each function is fully operational Search for more errors in the functions These tests are conducted at the software interface Whether input is properly accepted and output is correctly produced Integrity of external information (database) is maintained Examines fundamental aspects of the system with little regard to internal logical structure

The attributes of both black-box and white-box testing can be combined to provide an approach that validates the software interface and selectively ensures that the internal workings of the software are correct

White Box Testing Is also known as “Glass Box Testing” Uses the control structure of the procedural design to derive test cases If a= b then do c else do v Glass Box

White Box Testing Guarantees that all independent paths within a module have been exercised at least once Exercise all logical decisions on their true and false sides Exercise all loops at their boundaries and within their operational bounds Exercise internal data structures to ensure their validity

Test Case Design "Bugs lurk in corners and congregate at boundaries ..." Boris Beizer OBJECTIVE to uncover errors CRITERIA in a complete manner CONSTRAINT with a minimum of effort and time

Why Cover? logic errors and incorrect assumptions are inversely proportional to a path's execution probability we often believe that a path is not likely to be executed; in fact, reality is often counter intuitive typographical errors are random; it's likely that untested paths will contain some

Why need White-box testing? Logic errors occur more frequently in the less probable paths of execution Everyday processing tends to be well understood while “special case “ processing tends to fall into cracks We often believe that a logical path is not likely to be executed when , in fact it may be executed on a regular basis Typos are random. It is as likely that a type will exist on an obscure logical path as on a mainstream path.

White-Box Testing ... our goal is to ensure that all statements and conditions have been executed at least once ...

Basis Path Testing It is white-box testing technique Proposed by Tom Macabe Derives a logical complexity measure Test cases derived to exercise the basis set are guaranteed to execute every statement in the program at least once during testing Uses flow-graph notation Each structured construct has corresponding flow graph symbol

Structured Constructs if If-then-else else then Sequence Selection While-do Repeat until case Loops

Flow Graph Symbols if while Sequence until case

Flow Graph Symbols Each circle is called a flow graph node , represents one or more procedural statements A sequence of process boxes and a decision diamond can map into a single node An edge (arrow) must terminate at a node Areas bounded by edged and nodes are called regions Area outside graph is also a region

Flow Graph for Compound Conditions A compound condition occurs when one or more Boolean operators (AND,OR) is present in a conditional statement Separate node is created for each condition Each condition node is known as predicate node Each predicate node has two or more edges leaving it x b x y If a or b then procedure x else procedure y endif

Cyclomatic Complexity Very useful software metric Foundation on graph theory Quantitative measure of the logical complexity of the program It defines the number of independent paths in the basis set of a program Provides s with an upper bound for the number of tests to be conducted Ensures that all statements have been executed at least once

Cyclomatic Complexity A number of industry studies have indicated that the higher V(G), the higher the probability or errors. modules V(G) modules in this range are more error prone

Computation of Cyclomatic Complexity V(G) for flow - graph G There are three ways: V(G) = P+1 where P - no. of predicate nodes in G V(G) = E - N + 2 where W - no. of edges in G N - no. of nodes in G V(G) = R where R- no. of regions in G Given flow-graph has 4 regions V(G) = 11-9 + 2 = 4 V(G) = 3 + 1 = 4

Independent Path Path 1: 1-11 Path 2: 1-2-3-4-5-10-1-11 Any path through the program that introduces at least one new set of processing statements or a new condition Must move along at least one edge that has not been traversed before the path is defined Each path introduces a new edge There are Four paths for the given example: Path 1: 1-11 Path 2: 1-2-3-4-5-10-1-11 Path-3: 1-2-3-6-8-9-10-1-11 Path 4: 1-2-3-6-7-9-10-1-11

Basis Path Testing Next, we derive the independent paths: 1 2 3 4 5 6 7 8 independent paths: Since V(G) = 4, there are four paths Path 1: 1,2,3,6,7,8 Path 2: 1,2,3,5,7,8 Path 3: 1,2,4,7,8 Path 4: 1,2,4,7,2,4,...7,8 Finally, we derive test cases to exercise these paths.

Deriving Test Cases Using the code as the foundation , draw a corresponding flow graph Determine the cyclomatic complexity of the flow graph Determine a basis set of linearly independent paths Prepare test cases that will force execution of each path in the basis set Execute each test case and compared to expected results Some paths cannot be tested independently and must be tested as part of another path

Condition Testing It is a design method that exercises the logical conditions (Consisting of arithmetic, relational and Boolean expressions and operators) in a program module Types of error found in a condition: Boolean operator error Boolean variable error Boolean parenthesis error Relational operator error Arithmetic expression error

Condition Testing Branch Testing Domain Testing For a compound condition C, the true and false branches of C and every simple condition in C need to be executed at least once Domain Testing Requires three or four tests for a relational expression To detect errors in E1<relational-operator>E2 three tests are required to detect error in the operator putting as minimum possible different values in E1 and E2 will help to detect errors in E1and E2

BRO Testing All relational operators and boolean variables must occur only once Uses condition constraint for a condition C which is defined as (D1, D2, …Dn) where Di specifies thea constraint on he outcome of ith condition of C Boolean variables have constraint value True or False For relational expression symbols <,>, = are used to specify the conditions

BRO Testing - Examples Example 1: B1 & B2 C1= (D1,D2) = {(t,t),(t,f),(f,t)} Example 2: B1 & (E3=E4) C2= (D1,D2) = {(t,=),(t,>),(t,<),(f,=)} Example 3: (E1 > E2) & (E3=E4) C2= (D1,D2) = {(>,=),(<,=),(>,>),(>,<),(=,=)}

Dataflow Testing Selects test paths according to the location of definitions and uses of variables in the program (S-statement no.) DEF(S) = {X| statement S contains a definition of X} USE(S) = (X| statement S contains a use of X} Assume For if or loop statement DEF set is always empty The definition of var X at S is said to be live at statement S’ if the path S-S’ contains no other definition of X A definition-use chain of var Xis of the form [X,S,S’] where X is DEF(S) and USE(S’) and X is live at S’.

DU Testing Every DU chain should be covered at least once Useful for programs with nested if and loop statements Effective for error-detection Selecting test-paths is quite difficult Does not guarantee all branches of a program (e.g. if statement with no else statement and no definition in the then part.

DU Testing proc x b1: do while C1 if C2 then if C4 then B4; else B5; endif; else if C3 then B2; else B3; enddo; B6; end proc; Var X is defined in the last statement of blocks B1, B2, B3, B4, B5 and used in the first statements of B2, B3, B4, B5, B6 There are 25 DU chains of var X We need to cover only five paths from each B1, B2, B3, B4, B5 to B6

Loop Testing Loops are he cornerstone of vast majority of all software Focuses exclusively on the validity of loop constructs Four different classes of loops are defined: Simple loops Nested loops concatenated loops unstructured loops

Loop Testing Simple loop Nested Loops Concatenated Loops Unstructured

Simple Loops Set of Tests Skip the loop entirely Only one pass through the loop Two passes through the loop M passes through the loop where m < n (n – max no. pf allowable passes) N-1, n, n+1 passes through the loop

Nested Loops Number of possible test is impractically large if simple loop approach is adopted. Start at the innermost loop. Set all other loops to minimum value Conduct simple loop test for the innermost loop while outer loops have the minimum loop counter Work outward, conducting tests for the next loop , keeping outer loops at minimum value and nested loops at typical values Continue until all loops have been tested

Concatenated Loops Simple loop approach can be adopted if each of the loops is independent of the others If loop counter value of one loop is used as the initial loop counter value of another, then the loops are not independent. In such cases nested loop approach should be adopted