Testing an individual module Unit Testing Testing an individual module

Rules of Testing Testing is a process of executing a program with the intent of finding an error. A good test case is one that has a high probability of finding an as yet undiscovered error. A successful test is one that uncovers an as yet undiscovered error. Testing cannot show the absence of defects - it can only show that defects are present. Testing is a process of finding errors - if no errors are found then the tests are inadequate.

Testing objective The objective of testing is to design tests that systematically uncover different classes of errors and do so with a minimum amount of time and effort Secondary benefit is that testing demonstrates that the software works according to the specification

Debugging Testing establishes the presence of defects, debugging is the process of locating and correcting the defects. Debugging is the most unpredictable part of the testing process - an error can take 1 hour, 1 day, or 1 week to locate and correct. It is the uncertainty in debugging that makes testing unpredictable and difficult to schedule.

Testing process Evaluation Testing Debug Test Results Expected Results Test Cases Test Results Expected Results Defects Corrections

Designing Tests A test case is a particular test that the software is to perform. A test case includes: test conditions - what is being tested. test data - data that is input to the software to make sure that the test conditions will happen. expected results - what the software, if working correctly, is expected to do.

Test Case Design Methods There are two main methods of testing: Black box testing (functional testing) knowing the specified function that a system has been designed to perform, tests can be constructed to demonstrate that each function is working properly. White box testing (structural testing) knowing the internal workings of a system, tests can be constructed to make sure that the internal operation of the system performs according to the specification.

White Box Testing White box testing uses close examination of internal procedural detail, i.e. How the software works. Test cases are derived to ensure that all statements in the program have been executed at least once. Appears that white box testing leads to 100% correct programs. Exhaustive testing requires every possible input to a program to be executed. Example, a program to calculate the number of days from a given date. Specified date input in day, month, year. Caters from 1900 to 2000. Exhaustive testing is 31*12*1000 = 370,000 tests. 9

Type of white box testing. In practice this is impossible as even small programs number of possible paths can be very large. A limited number of important logical paths are selected and tested. Type of white box testing. Path testing. Condition testing. Loop testing.

Path Testing Path testing means each independent path within a program. Does not test all possible combinations of all paths through a program which is impossible except for simple programs without loops. Starting point is a flow graph - a skeletal model of all paths through a program. Nodes represent decisions. Lines represent flow of control. Constructed by replacing program control statements with equivalent diagrams. Procedure Sort 1. do while not end of records 2 read record 3. if record field 1 = 0 4. then process record 5 store in buffer 6 increment counter 7. else if record field 2 = 1 8 then reset counter 9 else process record 10 store in file 11 endif 12 endif 13 enddo 14 end Example above CC = 3: Path 1: 1 2 3 4 5 6 13 1 2 14 Path 2: 1 2 3 7 8 11 12 13 1 2 14 Path 3: 1 2 3 7 9 10 11 12 13 1 2 14 10

A Flow Graph… If-Then- Else Repeat-Until Do-While Case

Cyclomatic Complexity is a measure that tells us how many paths to look for CC(G) = E - N + 1 E = number of edges N = number of nodes Prepare test data and expected results for each path

Condition Testing Steps to design tests using path testing: using the design or code as a foundation, draw a corresponding flow graph determine the cyclomatic complexity to find the number of paths identify the actual paths define test data and expected result for each path (test case) Examples b1 AND b2 (T,T), (T,F), (F,T) * b1 AND (e1 = e2) (T,=), (F,=), (T,<), (T,>) (e1 > e2) AND (e3 = e4) (>,=), (=,=), (>,<), (>, >), (<,=) * (F,F) not needed as would be caught in test. E.g. if (Green AND Green) then GO, use tests (Green, Green), (Red, Green), (Green, Red). If coded incorrectly as if (Red AND Red) then GO, test (Green, Green) would fail. 11

Condition testing focuses on testing each condition in a program. A simple condition is a Boolean variable, variable with value true or false, or a relational expression - A operator B where operator is one of <,=,>,<>,<=,>=. a compound condition is one or more simple conditions, using operators AND, OR, NOT e.g.. (a = x) AND (b < y) Many potential errors in a compound expression.

Design test conditions to test each true or false branch for Boolean expression or variable. execute each simple condition at least once. test each possible outcome from a relational expression. e.g. e1 < e2 outcomes are <,>,=.

Loop Testing A white box testing technique that focuses on testing loops 4 different types of loops Simple Nested Concatenated and Unstructured nested loop conditions - impractical to apply conditions for simple loop as the number would grow geometrically 12

Simple Loop test conditions for a simple loop ( where n - max allowable passes thru loop): skip loop entirely. only one pass through loop. two passes through loop. m passes through loop where m<n. n, n+1 passes through loop.

Nested loop - test conditions Start at innermost loop, set all others to minimum counter values and conduct simple loop tests for inner loop. Perform random tests for out of range or excluded values. Work outwards, keeping outwards loops at minimum counter values and inner loops at typical values.

Concatenated and unstructured loops independent - test as simple loops dependent - test as nested loops unstructured loops - redesign

Black Box Testing System is like a black box whose behaviour is determined by examining inputs and the related outputs. Black box tests are designed to demonstrate that: The software functions are operational. Input is properly accepted. Output is correctly produced. The integrity of external information (e.g. Data files) is maintained; Test cases are derived from the specification. Example of a test case: Module that reads in numbers from a file one by one and writes out the numbers to the screen in reverse order. Condition: normal execution Test data: file with 2,6,4,9 Expected results: numbers 9,4,6,2 written out to screen in that order. Condition: Empty file Test data: create an empty file, no records Expected result: Message output that the file was empty 5

A Black Box Test: Attempts to find errors in the following categories Incorrect or missing functions Interface errors Errors in data structure or external database access Performance errors Initialisation and termination errors Three types of black box testing Equivalence partitioning Boundary value analysis Comparison testing

Equivalence Partitioning A testing method that divides the input domain of a program into classes of data from which test cases can be derived. A test case uncovers a class of errors (e.g. incorrect processing of negative numbers) that might otherwise require a lot of individual tests to identify. Design of tests requires identification of equivalence classes for specific input conditions. Equivalence class represents a set of valid or invalid states for input conditions. Example: of equivalence partitioning Example: Online banking, customer can phone the bank, provide a password and can operate specific commands. Input from user is - account number (6 digit number not starting with 0,1 or 9) - password (4 digit number) - commands (account enquiry, order cheques, bill pay etc.) Input conditions: account number: range, value > 200000 and < 900000 password: boolean, may or may not be present value, 4 digit string command: set, containing valid commands Equivalence classes: inputs with account numbers > 200000 and < 900000 inputs with account numbers <= 200000 inputs with account numbers >= 900000 inputs with a valid password inputs with an invalid password inputs with no password inputs with valid command inputs with an invalid command 6

Guidelines - an Input Condition Typically Is Range of values which results in 3 classes, valid input within range inputs above range inputs below range Specific value - 2 classes, valid value invalid value Set of values - 2 classes, Inputs in the set Input outside the set Boolean value - 2 classes, Inputs that are true Inputs that are false

Boundary Value Analysis A greater number of errors tend to occur at boundaries of the input domain than in the centre. BVA leads to tests that exercise bounding values. Compliments equivalence partitioning- rather than selecting any element from an equivalence class, select the elements that are at the edges of the class. Example : use values of 20000, 200001, 900000, 900001 7

Guidelines range from a to b - select a, b and values just above a and b a number of values - use the minimum and maximum numbers and numbers close to min and max data structures with limits - select values that test boundary of size, e.g.array Also apply guidelines to output conditions, e.g. A report that has a table of results: design tests that will ensure producing the max and min entries in the table

Comparison Testing In situations where reliability of software is critical, (e.g. aircraft avionics, nuclear power plant control) redundant hardware and software can be used to minimize the possibility of error. Separate teams develop independent versions of the system from the same specification. Each version is tested with the same test data (developed from other black box techniques) to ensure identical output . If output is different, each application is examined to identify source of error.

This is called comparison testing or back-to-back testing. Not foolproof, as multiple versions developed from same specification which could be incorrect. All versions could display same incorrect output. Can be used with an existing system also - test new software against existing software.

Unit Testing Technique A practical testing technique that uses aspects of different testing methods. Objective of unit testing is to test that each module in a larger system works independently. 13

Method Start from the program algorithm or detail design. Identify test conditions. Identify test cycles or cases - a run of the program. For each test case: Identify test data - data that will be input to force the conditions. Identify expected results - the results that are expected if the program worked correctly. Document a test plan - instructions documenting the actions, test data and expected results.

Identify test conditions Identify the different conditions to be tested using a combination of the different methods Condition testing Loop testing File handling - conditions to cater for end of file, not end of file and empty or nonexistent files Array handling - conditions to cater for empty arrays, single value arrays; Arrays of different sizes, accessing the first, middle and last element in array Error handling - non normal processing Different branches (paths) in the program automatically handled due to (i), (ii) and (iii) pgs 40-43 in 95/96 notes got through the example - hand out copies of some to class. 14