UNIT TESTING

Plan project Integrate & test system Analyze requirements Design Maintain Test units Implement Software Engineering Roadmap Identify corporate practices Test units (parts) separately - use implementations - apply discipline - gain coverage Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Learning Goals Understand meaning of unit testing Distinguish black box vs. white box testing Attain proper test coverage Learn a testing standard Inspect a unit test plan Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

1. Introduction to unit testing

Golden Rules of Testing Goal of testing: maximize the number and severity of defects found per dollar spent … thus: test early Limits of testing: Testing can only determine the presence of defects, never their absence –use proofs of correctness to establish “ absence ” Who should test: Someone other than the developer. –Why? Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Testing: the Big Picture Methods Combinations of methods in class Packages of classes OO: Include use-cases Function Module Module combination 2. Integration tests 3. System tests 1. Unit tests

Elaboration Unified Process InceptionConstructionTransition Requirements Analysis Design Implemen- tation Test Jacobson et al: USDP Prelim. iterations Iter. #1 Iter. #n Iter. #n+1 Iter. #m Iter. #m+1 Iter. #k ….. Unit Tests Integration tests... System tests

RoadMap for Unit Testing 1. Plan for unit testing Requirements Unit test plan 2. Design test cases and acquire test I/O pairs Generate I/O pairs (often products of prior testing) 3. Execute unit test Test set Test results Code under test Detailed design Identify largest trouble spots IEEE, 1986

2. Test types

Black-, Gray-, & White-box Testing Black box … requirements Actual output compared with required output White box Gray box … requirements & key design elements Input determined by... Result …design elements Confirmation of expected behavior As for black- and white box testing Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Black-box testing

Equivalence Partitioning Input data and output results often fall into different classes where all members of a class are related Each of these classes is an equivalence partition where the program behaves in an equivalent way for each class member Test cases should be chosen from each partition

Equivalence Partitioning

Test Input Possibilities interest rate 0% 25% principal $100$100M inflation estimate 1% 20% Infinitely many legal values: choose a finite sample. Infinitely many illegal values: choose a finite sample. Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Test Input Partitioning and Boundaries interest rate 0% 25% principal $100$100M inflation estimate Boundaries 1% 20% Equivalence partitions An illegal region Range of valid inputs Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Testing Ranges: Elementary Cases 1. within range 2. at the boundaries of the range 3. outside the range ( “ illegal ” ) range Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Requirement: The system must accept a 5- digit integer between and and perform various functions on the values based on the following equivalence partitions: = Which test cases should be chosen? Consider the boundaries of these sets … 00000, 09999, 10000, 99999, , Equivalence Partitioning – Example

Equivalence Partitions Between and Less than More than Input values

White Box Testing Every statement of code should be covered by at least one test However, this is not sufficient since the correct opeation of a unit of code depends upon sequences of statements

Covering Every Statement is Not Sufficient (Myers) u>1 and v==0 x = x/u u==2 or x>0 ++x No Yes No Yes Required program

Covering Every Statement is Not Sufficient (Myers) u>1 and v==0 x = x/u u==2 or x>0 ++x No Yes Code attempt to implement flowchart if( (u>1) && (v==0) )(1) x = x/u; (2) if( (u==2) || (x>3) )(3) ++x; (4) u=2, v=0 and x=3 executes every line (1) - (4) gives the correct output x= 2.5 However, line (3) is wrong SO THIS IS NOT A THOROUGH TEST No Yes Required program

Paths to be Checked Parameter & settings make sense? Parameter name too long? N YN Set _name to “ defaultName" Y Truncate name Set _name to parameter

Paths to be Checked Parameter & settings make sense? Parameter name too long? N YN Decision Coverage Set _name to “ defaultName" Y Truncate name Set _name to parameter Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Decision Coverage via Path testing The objective of path testing is to ensure that the set of test cases is such that each path through the program is executed at least once The starting point for path testing is a program flow graph that shows nodes representing program decisions and arcs representing the flow of control Statements with conditions are therefore nodes in the flow graph

Describes the program control flow. Each branch is shown as a separate path and loops are shown by arrows looping back to the loop condition node Used as a basis for computing the cyclomatic complexity Cyclomatic complexity = Number of edges - Number of nodes +2 Program flow graphs

The number of tests to test all control statements equals the cyclomatic complexity Cyclomatic complexity equals number of conditions in a program Useful if used with care. Does not imply adequacy of testing. Although all paths are executed, all combinations of paths are not executed Cyclomatic complexity

Binary search (Java)

Binary search flow graph

1, 2, 3, 8, 9 1, 2, 3, 4, 6, 7, 2 1, 2, 3, 4, 5, 7, 2 1, 2, 3, 4, 6, 7, 2, 8, 9 Test cases should be derived so that all of these paths are executed A dynamic program analyser may be used to check that paths have been executed Independent paths

Grey Box Testing Combination of Black and White box testing Gray box … requirements & key design elements Correct output and expected behaviour

Pre-conditions satisfied, key element in array Pre-conditions satisfied, key element not in array Pre-conditions unsatisfied, key element in array Pre-conditions unsatisfied, key element not in array Input array has a single value Input array has an even number of values Input array has an odd number of values Grey Box Testing Example Binary search - equiv. partitions

Binary search equiv. partitions

Binary search - test cases

3. Planning unit tests

Plan for Unit Testing 1. Decide on the philosophy for unit testing –individual engineer responsible (common)? –reviewed by others? –designed & performed by others? 2. Decide what / where / how to document –individual ’ s personal document set (common)? –how / when to incorporate into other types of testing? –incorporate in formal documents? –use tools / test utilities? 3. Determine extent of unit testing (i.e., in advance). –do not just “ test until time expires ” –prioritize, so that important tests definitely performed 4. Decide how and where to get the test input 5. Estimate the resources required –use historical data if available 6. Arrange to track time, defect count, type & source Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

4. Checklists and examples for Method testing

Perform Method Testing (Humphrey) 1/2 1. Verify operation at normal parameter values (a black box test based on the unit ’ s requirements) 2. Verify operation at limit parameter values (black box) 3. Verify operation outside parameter values (black box) 4. Ensure that all instructions execute (statement coverage) 5. Check all paths, including both sides of all branches (decision coverage) 6. Check the use of all called objects 7. Verify the handling of all data structures 8. Verify the handling of all files Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Perform Method Testing (Humphrey) 2/2 9. Check normal termination of all loops (part of a correctness proof) 10. Check abnormal termination of all loops 11. Check normal termination of all recursions 12. Check abnormal termination of all recursions 13. Verify the handling of all error conditions 14. Check timing and synchronization 15. Verify all hardware dependencies Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

5. Checklists and examples for class testing

1. Exercise methods in combination –2-5, usually –choose most common sequences first –include sequences likely to cause defects –requires hand-computing the resulting attribute values 2. Focus unit tests on each attribute –initialize, then execute method sequences that affect it 3. Verify that each class invariant is unchanged –verify that the invariant is true with initial values –execute a sequence (e.g., the same as in 1.) –verify that the invariant still true 4. Verify that objects transition among expected states –plan the state / transition event sequence –set up the object in the initial state by setting variables –provide first event & check that transition occurred. etc. Perform Class Unit Tests Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Encounter State-Transition Test Sequence 1 of 2 Waiting Preparing test step 1 Verify that the game is initially in Preparing state (by checking on the class membership of gameStateI) Player dismisses qualities menu

Encounter State-Transition Test Sequence 1 of 2 Waiting Preparing test step 1 test step 2 test step 3 Verify that the game is initially in Preparing state (by checking on the class membership of gameStateI) Dismiss the quality menu, and verify that the game is in Waiting state. Move the player character to an adjacent area, and verify that the game is still in Waiting state. Player dismisses qualities menu Move to adjacent area Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Player dismisses qualities menu Character enters area inhabited by an opponent Move to adjacent area Complete Encounter State-Transition Test Waiting Preparing Engaging Reporting Player dismisses encounter report menu Encounter completed 6 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Defect testing and debugging are distinct processes Inspection and testing is concerned with establishing the existence of defects in a program Debugging is concerned with locating and repairing these errors Debugging involves formulating a hypothesis about program behaviour then testing these hypotheses to find the error Testing and debugging

The debugging process