CSC 480 Software Engineering Testing - I

Plan project Integrate & test system Analyze requirements Design Maintain Test units Implement Software Engineering Roadmap: Identify corporate practices Test units (parts) separately - use implementations - apply discipline - gain coverage Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Learning Goals Understand meaning of unit testing Distinguish black box vs. white box testing Attain proper test coverage Validation and Verification Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Verification: are we building the thing right? Validation: are we building the right thing? Meaning of “V&V” (Boehm) Graphics reproduced with permission from Corel.

1. Introduction to unit testing

Golden Rules of Testing Goal of testing: maximize the number and severity of defects found per dollar spent –thus: test early

Golden Rules of Testing Goal of testing: maximize the number and severity of defects found per dollar spent –thus: test early Limits of testing: Testing can only determine the presence of defects, never their absence –use proofs of correctness to establish “absence” Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Testing: the Big Picture Function Module Module combination 2. Integration tests 3. System tests 1. Unit tests

Testing: the Big Picture Methods Combinations of methods in class Packages of classes OO: Include use-cases Function Module Module combination 2. Integration tests 3. System tests 1. Unit tests

Elaboration Unified Process InceptionConstructionTransition Requirements Analysis Design Implemen- tation Test Jacobson et al: USDP Prelim. iterations Iter. #1 Iter. #n Iter. #n+1 Iter. #m Iter. #m+1 Iter. #k …..

Elaboration Unified Process InceptionConstructionTransition Requirements Analysis Design Implemen- tation Test Jacobson et al: USDP Prelim. iterations Iter. #1 Iter. #n Iter. #n+1 Iter. #m Iter. #m+1 Iter. #k ….. Unit Tests Integration tests... System tests

RoadMap for Unit Testing 1. Plan for unit testing -- see section SSS Requirements Test results Detailed design..

RoadMap for Unit Testing 1. Plan for unit testing -- see section SSS Requirements Unit test plan 2. Acquire test set -- see section SSS Products of prior testing 3. Execute unit test -- see section SSS Test set Test results Code under test Detailed design Identify largest trouble spots IEEE, 1986

2. Test types

Black-, Gray-, & White-box Testing Black box … requirements* Actual output compared with required * from previous phase Input determined by... Result

Black-, Gray-, & White-box Testing Black box … requirements Actual output compared with required output White box Gray box … requirements & key design elements Input determined by... Result …design elements Confirmation of expected behavior As for black- and white box testing Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Test Input Possibilities interest rate 0% 25% principal $100$100M inflation estimate 1% 20% Infinitely many legal values: choose a finite sample. Infinitely many illegal values: choose a finite sample. Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Test Input Partitioning and Boundaries interest rate 0% 25% principal $100$100M inflation estimate Boundaries 1% 20% Equivalence partitions An illegal region Range of valid inputs Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Testing Ranges: Elementary Cases 1. within range 2. at the boundaries of the range 3. outside the range (“illegal”) range Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Covering Every Statement is Not Sufficient (Myers) u>1 and v==0 x = x/u u==2 or x>0 ++x No Yes No Yes Required program

Covering Every Statement is Not Sufficient (Myers) u>1 and v==0 x = x/u u==2 or x>0 ++x No Yes Code attempt to implement flowchart if( (u>1) && (v==0) )(1) x = x/u; (2) if( (u==2) || (x>3) )(3) ++x; (4) u=2, v=0 and x=3 executes every line (1) - (4) gives the correct output x= 2.5 However, line (3) is wrong No Yes Required program

Paths to be Checked Parameter & settings make sense? Parameter name too long? N YN Set _name to “defaultName" Y Truncate name Set _name to parameter

Paths to be Checked Parameter & settings make sense? Parameter name too long? N YN Decision Coverage Set _name to “defaultName" Y Truncate name Set _name to parameter Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Flow Chart vs. Flow Graph

Binary search (Java) This is an encapsulation of a binary search function that takes an array of ordered objects and a key and returns an object with 2 attributes namely index - the value of the array index found - a boolean indicating whether or not the key is in the array An object is returned because it is not possible in Java to pass basic types by reference to a function and so return two values the key is -1 if the element is not found

Binary search flow graph

1, 2, 3, 8, 9 1, 2, 3, 4, 6, 7, 2 1, 2, 3, 4, 5, 7, 2 1, 2, 3, 4, 6, 7, 2, 8, 9 Test cases should be derived so that all of these paths are executed A dynamic program analyser may be used to check that paths have been executed Independent paths

Cyclomatic Number In graph-theoretic terms, the cyclomatic number of a graph G is defined as C = E – N + 2P where –N is the number of nodes –E is the number of edges –P is the number of connected components (usually, 1)

Cyclomatic Complexity McCabe suggested that the number C can be used as a source-code metric –that reflects the complexity of a program’s control flow C defines the number of linearly independent paths in a program –It can be used as an upper bound for the number of tests that must be conducted

Alternative Ways to Compute C There are other ways that can be used to determine the cyclomatic number C –C = P + 1 where P is the number of predicate nodes contained in the flow graph –C = R where R is the number of regions into which the flow graph divides the plane