1. 8. UNIT TESTING

2. Plan project Integrate & test system Analyze requirements Design Maintain Test units Implement Software Engineering Roadmap: Chapter 8 Focus Identify corporate practices Test units (parts) separately - use implementations - apply discipline - gain coverage Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

3. Learning Goals of This Chapter Understand meaning of unit testing Distinguish black box vs. white box testing Attain proper test coverage Learn a testing standard Inspect a unit test plan Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

4. 1. Introduction to unit testing

5. Golden Rules of Testing Goal of testing: maximize the number and severity of defects found per dollar spent –thus: test early

6. Golden Rules of Testing Goal of testing: maximize the number and severity of defects found per dollar spent –thus: test early Limits of testing: Testing can only determine the presence of defects, never their absence –use proofs of correctness to establish “absence” Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

7. Testing: the Big Picture Function Module Module combination 2. Integration tests 3. System tests 1. Unit tests

8. Testing: the Big Picture Methods Combinations of methods in class Packages of classes OO: Include use-cases Function Module Module combination 2. Integration tests 3. System tests 1. Unit tests

9. Elaboration Unified Process InceptionConstructionTransition Requirements Analysis Design Implemen- tation Test Jacobson et al: USDP Prelim. iterations Iter. #1 Iter. #n Iter. #n+1 Iter. #m Iter. #m+1 Iter. #k …..

10. Elaboration Unified Process InceptionConstructionTransition Requirements Analysis Design Implemen- tation Test Jacobson et al: USDP Prelim. iterations Iter. #1 Iter. #n Iter. #n+1 Iter. #m Iter. #m+1 Iter. #k ….. Unit Tests Integration tests... System tests

11. RoadMap for Unit Testing 1. Plan for unit testing -- see section SSS Requirements Test results Detailed design..

12. RoadMap for Unit Testing 1. Plan for unit testing -- see section SSS Requirements Unit test plan 2. Acquire test set -- see section SSS Products of prior testing 3. Execute unit test -- see section SSS Test set Test results Code under test Detailed design Identify largest trouble spots IEEE, 1986

13. 2. Test types

14. Black-, Gray-, & White-box Testing Black box … requirements* Actual output compared with required * from previous phase Input determined by... Result

15. Black-, Gray-, & White-box Testing Black box … requirements Actual output compared with required output White box Gray box … requirements & key design elements Input determined by... Result …design elements Confirmation of expected behavior As for black- and white box testing Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

16. Test Input Possibilities interest rate 0% 25% principal $100$100M inflation estimate 1% 20% Infinitely many legal values: choose a finite sample. Infinitely many illegal values: choose a finite sample. Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

17. Test Input Partitioning and Boundaries interest rate 0% 25% principal $100$100M inflation estimate Boundaries 1% 20% Equivalence partitions An illegal region Range of valid inputs Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

18. Testing Ranges: Elementary Cases 1. within range 2. at the boundaries of the range 3. outside the range (“illegal”) range Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

19. Covering Every Statement is Not Sufficient (Myers) u>1 and v==0 x = x/u u==2 or x>0 ++x No Yes No Yes Required program

20. Covering Every Statement is Not Sufficient (Myers) u>1 and v==0 x = x/u u==2 or x>0 ++x No Yes Code attempt to implement flowchart if( (u>1) && (v==0) )(1) x = x/u; (2) if( (u==2) || (x>3) )(3) ++x; (4) u=2, v=0 and x=3 executes every line (1) - (4) gives the correct output x= 2.5 However, line (3) is wrong No Yes Required program

21. Paths to be Checked Parameter & settings make sense? Parameter name too long? N YN Set _name to “defaultName" Y Truncate name Set _name to parameter

22. Paths to be Checked Parameter & settings make sense? Parameter name too long? N YN Decision Coverage Set _name to “defaultName" Y Truncate name Set _name to parameter Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

23. public static boolean checkAssertion ( int loopSoFarP, int indexOfMaxP, double[] arrayP ) { // First establish following booleans: boolean b1M = true; /* means values of arrayP[ ] are lower than arrayP[ indexOfMaxP ] for all indices < indexOfMaxP */ if( indexOfMaxP != 0 ) // b1M true if max so far is first element for( int u = 0; u < indexOfMaxP; ++u ) b1M &= ( arrayP[u] < arrayP[ indexOfMaxP ] );.. Assertion-based testing for max() 1/2 Defining the assertion checker

24. public static boolean checkAssertion ( int loopSoFarP, int indexOfMaxP, double[] arrayP ) { // First establish following booleans: boolean b1M = true; /* means values of arrayP[ ] are lower than arrayP[ indexOfMaxP ] for all indices < indexOfMaxP */ if( indexOfMaxP != 0 ) // b1M true if max so far is first element for( int u = 0; u < indexOfMaxP; ++u ) b1M &= ( arrayP[u] < arrayP[ indexOfMaxP ] ); boolean b2M = true; /* means vals. of arrayP[ ] no higher than arrayP[ indexOfMaxP ] for indices indexOfMaxP … loopSoFarP */ for( int v = indexOfMaxP; v <= loopSoFarP; ++v ) b2M &= ( arrayP[v] <= arrayP[ indexOfMaxP ] ); Assertion-based Testing for max() Defining an Assertion Checker: 1 of 2: Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

25. if ( // Loop has progressed up to index loopSoFarP ( 0 <= loopSoFarP ) && (loopSoFarP < arrayP.length ) && // indexOfMaxP is the index <= loopSoFarP... ( 0 <= indexOfMaxP ) && ( indexOfMaxP <= loopSoFarP ) && b1M && b2M //... where the first max occurs ) {System.out.println( "Assertion valid" ); return true; } else {System.out.println( "Assertion invalid" ); return false; } Defining an Assertion Checker for max(): 2 of 2 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

26. /** Finds index & value of first of the largest array elements... */ public static void main( String[] mainArg ) { double[] a = getArray(); // Let I be the assertion … (see section tbd of chapter 7)... Establish I int i = 0; int k = 0; boolean validityM = checkAssertion( i, k, a ); // assertion test // Following preserves I.… terminates.… (section tbd of chapter 7) while( i != a.length - 1 ) { ++i; if( a[i] > a[k] ) k = i; validityM = validityM && checkAssertion( i, k, a ); // assertion test } System.out.println( "First max value is " + a[k] + " at index " + k ); System.out.println( "Validity: " + validityM ); // assertion report } Applying Assertion-based testing to max() Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

27. 3. Planning unit tests

28. Plan for Unit Testing 1. Decide on the philosophy for unit testing –individual engineer responsible (common)? –reviewed by others? –designed & performed by others? 2. Decide what / where / how to document –individual’s personal document set (common)? –how / when to incorporate into other types of testing? –incorporate in formal documents? –use tools / test utilities? 3. Determine extent of unit testing (i.e., in advance). –do not just “test until time expires” –prioritize, so that important tests definitely performed 4. Decide how and where to get the test input –see section tbd. 5. Estimate the resources required –use historical data if available 6. Arrange to track time, defect count, type & source Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

29. 4. Checklists and examples for method testing

30. Perform Method Testing (Humphrey) 1/2 1. Verify operation at normal parameter values (a black box test based on the unit’s requirements) 2. Verify operation at limit parameter values (black box) 3. Verify operation outside parameter values (black box) 4. Ensure that all instructions execute (statement coverage) 5. Check all paths, including both sides of all branches (decision coverage) 6. Check the use of all called objects 7. Verify the handling of all data structures 8. Verify the handling of all files Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

31. Perform Method Testing (Humphrey) 2/2 9. Check normal termination of all loops (part of a correctness proof) 10. Check abnormal termination of all loops 11. Check normal termination of all recursions 12. Check abnormal termination of all recursions 13. Verify the handling of all error conditions 14. Check timing and synchronization 15. Verify all hardware dependencies Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

32. 4. Checklists and examples for method testing

33. Relating Tests to Requirements & Design (1) D-Requirements 3.2.EC.1.2 Qualities of Encounter characters Every game character has the same set of qualities. Each quality shall be a non- negative floating point number with at least one decimal of precision.... (2) Design GameCharacter Requirements An abstract class with attribute name.....

34. Relating Tests to Requirements & Design (1) D-Requirements 3.2.EC.1.2 Qualities of Encounter characters Every game character has the same set of qualities. Each quality shall be a non- negative floating point number with at least one decimal of precision.... Characters GameCharacter EncounterCharacter adjustQuality() (2) Design GameCharacter Requirements An abstract class with attribute name... Encounter Characters... against this requirement Test this method... Test this class...... against this requirement Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

35. Unit test Applied to... ConcentrationStamina Results in a zero value? 1. Within range No: Test 1.1 Test 1.1.1 etc. Yes: Test 1.2 Test 1.2.1 etc. Partitioning of Range for Unit Testing 1 of 2 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

36. Unit test Applied to... ConcentrationStamina Results in a zero value? 2. Outside range No: Test 2.1 Test 2.1.1 etc. Yes: Test 2.2 Test 2.2.1 etc. Partitioning of Range for Unit Testing 2 of 2 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

37. 5. Checklists and examples for class testing

38. 1. Exercise methods in combination –2-5, usually –choose most common sequences first –include sequences likely to cause defects –requires hand-computing the resulting attribute values 2. Focus unit tests on each attribute –initialize, then execute method sequences that affect it 3. Verify that each class invariant is unchanged –verify that the invariant is true with initial values –execute a sequence (e.g., the same as in 1.) –verify that the invariant still true 4. Verify that objects transition among expected states –plan the state / transition event sequence –set up the object in the initial state by setting variables –provide first event & check that transition occurred. etc. Perform Class Unit Tests Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

39. Encounter State-Transition Test Sequence 1 of 2 Waiting Preparing test step 1 Verify that the game is initially in Preparing state (by checking on the class membership of gameStateI) Player dismisses qualities menu

40. Encounter State-Transition Test Sequence 1 of 2 Waiting Preparing test step 1 test step 2 test step 3 Verify that the game is initially in Preparing state (by checking on the class membership of gameStateI) Dismiss the quality menu, and verify that the game is in Waiting state. Move the player character to an adjacent area, and verify that the game is still in Waiting state. Player dismisses qualities menu Move to adjacent area Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

41. Player dismisses qualities menu Character enters area inhabited by an opponent Move to adjacent area Complete Encounter State-Transition Test Waiting Preparing Engaging 1 2 3 4 5 Reporting Player dismisses encounter report menu Encounter completed 6 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

42. 6. Summary

43. Unit Testing: Summary Unit testing =  “pieces” Other testing =  “assemblies” Black box: input / output only White box: verifies processing –Several ways –Ensure completeness Test planning earlier / better –helps clarify requirements Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

44. Case Study EncounterCharacter.java

45. /** To test this class. * @paramargsPdestination of method test log, class test log respectively */ public static void main( String[] argsP ) { // Default files on which to write test output & run tests String methodOutputFileNameM = "methodOutput.txt"; String classOutputFileNameM= "classOutput.txt"; if( argsP != null && argsP.length == 2 ) // use defaults if input improper {methodOutputFileNameM = argsP[0]; classOutputFileNameM = argsP[1]; } // 1. EXECUTE TESTS WHICH DO NOT REQUIRE HUMAN INTERVENTION // Test methods individually, then test class try {testEncounterCharacterMethods( methodOutputFileNameM ); testEncounterCharacterClass( classOutputFileNameM ); } catch( IOException eP ) {System.out.println( eP ); } Listing page 1 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

46. // 2. EXECUTE TESTS WHICH DO REQUIRE HUMAN INTERVENTION Frame[] imageTests = { // Display test cases. new testCharacterImage( // Missing image. new EncounterCharacter( "GuyWithNoImage", null ) ), new testCharacterImage( // Image is present. new EncounterCharacter( "Elena", "elena.gif" ) ) }; for( int i = 0; i < imageTests.length; i++) { // Display each test window. imageTests[i].setSize(400, 250); // Adequate size for character. imageTests[i].setVisible(true); imageTests[i].show(); } try { // Let user examine windows. Thread.currentThread().sleep( 30*1000 ); } catch( Exception exc ) { } for( int i = 0; i < imageTests.length; i++ ) // Shut the windows. imageTests[i].dispose(); System.exit( 0 ); } Listing page 2 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

47. /** Tests this class by executing its methods in combination. * @paramdestinationPLocation to write results. * @exceptionIOException If there’s a problem opening or accessing destinationP */ public static void testEncounterCharacterClass( String destinationP ) throws IOException { /* Prepare for the test */ PrintWriter outM = new PrintWriter( new FileOutputStream( destinationP ) ); System.out.println( "\nEncounterCharacter class test results on " + destinationP + "\n" ); /* * The following methods will be tested in sequences: * * a. adjustQuality( String qualityP, float qualityValueP ) * d. deleteFromEncounterCharacters( EncounterCharacter encounterCharacterP ) * ge. EncounterCharacter getEncounterCharacter( String nameP ) * gq. float getQualityValue( String qualityP ) * gt. float getTolerance() * io. int indexOf( String qualityP ) * ii. insertIntoEncounterCharacters( EncounterCharacter encounterCharacterP ) * m. int maxNumCharsInName() * sq. setQuality( String qualityP, float qualityValueP ) * so. float sumOfQualities() Listing page 3 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

48. * * The following sequences occur commonly: * ge-aq-so * ge-sq-a-gq *..... * The following sequences have a high potential for defects: * ge-aq-aq-gq-so *..... */ /* Test C1: ge-aq-so */ EncounterCharacter eC1M = new EncounterCharacter( "CharForTestC1" ); // method “ge” eC1M.adjustQuality(QUAL_STRENGTH, 40.0f ); // aq TestExecution.printReportToFile( outM, "Class test ge-aq-so", eC1M.sumOfQualities(), 100.0f ); // so /* Test C2: ge-aq-aq-gq-so */ EncounterCharacter eC2M = new EncounterCharacter( "CharForTestC2" ); // ge eC2M.adjustQuality(QUAL_STRENGTH, 40.0f ); // aq eC2M.adjustQuality(QUAL_STAMINA, 20.9876f ); // aq Listing page 4 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

49. TestExecution.printReportToFile( outM, "Class test ge-aq-aq-gq-so: part 1", eC2M.getQualityValue( QUAL_STAMINA ), 20.9876f ); // gq TestExecution.printReportToFile( outM, "Class test ge-aq-aq-gq-so: part 2", eC2M.sumOfQualities(), 100.0f ); // so /* INVARIANT-ORIENTED TESTS * Check for the invariant “qualValueI[i] >=0” * -- after executing the sequences of methods executed above */ boolean truthM = true; for( int i = 0; i < qualityTypeS.length; ++i ) { /* Set truthM false if any entry in eC1M.qualValueI not >= 0 */ truthM = truthM && ( eC1M.qualValueI[i] >= 0.0f ); } TestExecution.printReportToFile( outM, "Class test for the invariant 'qualValueI[i] >=0'", truthM, true ); /* Conclude */ outM.close(); System.out.println( "\nClass tests of EncounterChar class concluded." ); } // end of testEncounterCharacterClass Listing page 5 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

50. /** Tests all the methods of this class one at a time * @param destinationP Location to write results. * @exceptionIOException If there’s a problem opening or accessing destinationP */ public static void testEncounterCharacterMethods( String destinationP ) throws IOException { /* Prepare for the test */ FileWriter outM = new FileWriter( new File( destinationP ) ); System.out.println( "EncounterCharacter method test results on " + destinationP + "\n" ); /* Tests for getEncounterCharacter() */ EncounterCharacter eCNorM = new EncounterCharacter( "qwerty" ); // normal TestExecution.reportToFile( outM, "GetCharacter Test 1: nominal value", eCNorM.getName(), "qwerty" ); EncounterCharacter eCNullM = new EncounterCharacter( null ); // null TestExecution.reportToFile( outM, "GetCharacter Test 2: null parameter", eCNullM.getName(), GameCharacter. DEFAULT_NAME); Listing page 6 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

51. String tooLongM = "1234567890123456789012345678901234567890"; EncounterCharacter eCTooLongM = new EncounterCharacter(tooLongM); // too long TestExecution.reportToFile( outM, "GetCharacter Test 3: Limit parameter values, " + "max name len = " + eCTooLongM.maxNumCharsInName(), eCTooLongM.getName(), tooLongM.substring(0, eCTooLongM.maxNumCharsInName()) ); EncounterCharacter eCZeroM = new EncounterCharacter( "" ); // zero-len TestExecution.reportToFile( outM, "GetCharacter Test 4: zero-length", eCZeroM.getName(), GameCharacter. DEFAULT_NAME); EncounterCharacter eCPuncM = new EncounterCharacter( "a+b" ); // bad chars TestExecution.reportToFile( outM, "GetCharacter Test 5: bad char '+' ", eCPuncM.getName(), GameCharacter. DEFAULT_NAME); Listing page 7 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

52. /* Tests for indexOf() for every valid quality name. */ for( int i = 0; i < qualityTypeS.length; ++i ) try { TestExecution.reportToFile( outM, "indexOf() Test 1." + i + ": valid name: " + qualityTypeS[i], indexOf(qualityTypeS[i]), i ); } catch( Exception eP ) { TestExecution.reportToFile( outM, "indexOf() Test 1: valid name: compare ", "indexOf('" + qualityTypeS[i] + "')", "with expected " + i ); } /* Tests for indexOf() for an invalid quality name. */ try { TestExecution.reportToFile( outM, "indexOf() Test 2: invalid name: zorch", indexOf("zorch"), -1 ); } catch( Exception eP ) { TestExecution.reportToFile( outM, "indexOf() Test 2: valid name: compare ", "indexOf(\"zorch\")", "with expected -1" ); } Listing page 8 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

53. /* Tests for setQuality() */ // Set up for test EncounterCharacter hank = new EncounterCharacter( "Hank" ); // Nominal value hank.setQuality(QUAL_STRENGTH, 10.3f ); TestExecution.reportToFile( outM, "setQuality() Test 1: nominal value", hank.getQualityValue( QUAL_STRENGTH ), 10.3f ); // Out of range value hank.setQuality( QUAL_PATIENCE, -6.2f ); TestExecution.reportToFile( outM, "setQuality() Test 2: nominal value", hank.getQualityValue(QUAL_PATIENCE ), 0.0f ); // Value below close-to-zero threshold. hank.setQuality( QUAL_STAMINA, getTolerance() * 0.9f ); TestExecution.reportToFile( outM, "setQuality() Test 3: value close to zero", hank.getQualityValue(QUAL_STAMINA), 0.0f ); Listing page 9 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

54. // Tests for adjustQuality(). // Set up for test and verify: Values should be 20 each. EncounterCharacter harvey = new EncounterCharacter( "Harvey" ); TestExecution.reportToFile( outM, "adjustQuality() test 0: verify that values add to 100", harvey.sumOfQualities(), 100.0f ); // Nominal adjustment harvey.adjustQuality(QUAL_STRENGTH, 30.0f ); // strength 30 rest 70/4 each TestExecution.reportToFile( outM, "adjustQuality() test 1: values sum to 100 after adjusting", harvey.sumOfQualities(), 100.0f ); TestExecution.reportToFile ( outM, "adjustQuality() test 2: values adjusted as commanded", harvey.getQualityValue(QUAL_STRENGTH ), 30.0f ); // Adjustment resulting in a zero value harvey.adjustQuality( QUAL_STAMINA, 99.0f ); TestExecution.reportToFile( outM, "adjustQuality() test 3: verify low value reverts to zero", harvey.getQualityValue( QUAL_STRENGTH ), 0.0f ); // Conclude outM.close(); System.out.println( "\nMethod tests of EncounterCharacter class concluded." ); } Listing page 10 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

55. /** Class to test repainting of characters. Creates a window, which will contain * several copies of the character image. */ private static class testCharacterImage extends Frame { /** Instance attribute that remembers which character image to display. */ private EncounterCharacter characterI; /** Basic constructor -- create a window for testing some character's image. * @paramcharacterPCharacter whose image is to be tested. */ testCharacterImage(EncounterCharacter characterP) { super(characterP.getName());// Do all normal Frame initialization. characterI = characterP;// Remember which character we're testing. } Listing page 11 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 20001), with permission. Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

56. /** Repaint the display areaof the frame. * @paramdrawPGraphics context for drawing the character. */ public void paint(Graphics drawP) {Dimension frameSizeM = getSize(); // Size of the window area. int widthUnitM = frameSizeM.width / 5; // Convenient divisions of window. int heightUnitM = frameSizeM.height / 5; characterI.showCharacter(this, drawP, // Drawn small, facing right. new Point(widthUnitM, heightUnitM), heightUnitM, false); characterI.showCharacter(this, drawP, // Drawn large, facing left. new Point(widthUnitM*4, heightUnitM*3), heightUnitM*2, true); characterI.showCharacter(this, drawP, // Drawn large, facing right. new Point(widthUnitM*2, heightUnitM*2), heightUnitM*2, false); characterI.showCharacter(this, drawP, // Drawn small, facing left. new Point(widthUnitM*3, heightUnitM*4), heightUnitM, true); } } // End of testCharacterImage inner class Listing page 12 Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.