1. Testing

2. RHS – SOC 2 Testing So far, we have not really created tests in a systematic manner Testing has been more like ”probing” Making a thorough test is often very labor- intensive – many SW companies employ more testers than developers… Once it is done, it is a very useful tool to use during development

3. RHS – SOC 3 Testing The ideal test: When the test is passed, the product is ready for delivery! Ideal – but (almost) impossible –Number of test cases is often very large –How does one test a GUI? –Is functional correctness all that matters?

4. RHS – SOC 4 Testing In general, test goes hand in hand with requirement specification If we cannot specifiy what the software should do, how can we test it…? –”It must be easy to use the software” –”The software should respond quickly” –”The software must never crash”

5. RHS – SOC 5 Testing Requirements are speci- fied at many leves –Business level –User level –Functional level So we must also perform tests at many levels

6. RHS – SOC 6 Testing

7. RHS – SOC 7 Unit testing A Unit Test is aimed at testing a well- defined code module, in Java usually a single class Unit tests are at the functional level –Define test cases in terms of input to class methods (public and private) –Define the expected output for each case –Run the test –Compare expected and actual output

8. RHS – SOC 8 Unit testing Unit testing is – more or less – what we have done so far, but informally NetBeans can create a more ”formal” unit test framework for a project Relies on a Java framework called Junit (see www.junit.org)www.junit.org

9. RHS – SOC 9 Unit testing in NetBeans Consider our ”classic” BankAccount class, with three methods: –deposit –withdraw –getbalance Having created the class, we can now create a unit test for the class

10. RHS – SOC 10 Unit testing in NetBeans

11. RHS – SOC 11 Unit testing in NetBeans

12. RHS – SOC 12 Unit testing in NetBeans

13. RHS – SOC 13 Unit testing in NetBeans There are quite a lot of options to choose from when generating a test class… For now, just leave them as-is When pressing Finish, a unit test class is generated for us, called BankAccount- Test (just choose ”Junit 4.x”) The test class is placed under Test Packages

14. RHS – SOC 14 Unit testing in NetBeans The generated test class does look a bit complex (remember we chose all options) However, we are free to edit it! Remove whatever you do not need NetBeans can only generat a ”skeleton” for the test class – we must complete it

15. RHS – SOC 15 Unit testing in NetBeans @BeforeClass public static void setUpClass() throws Exception {} @AfterClass public static void tearDownClass() throws Exception {}

16. RHS – SOC 16 Unit testing in NetBeans The two methods setUpClass and tearDownclass allows us to include any actions needed before and after running all the test class methods, respectively –Dependencies to other classes –Database connection –Etc. Is often not used – then delete it!

17. RHS – SOC 17 Unit testing in NetBeans @Before public void setUp() {} @After public void tearDown() {}

18. RHS – SOC 18 Unit testing in NetBeans The two methods setUp and tearDown allows us to include any actions needed before and after running each of the test class methods, respectively –Initialising/resetting variable values –Cleaning up data structures –Etc. Is often not used – then delete it!

19. RHS – SOC 19 Unit testing in NetBeans @Test public void testGetBalance() { System.out.println("getBalance"); BankAccount instance = new BankAccount(); int expResult = 0; int result = instance.getBalance(); assertEquals(expResult, result); // TODO review the generated test code and // remove the default call to fail. fail("The test case is a prototype."); }

20. RHS – SOC 20 Unit testing in NetBeans Notice that a test method does not return a value (true/false) Instead, so-called assertions are used during the test An assertion can succeed or fail A failed assertion throws an exception, and the test case is considered failed

21. RHS – SOC 21 Unit testing in NetBeans Examples of assertions: –assertEquals(expectedValue, ActualValue) –assertTrue(condition) –assertFalse(condition) –assertNotNull(object) –assertNull(object) –assertSame(object, object) –assertNotSame(object, object) –fail() // ALWAYS fails

22. RHS – SOC 22 Unit testing in NetBeans If you inspect the generated test code, you will find that it is not very useful We must – almost always – implement the body of the test methods ourselves We are free to add more test methods than those initially generated – the test framework will run them automatically

23. RHS – SOC 23 Unit testing in NetBeans Once the test methods have been defined properly, we can run the test Choose Run | Test Project, or just press Alt + F6 Result of test is displayed in the output window, with indicative colors

24. RHS – SOC 24 Unit testing in NetBeans

25. RHS – SOC 25 Exercises Create a new Java project called BankAccount, and implement it as described in the presentation Generate a Unit test class for BankAccount, as described in the presentation Now change the code in the generated test class, in order to make it a useful test for the BankAccount class. Note that this may involve not only changing the code in the test methods, but perhaps also adding code to some of the setUp/tearDown methods Run the test, and keep changing the source code and/or test code until all tests are passed (all lights are green ) – how confident do you now feel about the correctness of the BankAccount class? Review the test – did you have to make some assumptions during implementation of the test, that were perhaps questionable?

26. RHS – SOC 26 Unit testing details Using assertions for testing is the most common way of performing a unit test (also see http://junit.org/junit/javadoc/4.5)http://junit.org/junit/javadoc/4.5 However, there are other aspects of test –Exceptions –Execution time These can also be tested with Unit test

27. RHS – SOC 27 Unit testing details In some cases, the correct response from a method could be to throw an exception We cannot ”capture” this behavior with an assert statement Instead, a special ”annotation” is used on the test method (expected = name of exception class)

28. RHS – SOC 28 Unit testing details @Test (expected = IllegalArgumentException.class) // Test fails if the specified exception is NOT // thrown public void testNegativeDeposit() { // Note that test as such always succeeds theAccount.deposit(-100); }

29. RHS – SOC 29 Unit testing details Also, a usual test cannot ”capture” if a method call takes too long (as defined by us) to complete This is also handled using a special anno- tation to the test method (timeout = M) Method must complete within M milli- seconds, otherwise it has failed

30. RHS – SOC 30 Unit testing details @Test (timeout = 100) // Test fails if NOT completed within 100ms public void testFibonacci() { // Calculate the first 10000 Fibonacci numbers // Note that test as such always succeeds theFib.calculateFibonacci(10000); }

31. RHS – SOC 31 Unit testing considerations In the ideal scenario, all units tests should be completely self-contained Testing of a particular class should not depend on other classes Testing of a particular method should not depend on other methods Isolates cause of failed tests

32. RHS – SOC 32 Unit testing considerations @test public void testDeposit() { int b = theAcc.getBalance(); theAcc.deposit(500); int a = theAcc.getBalance(); int diff = a – b; assertEquals(diff, 500); } BankAccount deposit withdraw getBalance

33. RHS – SOC 33 Unit testing considerations Suppose now that testDeposit fails Which method in BankAccount contains an error…? Is it deposit, or getBalance...? You are wrong! No, you are wrong!

34. RHS – SOC 34 Unit testing considerations @test public void testCubeVolume() { int volume = theCube.getVolume(); int expVolume = theMathLib.calcCube(theCube.getSide()); assertEquals(volume, expVolume); } Cube getSide getVolume MathLibrary calcCube …

35. RHS – SOC 35 Unit testing considerations Suppose now that testCubeVolume fails Which class contains an error…? Is it Cube or MathLibrary…? You are wrong, again! No, you are wrong again!

36. RHS – SOC 36 Unit testing considerations Testing one functionality often assumes that some other functionality already works correctly… This is quite hard to avoid in practice A rigorous approach is to useso-called test stubs

37. RHS – SOC 37 Unit testing considerations A test stub is a ”simulation” of the behavior of a real class or method (Martin Fowler): Test stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test

38. RHS – SOC 38 Unit testing considerations Making a test stub –Write the test, calling any external methods that are needed –Substitute all calls to external methods with calls to stub methods (Proxy…?) –Implement the stubs as returning the desired answer without any calculation

39. RHS – SOC 39 Unit testing considerations @test public void testCubeVolume() { int volume = theCube.getVolume(); int expVolume = theMathLibStub.calcCube(theCube.getSide()); assertEquals(volume, expVolume); }... // Code in MathLibraryStub // Only called with input = 8 in test public int calcCube(int input) { return 512; }

40. RHS – SOC 40 Unit testing considerations Creating a test using stubs consequently can be done – but is quite labor-intensive More pragmatic approach is to use a bottom-up approach –Test basic methods first (methods that do not call other methods) –When basic methods work, test methods that only use basic methods –And so on (dependency tree)

41. RHS – SOC 41 Testing – final remarks We can (almost) never expect to create a completely covering test  Testing is about building confidence in the correctness of the program Always a compromise between level of confidence and required effort

42. RHS – SOC 42 Testing – final remarks Confidence Effort Student assignment Commer- cial word processor Space Shuttle software

43. RHS – SOC 43 Testing – final remarks Further reading: JUnit test in NetBeans http://www.netbeans.org/kb/docs/java/junit-intro.html More about Junit in general www.junit.org …and the Net contains a lot of material about test in general!

44. RHS – SOC 44 Exercises Download the NetBeans project IntegerStack from the website (under Classes, week 44) The project contains a single class IntegerStack, which has a simple interface with five methods Implement a Unit Test for the IntegerStack class, which tests all five methods. Use a bottom-up approach, so you test the simplest methods first (you do not need to use stubs) Use the test to discover any errors in the implementation of the IntegerStack class – if you find any, fix them! Keep fixing errors until all test cases are green (i.e passed)