1. CS4723 Lecture 3 Unit Testing

2. 2 Unit testing  Testing of an basic module of the software  A function, a class, a component  Typical problems revealed  Local data structures  Algorithms  Boundary conditions  Error handling

3. 3 Unit test framework  xUnit  Created by Kent Beck in 1989  This is the same guy we mentioned in XP, design patterns  The first one was sUnit (for smalltalk)  Junit  The most popular xUnit framework  There are about 70 xUnit frameworks for corresponding languages  Never in the annals of software engineering was so much owed by so many to so few lines of code -------Martin Fowler

4. 4 An example: w riting test cases without a test framework  Class Definition  Consider a class that handles and manages a Fibonacci array  Initialized the class  Fibonacci f = new Fibonacci ();  Extend to a certain length  f. extend (length)  Provide the number at certain index  f. get(index)  Get Length  f.getLength()  Provide a range of numbers  f. getRange(start, end)

5. 5 An example: writing automatic test cases without a test framework public static void main(String args[]){ Fibonacci f = new Fibonacci(); if(f.getLenght != 0){ System.err.print(“wrong results …”); } f.extend(50); if(f.getLenght != 0){ System.err.print(“wrong results …”); } int num = f.get(50); if(num != 12586269025){ System.err.print(“wrong results …”); } int[] range = f.getRange(40, 50) if(!ValueEquals(range, …)){ System.err.print(“wrong results …”); } … } Not Enough! What happened if an exception is thrown

6. 6 An example: writing automatic test cases without a test framework public static void main(String args[]){ Fibonacci f = new Fibonacci(); if(f.getLenght != 0){ System.err.print(“wrong results …”); } f.extend(50); if(f.getLenght != 0){ System.err.print(“wrong results …”); } int num = f.get(50); if(num != 12586269025){ System.err.print(“wrong results …”); } int[] range = f.getRange(40, 50) if(!ValueEquals(range, …)){ System.err.print(“wrong results …”); } … } Potential test interference! Consider if the get method does an extension when length is smaller than index: while in getRange, such extension is forgotten to be added

7. 7 Common Operations  Comparison between actual results and expected results  Report error messages  Catch Exceptions  Re-initialize the class to be tested

8. 8 Unit Test Framework  System under test: the system/module/component we are testing  Test Fixture: SUT + DOC  Test Method: The actual code of the test  Test Case: A collection of tests with common purpose/setup

9. 9 Writing a Test Case public class FibonacciTest extends TestCase { protected Fibonacci fTest; protected static int groudtruth[] = {0, 1,...,12586269025}; public void setUp(){ fTest = new Fibonacci(); } @Test public testInitialize(){ assertEquals(“Length of fTest after initialization”, 0, fTest.getLength()); } @Test public testExtend(){ fTest.extend(50); assertEquals(“…”, 50, fTest.getLength()); } @Test public testGet(){ fTest.extend(50); assertEquals(“…”, groudtruth[49], fTest.get(50)); } … }

10. 10 Benefits of using test framework  Clear structure of testing  Each test method for one feature  Common initialization to setup  Assertion methods  You can always define new assertion methods  Reduced Interference  Try catch to run all test methods  Always do re-initialization in setup

11. 11 Tips of writing test cases  Bottom Up  If you know that obj.m1 calls obj.m2, test obj.m2 first  Why?  You may not understand the error in m1 if you don’t know whether m2 is working or not  Externalize data  Externalize expected results to files and load them at the test case constructor  Why?  Easier to maintain if you want to reuse them in several places

12. 12 Tips of writing test cases  Use development Database  Do not use real database for testing  Why? Obvious  Do enough logging in the code during testing  Why?  Help debugging  Testing for exceptions  How? public testException(){ try{ do(invalid); fail(); }catch(xxxException e){} }

13. 13 Writing Assertions public void testCapacity() { // a test method …. assertTrue(fFull.size() == 100+size); //assertion } If assertion fails: Assertion failed: myTest.java:150 (expected true but was false) Not so good! Try: assertEquals(100+size, fFull.size()); //expected value first Assertion failed: myTest.java:150 (expected 102 but was 103) Better! Try: assertEquals(“list length”, 100+size, fFull.size()); Assertion failed: myTest.java:150 (list length expected 102 but was 103)

14. 14 Assertions  Extend TestCase and write your own assertions  AssertStringContains, AssertArrayEquals, …  Use fail(String) to fail a test with certain message  Feel free to change the fixture  Each test will re-setup public void testRemoveAll() { fFull.removeAllElements(); fEmpty.removeAllElements(); assertTrue(fFull.isEmpty()); assertTrue(fEmpty.isEmpty()); }

15. 15 State machine of JUnit Initialization Of Test Class setUp try{ testMethod } catch {…} tearDown

16. Tear down  Consider the following test code void setUp() { File f = open(“foo”); File b = open(“bar”); } void testAAA() { use f and b } void testBBB(){ use f and b } Problems? void setUp() { File f = open(“foo”); File b = open(“bar”); } void testAAA() { try { use f and b } finally { clean&close f, b } void testBBB() { try { use f and b } finally { clean&close f, b } Better?

17. 17 Tear down  Consider the following test code void setUp() { File f = open(“foo”); File b = open(“bar”); } void testAAA() { use f and b } void testBBB(){ use f and b } void tearDown{ clean&close f, b } Problems? void setUp() { File f = open(“foo”); File b = open(“bar”); } … void tearDown{ try{ clean&close f, b }catch{ … } void setUp() { File f = open(“foo”); File b = open(“bar”); } … void tearDown{ try{ clean&close f }catch{ … } the same for b } Better?

18. 18 Tear down  Do some cleanup job after the test is executed  Close a file  Clear a global data structure  Revert changes to the database  Close network connections  …

19. 19 Tear down  Be careful about tear down  If tear down is not complete, a test failure may affect the following test cases  Recover the changes done to global data that are not well handled by the setup  Database, files, network, global variables  Clean resources  Caution of exceptions in tear down itself

20. 20 The Practical Difficulty in Unit Testing  Decoupling  It is rare that you are writing test cases for a class with no dependencies (e.g., fibonacci array)  Dependencies are everywhere  Complex structures  System calls  Databases  File systems  Network  …

21. 21 Problem of dependencies  Maintenance during setUp and tearDown  Expensive to setUp  Network, loading from file, start a database…  Hard to find bugs  Bugs may be related to the dependencies  They are still bugs, but should not be considered until system testing

22. 22 Handling dependencies  Explicitly show dependencies  Hidden dependencies are dangerous  Dependency injection  Remove unnecessary dependencies  Reasonable design  Double dependencies  When dependency is large or not ready yet  Use mock object

23. 23 A story from google A new developer go to a group and want to try on some methods testCreditCardCharge() { CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100); } This code: Only works when you run as part of the suite. When run in isolation, throws NullPointerException. When you get your credit card bill, you are out $100 for every time the test runs.

24. 24 A story from google After a lot of digging, you learn that you need to initialize the CreditCardProcessor. testCreditCardCharge() { CreditCardProcessor.init(); CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100); } Still not working 

25. 25 A story from google After a lot of digging, you learn that you need to initialize the OfflineQueue. testCreditCardCharge() { OfflineQueue.init(); CreditCardProcessor.init(); CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100); } Still not working 

26. 26 A story from google After a lot of digging, you learn that you need to initialize the Database. testCreditCardCharge() { Database.init(); OfflineQueue.init(); CreditCardProcessor.init(); CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100); } But sometimes you have to find out the correct sequence!

27. 27 Solution with dependency injection testCreditCardCharge() { Database db = Database(); OfflineQueue q = OfflineQueue(db); CreditCardProcessor ccp = new CreditCardProcessor(q); CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(ccp, 100); }

28. 28 Lessons Learned  Always try to make dependencies explicitly shown  Never do testing with real credit card number,  Or at least try $1 first

29. 29 Remove Dependencies  Thin method interfaces  Example:  Get the total rent of a car  getRent(car, customer) -> getRent(car.getDays(), car.getPrice(), customer.getDiscount())  Duplicate common dependencies  Example:  The class for Graduate students and under-graduate students both depend on a calendar module  It says, “if(student.isGraduate){…}else{…}”  Split it to two parts and put them into the class for both types of students

30. 30 Double Dependencies  Removing Dependencies is not always doable  Lots of necessary dependencies  It is mostly something in design, so maybe unchangeable at the testing time  It is not always good to remove all unnecessary dependencies  increase maintenance effort  reduce the benefit of software reuse  make code more complex and hard to read

31. 31 Double Dependencies  Solution: Double it!  Double: highly trained replacement, vaguely resembling the actor  No need to be a good actor  May require different skills for different cases

32. 32 Types of test doubles  Dummies  Test stubs  Fake objects  Mock objects

33. 33 Dummies  Simplest test double  No behavior, just serve as an argument  Null can be viewed as a simple double  Example public class Order{ private Shop sp; public Order (Shop sp){ this.sp = sp; } public Shop getShop(){ return sp; } … } public class ShopDummy extends Shop{... } public class OrderTest{ @Test public void testInit(){ Order o = new order(new ShopDummy());... }

34. When to use dummies  SUT does not invoke any method on the dependency  The dependency class is an interface or abstract class with no implementation or  The dependency class is hard to initialize so that you can put the initialization in the dummy once  E.g., Shop sp = new Shop(new A(…), new B(…), new C(…), new D(…), new E(…));  Can put the whole thing in ShopDummy

35. 35 Test Stubs  More powerful than dummies  Provide a fix value or fixed behavior for a certain method invocation  Example:  Always return 0 for a integer method  Do nothing for a void method

36. 36 Hard-coded Test Stubs  The value or behavior is hard coded in the Stub Class public class OrderTest{ @Test public void test(){ Order o = new order(new ShopStub()); o.add(1122, 3);... AssertEquals(expect, o.getTotal()); o.save(); } public class ShopStub extends Shop{ public void save(Order o){ } public double getShopDiscount(){ return 0.9; }

37. 37 Configurable Test Stubs  You may set different values for different test cases public class ShopStub extends Shop{ private Exception saveExc; private discount; public setException(Exception e){ this.saveExc = e; } public setDicount(Float f){ this.discount = f; } public void save(Order o){ if(this.saveExc!=null){throw saveExc;} } public double getShopDiscount(){ return 0.9; } public class OrderTest{ @Test public void testAbnormalDiscount(){ ShopStub stub = new ShopStub(); stub.setDiscount(1.1); Order o = new order(stub); o.add(1122, 3);... AssertEquals(expect, o.getTotal()); o.save(); }

38. Fake Objects  More powerful than stubs  A simplified implementation of the DOC  Example: a data table to fake a database  Example: use a greed algorithm to fake a complex optimized algorithm  Guidelines  Slow -> Fast  Complex -> Simple

39. Fake Objects  Tips for fake objects  As simple as possible (as long as not too time-consuming)  Go to a higher level if some object is hard to fake  Example: URLStatus sts = HttpConnection.open("http://api.dropbox.com/files/myfile"); if(sts.status == 200){ return sts.data; }else{ return “Error”; } Need to double 1.Difficult to reproduce 2.Maybe slow 3.Affected by lots of factors

40. Fake Objects Example (1)  Fake HttpConnection  We need to fake URLStatus also public class FakeUrlStatus{ public FakeUrlStatus(int status, string data){...}; public int getStatus(){...}; public String getData(){...}; } public class FakeDropBoxApi{ private files = { }; public FakeUrlStatus read(fname){ if(files.contain(fname)){ return FakeUrlStatus(200, files[fname]); }else{ return FakeUrlStatus(-1, "Error"); } FakeURLStatus sts = FakeDropBoxApi.read("myfile"); if(sts.status == 200){ …

41. Fake Objects Example (2)  Fake the HttpConnection + URLStatus public class FakeDropBoxHigherApi{ private files = { }; public String read(fname){ if(files.contain(fname)){ return files[fname]; }else{ return "Error"; } return FakeDropBoxHigherApi.read("myfile");

42. Fake Objects  Choose a reasonable abstraction level to do the faking  Do not need to fake lots of dependencies (put them in one box)  Write simpler code (less potential bugs)

43. 43 Mock objects  Mock objects is not similar to any of the above  For the ability to imitate DOC, it is similar to configurable stubs  But it does some different things  Mock objects do behavior-based testing  Usually we only check return values or status  AssertEquals (expected, actual);  AssertEquals (expected, array.length);  Can we do something like this? Why?  Assert ( testObject.f1 calls DOC.f)

44. 44 Mock objects  Go back to the order example 44 public class OrderTest{ @Test public void test(){ Order o = new order(new ShopStub()); o.add(1122, 3);... AssertEquals(expect, o.getTotal()); o.save(); } public class ShopStub extends Shop{ public void save(Order o){ } public double getShopDiscount(){ return 0.9; } Problems???

45. Mock objects  Do behavioral assertions for SUT  Only cares about whether the unit to be tested is correct  Do not care about the correctness of DOC Test Code SUT DOC DON’T CARE!!

46. Mock objects  Example  EasyMock @Test public void testOrder() { Order o = new Order(sp); o.add(1234, 1); o.add(4321, 3); //initialize Shop sp = EasyMock.CreateMock(Shop.class); //record EasyMock.expect(sp.getDiscount()).andReturn(0.9); EasyMock.expect(sp.save(o)); //replay EasyMock.replay(sp); AssertEquals(expect, o.getTotal()); o.Save(); EasyMock.verify(sp) }

47. 47 Mock objects  Procedure I am a mock object, just generated Record: What I am expected to be invoked, and what should I return Expect Methods Replay: See how SUT invoked me SUT Verification Results

48. 48 Verification  Verifies whether the expected methods are actually invoked  Exception: missing, expected save(0xaaaa)  Verifies whether the expected methods are invoked in an expected way  Exception: save(null) expected save(0xaaaa)

49. 49 Richer ways to do expectation  Different frameworks support it in different ways  Easymock:  isA: ignore the value of argument  EasyMock.expect(sp.save(isA(Order.class)))  Find: expect the argument to contain a certain substring  EasyMock.expect(mock.call(find(“pattern”)))  Geq: expect a number larger than the given value  EasyMock.expect(mock.call(Geq(1000)))

50. 50 Orders of expected method calls  Different frameworks support it in different ways  Easymock  Order is not checked by default  createStrictMock to check order  Mockito  Order is not checked by default  Create a InOrder object at the verification phase and add the expected invocations that should happen in order  jMock  More complex order control

51. 51 Fluent Mocking  Record phase:  Read expectation as specifications  Instead of directly generating code, the mock object generates an internal presentation, e.g. Automaton  Replay phase:  Check the real invocations with the internal presentation

52. 52 Fluent Mocking: jMock

53. 53 Types of test doubles  Dummies  Simplest, only for arguments  Test stubs  Return fixed value  Can be configurable at runtime  Fake objects  A simplified implementation  Mock objects  Similar to configurable stubs  Behavior checking  Automatic framework support

54. 54 Review of Unit Testing  Using unit test framework to do unit testing  It does all the common things, reduce test interference  Inside test framework  setUp -> test -> assert -> tearDown  Writing informative assertions  Writing complete tear downs  Dependencies are evil!  Dependency injection  Remove dependencies  Test doubles

55. 55 Announcement of Assignment I  Task  Writing unit test cases for a real-world Java class  One of two classes from the Apache project  org.apache.commons.email.Email  org.apache.commons.imaging.ImageParser  Write test method for only the listed 12 methods  Work individually

56. 56 Announcement of Assignment I  Deliverable  Send the test code, and required dependencies to my email account  Dependencies include  JUnit  Mock Framework if used  The tested code and required dependencies  Comment your test code properly  Due Feb 28 th 2014

57. 57 Announcement of Assignment I  Evaluation (20 points total)  Test code compiles and has no runtime error (failing test cases are not counted as runtime errors) – 8 points  Proper setUp, tearDown, and assertions – 4 points  Proper test doubles – 4 points  Good test coverage – 4 points

58. 58 Demo  Writing a JUnit testcase for a class in your project  Run it as a JUnit test case