1. Chapter 11, Testing: Unit Testing with JUnit 4
Note to Instructor:
The material in this slide set is not contained in the 3rd edition of the text book. It is planned for the 4th edition. However, because the major differences between to old JUnit 3 and the new JUnit 4, it might be advisable to use at least the JUnit 4 material.

2. Outline of the next 2 Lectures
Testing Activities
Unit testing with jUnit
Java Annotations and Assertions
Object-Oriented Test Modeling
Using object-oriented concepts also for the test model
Object-oriented Testing Patterns
Mock object pattern
Dependency injection pattern (not only a testing pattern)
Inner class test pattern
Reflection test pattern
Test exception pattern
4 stage testing pattern
Today
Next week
Test model as a new abstraction on the same level as the system model
Dependency injection pattern (not only a testing pattern, good for your system health )

3. Requirements Analysis
Testing Activities
Object Design
Document
System Design
Document
Requirements Analysis
Document
Client
Expectation
Unit Testing
Integration Testing
System Testing
Acceptance Testing
Developer
Client

4. Types of Testing Unit Testing Integration Testing System Testing
Individual component (class or subsystem)
Carried out by developers
Goal: Confirm that the component or subsystem is correctly coded and carries out the intended functionality
Integration Testing
Groups of subsystems (collection of subsystems) and eventually the entire system
Goal: Test the interfaces among the subsystems.
System Testing
The entire system
Carried out by developers
Goal: Determine if the system meets the requirements (functional and nonfunctional)
Acceptance Testing
Evaluates the system delivered by developers
Carried out by the client. May involve executing typical transactions on site on a trial basis
Goal: Demonstrate that the system meets the requirements and is ready to use.

5. JUnit: Overview A Java framework for writing and running unit tests
Test cases and fixtures
Test suites
Test runner
Written by Kent Beck and Erich Gamma
Written with “test first” and pattern-based development in mind
Tests are written before coding the SUT
Observe the ones that fail
Write new SUT code or fix existing SUT code to make the test pass
Allows for regression testing
Facilitates refactoring
JUnit is Open Source
JUnit Version 4, released Mar 2006
The JUnit framework is very easy to understand. A basic interface exists (Test), and the test classes must implement this interface. There are two types of test classes: TestCase and TestSuite:
TestCase is the main class that we extend for writing all our tests. It contains the methods that implement the individual tests with the signature public void testMethodName() and the optional private setup() and teardown() methods.
TestSuite assembles a collection of TestCase(s), other TestSuite(s), or a combination of the two.
Assert is the superclass of TestCase. It provides all the methods for our tests to check the values or method calls. A test can pass, fail, or have errors.
TestResult accumulates the results of individual tests in a run. It keeps information about the start and the end of the test and also about the failures and errors.
TestFailure provides access to the failure or error that occurs during the test. This includes detailed information on the exception.
Failures are the anticipated problems with the code. We use assertions to check for the possibility of failures. Errors are unanticipated problems. If a test method does not catch an exception, that exception is raised to the JUnit framework and it reports that as an error.
It is important to understand that JUnit reports the first failure in a single test. It executes each test within a separate instance of the test class. It reports failure on each test. It is better to split the assertions into individual tests rather than use JUnit for complete functional testing using one test method.
The framework defines an error class called AssertionFailedError, which is thrown whenever an assertion fails. The JUnit framework catches the error and reports that the test is failed.
TestRunners display the information contained within the object to the user.
TestListener is an interface that can be used by any class to monitor the progress of a test run. It contains methods for recording the starting state, the end state, and any failures or errors in a test. We discussed the three types of TestRunners in a previous section. Each of them extends a BaseTestRunner and provides the interfaces to the JUnit framework.

6. JUnit 4 and xUnit Frameworks
Version 4:
Annotation-based
Simplified test setup
Extension hooks for new runners
AssertEquals() for comparing arrays
„xUnit“ frameworks
nUnit (.NET)
pyUnit (Python)
cppUnit (C++)
dUnit (Delphi)

7. A Java Example class Money { private int fAmount;
private String fCurrency;
public Money(int amount, String currency) {
fAmount= amount;
fCurrency= currency; }
public int amount() {
return fAmount;
public String currency() {
return fCurrency;
public Money add(Money m) {
return new Money(amount()+m.amount(), currency()); }

8. Unit Testing add() with JUnit 4.0
The unit test MoneyTest tests that the sum of two Moneys with the same currency contains a value that is the sum of the values of the two Moneys
Static import of Assertion package (Older JUnit versions used inheritance)
import org.junit.Test;
import static org.junit.Assert.*;
public class MoneyTest {
@Test public void simpleAdd() {
Money m12CHF= new Money(12, "CHF"); 
Money m14CHF= new Money(14, "CHF"); 
Money expected= new Money(26, "CHF"); 
Money observed= m12CHF.add(m14CHF); 
assertTrue(expected.equals(observed)); }
Calling the SUT Method
Assertion: Returns True if parameter of type Boolean evaluates to True

9. Assertions in JUnit 4.0 assertTrue(Predicate); fail(String) 
Returns True if Predicate evaluates to True
fail(String) 
Let the method fail, useful to check that a certain part of the code is not reached.
assertsEquals([String message], expected, actual)
Returns message if the values are the same
assertsEquals([String message], expected, actual, tolerance)
Used for float and double; tolerance specifies the number of decimals which must be the same
assertNull([message], object)
Checks if the object is null and prints message if it is
assertNotNull([message], object)
Check if the object is not null
assertSame([String], expected, actual)
Check if both variables refer to the same object
assertNotSame([String], expected, actual)
Check that both variables refer not to the same object
assertTrue([message], boolean condition)
Check if the boolean condition is True
try {a.shouldThroughException(); fail("Failed")} catch (RuntimeException e) {assertTrue(true);}
Alternative way for checking for exceptions

10. Unit Testing add() with JUnit 4.0
The unit test MoneyTest tests that the sum of two Moneys with the same currency contains a value that is the sum of the values of the two Moneys
Static import of Assertion package (Older JUnit versions used inheritance)
import org.junit.Test;
import static org.junit.Assert.*;
public class MoneyTest {
@Test public void simpleAdd() {
Money m12CHF= new Money(12, "CHF"); 
Money m14CHF= new Money(14, "CHF"); 
Money expected= new Money(26, "CHF"); 
Money observed= m12CHF.add(m14CHF); 
assertTrue(expected.equals(observed)); }
Annotation: Declaration of a Test Method simpleAdd()
Calling the SUT Method
Assertion: Returns True if parameter of type Boolean evaluates to True

11. Annotations in JUnit 4.0 @Test public void foo()
identifies that foo() is a test method
@Before public void bar()
Perform bar() before executing a test method
@After public void foobar()
A test method must finish with call to foobar()
@BeforeClass public void foofoo()
Perform foofoo() before the start of all tests. Used to perform time intensive activities, e.g. to connect to a database
@AfterClass public void blabla()
Perform blabla() after all tests have finished. Used to perform clean-up activities, e.g. to disconnect to a database
@Ignore(string S)
Ignore the test method prefixed print out the string S instead. Useful if the code has been changed but the test has not yet been adapted
@Test(expected=IllegalArgumentException.class)
Tests if the test method throws the named exception
@Test(timeout=100)
Fails if the test method takes longer then 100 milliseconds

12. @Ignore: Omitting Tests
There are situations where certain tests should not be executed by the test harness
Example: the current release of a third-party library used in the SUT has a bug
public class CalculatorTest {
@Ignore(„Don‘t run now this test until bug in Foo is fixed")
@Test
public void testSpecialFunctionality() {
... }

13. @Test(timeout): Making Sure Tests are short
Unit tests should be short
But some tests take their time, particularly if network connectivity is involved
In these cases it is recommended to set an upper bound for the test
@Test(timeout=5000)
public void testLengthyOperation() {
... }

14. Another JUnit Example
import org.junit.Test; import static org.junit.Assert.*; public class CalculatorTest { private int x = 1; private int y = public void add() { int z = x + y; assertEquals(2, z); }
In JUnit 4, the unit tests don’t have to be in a special test class anymore. They can be anywhere in the code
Identifies that add() is a test method
Assertion

15. Annotations in JUnit 4.0 @Test public void foo()
identifies that foo() is a test method
@Before public void bar()
Perform bar() before executing a test method
@After public void foobar()
A test method must finish with call to foobar()
@BeforeClass public void foofoo()
Perform foofoo() before the start of all tests. Used to perform time intensive activities, e.g. to connect to a database
@AfterClass public void blabla()
Perform blabla() after all tests have finished. Used to perform clean-up activities, e.g. to disconnect to a database
@Ignore(string S)
Ignore the test method prefixed print out the string S instead. Useful if the code has been changed but the test has not yet been adapted
@Test(expected=IllegalArgumentException.class)
Tests if the test method throws the named exception
@Test(timeout=100)
Fails if the test method takes longer then 100 milliseconds

16. @Before and @After: Ensuring Pre- and Post Conditions
Any Method can be decorated and @After:
public class CalculatorTest {
@Test public void add()
@Test public void sub()
@Before public void setupTestData(){ } //executed before every add/sub
@After public void teardownTestData() {} //executed after every add/sub }
A Class containing a set of tests can be decorated
This is useful for expensive setups that do not need to be run for every test, such as setting up a database connection.
public class CalculatorTest {
@BeforeClass // executed at instantiation of class
public static void setupDatabase Connection() { ... }
@AfterClass // executed after remove instance of class
public static void teardownDatabase Connection() { ... } }

17. Outline of the Lecture Test Model Model-Based Testing
Generation of a test model from the system model
Model-Driven Testing
Distinction: platform independent and platform dependent tests
Testing Activities
Unit testing with JUnit
Java Annotations and Assertions
Object-Oriented Test Modeling
Using object-oriented concepts also for the test model
Object-oriented Testing Patterns
Mock object pattern
Dependency injection pattern (not only a testing pattern)
Inner class test pattern
Reflection test pattern
Test exception pattern
4 stage testing pattern
Test model as a new abstraction on the same level as the system model
Dependency injection pattern (not only a testing pattern, good for your system health )

18. Object-Oriented Test Modeling
We start with the system model
The system contains the SUT (the unit we want to test)
The SUT does not exist in isolation, it collaborates with other objects in the system model
The test model is derived from the SUT
To be able to interact with collaborators, we add objects to the test model
These objects are called test doubles
Mock objects -> not synonyms
System Model
Test Model
System under Test
(SUT)
Double 1
Double 2
Double 3
Collaborators
(Objects interacting with the SUT)

19. Object-Oriented Test Modeling
We start with the system model
The system contains the SUT (the unit we want to test)
The SUT does not exist in isolation, it collaborates with other objects in the system model
The test model is derived from the SUT
To be able to interact with collaborators, we add objects to the test model
These objects are called test doubles
These doubles are substitutes for the Collaborators during testing
Mock objects -> not synonyms
System Model
Test Model
System under Test
(SUT)
Double 1
Double 2
Double 3
Collaborators
(Objects interacting with the SUT)
Collaborators
(Objects interacting with the SUT)

20. Subclasses of Test Doubles
A test double is like a double in the movies („stunt double“) replacing the movie actor, whenever it becomes dangerous
A test double is used if the collaborator in the system model is awkward to work with
There are 4 types of test doubles. All doubles try to make the SUT believe it is talking with its real collaborators:
Dummy object: Passed around but never actually used. Dummy objects are usually used to fill parameter lists
Fake object: A fake object is a working implementation, but usually contains some type of “shortcut” which makes it not suitable for production code (Example: A database stored in memory instead of a real database)
Stub: Provides canned answers to calls made during the test, but it is not able to respond to anything outside what it is programmed for
Mock object: Mocks are able to mimic the behavior of the real object. They know how to deal with sequence of calls they are expected to receive.
For more details see Martin Fowler‘s online paper:
XXX title difference between superclass and subclasses?
Mock objects -> behaviour, interaction
rest -> state

21. Motivation for the Mock Object Pattern
Let us assume we a system model for an auction system with 2 types of policies. We want to unit test Auction, which is our SUT
Instanzen
Dynamic polymorphism.
PersonDummy -> later in animation -> Mock
Real objects in addition

22. Motivation for the Mock Object Pattern
Let us assume we a system model for an auction system with 2 types of policies. We want to unit test Auction, which is our SUT
The mock object test pattern is based on the idea to replace the interaction with the collaborators in the system model, that is Person, the Bidding Policy and the TimingPolicy by mock objects
These mock objects can be created at startup-time with a factory pattern.
Instanzen
Dynamic polymorphism.
PersonDummy -> later in animation -> Mock
Real objects in addition

23. Mock-Object Pattern
In the mock object pattern a mock object replaces the behavior of a real object called the collaborator and returns hard-coded values
These mock objects can be created at startup-time with a factory pattern
Mock objects can be used for testing state of individual objects as well as the interaction between objects, that is, to validate that the interactions of the SUT with collaborators behave is as expected.

24. Example: System Model of a Simple Inventory System

25. Java Code for the Simple Inventory System
public class InventorySystem { private static String TALISKER = "Talisker"; private int totalStock; private Warehouse warehouse = new WarehouseImpl(); public void addToWarehouse(String item, int amount) { warehouse.add(item, amount); totalStock += amount; } public int getTotalStock() { return totalStock; public boolean processOrder(Order order) { order.fill(warehouse); return order.isFilled(); public static void main(String[] args) { InventorySystem inventorySystem = new InventorySystem(); inventorySystem.addToWarehouse(TALISKER, 50); boolean order1success = inventorySystem.processOrder(new OrderImpl(TALISKER, 50)); boolean order2success = inventorySystem.processOrder(new OrderImpl(TALISKER, 51)); System.out.println("Order1 succeeded? " + order1success + " - Order2 succeeded? " + order2success); ?

26. UML Model of the InventorySystem
Note that in our Java example, the Client is realized as the main() method in the class InventorySystem

27. Java Code for the Warehouse
public interface Warehouse { public boolean hasInventory(String item, int amount); public int getInventory(String item); public void add(String item, int amount); public void remove(String item, int amount); } public class WarehouseImpl implements Warehouse { private Map<String, Integer> inventory = new HashMap<String, Integer>(); public void add(String item, int amount) { inventory.put(item, amount); public int getInventory(String item) { return inventory.get(item); public boolean hasInventory(String item, int amount) { return inventory.get(item) >= amount; public void remove(String item, int amount) { inventory.put(item, inventory.get(item) - amount);

28. Java Code for the Order
public interface Order { public boolean isFilled(); public void fill(Warehouse warehouse); } public class OrderImpl implements Order { private String item; private int amount; private boolean filled = false; public OrderImpl(String item, int amount) { this.item = item; this.amount = amount; public void fill(Warehouse warehouse) { if(warehouse.hasInventory(item, amount)) { warehouse.remove(item, amount); filled = true; public boolean isFilled() { return filled;

29. Test Model of the InventorySystem
Assume we now want to unit testthe class Order, our SUT

30. A test case for testing the State
public class OrderStateTester { private static String TALISKER = "Talisker"; private Warehouse warehouse = new public void setUp() throws Exception { warehouse.add(TALISKER, 50); public void orderIsFilledIfEnoughInWarehouse() { Order order = new Order(TALISKER, 50); order.fill(warehouse); assertTrue(order.isFilled()); assertEquals(0, public void orderDoesNotRemoveIfNotEnough() { Order order = new Order(TALISKER, 51); assertFalse(order.isFilled()); assertEquals(50, warehouse.getInventory(TALISKER));
XXX Collaborator -> external Actor?
XXX Mocks as Proxy? +

31. What if we not only want to test the state, but also the interaction of the SUT (Order) with its collaborator (Warehouse)?
Here the Mock Object Pattern comes into play.
It will be covered in more detail in the next lecture.

32. Readings
Kent Beck, Erich Gamma, Junit Cookbook JUnit Source Forge: JUnit 4: JUnit Fixtureshttp:// Num=5 Martin Fowler, Mocks are not Stubs

33. Additional Slides

34. Difference between Stub and Mock
Assume the real systems send an message if a user failed to fill an order using this interface:
public interface MailService { public void send (Message msg); }
During testing we don't want to send actual messages out to customers. So we want to create a test double
If we were writing a test for the mailing behavior, we can write a simple stub and driver:
Stub:
public class MailServiceStub implements MailService { private List<Message> messages = new ArrayList<Message>(); public void send (Message msg) { messages.add(msg); }
public int numberSent() { return messages.size();} }
Driver:
class OrderStateTester...
public void testOrderSendsMailIfUnfilled() {
Order order = new Order(TALISKER, 51);
MailServiceStub mailer = new MailServiceStub();
order.setMailer(mailer);
order.fill(warehouse);
assertEquals(1, mailer.numberSent()); }

35. Applicability of the Mock Object Pattern
The unit test needs to test an object with nondeterministic behavior
Violation of good test design: Repeatability
Example: Current weather temperature
An object is difficult to set up
Violation of good test design: Performance
Example: The set up takes too long for running many unit tests
A specific behavior is hard to trigger
Example: network error
The methods of an object are very slow
Example: Climate modeling
The object has a user interface or is the user interface itself
The unit test needs to confirm that a callback function was actually called
The real object cannot be tested, because it does not (yet) exist
This is a common problem when interfacing with subsystems built by other teams or when interfacing to new hardware systems.
Problems in unit testing: