Test Driven Development Lecture 8.1 Test Driven Development

Automated Software Testing A process in which software tools execute pre-scripted tests on a software application before it is released into production The objective of automated testing is to simplify as much of the testing effort as possible with a minimum set of scripts Automated testing tools are capable of: Repeatedly executing tests Reporting outcomes Comparing results with earlier test runs The method or process being used to implement automation is called a test automation framework.

Manual vs Automated Software Testing Manual Testing Automatic Testing Definition: executing tests manually without any tool support. Definition: using tool support and executing the test cases by using an automation tool Time-consuming − executing manually is slow, tedious, and harder to track Fast − Automation tools execute tests significantly faster than human resources, and easier to track Less reliable − Manual testing is less reliable, since it has to account for human errors More reliable − Automation tests are precise and reliable. Non-programmable − No programming can be done to write tests to test hidden (private) information Programmable − Testers can implement tests to bring out hidden (private) information as well Increased Cost – As test cases need to be executed manually, more testers are required in manual testing Reduced Cost - Test cases are executed using automation tools, which requires less testers

Test Driven Development Test First Design: First, write a test case and then implement the code! Once code passes the written test, write another. Requirements on the code is enforced by writing a test Test Case implementation enforces the requirement Writing tests is done by creating Unit Tests that enforce: Pre-conditions: A condition that is true before method invocation Post-conditions: A condition that is after before method invocation Invariants: A condition that true before and after method invocation Unit Tests are the testing functions that test the program components For Java JUnit, for C++ CUnit!

Benefits of Test Driven Development Gives a way to the programmer to express correctness criteria on the code Using the terminology of invariant, pre-conditions and post-conditions Creates a set of Test Cases that ensure the validation of correctness: Errors caused by code and design and corrected due to the testing stage Allows easier refactoring and code modifications Due to deeper knowledge of the code Facilitate Design Improvements: Complicated classes that depends on too many things, makes testing difficult Provides an incentive to modify the design to one that allows easier testing Documentation: Test classes provide examples of code usage and useful information looked at as documentation

JUnit - Java Unit Testing Framework Provides tools which allow an easier program testing Friendly interface – with a single click, all tests can be executed and results returned in an easily read manner: Annotations for test methods Assertions for testing results Runners to run tests Unit Testing Expressions: Test Suite – A class bundling a few unit test cases together for execution Unit Test Case - A method testing a single use case for a specific method Positive Test Case – a test case that ensures correct behavior given correct input Negative Test Case – a test case that ensures a failed behavior given incorrect input Object Under Test  - the object being tested by the test suite through that object we are able to test all of its class methods! Positive test case – adding an element to an empty slist ends with isEmpty() returning false Negative test case – attempting to remove an element from an empty list ends up with a designed exception

JUnit Execution Flow

setUp/tearDown setUp()/tearDown() Functions which will run before and after each test. setUp(): creating new objects tearDown(): clearing data, resetting data, freeing memory (C++) If self destruction is good enough, there is no need to implement it. setUpBeforeClass()/tearDownAfterClass() Functions which will run once before and after all the tests. Can be used to initialize objects which will be used throughout the tests. Changes done in first test, can be used as input data for the upcoming one.

JUnit Assert Functions Each assert method has parameters like these: message, expected-value, actual-value assertTrue(String message, Boolean test) assertFalse(String message, Boolean test) assertNull(String message, Object object) assertNotNull(String message, Object object) assertEquals(String message, Object expected, Object actual) (uses equals method) assertSame(String message, Object expected, Object actual) (uses == operator) assertNotSame(String message, Object expected, Object actual) Assert methods dealing with floating point numbers get an additional argument, a tolerance (for rounding) Each assert method has an equivalent version that does not take a message – however, this use is not recommended because: messages helps documents the tests messages provide additional information when reading failure logs

JUnit Example: Counter Class Test Suite 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 public class CounterTest { Counter counter; // default constructor public CounterTest() { } @Before protected void setUp() { counter = new Counter(); } @Test public void testIncrement() { assertTrue(counter.increment() == 1); assertTrue(counter.increment() == 2); public void testDecrement() { assertTrue(counter.decrement() == -1); Each test begins with a brand new counter instance No need to worry about the order of test execution! Counter Interface: int increment(); Increments counter and returns new value int decrement(); Decrements counter and returns new value Questions: Why no tearDown()? Why testDecrement() works correctly? How many times setUp() is executed? How many tests we executed? Any of those fail? In java – deallocating memory is done automatically Because a new counter is instantiated after each @Test 2 3 None fail

JUnit Example: Counter Class Test Suite 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 public class CounterTest { Counter counter; // default constructor public CounterTest() { } @Before protected void setUp() { counter = new Counter(); } @Test public void testIncrement() { assertTrue(counter.increment() == 1); assertTrue(counter.increment() == 2); public void testDecrement() { assertTrue(counter.decrement() == -1); Each test begins with a brand new counter instance No need to worry about the order of test execution! Counter Interface: int increment(); Increments counter and returns new value int decrement(); Decrements counter and returns new value Questions: Why no tearDown()? In java – deallocating memory is done automatically Why testDecrement() works correctly? Because a new counter is instantiated after each @Test How many times setUp() is executed? 2 How many tests we executed? 3 Any of those fail? None In java – deallocating memory is done automatically Because a new counter is instantiated after each @Test 2 3 None fail

Test First Design Cycle Defining Object Under Test (The class to be tested) Defining the Interface Defining DbC for each method Specifying pre-conditions and post-conditions for each method and invariant Writing Test Cases – For each invariant, pre-condition and post-condition for each method Implementing the interface – Execute tests, modify code - until all tests are passed Refactoring Stage Making code simpler Due to code change – testing process is repeated As needed – change contract, change DbC conditions and their test cases

Step by Step: Unit Testing for Stack Implementation Stack object – design: A container of Objects – we opt to make it generic Objects are ordered in Last-In-First-Out order (LIFO) Adding an object is done to the top (end) of the Stack Removing an object is done by removing the top (end) of the Stack Stack Interface: 1 2 3 4 5 6 public class Stack<E>{ void push(E element); E pop(); boolean isEmpty(); }

Step by Step: DbC Conditions 1 2 3 4 5 6 7 8 9 10 public interface Stack<E> { /** * Add E obj to the top of the stack * @param obj - any non null E * @pre: none * @post: this.isEmpty() == false * @post: this.pop(); this == @pre(this) * @post: this.pop() == @param obj */ void push(E obj); 1 2 3 4 5 6 7 8 /** * Remove top E from the stack and return it * @return the the most top E, if exists. Otherwise, null * @throws Exception * @pre: this.isEmpty() == false * @post: none */ E pop() throws Exception; 1 2 3 4 5 6 7 8 9 /** * Checks whether Stack is empty or not * @return True if the Stack is empty * False if the Stack contains E * @pre: none * @post: none */ boolean isEmpty(); }

Step by Step: Implementing a Skeleton Test Class 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 /** * Unit Test for the Stack interface */ public abstract class StackTest<T> { /** Object under test. */ private Stack<T> stack; /** * Test method for {@link spl.util.Stack#isEmpty()}. */ @Test public void testIsEmpty() { fail("Not yet implemented"); } * Test method for Stack#pop() @Test public void testPop() { fail("Not yet implemented"); } * Test method for Stack#push(T). @Test public void testPush() { fail("Not yet implemented"); } }

Step by Step: Defining Test Cases Test Cases need to provide 100% coverage of application scenarios! See https://node.green/ testIsEmpty() Stack instantiation  isEmpty() returns true push() isEmpty() returns false pop()  isEmpty() returns true testPop() Stack instantiation pop()  exception is thrown testPush() Stack instantiation, push(x) pop()  return value == x

Step by Step: Implementing the Test Suite: testIsEmpty() 1 2 3 4 5 6 7 8 9 10 11 1213141516 public class StackTest { /** Object Under Test */ private Stack<Integer> stack; @Before protected void setUp() throws Exception { this.stack = new ArrayListStack<Integer>(); } @Test public void testIsEmpty() { Integer i; assertTrue(stack.isEmpty()); stack.push(i); assertFalse(stack.isEmpty());

Step by Step: Implementing the Test Suite: testPop() 1 2 3 4 5 6 7 8 9 10 11 12 13 14 public class StackTest { @Test public void testPop() { Integer i; stack.push(i); try { stack.pop(); } catch (Exception e) { fail("Unexpected exception: " + e.getMessage()); } assertTrue(true);

Step by Step: Implementing the Test Suite: testPush() 1 2 3 4 5 6 7 8 9 10 11 12 13141617 public class StackTest { /** * Test method for Stack#pop() * This is a negative test - causes an exception to be thrown. * The test ensures failure if pre-condition is not met! */ @Test public void testPop() { try { stack.pop(); fail("Exception expected!"); } catch (Exception e) { assertTrue(True); }

Test Coverage: Cases in Test Driven Development To provide a complete coverage, these cases must be covered: Normal Use Cases – output and input cases – cases that frequently occur These cases are related to parameters, exceptions, and method return values Edge Cases – cases rarely happen, in special scenarios Handling null cases, and exceptions Complex Cases – a combination of simpler cases together Stack Example: push-pop-push, push-push-pop, etc… Programmer must decide on changing an interface – as needed, or detail the limitations enforced by the proposed interface Example: push() not returning a success/failure message in its return value

Step by Step: Implementing the Stack Opt to write minimum amount of code to pass written tests Once all tests pass – our code is a valid Stack implementation Implementation Cycle: Implement one method in the interface Execute its Test Cases If cases fail, debug until all cases pass Repeat until all methods are implemented And all test cases pass! Once done – we have a complete and valid Stack implementation!

Code Refactoring: The Six Principles Separate Commands and Queries Separate Basic Queries from Derived Queries For each Basic Command write Post-Conditions that use Basic Queries Define Derived Queries in terms of Basic Queries For each Basic Command and Query express pre-conditions in terms of basic queries Specify class invariants that impose always-true constraints on basic queries

Commands and Queries Queries: Commands: They request information – and return a value They do not change the visible state of the object They are read only methods – isEmpty() Queries do not have post-conditions Commands: They change the internal state of the object They do not return values They are methods with side effects – push() Commands always have post-conditions

Separate Commands and Queries pop()has two parts: Removes an item from the stack – command! Returns the value of head of stack – query! pop() does not stand by this rule: It is a combination of command and query! Separation: Add two new methods to the interface! top()- returns the value of the top object - query remove()- removes top object from the stack - command New Implementation: T pop() { T top = top(); remove(); return top; }

Separate Basic Queries from Derived Queries isEmpty() has two parts: Checks the number of elements found in the Stack If the value is zero, returns true, otherwise returns false This is a derived query! Separation: Use a basic query to provide the number of elements found in the Stack Adding a new method to the interface: int size(); 1 2 3 45 /** * @returns The number of elements * found in the Stack. */ int size(); 1 2 3 4 5 6 /** * @return True if the Stack is empty * False if the Stack contains at least one element E. * @post: @return (size()==0) */ boolean isEmpty();

For each Basic Command, write Post-Conditions that use Basic Queries The addition of a new basic command remove() requires adding a new post-condition. Remove requires counting! isEmpty() provides a binary value. We can use the size()basic query for the post-condition 1 2 3 4 5 6 7 /** * remove the top object from the stack. * @throws Exception * @pre: isEmpty() == false * @post: size() == @pre(size()) - 1 */ void remove() throws Exception;

Define Derived Queries in terms of Basic Queries This rule aids the programmer to avoid code repetition! If two queries are partially implemented using the same code Then these two queries are derived And the repeated code can be moved to a basic query! isEmpty() has two parts to implement: Checks the number of elements found in the Stack Implemented using the basic query - size() If the value is zero, returns true, otherwise returns false

For each Basic Command and Query express pre-conditions in terms of basic queries pop() uses a basic command and a basic query: top()- returns the value of the top object – basic query remove()- removes top object from the stack – basic command 1 2 3 4 5 6 /** * returns the top object from the stack * @throws Exception * @pre: size() > 0 */ E top() throws Exception; 1 2 3 4 5 6 7 /** * remove the top object from the stack * @throws Exception * @pre: size() > 0 * @post: size() == @pre(size()) - 1 */ void remove() throws Exception;

Specify class invariants that impose always-true constraints on basic queries In the Stack example, we can state at least one class invariant: The container cannot contain a negative number of elements Class invariants must hold for all commands that affect size() The only candidate that could do this is remove() - since it decreases size() This can be done by adding a pre-condition that prevents size() from changing to a negative value 1 2 3 /** * @inv: size() >= 0 */