1. Software Quality and Testing (CSC 4133)
Unit Testing

2. Outline Concept of Unit Testing Static Unit Testing Defect Prevention
Dynamic Unit Testing
Mutation Testing
Debugging
Unit Testing in eXtreme Programming
Tools For Unit Testing

3. Concept of Unit Testing
Unit testing tests a small software unit at a time, which is typically performed by the individual programmer who implemented the unit.
Unit testing typically focuses on the implementation details and uses WBT techniques, with various coverage criteria as the exit criteria.
Unit testing typically focuses on the executable statements and related control & data elements.
BBT could also be performed on the unit, while focusing on the input-output relations.

4. Unit Testing
A unit is the smallest testable part of an application like functions/methods, classes, procedures.
Unit testing is a method by which individual units of source code are tested to determine if they are fit for use.
Unit tests are basically written and executed by software developers to make sure that code meets its design and requirements and behaves as expected.
The goal of unit testing is to segregate each part of the program and test that the individual parts are working correctly.

5. Unit Testing What method/technique is used for unit testing?
Unit Testing is performed by using the White Box Testing method.
When is it performed?
Unit Testing is the first level of testing and is performed prior to Integration Testing.
Who performs it?
Unit Testing is normally performed by software developers themselves or their peers. In rare cases it may also be performed by independent software testers.

6. Concept of Unit Testing
Static Unit Testing
Code is examined over all possible behaviors that might arise during run time
Code of each unit is validated against requirements of the unit by reviewing the code
Dynamic Unit Testing
A program unit is actually executed and its outcomes are observed
One observe some representative program behavior, and reach conclusion about the quality of the system
Static unit testing is not an alternative to dynamic unit testing
Static and Dynamic analysis are complementary in nature
In practice, partial dynamic unit testing is performed concurrently with static unit testing. It is recommended that static unit testing be performed prior to the dynamic unit testing.

7. Static Unit Testing
In static unit testing, code is reviewed by applying techniques:
Inspection: It is a step by step peer group review of a work product, with each step checked against pre-determined criteria
Walkthrough: It is review where the author leads the team through a manual or simulated executed of the product using pre-defined scenarios
The idea here is to examine source code in detail in a systematic manner
The objective of code review is to review the code, and not to evaluate the author of the code
Code review must be planned and managed in a professional manner
The key to the success of code is to divide-and-conquer
An examiner inspect small parts of the unit in isolation
Nothing is overlooked
Correctness of all examined parts of the module implies the correctness of the whole module

8. Static Unit Testing (Code Review)
Step 1: Readiness –The author of the unit ensures that the unit under test is ready for review. A unit is said to be ready if it satisfies the following criteria –
Completeness
Minimal functionality
Readability
Complexity
Requirements and design documents

9. Static Unit Testing (Code Review)
Step 2: Preparation –Each reviewer develops the following:
List of questions
Potential Change Request (CR)
Suggested improvement opportunities

10. Static Unit Testing (Code Review)
Step 3: Examination – the examination process consists of the following:
The author makes a presentation
The presenter reads the code
The record keeper documents the CR
Moderator ensures the review is on track

11. Static Unit Testing (Code Review)
Step 4: Re-work – At the end of the meeting, the record keeper produces a summary of the meeting that includes the following information:
Make the list of all the CRs
Make a list of improvements
Record the minutes of the meeting (optional)
Author works on the CRs to fix the issue

12. Static Unit Testing (Code Review)
Step 5: Validation
CRs are independently validated
Step 6: Exit
A summary report of the meeting minutes is distributed
A Change Request (CR) includes the following details:
Give a brief description of the issue
Assign a priority level (major or minor) to a CR
Assign a person to follow it up
Set a deadline for addressing a CR

13. Steps in the Code Review process

14. Static Unit Testing (Code Review)
Roles of the Code Review Group
Moderator: A review meeting is chaired by the moderator.
Author: Person who has written the code to be reviewed.
Presenter: A presenter is someone other than the author of the code.
Record keeper: Should be different from the author & moderator. The record keeper documents the problems found during the review process & the follow-up actions suggested.
Reviewers: These are the experts in the subject area of the code under review.
Observer : These are the people who want to learn about the code under review.

15. Static Unit Testing (Code Review)
Code Review Metrics: The following metrics can be collected from a code review:
The number of lines of code (LOC) reviewed per hour
The number of CRs generated per thousand lines of code (KLOC)
The number of CRs generated per hour
The total number of hours spend on code review process

16. Static Unit Testing (Code Review)
The code review methodology can be applicable to review other documents.
Five different types of system documents are generated by engineering department
Requirement
Functional Specification
High-level Design
Low-level Design
Code

17. Defect Prevention
It is the best interest of the programmers in general to reduce the number of CRs generated during code review( Because CRs are indications of potential problems in the code, and those problems must be resolved before different program units are integrated).
Preventive measures can be taken during code development to reduce the number of faults in a program.
It is essential to adopt the concept of defect prevention during code development.
In practice, defects are inadvertently introduced by programmers. Those accidents can be reduced by taking preventive measures.
It is useful to develop a set of guidelines to construct code for defect minimization.

18. Dynamic Unit Testing
Execution-based unit testing is referred to as dynamic unit testing.
In this testing, a program unit is actually executed in isolation, and the outcomes of program execution are observed.
However, this execution differs from ordinary execution in the following way:
A unit under test (UUT) is taken out of its actual execution environment
The actual execution environment is emulated by writing more code so that the unit and the emulated environment can be compiled together
The above compiled aggregate is executed with selected inputs

19. Dynamic Unit Testing
The environment of a unit is emulated and tested in isolation
The caller unit is known as test driver
A test driver is a program that invokes the unit under test (UUT)
It provides input data to UUT and report the test result
The emulation of the units called by the UUT are called stubs
It is a “dummy sub-program” that replaces a unit that is called by the UUT
The test driver and the stubs are together called scaffolding
The low-level design document provides guidance for selection of input test data

20. Dynamic Unit Testing
Figure : Dynamic unit test environment

21. Dynamic Unit Testing
The low-level design document provides guidance for the selection of input test data that are likely to uncover faults. Selection of test data is broadly based on the following techniques:
Control flow testing
Draw a control flow graph (CFG) from a program unit
Select a few control flow testing criteria
Identify a path in the CFG to satisfy the selection criteria
Derive the path predicate expression from the selection paths
By solving the path predicate expression for a path, one can generate the data

22. Dynamic Unit Testing Data flow testing
Draw a data flow graph (DFG) from a program unit and then follow the procedure described in control flow testing
Domain testing
Domain errors are defined and then test data are selected to catch those faults
Functional program testing
Input/output domains are defined to compute the input values that will cause the unit to produce expected output values

23. Mutation Testing
Mutation testing is a technique that focuses on measuring the adequacy of test data (or test cases).
The original intention behind mutation testing was to expose and locate weaknesses in test cases.
So, Mutation testing is a way to measure the quality of test cases, and the actual testing of program is an added benefit
Mutation testing is not a testing strategy like control flow or data flow testing. It should be used to supplement traditional unit testing techniques.

24. Mutation Testing
A mutation of a program is a modification of the program created by introducing a single, small, legal syntactic change in the code.
A modified program is called mutant.
A mutant is said to be killed when the execution of test case causes it to fail. The mutant is considered to be dead.
A mutant is an equivalent to the given program if it always produce the same output as the original program.
A mutant is called killable or stubborn, if the existing set of test cases is insufficient to kill it.
A mutation score for a set of test cases is the percentage of non-equivalent mutants killed by the test suite.
The test suite is said to be mutation-adequate if its mutation score is 100%

25. Mutation Testing Consider the following program P main(argc,argv)
int argc, r, i;
char *argv[];
{ r = 1;
for i = 2 to 3 do
if (atoi(argv[i]) > atoi(argv[r])) r = i;
printf(“Value of the rank is %d \n”, r);
exit(0); }
Test Case 1:
input: 1 2 3
output: Value of the rank is 3
Test Case 2:
input: 1 2 1
output: Values of the rank is 2
Test Case 3:
input: 3 1 2
output: Value of the rank is 1
Mutant 1: Change line 5 to for i = 1 to 3 do
Mutant 2: Change line 6 to if (i > atoi(argv[r])) r = i;
Mutant 3: Change line 6 to if (atoi(argv[i]) >= atoi(argv[r])) r = i;
Mutant 4: Change line 6 to if (atoi(argv[r]) > atoi(argv[r])) r = i;
Execute modified programs against the test suite, you will get the results:
Mutants 1 & 3: Programs will pass the test suite, i.e., mutants 1 & 3 are not killable
Mutant 2: Program will fail test cases 2
Mutant 1: Program will fail test case 1 and test cases 2
Mutation score is 50%, assuming mutants 1 & 3 non-equivalent

26. Mutation Testing Mutation testing makes two major assumptions:
Competent Programmer hypothesis
Programmers are generally competent and they do not create “random” programs
Coupling effects
Complex faults are coupled to simple faults in such a way that a test suite detecting simple faults in a program will detect most of the complex faults

27. Debugging
The process of determining the cause of a defect and correcting it is known as debugging. Debugging occurs as a consequence of a test revealing a defect.
After a program failure, the programmer identifies the corresponding fault and fixes it. Programmers can locate faults by debugging a unit.
The objective of debugging is to precisely identify the cause of a failure. Given the symptoms of a problem, the purpose is to isolate & determine its specific cause.
Debugging is a time consuming and error-prone process.
Debugging involves a combination of systematic evaluation, intuition and a little bit of luck.

28. Debugging There are three approaches to debugging Brute force
Cause elimination
Backtracking

29. Unit Testing in eXtreme Programming
Pick a requirement, i.e., a story
Write a test case that will verify a small part of the story and assign a fail verdict to it
Write the code that implement particular part of the story to pass the test
Execute all tests
Rework on the code, and test the code until all tests pass
Repeat step 2 to step 5 until the story is fully implemented
Figure : Test-first process in XP

30. Unit Testing in eXtreme Programming
A TDD(test driven development) approach to code development is used in XP methodology. The key aspect of TDD approach is that a programmer writes low-level tests before writing production code. This is referred to as test-first in software development.
In XP, a few unit tests are coded first, then a simple, partial system is implemented to pass the tests. Then, one more new unit test is created, and additional code is written to pass the new test, but not more, until a new unit test is created. The process is continued until nothing is left to test.
In the XP process, unit tests are created prior to coding – this is known as test-first. The test-first approach sets up checks and balances to improve the chances of getting things right the first time.

31. Unit Testing in eXtreme Programming
Pair programming:
In XP, the code is being developed by two programmers working together side by side.
The two programmers sit side by side in front of the monitor. One person develops the code tactically and the other one inspects it methodically by keeping in mind the story they are implementing.

32. JUnit – A Framework for Unit Testing
JUnit: It is a framework for performing unit testing of Java programs.
Other frameworks: NUnit (C#), CPPUnit (C++), fUnit (Fortran)
Intuitive steps to test a method in Java (Ex. Move() method of PlanetClass)
Create an object instance of PlanetClass. Call it Mars.
Select values of all input parameters of Move().
Compute the expected value to be returned by Move(). Let it be y.
Execute method Move() on Mars with the selected input values.
Let Move() return a value called z
Compare the actual output (z) returned by Move() with the expected value (y).
If (z == y), Move() passes the test; otherwise it fails.
 Report the result.

33. Tools For Unit Testing
Programmers can benefit from using tools in unit testing by reducing testing time without sacrificing thoroughness.
The well-known tools in everyday life are an editor, a compiler, an operating system, and a debugger.
However, in some cases, the real execution environment of a unit may not be available to a programmer while the code is being developed. In such cases, an emulator of the environment is useful in testing and debugging the code.
Other kinds of tools that facilitate effective unit testing are in the next slide…

34. Tools For Unit Testing Static code (path) analyzer
Code auditor
Documenters
Interactive debuggers
Static code (path) analyzer
Software inspection support
Test coverage analyzer
Test data generator
Test harness
Performance monitors
Network analyzers
Simulators and emulators
Version control