Annoucements  Next labs 9 and 10 are paired for everyone. So don’t miss the lab.  There is a review session for the quiz on Monday, November 4, at 8:00 in rooms and  The notes for the lectures are available online.  Additional handouts on OOD and things discussed last week and this week are available outside my office.

What is Testing?  Testing is the process of finding errors in the system implementation.  The intent of testing is to find problems with the system.

What is Debugging?  Debugging is the process of finding the source of errors and fixing such errors.  Testing is done before debugging.

Why Test?  The purpose of testing is to identify implementation errors before the product is shipped.  The errors may be:  Actual bugs in the code.  Incorrect implementation of the requirements or functional specifications. Misunderstandings Incomplete requirements or functional specifications.

What Testing is not  Testing is not a random process.  There is a rhyme and reason to the process.  Testing is not debugging.  Testing identifies the problems.  Debugging finds the location of a problem and fixes the problem.

Who is responsible for Testing?  Multiple people are responsible for testing a system.  Initially the programmers are responsible for testing their implementation but this is not systems testing.  Usually a testing team will perform the majority of the tests, particularly at the system level.  The customer will also test the entire system. Alpha and Beta testing.

Who is responsible for Debugging?  There is also a number of people who are responsible for debugging.  If there is a testing team responsible for testing the system, this team will also attempt to precisely identify the problem and report it to the appropriate programmer.  The programmer is responsible for determining the actual problem and repairing it.  The customer should not debug a system.

When does Testing Begin?  Testing begins during the implementation phase.  The programmer is responsible for testing their unit to ensure the code meets the design and functional specifications.  As multiple units become available and can be combined, system testing can begin by the testing team. It is not unusual for implementation and testing phases to overlap. This is particularly true with today’s shorter development cycles.

What is tested?  The system is tested by:  Units of the system. In object oriented programming the classes would be tested.  Related units of the system.  The entire system.

How are Tests Passed?  A system passes the tests if it produces results that are consistent with the functional specification and requirements.  The program does what it is supposed to do.  The program does not do anything it is not supposed to.

How are Tests Passed?  If any single unit test fails, then the entire system is not correct.  If all unit tests pass, then there is a good probability that the entire system will work together.

Types of Testing  Formal verification is a process that uses mathematical and logical assertions to prove that the program is correct.  Formal verification is difficult to do.

Types of Testing  Empirical testing is the process of generating test cases and running the tests to show that errors exist.  Empirical testing involves observing the results of using the system.  Empirical testing can only prove that an error exists. It can not prove that there are no errors.

Empirical Testing  There are two types of Empirical Testing.  White box testing: Requires access to the actual implementation code. Requires the development of test cases that will exercise each unit of the system, and possible “flows” through the system based upon the actual implementation. All statements, all decisions, all conditions, and all inputs. This type of testing is not very practical but sometimes it is required.

Empirical Testing  Methodologies that are used for White Box testing are: Statement coverage Decision coverage Condition coverage Decision/condition coverage Multiple-condition coverage

Empirical Testing  Black Box Testing: Typically a testing team develops use cases based upon the requirements and functional specification without looking at the actual implementation. Tests valid and invalid inputs but can not possibly test all inputs. Must determine what subset of inputs will sufficiently cover all inputs. You want to break the system, it is your job with Black Box Testing.

Empirical Testing  Methodologies for Black Box Testing: Equivalence partitioning: A set of inputs that are processed identically by the program Legal input values Numeric/non-numeric values Boundary Testing Error Guessing What is the difference between white box and black box testing?

Testing Statement Coverage  Statement coverage tests that each statement in the system is executed at least once by the test data.  Testing the statement coverage is necessary but is not sufficient.

Testing Statement Coverage  What problems can you find? if (a > 1) && (b == 2){ x = x / a; } if (a == 2) || ( x > 1) { x++; } Assume that a = 2, b = 2, and that x is properly defined and initialized.

Testing Decision Coverage  Testing for Decision Coverage requires testing every decision for both a true and false outcome.

Testing Decision Coverage if (a > 1) && (b == 2){ x = x / a; } if (a == 2) || ( x > 1) { x++; } Assume a = 2, b = 2, x > 1 Assume a = 1, b = 2, x = 0

Testing Decision Coverage  Testing for Decision Coverage also tests for statement coverage in modern languages.  This is not true of languages that have multiple entry points, contain self- modifying code, etc.

Testing Condition Coverage  Testing Condition Coverage requires testing each possible outcome for every condition within a decision at least once.  The Decision Coverage testing only covered half of the cases in the previous example. The cases are: a > 1 (1) b == 2 (2) a == 2 (3) x > 1 (4)

Testing Condition Coverage if (a > 1) && (b == 2){ x = x / a; } if (a == 2) || ( x > 1) { x++; } Assume a = 2, b = 2, x = 4 Assume a = 1, b = 3, x = 1

Multiple-Condition Coverage  Testing for Multiple-condition coverage requires test cases that test all possible combinations of condition outcomes for every decision tested.  This type of testing will generate many test cases.

Debugging  Debugging should be a formal process of attempting to narrow down the location of the problem and then identifying the problem.  Debugging does not mean simply changing code until the problem goes away.  Debugging requires thinking about what might be the problem.

Debugging  Methods of determining the location of a bug:  Use extra output statements in the program to trace the program execution.  Use a debugger to trace the program execution.  Possibly write special test code to exercise parts of the program in special ways that will allow you to better understand the error.

Debugging  Potentially test a certain range of values to see which ones fail.  Attempt to eliminate parts of the program as the problem, thus narrowing your search.  Check that the data is valid.  Many times, the location where you see the first instance of the bug is not the source of the bug.

Fixing Bugs  Steps for fixing bugs.  Fix only one bug at a time and then rerun the same exact tests. Changing multiple things makes id difficult to identify which change caused the behavior change.  If the problem appears to be fixed, still run a full test suite to ensure the “fix” did not break something else.

General Rules to follow  Test your code as your write it:  Test the code boundaries.  Test pre- and post- conditions. The necessary or expected properties before and after the code is executed.  Use assertions (if you are programming in C or C++)  Program defensively by adding code to handle the “can not happen” cases.  Check error returns

General Rules to Follow  Steps for Systematic Testing  Test incrementally by writing part of the system, test it, then write some more code, test that code, etc.  Test the simple parts of the system first.  Know what output you are expecting.  Compare independent implementations of a library or program provide the same answers.

General Rules  Ensure that testing covers every statement of the program. Every line of the program should be exercised by at least one test.