Presentation is loading. Please wait.

Presentation is loading. Please wait.

What is Testing? Testing is the process of finding errors in the system implementation. –The intent of testing is to find problems with the system.

Similar presentations


Presentation on theme: "What is Testing? Testing is the process of finding errors in the system implementation. –The intent of testing is to find problems with the system."— Presentation transcript:

1 What is Testing? Testing is the process of finding errors in the system implementation. –The intent of testing is to find problems with the system.

2 What is Debugging? Debugging is the process of finding the source of errors and fixing such errors. –Testing is done before debugging.

3 Some Other Definitions An error is an improper action of a software developer in the implementation. A fault is a result of improper logic in the software that could potentially cause a problem. A failure is an improper action of the software during execution.

4 Why Test? The purpose of testing is to identify implementation errors before the product is shipped. The errors may be: –Actual bugs in the code. –Incorrect implementation of the requirements or functional specifications. Misunderstandings Incomplete requirements or functional specifications.

5 What Testing is not Testing is not a random process. –There is a rhyme and reason to the process. Testing is not debugging. –Testing identifies the problems. –Debugging finds the location of a problem and fixes the problem.

6 Who is responsible for Testing? Multiple people are responsible for testing a system. –Initially the programmers are responsible for testing their implementation but this is not systems testing. –Usually a testing team will perform the majority of the tests, particularly at the system level. –The customer will also test the entire system. Alpha and Beta testing.

7 Who is responsible for Debugging? There are also a number of people who are responsible for debugging. –If there is a testing team responsible for testing the system, this team will also attempt to precisely identify the problem and report it to the appropriate programmer. –The programmer is responsible for determining the actual problem and repairing it. –The customer should not debug a system.

8 When does Testing Begin? Testing begins during the implementation phase. –The programmer is responsible for testing their unit to ensure the code meets the design and functional specifications. –As multiple units become available and can be combined, system testing can begin by the testing team. It is not unusual for implementation and testing phases to overlap. This is particularly true with today’s shorter development cycles.

9 Unit Testing Unit testing is the process of testing the individual classes. –Unit testing is easier to complete for a single class because: It is easier to generate stimuli that are guaranteed to produce the desired response. Bugs that are found earlier are always cheaper and easier to fix.

10 What is tested? The system is tested by: –Units of the system. In object oriented programming the classes would be tested. –Related units of the system. –The entire system.

11 How are Tests Passed? A system passes the tests if it produces results that are consistent with the functional specification and requirements. –The program does what it is supposed to do. –The program does not do anything it is not supposed to.

12 How are Tests Passed? If any single unit test fails, then the entire system is not correct. If all unit tests pass, then there is a good probability that the entire system will work together.

13 Types of Testing Formal verification is a process that uses mathematical and logical assertions to prove that the program is correct. –Formal verification is difficult to do.

14 Types of Testing Empirical testing is the process of generating test cases and running the tests to show that errors exist. –Empirical testing involves observing the results of using the system. –Empirical testing can only prove that an error exists. It can not prove that there are no errors.

15 White Box Testing There are two types of Empirical Testing. –White box testing: Requires access to the actual implementation code. Requires the development of test cases that will exercise each unit of the system, and possible “flows” through the system based upon the actual implementation. –All statements, all decisions, all conditions, and all inputs. This type of testing is not very practical but sometimes it is required.

16 White Box Testing –Methodologies that are used for White Box testing are: Statement coverage Decision coverage Condition coverage Decision/condition coverage Multiple-condition coverage

17 White Box Testing White Box Testing occurs with knowledge of the implementation. –Conduct a test using the debugger to directly set the state of an object and observe the reaction. –Conduct a test that queries an object by directly observing the private attributes.

18 Testing Statement Coverage Statement coverage tests that each statement in the system is executed at least once by the test data. –Testing the statement coverage is necessary but is not sufficient.

19 Testing Statement Coverage What problems can you find? if (a > 1) && (b == 2){ x = x / a; } if (a == 2) || ( x > 1) { x++; } Assume that a = 2, b = 2, and that x is properly defined and initialized.

20 Testing Decision Coverage Testing for Decision Coverage requires testing every decision for both a true and false outcome.

21 Testing Decision Coverage if (a > 1) && (b == 2){ x = x / a; } if (a == 2) || ( x > 1) { x++; } Assume a = 2, b = 2, x > 1 Assume a = 1, b = 2, x = 0

22 Testing Decision Coverage Testing for Decision Coverage also tests for statement coverage in modern languages. –This is not true of languages that have multiple entry points, contain self-modifying code, etc.

23 Testing Condition Coverage Testing Condition Coverage requires testing each possible outcome for every condition within a decision at least once. –The Decision Coverage testing we did only covered half of the cases in the previous example. The cases are: –a > 1 (1) –b == 2 (2) –a == 2 (3) –x > 1 (4)

24 Testing Condition Coverage if (a > 1) && (b == 2){ x = x / a; } if (a == 2) || ( x > 1) { x++; } Assume a = 2, b = 2, x = 4 Assume a = 1, b = 3, x = 1

25 Multiple-Condition Coverage Testing for Multiple-condition coverage requires test cases that test all possible combinations of condition outcomes for every decision tested. –This type of testing will generate many test cases.

26 Black Box Testing –Black Box Testing: Typically a testing team develops use cases based upon the requirements and functional specification without looking at the actual implementation. Tests valid and invalid inputs but can not possibly test all inputs. –Must determine what subset of inputs will sufficiently cover all inputs. You want to break the system, it is your job with Black Box Testing.

27 Black Box Testing –Methodologies for Black Box Testing: Equivalence partitioning: –A set of inputs that are processed identically by the program »Legal input values »Numeric/non-numeric values Boundary Testing Error Guessing

28 Black Box Testing Black Box Unit Testing tests contracts. –Create tests that call functions with carefully chosen parameters. –Create tests that call functions in different orders. –Follow the above two types of tests with calls to the query and accessor functions to determine the state of the object.

29 What You Need for Testing To test a class you need two things: –A skeleton, scaffold, or driver program that emulates the class’s clients. Such a program provides enough support and interface to the rest of the system that the section being tested will run. This program provides the test cases. This program should also include queries that report the results of the test cases.

30 What You Need for Testing –Stubs that represent the class’s suppliers (potentially including hardware) The calls to the stubs should also be reported.

31 Automated Tools In order to complete testing of a full system there are lots of tests with many inputs and many output comparisons. –Very tedious to humans, use automation when possible.

32 Automated Tools Types of testing that can be automated are: –Regression testing attempts to ensure that the behavior of a new software release hasn’t changed from the previous release. Only works for behaviors that were implemented in the previous release. –Create self-contained tests that have inputs and expected outputs.

33 Other Testing Tools You should never throw away a test, you may need to go back and verify the test again. –Use a database or some other tool to store your tests and their results. Tracking tools are used to record and track bugs, as well as list code changes and fixes. –Never remove items from the tracking tool until the product is complete.

34 Automated Tools There are automated tools available to perform testing. –Parasoft C++ Test can be used for unit testing. –Rational test studio includes: Purify that looks for memory management problems. Pure Coverage that test statement coverage.

35 Debugging Debugging should be a formal process of attempting to narrow down the location of the problem and then identifying the problem. –Debugging does not mean simply changing code until the problem goes away. –Debugging requires thinking about what might be the problem.

36 Debugging Methods of determining the location of a bug: –Use extra output statements in the program to trace the program execution. –Use a debugger to trace the program execution. –Possibly write special test code to exercise parts of the program in special ways that will allow you to better understand the error.

37 Debugging –Potentially test a certain range of values to see which ones fail. –Attempt to eliminate parts of the program as the problem, thus narrowing your search. –Check that the data is valid. –Many times, the location where you see the first instance of the bug is not the source of the bug.

38 Fixing Bugs Steps for fixing bugs. –Fix only one bug at a time and then rerun the same exact tests. Changing multiple things makes it difficult to identify which change caused the behavior change. –If the problem appears to be fixed, still run a full test suite to ensure the “fix” did not break something else.

39 General Rules to Follow Test your code as your write it: –Test the code boundaries. Test loop, conditional, list, stack, queue and array boundaries. Make sure conditional branches branch the proper way and that loops are executed the proper number of times. Test the natural program and data boundaries by testing for non-existent or empty input, a single input item, an exactly full array, attempt to overfill the array, etc.

40 General Rules to Follow –Test pre- and post- conditions. Ensure that the necessary or expected properties before and after the code is executed hold. –Use assertions Assertions are useful for validating properties of interfaces because they draw attention to inconsistencies between the caller function and the callee function. Assertions may also indicate which is at fault.

41 General Rules to Follow –Program defensively by adding code to handle the “can not happen” cases. Ensure that the program protects itself against incorrect use or illegal data. –Check error returns from library functions and system calls.

42 General Rules to Follow Steps for Systematic Testing –Test incrementally by writing part of the system, test it, then write some more code, test that code, etc. –Test the simple parts of the system first. Test the simplest most frequently used code sections. Only move on when these sections work properly. –Know what output you are expecting otherwise you are wasting your time.

43 General Rules to Follow –Compare independent implementations of a library or program provide the same answers. –Ensure that testing covers every statement of the program. Every line of the program should be exercised by at least one test.


Download ppt "What is Testing? Testing is the process of finding errors in the system implementation. –The intent of testing is to find problems with the system."

Similar presentations


Ads by Google