Software Development Software Testing

Testing Definitions There are many tests going under various names. The following is a general list to get a feel for the different types of testing required by during typical high quality software development.

Acceptance Test The test performed by users of a new or changed system in order to approve the system and go live Usually carried out at the customer’s premises or using customer’s data with the customer being present This is usually the final phase of development and results in the handover of the software product to the customer

Active / Passive Testing An active test introduces specific test data and the results of processing that data are observed for correctness Passive testing works on real data and the outputs are observed for correctness

Alpha and Beta Testing Alpha testing is the first test phase carried out by the developers in the lab Beta testing is carried out when alpha testing is complete and the developers can find no errors (ideally). Beta testing is carried out by selected real users who report back to the developers. This enables the users to get early exposure to the software and the developers get usability and error feedback

Automated Testing This is when software is used to test software. This enables test repeatability and is faster as the user is not generally required to enter lots of data manually If the code is changed in a single module, all the tests would be re-run on the whole application to ensure that the changes have not impacted on any other modules. Retesting everything in this way is often referred to as “regression testing”

White and Black Box Testing White box testing requires knowledge of the internal operation of the software. This enables the test to chack all paths through a function for example Black box testing assumes no knowledge of the internal operation and based on the specification will input test data at a system level and observe the correctness of the output

Dirty or Negative Testing This method requires data to be input such that it will maximise the chances of failure so is used to test the application’s error recovery effectiveness Examples could be user input validation through to memory allocation failure

Functional Testing Does the application do what it has been specified to do? – Check all menu and tool bar options – Check all keyboard input Likely to take the form of a task list which a tester goes through to check all options and navigation paths. This is not a usability test but initial feedback on usability could be collected at the same time

Recovery Testing How well does the application recover from hardware and software failure? – Reading a data file which you know is always available will still need error recovery code in case the disk you are reading from fails – How well does the system recover from the dirty test phase? – What happens if the device your application is trying to communicate with does not respond?

Test Case Testing This consists of a set of test data, test programs (test harnesses) and a set of expected results Typically a set of test cases is prepared (this is a test scenario). A file of input test data and expected results for example is read by the test harness (test driver program). A function (unit of code so is often referred to as a unit test) for example is called with the test data and the results are compared with the expected results. The success or failure of the test is reported and the next test is carried out. This method of testing tends to be totally automated if possible

Test Suite This is a collection of test scenarios which themselves are a collection of test cases In oop for example, a test suite may consist of a test scenario for each class. The test scenario may consist of a series of tests for each of the member functions

Usability Testing A series of tests carried out by users or subject experts to determine the ease of use Various techniques such as – Task driven – Observation – Keystroke and mouse recording – Interview – Think aloud – Questionnaires

Test Driven Development This is not a test method as such but is test oriented. Based on the system requirements, a series of tests is specified which, when they pass, mean that a piece of working code has been produced. i.e. the test is specified before the software is written and the software is written to pass the test. Often used in Extreme Programming (XP) but can be used in its own right as a development method

Compatibility Testing Ensures that the software is compatible with the hardware, operating systems, and other software packages that it will be working with such as: – Works on required software platforms such as Windows and Linux – Works on required hardware platform such as 16, 32 and 64 bit systems – Works with various software releases if required such as Win 95, XP, Vista

Performance Testing Ensures that the software runs fast enough to meet speed requirements e.g. executing algorithms in time specified Ensures that the software does not consume more memory than anticipated

Function Stubs When developing an application as part of a team, you may need to call functions (or objects) that do not yet exist (because someone is still writing them – Write a function which does nothing other than enable the caller to call it and return a value or series of values depending on the function spec. – The caller can test the calling function’s response to a variety of return values from the stubs

Test Harness Example Performs 16 and 32 bit compatibility tests Performs Test Case Testing on a function Uses Automated Testing Uses Active Testing Data and results taken from a Test Scenario Function under test is a C function to convert a String to an Integer (like atoi but with extra features). Errors found would need source header updating with mod date and author and an overview of the problem and fix