COSC 4506/ITEC 3506 Software Engineering Classic Testing (3)

Outline Testing in the large Module testing Integration testing System testing Acceptance testing Analysis

Testing in the large Module (Unit) testing testing a single module Integration testing integration of modules and subsystems System testing testing the entire system Acceptance testing performed by the customer

Review: V Graph Requirements System testing Analysis High-Level Design Integration testing Low-Level Design Unit testing Here shows the development activities and the associated testing activities for checking the correctness. Coding Unit testing Delivery Acceptance testing Maintenance Regression testing

Module testing Verify: given module has been implemented correctly (it expected external behavior) Example Mi USES Mh, Mi USES Mk, Mh and Mk not available how to test Mi in isolation? Provide temporary context that simulates the real context to be provided by Mh and Mk

Testing a functional module

Module testing stubs: a piece of code (dummy unit) that stimulates the activity of the missing component, but a highly simplified code it answers the calling sequence and passes back output data that lets the testing process continue It might produce its expected results by reading them from a file, returning some random values etc

Module testing driver module (containing main method) activating the module under test a piece of code, simulates use of module being tested by others, and passes test cases to another piece of code

Example Develop a “calculator” that converts between different units (feet to inches) “stubbing out the UnitConverter class All non-void methods return an appropriate default so the class can be compiled public class UnitConverter { public UnitConverter () {} public double convert (double value, String from, String to) { return 0.0;} public double getMultiplier (String from, String to) { return 0.0} // return the multiplier needed for a conversion }

Example public class Driver { public static void main (String [] args) We need to write a preliminary driver to test the UnitConverter class // all the methods calls are all hard coded public class Driver { public static void main (String [] args) double converted, original =10.0; String from, to; UnitConverter calculator = new UnitConverter (); from = “inches”; to = “feet”; converted = calculator.convert (original, from, to); System.out.println (original+” ” +from+ “ =“ +” “ + to); } } // this will compile and run

Example Next version has a complete implementation of convert method public class UnitConverter { public UnitConverter () {} public double convert (double value, String from, String to) double result; result = value * getMultiplier (from, to); return result; } public double getMultiplier (String from, String to) return 1.0/12.0; } // stub

Next version has a complete implementation of getMultiplier method public class UnitConverter { …… public double getMultiplier (String from, String to) { double multiplier =1.0; if (from.equals(“inches”) { if (to.equals (“feet”)) multiplier =1.0/12.0; else if (to.equals(yards”) multiplier =1.0/12.0/3.0; } else if (from.equals (“feet”)) { if (to.equals (“inches”)) multiplier =12.0; else if (to.equals(yards”) multiplier =1.0/3.0; else if (from.equals (“yards”)) { if (to.equals (“inches”)) multiplier =3*12.0; else if (to.equals(“feet”) multiplier =3.0; return multiplier;

Example { public class Driver public static void main (String [] args) We need to revise the driver to have more test cases public class Driver { public static void main (String [] args) double converted, original =10.0; String from, to; UnitConverter calculator = new UnitConverter (); from = “inches”; to = “feet”; converted = calculator.convert (original, from, to); System.out.println (original+” ” +from+ “ =“ +” “ + to); …………………… } }

Example This is a complete version of UnitConverter public class Driver { public static void main (String [] args) { double converted, original; int i, j; String from, to; String [] units ={“inches”, “feet”, “yard”); UnitConverter calculator = new UnitConverter (); original =10.0; for (i=0; i<units.length; i++) for (j=0; j<units.length; j++) { from = units[i]; to = units[j]; converted = calculator.convert (original, from, to); System.out.println (original+” ” +from+ “ =“ +” “ + to); } } }

Integration build strategies All at once Big Bang: All coding precedes all integration Growing Strategies (Incremental Approach) Bottom Up: Start at low-level utility modules Top Down: Start at high-level control modules Sandwich: Integrate control modules top down and utility modules bottom up

Integration testing Big-bang approach code & test each module separately link all 13 modules together, test entire product

Integration testing problems lack of fault isolation product as a whole fails for a test case – where? could be in any of 13 interfaces or 13 modules

Integration testing Problems of Big-Bang Critical modules receive no extra testing Integration has to wait for coding to finish Major design flaws are discovered very late No flexibility in scheduling

Integration testing Incremental approach modules are progressively integrated and tested Advantages Easy to discover errors at early stages easy to localize errors incrementally integration of n modules, then n+1 reduce the need for stubs or drivers can proceed both top-down and bottom-up according to the USES relation Wrt  w.r.t.

Integration testing How about the stubs and drivers? Top-down ? if module mAbove calls module mBelow, mAbove implemented & integrated before mBelow one ordering a, b, c, d, e, f, g, h, i, j, k, l, & m alternative in parallel [a] b, e, h [a] c, d, f, i [a d] g, j, k, l, m How about the stubs and drivers?

Top Down Integration Implement highest level component Create stubs for called component Test the component with the stubs Implement and test stubs one by one use stubs for any called components Repeat until all stubs are implemented

Top Down Integration Disadvantages important low-level utilities are not tested as well (reusable) simple high-level drivers may not need the extra testing Advantages no need to write drivers high-level drivers are well tested unimportant low-level utilities are not as well tested Decision modules test first, and more time

Integration testing How about the stubs and drivers? Bottom-up ? if module m1 calls m2, then m2 is implemented & integrated 1st one ordering l, m, h, i, j, k, e, f, g, b, c, d, a alternative in parallel h, e, b i, f, c, d l, m, j, k, g [d] a [b, c, d] How about the stubs and drivers?

Bottom Up Integration Implement a module Implement a test driver often a quick prototype Execute tests Replace test driver with its implementation Implement a test driver for new module Repeat until all modules are integrated

Bottom Up Integration Advantages no need to write stubs low level utilities are well tested (reusable) high-level (often simple) drivers are not as well tested Disadvantages not all low level utilities are important emphasis on low level functionality major design flaws are discovered late

Integration testing Sandwich ? logic modules implemented & integrated top-down operational modules implemented & integrated bottom-up finally, interfaces between the two groups tested

Integration testing Sandwich approach Advantages major design flaws are discovered earlier low level utilities are well tested (reusable) Fault isolation at all times

Testing in the large: System Testing A series of tests to verify that all system elements have been properly integrated. Purpose of integration testing: to ensure that the code implemented the design properly Purpose of System Testing: to ensure that the system does what the customer wants it to be

Testing in the large: System Testing Function Testing example Performance Testing: Some typical performance testings Recovery Testing: addresses response to 1) presence of faults 2) loss of data, devices, service loss system resources and see if it recovers properly

Testing in the large: System Testing Performance Testing: Overload testing evaluate when goes beyond limit quality testing evaluate system’s reliability, maint. Availability robustness testing test under unexpected cond. Like erroneous user commands, power failure Regression testing When a change is made to a large application, we need to validate the change did not disturb the existing functionality

Testing in the large: System Testing Installation testing to allow users to exercise system functions and document additional problems that result from being the actual site Acceptance testing tested by the customer according to their understanding of requirements sometimes alpha test by users within org. before beta test by customer (OS)

Alpha & Beta test software developer site customer site software customer tests Alpha test Beta test software developer reviews customer tests developer site customer site

Definitions Verification determining if phase completed correctly “are we building the product right?” “Testing” (Execution-based testing) running test cases against executing code Analysis (Synthesis, Nonexecution-based testing) reviewing code or documents carefully why should we also do this for code?

Synthesis: Non-Execution-based Testing Underlying principles cannot review own work – why? team of reviewers – why? Two types informal synthesis walkthroughs inspections key difference? walkthroughs have fewer steps & are less formal inspections record detailed data & use it in later phases & projects Formal synthesis proof

Walkthroughs Team of 4-6 members, chaired by SQA (why?) spec writer & manager, client, designers, SQA more experienced & senior better – why? Distribute info in advance, each reviewer creates list of confusing items list of items in error Process detect suspected faults, don’t correct (why?) document-driven & interactive, not participant-driven verbalization leads to fault finding spontaneously by presenter

Software inspections Involve people examining the source representation with the aim of discovering anomalies and defects Do not require execution of a system so may be used before implementation May be applied to any representation of the system (requirements, design, test data, etc.) Very effective technique for discovering errors

Inspection success Many different defects may be discovered in a single inspection. In testing, one defect ,may mask another so several executions are required The reuse domain and programming knowledge so reviewers are likely to have seen the types of error that commonly arise

Inspections and testing Inspections and testing are complementary and not opposing verification techniques Inspections can check conformance with a specification but not conformance with the customer’s real requirements Inspections cannot check non-functional characteristics such as performance, usability, etc.