Illinois Institute of Technology CS487 Software Engineering Testing - Part II Software Testing Strategies Mr. David A. Lash © Carl J. Mueller, 2000

Why Software Testing Strategy? Some Test Strategy Issues: Should develop a formal test plan? Test entire program as a whole or various pieces? When should customer be included? Testing often accounts for more effort then development Usually strategy is defined in an overall testing strategy and procedure document (often by independent test group)

Elements Of Software Test Strategy Unit Testing - Tests the implemented source code of the component. Uses White Box. Integration Testing - focuses on the design, testing and integration of components. Black box and limited White Box. Validation Testing - Looks at requirements a validates software against it. Black Box Testing. System Testing - Looks at software and other system elements tested as a whole. (Other hardware, people, databases, processes.)

Unit Testing Unit Testing - Module test Use drivers (accepts data interfaces are tested to ensure information flows into and out of the program correctly. Data structures are tested. Independent paths (basis paths) are exercised) Error handling paths - A mistake to miss these. Use drivers (accepts data and tests and passes to module) and stubs (replace other modules)

Integration A systematic technique to uncover problems while putting the modules together. Composed of two activities: Building Compile and link all of the modules together Testing Verify interface function as specified Avoid the “big bang” approach A.K.A. non- incremental integration can yield chaotic results.

Implementation Process

Approaches to Integration Testing Top-down (depth-first or breadth-first) Test higher level modules first Require stubs (not just returns) Bottom-up Test lower-level modules first Require drivers (require logic) Sandwich Test highest level modules in conjunction with lowest level meeting in the middle Use stubs and drivers

Example System

Top Down (depth) Example Depth First Integrate M1, use stubs for M2, M3 and M5 . Test all M1 interfaces Next would test M2 and use operational stubs for M3, M4, M5, M6 .Test all M2 interfaces Repeat for M5, then M8, M6, and M3...

Top Down (depth) Example Integrate M1, use stubs for M2, M3 and M5 . Test all M1 interfaces Next would test M2 and use operational stubs for M3, M4, M5, M6 .Test all M2 interfaces Replace stub M5 with module M5. Use operational stubs for M3, M4, M6, M8. Test all the interfaces in M5 Repeat for M8, M6, and M3...

Top Down (breath) Example Breath First Order M1, M2, M3, M4, Next level would be M5, M6, ... Next Top down testing can have problems when low level modules are needed (delay full tests or develop better stubs or go bottom up)

Bottom Up Lower Items first using drivers : Using an M8 driver test module M8 Using an M7 driver test module M7 Using an M6 driver test module M6 Using an M5 driver test module M5

Bottom Up Example II In reality lower items are clustered together into sub-functions (aka builds).

Sandwich Example First Round Top-down Use operational stubs for M2, M3, M4 Test all the interfaces in M1 Bottom-up Using an M8 driver test module M8 Using an M7 driver test module M7 Using an M6 driver test module M6

Regression Testing The reexecution of some subset of tests to ensure changes have not propagated errors. Tests conducted when a previously tested software is delivered Continuing testing after repair New Version Partial testing old features when testing a maintenance release. Might include tests of features/functions that were unchanged doing the current release. Capture/playback tools enable re-execute of tests previously run.

Verification (1) The process of determining whether or not the products of a given phase of the software development cycle fulfill the requirements established during the previous phase (2) Formal proof of program correctness (3) The act of reviewing, inspecting, testing, checking, auditing, or otherwise establishing and documenting whether or not items, processes, services and documents conform to specified requirements

Functional Verification Verification of the Functional Design Specification Should be done by developers Normally done by user or testing group Can be combined with integration testing

Are we building the right product? Does it meet the requirements? Validation Are we building the right product? Does it meet the requirements?

Validation Phase

Validation The process of evaluating software at the end of the software development process to ensure compliance with software requirements. Test plans define classes of test to be completed

Validation Activities

Validation Process

System Tests

System Tests

System Tests

System Tests

Rules for High-Level Tests 1. Test are run on operational production hardware and software. 2. Tests must stress the system significantly 3. All interfaced systems must be in operation throughout the test. 4. Test duration should run a full cycle 5. Tests should exercise the system over a full range of inputs, both legal and illegal 6. All major functions and interfaces should be exercised. 7. If running the entire test cannot be completed, a new run should include a complete restart.

Acceptance Testing Depends on the type of product and the situation. When working with an a non-technical user, the following are recommended. First, train user on system with a number of test cases. Use a comparison test. Re-processing Parallel to existing system

Acceptance Testing Product Development Alpha testing Performed by a selected end-users usually in side the development environment. Beta testing Performed by a subset of actual customers outside the company, before the software is made available to the customer. Field Trial Large number of users, but not general release.

Acceptance Testing Work for Hire Training Programming Evaluate based on comments users general reaction to system. Usability Test Parallel Operation Comparison Testing New software used after old system process. Controlled deployment Representative monitors software at site.