1. Illinois Institute of Technology
CS487
Software Engineering
Testing - Part II
Software Testing Strategies
Mr. David A. Lash
© Carl J. Mueller, 2000

2. Why Software Testing Strategy?
Some Test Strategy Issues:
Should develop a formal test plan?
Test entire program as a whole or various pieces?
When should customer be included?
Testing often accounts for more effort then development
Usually strategy is defined in an overall testing strategy and procedure document (often by independent test group)

3. Elements Of Software Test Strategy
Unit Testing - Tests the implemented source code of the component. Uses White Box.
Integration Testing - focuses on the design, testing and integration of components. Black box and limited White Box.
Validation Testing - Looks at requirements a validates software against it. Black Box Testing.
System Testing - Looks at software and other system elements tested as a whole. (Other hardware, people, databases, processes.)

4. Unit Testing Unit Testing - Module test Use drivers (accepts data
interfaces are tested to ensure information flows into and out of the program correctly.
Data structures are tested.
Independent paths (basis paths) are exercised)
Error handling paths - A mistake to miss these.
Use drivers (accepts data
and tests and passes
to module) and stubs
(replace other modules)

5. Integration
A systematic technique to uncover problems while putting the modules together.
Composed of two activities:
Building
Compile and link all of the modules together
Testing
Verify interface function as specified
Avoid the “big bang” approach A.K.A. non- incremental integration
can yield chaotic results.

6. Implementation Process

7. Approaches to Integration Testing
Top-down (depth-first or breadth-first)
Test higher level modules first
Require stubs (not just returns)
Bottom-up
Test lower-level modules first
Require drivers (require logic)
Sandwich
Test highest level modules in conjunction with lowest level meeting in the middle
Use stubs and drivers

8. Example System

9. Top Down (depth) Example
Depth First
Integrate M1, use stubs for M2, M3 and M5 . Test all M1 interfaces
Next would test M2 and use operational stubs for M3, M4, M5, M6 .Test all M2 interfaces
Repeat for M5, then M8, M6, and M3...

10. Top Down (depth) Example
Integrate M1, use stubs for M2, M3 and M5 . Test all M1 interfaces
Next would test M2 and use operational stubs for M3, M4, M5, M6 .Test all M2 interfaces
Replace stub M5 with module M5. Use operational stubs for M3, M4, M6, M8. Test all the interfaces in M5
Repeat for M8, M6, and M3...

11. Top Down (breath) Example
Breath First Order
M1, M2, M3, M4,
Next level would be M5, M6, ...
Next Top down testing can have problems when low level modules are needed (delay full tests or develop better stubs or go bottom up)

12. Bottom Up Lower Items first using drivers :
Using an M8 driver test module M8
Using an M7 driver test module M7
Using an M6 driver test module M6
Using an M5 driver test module M5

13. Bottom Up Example II
In reality lower items are clustered together into sub-functions (aka builds).

14. Sandwich Example First Round Top-down
Use operational stubs for M2, M3, M4
Test all the interfaces in M1
Bottom-up
Using an M8 driver test module M8
Using an M7 driver test module M7
Using an M6 driver test module M6

15. Regression Testing
The reexecution of some subset of tests to ensure changes have not propagated errors.
Tests conducted when a previously tested software is delivered
Continuing testing after repair
New Version
Partial testing old features when testing a maintenance release.
Might include tests of features/functions that were unchanged doing the current release.
Capture/playback tools enable re-execute of tests previously run.

16. Verification
(1) The process of determining whether or not the products of a given phase of the software development cycle fulfill the requirements established during the previous phase
(2) Formal proof of program correctness
(3) The act of reviewing, inspecting, testing, checking, auditing, or otherwise establishing and documenting whether or not items, processes, services and documents conform to specified requirements

17. Functional Verification
Verification of the Functional Design Specification
Should be done by developers Normally done by user or testing group
Can be combined with integration testing

18. Are we building the right product? Does it meet the requirements?
Validation
Are we building the right product?
Does it meet the requirements?

19. Validation Phase

20. Validation
The process of evaluating software at the end of the software development process to ensure compliance with software requirements.
Test plans define classes of test to be completed

21. Validation Activities

22. Validation Process

23. System Tests

24. System Tests

25. System Tests

26. System Tests

27. Rules for High-Level Tests
1. Test are run on operational production hardware and software.
2. Tests must stress the system significantly
3. All interfaced systems must be in operation throughout the test.
4. Test duration should run a full cycle
5. Tests should exercise the system over a full range of inputs, both legal and illegal
6. All major functions and interfaces should be exercised.
7. If running the entire test cannot be completed, a new run should include a complete restart.

28. Acceptance Testing Depends on the type of product and the situation.
When working with an a non-technical user, the following are recommended.
First, train user on system with a number of test cases.
Use a comparison test.
Re-processing
Parallel to existing system

29. Acceptance Testing Product Development Alpha testing
Performed by a selected end-users usually in side the development environment.
Beta testing
Performed by a subset of actual customers outside the company, before the software is made available to the customer.
Field Trial
Large number of users, but not general release.

30. Acceptance Testing Work for Hire Training Programming
Evaluate based on comments users general reaction to system.
Usability Test
Parallel Operation
Comparison Testing
New software used after old system process.
Controlled deployment
Representative monitors software at site.