1. Chapter 18 Software Testing Strategies

2. Testing Strategy unit test integration test system validation test

3. Unit Testing
module
to be
tested
results
software
engineer
test cases

4. Unit Testing module to be tested interface local data structures
boundary conditions
independent paths
error handling paths
test cases

5. Unit Test Environment driver Module stub stub test cases RESULTS
interface
local data structures
Module
boundary conditions
independent paths
error handling paths
stub
stub
test cases
RESULTS

6. Integration Testing Strategies
Options:
• the “big bang” approach
• an incremental construction strategy

7. Top Down Integration A top module is tested with stubs B F G
stubs are replaced one at
a time, "depth first" C
as new modules are integrated,
some subset of tests is re-run D E

8. Bottom-Up IntegrationF G
drivers are replaced one at a
time, "depth first" C
worker modules are grouped into
builds and integrated D E
cluster

9. Sandwich Testing cluster A Top modules are tested with stubs B F G C
Worker modules are grouped into
builds and integrated D E
cluster

10. High Order Testing validation test system test alpha and beta test
other specialized testing

11. Debugging: A Diagnostic Process

12. The Debugging Process test cases results Debugging new test cases
regression
tests
suspected
causes
corrections
Debugging
identified
causes

13. Debugging Effort time required to diagnose the symptom and
determine the
cause
time required
to correct the error
and conduct
regression tests

14. Symptoms & Causes symptom cause symptom and cause may be
geographically separated
symptom may disappear when
another problem is fixed
cause may be due to a
combination of non-errors
cause may be due to a system
or compiler error
cause may be due to
symptom
assumptions that everyone
cause
believes
symptom may be intermittent

15. Consequences of Bugs infectious damage catastrophic extreme serious
disturbing
annoying
mild
Bug Type
Bug Categories:
function-related bugs,
system-related bugs, data bugs, coding bugs,
design bugs, documentation bugs, standards
violations, etc.

16. Debugging Techniques brute force / testing backtracking induction
deduction

17. Debugging: Final Thoughts1.
Don't run off half-cocked,
think
about the
symptom you're seeing. 2.
Use tools
(e.g., dynamic debugger) to gain
more insight. 3.
If at an impasse,
get help
from someone else. 4.
Be absolutely sure to
conduct regression tests
when you do "fix" the bug.

18. Testing approaches Architectural validation System demonstration
Top-down integration testing is better at discovering errors in the system architecture
System demonstration
Top-down integration testing allows a limited demonstration at an early stage in the development
Test implementation
Often easier with bottom-up integration testing
Test observation
Problems with both approaches. Extra code may be required to observe tests

19. Interface testing
Takes place when modules or sub-systems are integrated to create larger systems
Objectives are to detect faults due to interface errors or invalid assumptions about interfaces
Particularly important for object-oriented development as objects are defined by their interfaces

20. Interfaces types Parameter interfaces Shared memory interfaces
Data passed from one procedure to another
Shared memory interfaces
Block of memory is shared between procedures
Procedural interfaces
Sub-system encapsulates a set of procedures to be called by other sub-systems
Message passing interfaces
Sub-systems request services from other sub-systems

21. Interface errors Interface misuse Interface misunderstanding
A calling component calls another component and makes an error in its use of its interface e.g. parameters in the wrong order
Interface misunderstanding
A calling component embeds assumptions about the behaviour of the called component which are incorrect
Timing errors
The called and the calling component operate at different speeds and out-of-date information is accessed

22. Interface testing guidelines
Design tests so that parameters to a called procedure are at the extreme ends of their ranges
Always test pointer parameters with null pointers
Design tests which cause the component to fail
Use stress testing in message passing systems
In shared memory systems, vary the order in which components are activated

23. Stress testing
Exercises the system beyond its maximum design load. Stressing the system often causes defects to come to light
Stressing the system test failure behaviour.. Systems should not fail catastrophically. Stress testing checks for unacceptable loss of service or data
Particularly relevant to distributed systems which can exhibit severe degradation as a network becomes overloaded