1. Software Testing Strategies
“How do you organise your software tests?”
TCS2411 Software Engineering

2. TCS2411 Software Engineering
Lecture Objectives
To describe the process of planning for software testing
To understand the different levels of testing that can be done
To describe the different approaches to debugging software
TCS2411 Software Engineering

3. Software Testing Strategy
The approach towards software testing
A set of tests that is planned and conducted to ensure successful construction of software
Guideline or “road map” of testing activities - planning, design, execution, data collection, evaluation
TCS2411 Software Engineering

4. Examples of Software Testing Strategies
Testing begins at module level and works “outward” towards integration of the entire computer-based system
Different testing techniques at different points in time
Testing is conducted by developer and an independent test group
Testing and debugging are different activities, but debugging is accommodated
TCS2411 Software Engineering

5. A Software Testing Strategy
System
Engineering
Requirements
Design
Code
Unit
Test
Integration
Validation
TCS2411 Software Engineering

6. Software Testing Organization
Developer : unit testing
Ensure each functions performs according to its design
Sometimes conduct integration testing
Independent testing group (ITG)
Report to SQA team
Developer and ITG work closely to ensure thorough test is well conducted
Developer must be present during testing to correct uncovered errors.
TCS2411 Software Engineering

7. Software Testing Steps
High-order tests
Integration test
Unit test
Testing
direction
code
design
requirements
TCS2411 Software Engineering

8. TCS2411 Software Engineering
Unit Testing
Verification of the smallest unit in software - the module
White box oriented
Can be conducted in parallel for multiple modules
Unit test cases designed based on source code - examine likely causes for errors
Test environment includes driver and/or stubs
TCS2411 Software Engineering

9. TCS2411 Software Engineering
Unit Test Environment
Driver
Module to
be tested
Stub
…..
TCS2411 Software Engineering

10. Unit Testing Considerations
Interface - tested to ensure that information properly flows into and out of the unit
Local data structures - examined to ensure that data stored temporarily maintains its integrity in an algorithm’s execution
Boundary conditions - tested to ensure that module operates properly at boundaries
TCS2411 Software Engineering

11. Unit Testing Considerations (Continued)
Independent paths - to ensure all statements in a module executed at least once
Error-handling paths
clear and correct messages/descriptions
correct processing of errors
not interrupted by system
sufficient information to locate cause of error
TCS2411 Software Engineering

12. TCS2411 Software Engineering
Unit Test Environment
Driver
Interface
Local data structures
Boundary conditions
Independent path
Error Handling path
Test cases
Module to be tested
Stub
Act as main program
Accept test data
Past to the component to be tested
Print results
Replace module that are subordinate (called by) component to be tested
Use subordinates module interface
Do minimal data manipulation
Print verification entry
Return control to the module
Notes :
Driver and stub are software that must be written.
Both are not delivered with final product
RESULTS
TCS2411 Software Engineering

13. TCS2411 Software Engineering
Integration Testing
Systematic technique for constructing software while testing to uncover errors associated with interfacing
Take unit-tested modules and build the program structure based on the design
Nonincremental (“big bang”) - all modules combined in advanced  chaos!
Incremental integration - part by part, e.g. top-down or bottom-up
TCS2411 Software Engineering

14. TCS2411 Software Engineering
Top-Down Integration
Beginning with main module, modules are integrated by moving downward through the control hierarchy
Subordinate modules are added into the structure either depth-first or breadth-first
Verifies major control or decision points early in testing
Problem when lower level processes needed to test upper levels
TCS2411 Software Engineering

15. Top-Down Integration Diagram
TCS2411 Software Engineering

16. Top-Down Integration Process
Beginning at main module, stubs are replaced one at a time with actual modules
Tests are conducted as each module is integrated
On the completion of each set of tests, another stub is replaced with real module
Regression testing (conduct all or some of previous tests) may be done to ensure no new errors have been introduced
TCS2411 Software Engineering

17. Bottom-Up Integration
Begins construction and testing with atomic modules (lowest level in hierarchy)
No need for stubs, all subordinate modules tested and available
Test drivers required and can be shared as integration move towards the higher levels
TCS2411 Software Engineering

18. Bottom-Up Integration DiagramMc D1 D2 D3 Ma Mb
Cluster 1
Cluster 2
Cluster 3
TCS2411 Software Engineering

19. Bottom-Up Integration Process
Low-level modules are combined into clusters
A driver is written to coordinate test case input and output
The cluster is tested
Drivers are removed and clusters are combined moving upward in the program structure
TCS2411 Software Engineering

20. TCS2411 Software Engineering
Regression Testing
New module or major changes in software.
Helps to ensure changes does not introduce
Unintended behavior
Additional errors
Re-execute the subset of all test cases
Most advised : Use automated capture-play back tools
This tool enable the software engineer to capture test cases and results for subsequent playback and comparison.
TCS2411 Software Engineering

21. TCS2411 Software Engineering
Validation Testing
A series of black box tests that demonstrate conformity with requirements
Test that software functions as expected by customer
Test plan and details can be based on software requirements specification
TCS2411 Software Engineering

22. TCS2411 Software Engineering
Alpha Test
Conducted at developer’s site by customer
Software is used in a natural setting with the developer “looking over the shoulder”
Errors and usage problems recorded, correction can possibly be done quickly
Test conducted in a controlled environment
TCS2411 Software Engineering

23. TCS2411 Software Engineering
Beta Test
Conducted at one or more customer sites by end user(s)
Developer generally not present
“Live” application of software in uncontrolled environment
Customer records all problems and reports them to developer at regular intervals
Software modified based on problems reported before final release to customers
TCS2411 Software Engineering

24. TCS2411 Software Engineering
System Testing
A series of tests to fully exercise a computer-based system
Verify that all system elements have been properly integrated and perform allocated functions
Software is one element to be integrated
Software engineer to anticipate potential interfacing problems
TCS2411 Software Engineering

25. TCS2411 Software Engineering
Recovery Testing
Force the software to fail in a variety of ways
Verify that recovery is properly performed
If recovery is automatic, re-initialisation, checkpointing mechanisms, data recovery, and restart are evaluated for correctness
If recovery requires human intervention, mean time to repair is evaluated
TCS2411 Software Engineering

26. TCS2411 Software Engineering
Security Testing
Verify that protection mechanisms built into the system will protect it from improper penetration
Tester plays the role(s) of individual who desires to penetrate the system
System designer to make penetration cost greater than the value of information obtained
TCS2411 Software Engineering

27. Stress & Performance Testing
Stress Testing
Execute system in a manner that demands resources in abnormal quantity, frequency or volume
Performance Testing
Test run-time performance of software within context of an integrated system
TCS2411 Software Engineering

28. TCS2411 Software Engineering
Debugging
To find and correct the cause of a software error
Attempt to match symptom to cause
Bug characteristics:
Symptom and cause may be geographically remote
Symptom disappear (temporarily) when another error is corrected
TCS2411 Software Engineering

29. Debugging (Continued)
Symptom caused by nonerrors
Symptom caused by human errors
Symptom a result of timing problems
Difficult to accurately reproduce input conditions
Symptom may be intermittent
Symptom due to causes that are distributed across a number of tasks on different processors
TCS2411 Software Engineering

30. TCS2411 Software Engineering
Debugging Process
Test
Cases
Execution
Of cases
Results
Debugging
Additional
test
Suspected
causes
Regression
Corrections
Identified
TCS2411 Software Engineering

31. TCS2411 Software Engineering
Debugging Approaches
Brute force
“let the computer find the error”
memory dumps, run-time traces, lots of WRITE statements  clues to error
Backtracking
trace source code backwards from error
Cause elimination
induction/deduction, “binary partitioning”
isolate potential causes, test each one
TCS2411 Software Engineering

32. TCS2411 Software Engineering
Debugging Aids
Debugging Tools
Debugging compiler
Dynamic debugging aids (“tracers”)
Automatic test case generators
Memory dumps
Cross-reference maps
Good design documentation
Clear source code
Other people!
TCS2411 Software Engineering

33. TCS2411 Software Engineering
References
“Software Engineering: A Practitioner’s Approach” 5th Ed. by Roger S. Pressman, Mc-Graw-Hill, 2001
“Software Engineering” by Ian Sommerville, Addison-Wesley, 2001
TCS2411 Software Engineering