1. Software Testing Strategies

2. Software Testing Strategy
Integration of software test case design methods into a well-planned series of steps
Successful software construction
Provides a road map that describes
The steps to be conducted as part of testing
How much effort, time and resources will be required
Must incorporate test planning, test case design, test execution, and resultant data collection and evaluation

3. Software Testing Strategy
Should be flexible enough to promote a customized testing approach
Must be rigid enough to promote reasonable planning and management tracking as the project progresses
Different test methods are beginning to cluster themselves into distinct approaches and philosophies 3

4. Software Testing What is it? To uncover errors
How do we conduct the test?
Do we develop a formal plan?
Should we test the entire program as a whole or run tests on a small part of it?
Should we rerun tests we have already conducted as we add new components to a large system?
When should we involve customer?
These questions must be answered when we develop a software testing strategy. 4

5. Software Testing Who does it? Why is it important
The strategy is developed by the project manager, software engineers, and testing specialists.
Why is it important
Testing often accounts for more project effort than any other software engineering activity
If conducted haphazardly
time is wasted
Unnecessary effort is expended
Errors sneak through undected 5

6. Software Testing What are the steps?
Begins in the small and progresses to the large
Focus on a single component or a small group of related components
After components are tested they may be integrated until the complete system is constructed
As errors are uncovered, they must be diagnosed and corrected using a process that is called debugging 6

7. Software Testing What is the work product?
A test specification document
Defining a plan that describes overall strategy
A procedure that defines specific testing steps and the tests that will be conducted
How do I ensure that I’ve done it right?
By reviewing the Test specification prior to testing
Assess the completeness of test cases and testing tasks 7

8. Software Testing A Strategic Approach
Testing is a set of activities that can be planned in advance and conducted systematically
Conduct effective formal technical reviews
Begin at the component level and work “outward” toward the integration of the entire computer-based system
Different testing techniques at different points in time
Usually conducted by the software developer and for large projects by an independent test group
Testing and debugging are different activities, but debugging must be accommodated in any testing strategy 8

9. Software Testing Verification and Validation
Software testing is often referred to as V&V
Verification refers to the set of activities that ensures that software correctly implements a specific function
Are we building the product right?
Validation is a different set of activities that ensure that the software that has been built is traceable to customer requirements
Are we building the right product? 9

10. Software Testing Verification and Validation
Encompass a wide array of Software Quality Assurance (SQA) activities including
Formal technical reviews
Quality and configuration audits
Performance monitoring
Simulation
Feasibility study
Documentation review
Database review
Algorithm analysis
Development, usability, installation testing 10

11. Software Testing Organizing for Software Testing
Psychological point of view
The software engineer analyzes, models, and then creates a computer program and its documentation
From the software engineers perspective testing is destructive as it tries to break the thing that SE has built
There are often a number of misconceptions
The developer of the software should no do test
Software should be tossed over the wall to strangers who will test it mercilessly
Testers get involved with the project only when the testing steps are about to begin 11

12. Software Testing Organizing for Software Testing
The role of the independent test group
Remove the inherent problems associated with letting the builder test the thing that has been built
Removes the conflict of interest
The software engineer, however, does turn the program over to the ITG and walk away; both have to work closely throughout the project
The developer must be available to correct uncovered errors 12

13. Software Testing Strategy for conventional software architecture
Unit Testing
Concentrates on each unit (i.e. component) of the software as implemented in the source code
Integration Testing
Focus is on design and the construction of the software architecture
Validation Testing
Requirements are analysis are validated against the software that has been constructed
System Testing
Software and other elements are tested as a whole 13

14. Software Testing Strategy for Object Oriented Architecture
Focus on “testing in the small” and work outward toward “testing in the large”
Testing in the small changes from an individual module to a class encompassing attributes and operations; implies communication and collaboration
A series of regression tests are run to uncover errors due to communication and collaboration between classes 14

15. Software Testing Strategic Issues
Specify product requirements in a quantifiable manner long before testing
A good strategy not only looks for errors, but also asses other quality characteristics such as
Portability
Maintainability
Usability
State testing objectives explicitly
The specific objectives of testing should be stated in measurable terms
Test effectiveness
The cost to find and fix defects
Test coverage 15

16. Software Testing Strategic Issues
Understand the user of the software and develop a profile for each
Use-cases that describe the interaction scenario for each class of user can reduce overall testing effort by focusing testing on actual use of the project
Build robust software that is designed to test itself
Software should be designed in a manner that uses anti-bugging techniques. Software should be capable of diagnosing certain classes of errors. The design should be accommodated automated testing and regression testing 16

17. Software Testing Strategic Issues
Use effective formal technical reviews as a filter prior to testing
Reviews can reduce the amount of testing effort that is required to produce high-quality software
Develop a continuous improvement approach
The testing strategy should be measured. The metrics collected during testing should be used as part of a statistical process control approach for testing 17

18. Software Testing Test Strategies for Conventional Software
There are many strategies for testing software.
A software team could wait until the system is fully constructed and then conduct tests on the overall system.
At the other extreme software engineer can conduct tests on a daily basis, whenever any part of the system is constructed
A testing strategies falls between the two extremes
Begin with the testing of individual program units
Moving to tests designed to facilitate the integration of the units 18

19. Software Testing Test Strategies for Conventional Software
Unit Testing
Focuses verification effort on the smallest unit of software design – the software component or module
Integration Testing
“If they all work individually, why do you doubt that they’ll work when we put them together?”
The problem is “putting them together” – interfacing
Data may be lost
One module can have an inadvertent, adverse effect on another; sub-function when combined
Global data structure can present problems 19

20. Software Testing Test Strategies for Conventional Software
Regression Testing
Each time a new module is added as part of integration testing, the software changes
New data flow paths are established
New I/O and control logic is invoked
Regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects
Smoke testing
An integration testing 20

21. Software Testing Test Strategies for Conventional Software
Smoke testing
An integration testing
Software components that have been translated into code are integrated into a “build” – data files, libraries, reusable modules, and engineered components that are required to implement one or more product
The intent of testing here is to uncover show stoppers – errors that have the highest likelihood of throwing the software project behind schedule
The build is integrated with other builds and the entire product is smoke tested daily 21

22. Software Testing Test Strategies for OBJECT-ORIENTED SOFTWARE
Unit Testing
An encapsulated class is the focus of unit testing
Operations within the class are, however, the smallest unit
Integration Testing
Thread based testing
Integrates a set of classes required to respond to one input or event for the system
Used based testing
Begins the construction of the system by testing independent classes – use very few if any server classes
Testing dependent classes – which use independent classes 22

23. Software Testing Test Strategies for OBJECT-ORIENTED SOFTWARE
Integration Testing
Cluster Testing
A cluster of collaborating classes (determined by examining the CRC) to uncover errors in collaboration 23

24. Software Testing Validation Testing
At this level the distinction between conventional and object-oriented software disappears
Testing focuses on user visible actions and user recognizable output from the system
Can be defined in many ways
A simple definition is that validation succeeds when software functions in a manner that can be reasonably expected by the customer.
Alpha Testing
Conducted at the developer site by end-users
Beta Testing
Conducted at end user sites – the developer is generally not present 24

25. Software Testing System Testing
Ultimately, software is incorporated with other system elements
Hardware, people, information
A series of system integration and validation tests are conducted
A classic system testing problem – finger pointing
Software engineers should anticipate potential interfacing problems
Design error handling paths that test all information coming from other elements of the system
Conduct series of tests to simulate bad data
Record the results of testing as evidence
Participate in planning and design of systems tests to ensure that software is adequately tested 25

26. Software Testing System Testing Recovery Testing
Is a system test that forces the software to fail in a variety of ways and verifies that recovery is properly performed
If automatic recovery
Re-initialization
Check-pointing mechanisms
Data recovery
Restart are evaluated for correctness
If manual
Mean time to repair (MTTR) is evaluated 26

27. Software Testing Security Testing Stress Testing Performance Testing
It verifies that protection mechanisms built into a system will, in fact protect it from improper penetration
Stress Testing
It executes a system in a manner that demands resources in abnormal quantity, frequency, or volume
Performance Testing
Are often coupled with stress testing and usually require both hardware and software instrumentation.
To measure resource utilization 27