2 Why Software Testing Strategy? Some Test Strategy Issues:Should develop a formal test plan?Test entire program as a whole or various pieces?When should customer be included?Testing often accounts for more effort then developmentUsually strategy is defined in an overall testing strategy and procedure document (often by independent test group)
3 Elements Of Software Test Strategy Unit Testing - Tests the implemented source code of the component. Uses White Box.Integration Testing - focuses on the design, testing and integration of components. Black box and limited White Box.Validation Testing - Looks at requirements a validates software against it. Black Box Testing.System Testing - Looks at software and other system elements tested as a whole. (Other hardware, people, databases, processes.)
4 Unit Testing Unit Testing - Module test Use drivers (accepts data interfaces are tested to ensure information flows into and out of the program correctly.Data structures are tested.Independent paths (basis paths) are exercised)Error handling paths - A mistake to miss these.Use drivers (accepts dataand tests and passesto module) and stubs(replace other modules)
5 IntegrationA systematic technique to uncover problems while putting the modules together.Composed of two activities:BuildingCompile and link all of the modules togetherTestingVerify interface function as specifiedAvoid the “big bang” approach A.K.A. non- incremental integrationcan yield chaotic results.
7 Approaches to Integration Testing Top-down (depth-first or breadth-first)Test higher level modules firstRequire stubs (not just returns)Bottom-upTest lower-level modules firstRequire drivers (require logic)SandwichTest highest level modules in conjunction with lowest level meeting in the middleUse stubs and drivers
9 Top Down (depth) Example Depth FirstIntegrate M1, use stubs for M2, M3 and M5 . Test all M1 interfacesNext would test M2 and use operational stubs for M3, M4, M5, M6 .Test all M2 interfacesRepeat for M5, then M8, M6, and M3...
10 Top Down (depth) Example Integrate M1, use stubs for M2, M3 and M5 . Test all M1 interfacesNext would test M2 and use operational stubs for M3, M4, M5, M6 .Test all M2 interfacesReplace stub M5 with module M5. Use operational stubs for M3, M4, M6, M8. Test all the interfaces in M5Repeat for M8, M6, and M3...
11 Top Down (breath) Example Breath First OrderM1, M2, M3, M4,Next level would be M5, M6, ...Next Top down testing can have problems when low level modules are needed (delay full tests or develop better stubs or go bottom up)
12 Bottom Up Lower Items first using drivers : Using an M8 driver test module M8Using an M7 driver test module M7Using an M6 driver test module M6Using an M5 driver test module M5
13 Bottom Up Example IIIn reality lower items are clustered together into sub-functions (aka builds).
14 Sandwich Example First Round Top-down Use operational stubs for M2, M3, M4Test all the interfaces in M1Bottom-upUsing an M8 driver test module M8Using an M7 driver test module M7Using an M6 driver test module M6
15 Regression TestingThe reexecution of some subset of tests to ensure changes have not propagated errors.Tests conducted when a previously tested software is deliveredContinuing testing after repairNew VersionPartial testing old features when testing a maintenance release.Might include tests of features/functions that were unchanged doing the current release.Capture/playback tools enable re-execute of tests previously run.
16 Verification(1) The process of determining whether or not the products of a given phase of the software development cycle fulfill the requirements established during the previous phase(2) Formal proof of program correctness(3) The act of reviewing, inspecting, testing, checking, auditing, or otherwise establishing and documenting whether or not items, processes, services and documents conform to specified requirements
17 Functional Verification Verification of the Functional Design SpecificationShould be done by developers Normally done by user or testing groupCan be combined with integration testing
18 Are we building the right product? Does it meet the requirements? ValidationAre we building the right product?Does it meet the requirements?
27 Rules for High-Level Tests 1. Test are run on operational production hardware and software.2. Tests must stress the system significantly3. All interfaced systems must be in operation throughout the test.4. Test duration should run a full cycle5. Tests should exercise the system over a full range of inputs, both legal and illegal6. All major functions and interfaces should be exercised.7. If running the entire test cannot be completed, a new run should include a complete restart.
28 Acceptance Testing Depends on the type of product and the situation. When working with an a non-technical user, the following are recommended.First, train user on system with a number of test cases.Use a comparison test.Re-processingParallel to existing system
29 Acceptance Testing Product Development Alpha testing Performed by a selected end-users usually in side the development environment.Beta testingPerformed by a subset of actual customers outside the company, before the software is made available to the customer.Field TrialLarge number of users, but not general release.
30 Acceptance Testing Work for Hire Training Programming Evaluate based on comments users general reaction to system.Usability TestParallel OperationComparison TestingNew software used after old system process.Controlled deploymentRepresentative monitors software at site.