Chapter 9 Testing the System, part 2

Testing  Unit testing White (glass) box Code walkthroughs and inspections  Integration testing Bottom-up Top-down Sandwich Big Bang  Function testing  Performance testing  Acceptance testing  Installation testing

One-minute quiz  What is the difference between verification and validation?

One-minute quiz  What is meant by regression testing?  Is regression testing used for verification or validation?

Function Testing  Test cases derived from requirements specification document Black box testing Independent testers Test both valid and invalid input and the success of the test is determined by the produced output  Equivalence partitioning Boundary values

Performance Testing  Stress tests  Volume tests  Configuration tests  Compatibility tests  Regression tests  Security tests  Timing tests  Environmental tests  Quality tests  Recovery tests  Maintenance tests  Documentation tests  Human factors (usability) tests

Reliability, Availability, and Maintainability  Software reliability: operating without failure under given condition for a given time interval  Software availability: operating successfully according to specification at a given point in time  Software maintainability: for a given condition of use, a maintenance activity can be carried out within stated time interval, procedures and resources

Measuring Reliability, Availability, and Maintainability  Mean time to failure (MTTF)  Mean time to repair (MTTR)  Mean time between failures (MTBF) MTBF = MTTF + MTTR  Reliability R = MTTF / (1+MTTF)  Availability A = MTBF / (1+MTBF)  Maintainability M = 1 / (1+MTTR)

When to stop testing  Fault seeding Adding faults to the code to estimate the number of remaining faults. Suppose 50 faults have been seeded in the code. Regression testing identifies 60 faults, forty of which are the seeded faults. What is the estimate of the total number of undiscovered real faults remaining?

Acceptance Tests  Enable the customers and users to determine if the built system meets their needs and expectations  Written, conducted, and evaluated by the customers

Types of Acceptance Tests  Pilot test: install on experimental basis  Alpha test: in-house test  Beta test: customer pilot  Parallel testing: new system operates in parallel with old system

Test Documentation  Test plan: describes system and plan for exercising all functions and characteristics  Test specification and evaluation: details each test and defines criteria for evaluating each feature  Test description: test data and procedures for each test  Test report: results of each test

Test Documentation

Test Plan  Used to organize testing activities guides the scheduling of the programming explains the nature and extent of each test documents test input, specific test procedures, and expected outcomes

Defect Tracking Form

Quality Assurance  Quality Control Testing the quality of the program  Quality Assurance Building quality into the program Management level – proactive process Checklists

Testing Safety-Critical Systems  Recognize that testing cannot remove all faults or risks  Assume that every mistake users can make will be made Do not assume that low-probability, high- impact events will not happen  Emphasize requirements definition, testing, code and specification reviews, and configuration control Cleanroom testing

Different Levels of Failure Severity  Catastrophic: causes death or system loss  Critical: causes severe injury or major system damage  Marginal: causes minor injury or minor system damage  Minor: causes no injury or system damage