Software Failure: Reasons Incorrect, missing, impossible requirements * Requirement validation. Incorrect specification * Specification verification. Faulty system design * Design reviews. Faulty software code * Unit testing. Faulty hardware infrastructure

Human Factor Developers, analysts and users make ‘simple’ mistakes. These mistakes can be uncovered and corrected with the help review meetings, verification, validation and testing process. However, mistakes made due to miscommunication or misunderstanding are much harder to notice.

Fault Types Algorithmic Incorrect process steps Computation Incorrect formulas Precision Rounding errors, small data types Documentation Incorrect interface/API docs leads to program errors Load / Capacity Timing / Synchronization Performance

Fault Distribution (HP)

Test Process Organization

Test Types Unit testing: verifies that the component functions property (in a controlled environment) with the expected input. Integration testing: ensures that the system components work together as they are supposed. Function testing: determines if the functions described in the requirement are performed. Successful function test yields validated system. Acceptance testing: performed by customers, system is checked against the requirements. Deployment testing: the software is tested again once it is deployed in a new (clean) environment.

Views of the Test Objects Black Box: internal structure is unknown - Difficult to test comprehensively White Box/Clear Box: internal structure is known - Possible to devise specific & boundary tests

Code Examination Code Review Independent / external reviewers examine and critique the code Code Walkthrough You walk the reviewers through your code Code Inspection checks the code against predetermined list of concerns + Can uncover up to 90% of faults! ! It is best to use well prepared reviewers that are very well familiar with the product. - Does not uncover requirement or design faults.

Formal Proof Techniques 1.Write assertions that describes code’s input and output conditions. These should be logical statements. 2.Draw a flow chart diagram of the code. 3.Using the flow chart and assertions devise a series of theorems that trace logical transformations performed by the code for various paths through the flow chart. 4.Prove the theorems. + Develops understanding of the code + Ensures algorithm correctness - Difficult to set up and carry out the proofs - Impractical / impossible for complex algorithms

Choosing Test Cases Ideally we want to test every possible input. When the inputs are limitless or impractical to enumerate we need to separate data into the equivalence classes (when the object is white box). Then it would be sufficient to test data drawn from the equivalence class. Equivalence classes need to be picked out such that they cover all branches and switches in the code in order to maximize the code coverage. If the code under investigation maintains state we must devise a sequence of tests such that all states of the finite state machine are tried. We also need to test boundary conditions as well as invalid input.

Test Thoroughness Approaches Statement testing: every statement has to be tested. Branch testing: every brunch must be process in both directions. Path testing: every distinct path through the code must be tested. Definition-use testing: every path from every definition (variable) to every use of that definition must be tested.

Unit Testing Comparison

Read Chapter 8 from the white book.