1. Winter 2007SEG2101 Chapter 121 Chapter 12 Verification and Validation

2. Winter 2007SEG2101 Chapter 122 Contents What is verification and validation? Reviews Testing SDL systems Dynamic analysis of interfaces

3. Winter 2007SEG2101 Chapter 123 Verification and Validation Verification: –to establish the truth of correspondence between a software product and its specification –are we building the product right –scope of verification is process quality Validation –to establish the fitness or worth of a software product for its operational mission –are we building the right product –scope of validation is system quality

4. Winter 2007SEG2101 Chapter 124 Techniques for Verification and Validation Manual review Testing SDL systems Dynamic analysis of interfaces

5. Winter 2007SEG2101 Chapter 125 Reviews Reviews may be performed on several levels in a project. As a minimum, the quality of milestone results should be assessed. Two purposes of formal review: –to provide feedback to management on the status of products under review –to stimulate the development team towards producing quality results Commenting

6. Winter 2007SEG2101 Chapter 126 Testing SDL systems The purpose of testing Test environment Testing a single SDL process Test complete SDL system

7. Winter 2007SEG2101 Chapter 127 The Purpose of Testing Two distinct aspects to test: –the functionality –the implementation Purpose of functionality testing: –verify the functional design against the functional requirements –detect possible internal inconsistencies and errors in functional design –validate the functional design against the users and owners needs Purpose of implementation testing: –verify the functionality of the implementation against functional design –verify the non-functional properties of the implementation against the non- functional requirements

8. Winter 2007SEG2101 Chapter 128 Test Environment Requirements for testing environment: –Inputs, including time-outs, may be generated and replayed. –The responses can be traced. –The execution can be controlled. The signal interfaces of SDL hide the internal structures of blocks and processes from each other. There is no difference between the test environment and the real environment as seen from the processes under test. Two kinds of problems may remain after a test in simulated time: –the implementation may be too slow to handle the traffic load or meet response time requirement –there may be time-dependent synchronization problems with the real environment or with real timers

9. Winter 2007SEG2101 Chapter 129 Testing a Single SDL Process A beautiful property of pure FSM is that the effort needed to test an implementation exhaustively is proportional to the number of transition in the state transition diagram. Every possible behavior is explicitly described by states and transitions. The state transition diagram itself may serves as the test plan. Work through the diagram transition by transition and check that the specifies outputs are generated that the next state is reached. This must be done for some characteristic values. Values must be selected so that each branch of a decision is carefully tested. Each asterisk transition and each save must be tested. Normally there is no point in testing units smaller than an SDL process.

10. Winter 2007SEG2101 Chapter 1210 Testing Complete SDL Systems Most SDL systems consists of many processes operating concurrently and interacting with each other through a network of channels and signal routes. Implementation testing at this level means to test the implementation of concurrency support, synchronization, signal transfer mechanisms and timers. To verify that the functionality is right we need to compare the actual behavior on the system interfaces with the required behavior. MSC Before SDL processes are integrated it is well worth the effort of inspecting the signal interfaces manually. Dynamic problems: deadlock, livelock

11. Winter 2007SEG2101 Chapter 1211 Testing Complete SDL Systems (cont’) In principle, we must test all possible combinations of states and transitions in all the processes of the system. Test cases –Normal behavior - the system behaves as intended in all the normal situations. –Exceptional behavior/situations- try to provoke errors by giving inputs that are not specified and check the exception handling is correct. –Try to reveal dynamic interworking problems by running input sequences concurrently on different interfaces. –When can we stop testing? (cost of testing, cost of undetected bug)

12. Winter 2007SEG2101 Chapter 1212 Validating Interfaces The validation of interfaces has two aspects: –static checks on signals –dynamic behavior analysis By inspecting the states and transitions of the global behavior graph one may find errors. Deadlock will appear as states in the graph without further transitions. - reachability analysis.

13. Winter 2007SEG2101 Chapter 1213 Two SDL Processes

14. Winter 2007SEG2101 Chapter 1214 Global Behavior Graph

15. Winter 2007SEG2101 Chapter 1215 Example: Transition Coverage Tree

16. Winter 2007SEG2101 Chapter 1216 Example: Symbol Coverage Tree