1. 1 Systems V & V, Quality and Standards Dr Sita Ramakrishnan School CSSE Monash University

2. © S Ramakrishnan2 Documenting Software Testing zTopics x Documenting Software Testing xReferences: x B Brugge and A H Dutoit – Object-Oriented Software Engineering – Conquering Complex and Changing Systems Students - ch.9 on Testing and pp. 365-367 on documenting testing xIEEE Std 829-1998 – available for Monash Students using monash library web pages in the IEEE Explorer standards area xOther references (books included in the slides, conference & journal papers) given in the handout for the unit

3. © S Ramakrishnan3 Documenting Software Testing z Testing activities are documented as per IEEE Std 829 1998 in 4 types of documents. They are : x the test plan x the test case specification x the test incident report and x the test summary report z Test Plan: x focus is on managerial aspects of testing x documents the scope, resources, schedule of testing activities, approach, test requirements, components to be tested

4. © S Ramakrishnan4 Documenting Software Testing z Test case specification document x Each test with a test case spec. x contains the input, drivers, stubs, tasks to be done and expected output of the tests z Test Incident report x each execution of the test documented by an incidence report x lists actual test results & variance from expected results are recorded x if failure found, analysis report should contain enough information to allow the failure to be reproduced x failures from all test incident reports are collected & summarised in the test summary report

5. © S Ramakrishnan5 Documenting Software Testing z Test Summary Report y lists failures found during tests that must be checked y developers use the summary report to further analyse and prioritize each failure and plan for changes in the system and in the models x changes may result in need for more new test cases and new test executions z These test documents are under configuration management and are updated as the system models change. zIEEE 829-1998 standard for test documentation is more elaborate and is more suited to large organisation (Refer to the IEEE Std)

6. © S Ramakrishnan6 Test Plan template 1.Introduction 2.Relationship to other documents 3.System overview 4.Features to be tested / not to be tested 5.Pass/fail criteria 6.Approach 7.Suspension & Resumption 8.Testing materials (Hardware & Software requirements) 9.Test cases 10.Testing schedule

7. © S Ramakrishnan7 Test Plan template 1.Introduction – Section1 of Test Plan: yDescribes the test objectives & Scope yProvides a framework for managers & testers to plan & execute tests in a cost effective & timely fashion 2. Relationship to other documents produced during development such as RAD, ODD etc. z explains how all the tests are related to the functional & non functional requirements as well as the design in the respective documents zIncludes a naming scheme for relating requirements and its tests

8. © S Ramakrishnan8 Test Plan template 3. System overview zProvides overview of the system in terms of components that are tested during unit testing zFocus is on structural testing zDependencies between components spelled out here 4. Features to be tested / not to be tested zIdentifies features & combination of features to be tested zFocus is on functional testing zDescribes features not tested and reasons as to why?

9. © S Ramakrishnan9 Test Plan template 5. Pass/fail criteria zTest plan includes general pass/fail criteria. zTo be augmented with more in test design spec. 6. Approach zDiscusses the reasons for the integration testing strategy selected. Different strategies are required to test different parts of the system zDiagrams can be used to illustrate dependencies between the individual tests and their relationship in integration tests

10. © S Ramakrishnan10 Test Plan template 5. Pass/fail criteria zTest plan includes general pass/fail criteria. zTo be augmented with more in test design spec. 6. Approach zDiscusses the reasons for the integration testing strategy selected. Different strategies are required to test different parts of the system zDiagrams can be used to illustrate dependencies between the individual tests and their relationship in integration tests

11. © S Ramakrishnan11 Test Plan template 7. Suspension & Resumption zCriteria for suspending the testing on test items in the test plan, and test activities that must be redone when testing is resumed 8. Testing materials zResources for testing – H/W, S/W, test tools, office space 9. Test cases zCore of the test plan. Lists test cases used in testing zEach test case detailed in test case specification 10. Testing Schedule – staffing, training, risks, schedule

12. © S Ramakrishnan12 Test Case Specification Template 1.Test Case Specification ID – name of the TC 2.Test Items – lists components under test & features being exercised 3.Input Specifications – lists inputs reqd for the TCs 4.Output Specifications – lists the expected output 5.Environmental needs – H/W, S/W platform needed for running the tests, including any drivers & stubs 6.Special procedural requirements – constraints needed in executing the tests such as timing, load, operator intervention 7.Dependencies – list the dependencies with other test cases