1 Software Testing and Quality Assurance Lecture 15 - Planning for Testing (Chapter 3, A Practical Guide to Testing Object- Oriented Software)

2 Lecture Outline Adequacy of test cases. Testing Roles Testing Planning

3 A testing process: dimensions of software testing (cont...) How much testing is adequate? No testing, exhaustive testing (consider the expected lifetime of the software), is the software life-critical, Coverage: is a measure of how completely a test suite exercises the capabilities of a piece of software. For example, If every LOC executed at least once, the number of requirement that is checked by the test suite.

4 A testing process: adequacy of test cases Adequacy of test cases: test the software enough to be reasonably sure that the software works as it is supposed to.

5 A testing process: adequacy of test cases Adequacy can be measured based on coverage: Coverage based on the requirements: based on what the software is supposed to do how many of the requirements called out in the specification are tested Coverage based on the code: based on how the software actually works ― how much of the software was executed as a result of running test suites

6 A testing process: adequacy of test cases (cont...) Functional testing (specification-based or black box testing): test cases are constructed based solely on the software’s specification and not on how the software is implemented. Structural testing (implementation- based or white box testing): test cases are constructed based on the code that implements the software.

7 A testing process: adequacy of test cases (cont...) Rule of Thumb Use some combination of both approaches.

8 A testing process: adequacy of test cases ― risk analysis (cont...) Risk analysis in the testing process is applied to determine the level of details and amount of time to dedicate to testing a component. A reasonable scale of increasing risk for components is: Prototype components Production components Library components Risk increases

9 Roles in the testing process One or more people can assume each role. One person can assume multiple roles. Roles must be separate: Class tester: test individual classes as they are programmed. Integration tester: test a set of objects that are being brought together from different development resources.

10 Roles in the testing process System tester: responsible for independently verifying that the completed application satisfies the system requirements. Test manager: managing the test process.

11 Planning activities: scheduling testing activities Class tests: Scheduled based on the developer’s judgment as they become useful or necessary. Useful during coding Necessary when adding a component to the code base.

12 Planning activities: scheduling testing activities Integration tests: Scheduled as specific intervals, usually at the end of major iteration that signal the completion of an increment System tests: Performed on major deliverables at specified intervals throughout the project, usually specified in the project plan.

13 Planning activities: estimation Estimation: estimating resources ―cost, time and personnel Factors should be considered for estimation: Level of coverage: estimate the amount of effort for one use case then you can construct the estimate for the whole system Domain type: logic intensive vs. data intensive.

14 Planning activities: estimation Equipment required: environment as close as possible to the deployment environment. Costs of equipment for simulation must also be considered. Organizational model: buddy approach (two developers swap code with each other and test) or different approach. Testing effort estimate: effort for testing all classes: total number of classes times the effort per class.

15 Planning activities: document templates Project test plan Component test plan (one per important component/class). Use case test plan (one per use case) Integration test plan (one per increment) System test plan (only one)

16 Planning activities: document templates— Project test plan The purpose of project test plan is to summarize the testing strategy that is to be employed for the project: The steps in the development process at which testing will occur The frequency of testing with which the testing should occur Who is responsible for the activity

17 Planning activities: document templates— Project test plan The project test plan may be an independent document or it may be included in either the overall project plan or the project’s quality assurance plan.

18 Planning activities: document templates— Component test plan The purpose of the component test plan is to define the overall strategy and specific test cases that will be used to test a certain component. Two types of guiding information: Project criteria: standards that have been agreed upon as to how thoroughly each component will be tested (e.g. 100% of the critical components will be tested)

19 Planning activities: document templates— Component test plan Project procedure: identify techniques that have been agreed upon as the best way to handle a particular task (e.g. constructing a PACT class for each component that will be tested)

20 Planning activities: document templates— Use case test plan The purpose of the use case test plan is to describe the system-level tests to be derived from a single use case Three levels of use cases: High level: abstract use cases that are the basis for being extended to end-to-end system level use case End-to-end system

21 Planning activities: document templates— Use case test plan Functional sub-use cases: aggregated into end-to-end system-level use cases. Types of use cases: Functionality use cases: modified the data maintained by the system in some way Report use cases: accessed information, summarized it, and format it for the user.

22 Planning activities: document templates— Integration test plan Integration test plan is important in an iterative development environment. Specific sets of functionality will be delivered before others. Out of these increments the full system slowly emerges.

23 Planning activities: document templates— Integration test plan Since small, localized behavior should have already been tested. Integration tests should be more complex and more comprehensive than the typical component tests. Individual test plans are combined to form the integration test plan for a specific increment. Integration test depends on other test cases, so no template is provided.

24 Planning activities: document templates— System test plan The system test plan is a document that summarizes the individual use case test plans and provides information on additional types of testing that will be conducted at the system level.

25 Key points Adequacy of test cases: test the software enough to be reasonably sure that the software works as it is supposed to. Roles in the testing process: class tester, integration tester, system tester, and test manager. Factors should be considered for estimation: level of coverage, domain type, equipment required, organizational model, and testing effort estimate. Document templates