1. Software Testing and Quality Assurance
Lecture 29 – Testing Systems

2. Lecture Objectives Learn how to write a test plan
Learn how to write test cases from use cases
Learn how to quantify the quality of testing

3. Introduction
System testing: usually refers to the testing of a completed application to determine that it provides all of the behaviors required of the application.
In an incremental development effort,
system testing usually refers to successive rounds of testing end-user functionality with extensive feedback and interaction between the development staff and the test team

4. Introduction
System testing phase may be a multilevel activity: subsystems and system.

5. Introduction (cont...) Testing is not only search for defects.
The focus for testing at this point in the development process shifts from a search for defects that lead to program failure to a search for defects that are variance between the actual operation of the system and the requirements.

6. Introduction (cont...) Other types of testing: Performance testing:
Security testing
Deployment testing
Acceptance testing (special kind of system testing)

7. Defining the system test plan
System test plan is more formal and comprehensive than the component test plan.

8. Defining the system test plan: The IEEE 829 Standard Test Plan outline
Introduction
Test Items
Tested Features
Features Not Tested (per cycle)
Testing Strategy and Approach
5.1 Syntax
5.2 Description of Functionality
5.3 Arguments for Tests
5.4 Expected Output
5.5 Specific Exclusions
5.6 Dependencies
5.7 Test case Success/Failure criteria

9. Defining the system test plan: The IEEE 829 Standard Test Plan outline (cont...)
Pass/Fail Criteria for Complete Test Cycle
Entrance Criteria / Exit Criteria
Test Suspension Criteria and resumption Requirements
Test Deliverables/Status Communication Vehicles
Testing Tasks
Hardware and Software Requirements
Problem Determination and Correction Responsibilities
Staffing and Training Needs/Assignments
Test Schedules
Risks and Contingencies
Approvals

10. Defining the system test plan: test suspension criteria and resuspension requirements
If the system being tested contains too many defects, testing may need to be suspended before all planned tests have been run.
Testing is suspended if there is no use case for which we can successfully complete a test.

11. Defining the system test plan: test suspension criteria and resuspension requirements
Testing is resumed when sufficient development effort has been expended to cause a significant percentage of the use cases to pass tests.

12. Complementary strategies for selecting test cases
Two possible avenues for selecting test cases for the systems:
Types of defects that the system might contain and test cases to find these (guide the selection of the test cases for maximum defect-finding power)
Use the Orthogonal Defect Classification (ODC) approach

13. Complementary strategies for selecting test cases
How the system will be used and build test cases to take the system through its paces (determine how many test cases to apply to each use case)
Use the use profile approach.

14. Complementary strategies for selecting test cases: use profile
Operational profile: a listing of the relative frequency with which end user function in the system is used
Use profile uses priority of an operation rather than the frequency of the operation
Priority of a function is based on its frequency and criticality

15. Complementary strategies for selecting test cases: Orthogonal Defect Classification (ODC)
ODC captures information about the type of faults that are present in a software system under development

16. Key points System Testing
Performance testing:
Security testing
Deployment testing
Acceptance testing
System test plan is more formal and comprehensive than the component test plan.