Software Testing and Quality Assurance Lecture 29 – Testing Systems

Lecture Objectives Learn how to write a test plan Learn how to write test cases from use cases Learn how to quantify the quality of testing

Introduction System testing: usually refers to the testing of a completed application to determine that it provides all of the behaviors required of the application. In an incremental development effort, system testing usually refers to successive rounds of testing end-user functionality with extensive feedback and interaction between the development staff and the test team

Introduction System testing phase may be a multilevel activity: subsystems and system.

Introduction (cont...) Testing is not only search for defects. The focus for testing at this point in the development process shifts from a search for defects that lead to program failure to a search for defects that are variance between the actual operation of the system and the requirements.

Introduction (cont...) Other types of testing: Performance testing: Security testing Deployment testing Acceptance testing (special kind of system testing)

Defining the system test plan System test plan is more formal and comprehensive than the component test plan.

Defining the system test plan: The IEEE 829 Standard Test Plan outline Introduction Test Items Tested Features Features Not Tested (per cycle) Testing Strategy and Approach 5.1 Syntax 5.2 Description of Functionality 5.3 Arguments for Tests 5.4 Expected Output 5.5 Specific Exclusions 5.6 Dependencies 5.7 Test case Success/Failure criteria

Defining the system test plan: The IEEE 829 Standard Test Plan outline (cont...) Pass/Fail Criteria for Complete Test Cycle Entrance Criteria / Exit Criteria Test Suspension Criteria and resumption Requirements Test Deliverables/Status Communication Vehicles Testing Tasks Hardware and Software Requirements Problem Determination and Correction Responsibilities Staffing and Training Needs/Assignments Test Schedules Risks and Contingencies Approvals

Defining the system test plan: test suspension criteria and resuspension requirements If the system being tested contains too many defects, testing may need to be suspended before all planned tests have been run. Testing is suspended if there is no use case for which we can successfully complete a test.

Defining the system test plan: test suspension criteria and resuspension requirements Testing is resumed when sufficient development effort has been expended to cause a significant percentage of the use cases to pass tests.

Complementary strategies for selecting test cases Two possible avenues for selecting test cases for the systems: Types of defects that the system might contain and test cases to find these (guide the selection of the test cases for maximum defect-finding power) Use the Orthogonal Defect Classification (ODC) approach

Complementary strategies for selecting test cases How the system will be used and build test cases to take the system through its paces (determine how many test cases to apply to each use case) Use the use profile approach.

Complementary strategies for selecting test cases: use profile Operational profile: a listing of the relative frequency with which end user function in the system is used Use profile uses priority of an operation rather than the frequency of the operation Priority of a function is based on its frequency and criticality

Complementary strategies for selecting test cases: Orthogonal Defect Classification (ODC) ODC captures information about the type of faults that are present in a software system under development

Key points System Testing Performance testing: Security testing Deployment testing Acceptance testing System test plan is more formal and comprehensive than the component test plan.