CSE Senior Design II Test Planning Mike O’Dell Based on an earlier presentation by Mike O’Dell, UTA

1 CSE Test Planning … when?  Occurs throughout the product life cycle  Requirement completeness  Architectural integrity  Component design precision  Implementation correctness  But all too often we wait until the end to “validate” what we have already done.

1 CSE System Verification SRD: System Requirements ADS: Architecture Specification DDS: Detailed Design Specification Implementation Integration Test Component Test (a.k.a. Function Test) Unit Test System Validation Test System Verification System Definition MAP: All Specified Requirements MAP: All Module Interfaces & Interactions MAP: All Modules MAP: All Subsystem & Layer Interfaces & Interactions

1 System Verification  Unit Test: verifies that EVERY module (Hardware & Software) specified in the DDS operates as specified.  Component/Function Test: verifies integrity of ALL inter-module interfaces and interactions.  Integration Test: verifies integrity of ALL inter-subsystem/layer interfaces and interactions.  System Verification Test: verifies ALL requirements are met. CSE

1 System Test Plan Contents  Introduction  Executive Summary (purpose, scope, etc.)  References  SRD (requirements drive the system-level tests you do)  ADS (architecture drives the subsystem and integration testing)  Etc. (detailed design, standards, sample plans, etc. that are test drivers)  Test Item(s) – What will be tested CSE

1 System Test Plan Contents  Design decomposition (relationship of specific modules to test plan, test cases)  Limitations of product under test (restrictions, assumptions, caveats, etc.)  Other product-level restraints on testing  Risks (in test phase)  Specific risks that may affect testing/test outcome  Impact assessment and management plan as it relates to these risks CSE

1 System Test Plan Contents  Features to be tested  List of specific features (from the USERS’ viewpoint) that you will test  Features NOT to be tested  List of those features (USERS’ perspective) that cannot/will not be tested  Describe rationale for not testing each item (not in current release, insufficient test capability, low risk feature that is hard to test, not a user-visible feature) CSE

1 System Test Plan Contents  Approach (Strategy)  Overall test strategy (what, why, and how of your test plan)  Number of hardware and software configurations tested/not tested  Plan to deal with defects identified (re-integration, regression)  Identify metrics for overall success (number of bugs found/corrected/open, number of critical features passed, etc.)  Identify any special requirements for testing CSE

1 System Test Plan Contents  Item Pass/Fail criteria  List specific criteria for each unit/module/feature to be tested (number and severity of defects, any specific tests that indicate system failure, etc.)  Test Deliverables  List what is to be delivered by this test plan (plan document, test cases, error logs, etc.)  Incident/Bug/Problem Reports (IRs)  Test Schedule (should be consistent with your published project plan) CSE

1 System Test Plan - Resources  IEEE Std , IEEE Standard for Software Test Documentation IEEE Std  Test Plan Outline for IEEE Format (from Gerrard Consulting) Test Plan Outline CSE