Strategies For Software Test Documentation Jagadeesan.S Assistant Professor Department of CSE

Software Testing Software Testing is a process of evaluating a system against the requirements. The testing can be done either Manually or by using Automated testing tools. By giving Quality deliverables, Customer satisfaction is ensured.

Principle of Testing Objective of Software Tester Testing is the process of executing a program with an intent of finding error and it should be planned well before testing begins. Objective of Software Tester The goal of a Tester is to find defects. The goal of a Tester is to find defects as early as possible. The goal of a Tester is to find defects and fix them.

What is Defects ? There are various ways in which we can classify Defects. Below are some of the classifications: Wrong: The specifications have been implemented incorrectly. This defect is a variance from customer / user specification. (correctly mentioned in specification but wrongly implemented) Missing: A specified or wanted requirement is not in the built product. This can be a variance from specification, an indication that the specification was not implemented. (given in specification but missed out in application) Extra: A requirement incorporated into the product that was not specified. This is always a variance from specifications, But this could be desired feature for the product. (Good to have)

Verification and Validation (V&V) Verification: Refers to the set of activities that ensure the software correctly implements a specific function. "Are we building the product right " Validation: Refers to a different set of activities that ensure the software that has been built is traceable to customer requirements. "Are we building the right product "

Software Testing Documentation KISS principle  - Rule for testing processes (an abbreviation for Keep It Smart, Short ) Test Plan. Test Case. Test Procedure. Test Log. Test Report.

Testing Process Test Plan Define Test Cases Require Analysis & Design Create Test Data Track Defects Analyze results Execute Tests

Definition of Test Plan: A document describing the scope, approach, resources, and schedule of intended testing activities. Table of contents of a test plan might contain the following. Roles & Responsibilities Testing Schedules (testing tasks and milestones) Environment Needs (hardware and software) Test Items (what should be tested) Risk and Constraints Test Recording procedures (test results must be systematically recorded)

Sample Scenario for Test Cases

Definition of Test Case: A set of test inputs, executions, and expected results developed for a particular objective. Set of procedures written by a tester which execute in our system to find defect. -Positive test case. -Negative test case. A test case is said to be effective only when both positive and negative cases are prepared.

Definition of Test Procedure: A document, providing detailed instructions for the [manual] execution of one or more test cases. The procedure document describes how the tester will actually run the test, the physical set-up (Environment) required, and the procedures or steps that need to be followed.

Definition of Test Log: The Test Log records the details of what Test Cases have been run, the order of running, and the results of those tests. The results are either the test passed, meaning that the actual and expected results were identical, or it failed meaning that there was a discrepancy. If there is a discrepancy than one or more Test Incident Reports are raised or updated, and their identities recorded on the Test Log.

Definition of Test Report: To gratify the customer or client’s demand; one must perform complete software testing activities on the application before the deployment as its stated that “No Software Exists Without a BUG”; while performing those actions if any ambiguity found then the tester should report the bug and notify the developer. Elimination of bug from the software needs to follow the proper steps formerly known as BUG LIFE CYCLE; the structure of it varies from organization to organization but the basic flow will remain the same.

Bug life cycle

Bug Life Cycle and Description Open: Tester finds a ‘bug’ and posts it with the status OPEN. This bug is yet to be studied/assigned. Assign: The assigned Developer’s responsibility is now to fix the bug and have it RESOLVED or give valid reasons. Resolved: Developer fixes the bug that is ASSIGNED. Now, the ‘resolved’ bug needs to be verified by the Tester (retesting / regression .

Bug Life Cycle and Description Deferred / Waived: If lead / verifier / developer considers that the bug is “Not valid” then they should give valid reasons to change the status to Deferred or Waived. Re-Opened: If bug still exists not satisfied with the solution in the new build then tester should open the bug again by selecting the status as RE-OPENED. Closed: If bug is successfully resolved and verified by the tester then its status should be marked as CLOSED.

Priority and Severity Severity - Severity is how seriously the bug is impacting the application Priority - Priority is the order in which developer has to fix the bug. High Priority & High Severity: A show stopper error which occurs on the basic functionality of the application. (E.g.. A site maintains the student details on saving record if it doesn't allow to save the record then this is high priority and high severity bug.)

Dichotomies Testing & Debugging 2. Functional Vs Structural Testing 3. Designer vs Tester 4. Buyer vs Builder

Testing Vs Debugging Testing is to find bugs. Debugging is to find the cause or misconception leading to the bug. Their roles are confused to be the same. But, there are differences in goals, methods and psychology applied to these

Testing Debugging Starts with known conditions. Uses predefined procedure. Has predictable outcomes. Starts with possibly unknown initial conditions. End cannot be predicted. Planned, Designed and Scheduled Procedures & Duration are not constrained. A demo of an error or apparent correctness. A Deductive process. Proves programmer’s success or failure. It is programmer’s Vindication. Can be done by outsider to the development team. Must be done by an insider (development team) Test execution and design can be automated Debugging - Automation is a dream.

FUNCTIONAL VS STRUCTURAL TESTING Functional Testing: Treats a program as a black box. Outputs are verified for conformance to specifications from user’s point of view. Structural Testing: Looks at the implementation details: programming style, controlmethod, source language, database & coding details.

Designer/Tester Designer Tester Tests designed by designers With knowledge about internal test design, the tester can eliminate useless tests, optimize & do an efficient test design. Likely to be biased. Tests designed by independent testers are biasfree. Tries to do the job in simplest & cleanest way, trying to reduce the Complexity. Tester needs to suspicious, uncompromising, hostile and obsessed with destroying program.

BUYER VS BUILDER Builder: designs for & is accountable to the Buyer. Pays for the system and hopes to get profits from the services to the User. User: is the ultimate beneficiary of the system. User’s interests are guarded by the Tester

Cont… Tester: works towards the destruction of the software. The tester tests the software in the interests of the user & the operator. Operator: The operator lives with the mistakes of the builder, murky specs of Buyer, oversights of Tester and the complaints of User.

Summary Defects - Wrong, Missing and Extra Software Testing - A process of evaluating a system against the requirements. Software Tester - The goal of a Tester is to find defects as early and fix them. Defects - Wrong, Missing and Extra V&V - Verification "Are we building the product right " and Validation "Are we building the right product " Testing Documentation - Test Plan, Test Case, Test Procedure, Test Log, Test Report. Test Plan - A document describing the scope, approach, resources, and schedule of intended testing activities. Test Case - A set of test inputs, executions, and expected results developed for a particular objective. Test Procedure - A document, providing detailed instructions for the execution of one or more test cases. Test Log - The Test Log records the details of what Test Cases have been run, the order of running, and the results of those tests. Test Report - while performing test actions if any ambiguity found then the tester should report the bug and notify the developer.

Thank You !!!!!!!!