Test plans CSCI102 - Systems ITCS905 - Systems MCS Systems

2 Why Do We Need a Test Plan? A test plan isn't to test general stuff or stuff that comes to mind on the day –To ensure that the program does what it is meant to –To ensure that all of the program is tested –To ensure that all requirements have been met However, testing can only determine the presence of defects –Just like inductive proofs vs deductive proofs –You would need to use proofs of correctness to show ‘absence’ of defects

3 Some Test Methods Black - Box Testing White - Box Testing Gray - Box Testing

4 Black - Box Testing Program Inputs determined by requirements Actual results compared to expected results

5 Black - Box Testing Also known as functional testing The tester views the program as a black – box –Only know what they put in and what they get out They –Don’t see the code of the program –Only know the inputs and what the expected outcomes should be and not how the program arrives at those outputs Effectively the tester will be using the public interface of the software

6 Black - Box Testing Advantages –User centred testing –Tester doesn’t need to know how to code –Removal of bias due to autonomy of tester and designer –Test cases can be developed in combination with specifications and design Disadvantages –Redundancy if designer tests –Test cases can be difficult to design

7 White - Box Testing The tester examines the internal structure of the program –Can test at any and every point in the final program Also known as –Structural testing –Glass box –Clear box –Open box

8 White - Box Testing Usually applied to small units Uses specific knowledge of the code to examine outputs –Accurate only if the tester knows what the program is supposed to do –Tests to see if the program deviates from original goal

9 White - Box Testing Advantages –More detail than looking at the output from a program –Tests the actual structure of the program Better than black box testing on individual small modules Can discover “undocumented features” –e.g. non-documented illegal options supplied by a disreputable programmer Disadvantages –Doesn’t account for errors caused by omission

10 Gray - Box Testing Black Box testing can be combined with knowledge of internal coding to confirm functions and expected results Two approaches –Both use only the public interface Analyse code to target boundaries –Artefacts of the implementation –Not implied by the specification May use internal queries to check the results Combines the advantages of black and white box testing

11 Some Other Ways of Testing the Program Desk-checking –Mental check of the logic of the program Walkthroughs –code review by other programmers –source code logic is traced manually by a group with a small set of test cases State of program variables is manually monitored Used to analyse programmer's logic and assumptions Debugging –detect, locate, and correct mistakes in the program Testing –The program for the new system must be tested under a variety of conditions

12 Test Strategies A series of different tests that will fully test program May include –System Test/Black Box –Performance Test –Security Test –Automated Test –Stress and Volume Test –Recovery Test –Documentation Test –Beta Test –User Acceptance Test

13 Contents of a Test Plan Cover Sheet Executive Summary Table of Contents Introduction –Description of this Document Purpose of Report Scope Main Sections –Related Documents Remember the reader may NOT have seen previous reports –Schedule and Milestones

14 Contents of a Test Plan Resource Requirements –Hardware –Software Test Tools –Staffing Responsibilities Training

15 Contents of a Test Plan Test Strategies –Features to be tested Boundaries to be tested Justification Values used to test Expected results –Features not to be tested

16 Contents of a Test Plan Test Deliverables –Documents –Test cases –Requirements validation –etc

17 Contents of a Test Plan Dependencies/Risks Entrance/Exit Criteria –At what point will you suspend testing –Why Report Conclusion