1. Semester 2, 2003 Week 7 CSE9020 / 1 Software Testing and Quality Assurance With thanks to Shonali Krishnaswamy and Sylvia Tucker

2. Semester 1, 2004 Week 6 CSE9020 / 2 Reading and References n “ Software Testing - A Craftsman’s Approach”, (1995) by Paul C. Jorgensen n “The Complete Guide to Software Testing”, (1988), by Bill Hetzel

3. Semester 1, 2004 Week 6 CSE9020 / 3 Presentation Outline n Introduction n Terminology n Levels of Testing n Types of Testing n Defect Fixing Strategies n Test Process n Bottom line

4. Semester 1, 2004 Week 6 CSE9020 / 4 Testing and Quality n Hetzel - “Testing is the process of establishing confidence that a program or system does what it is supposed to do” n Quality “Meets requirements” n Hetzel proposes that Quality is not tangible

5. Semester 1, 2004 Week 6 CSE9020 / 5 Testing and Quality n The purpose of testing is to make quality visible (? And hence tangible ?) n Testing is the measurement of software quality (including inspection as a testing technique)

6. Semester 1, 2004 Week 6 CSE9020 / 6 Terminology n Test Plan: A document describing the testing approach –Description of the system to be tested –Objectives of testing

7. Semester 1, 2004 Week 6 CSE9020 / 7 Terminology - Test Plan Contents –Project team –Test environment –Resource requirements –Test Schedule

8. Semester 1, 2004 Week 6 CSE9020 / 8 Terminology n Test Procedure: A document defining the steps required to carry out some part of the test plan - that is, execute a set of test cases –Test case name, objective –Description of input required –Description of expected output

9. Semester 1, 2004 Week 6 CSE9020 / 9 Terminology - Test Procedure Some aspects that need attention : –Commands or Macros to run the test case –Location and format of the actual output –Assumptions, Conditions

10. Semester 1, 2004 Week 6 CSE9020 / 10 Terminology n Test Case: A specific set of test data and the corresponding expected results for a particular test objective. –Invalid and Valid test cases n Test Script: The code written to implement the test case

11. Semester 1, 2004 Week 6 CSE9020 / 11 Levels of Testing n Unit Testing: Testing of individual components of a system (sometimes called Basic Functionality Testing)

12. Semester 1, 2004 Week 6 CSE9020 / 12 Levels of Testing n System Testing: Testing in the wider scope - a integrated group of components and a matching sequence of operations

13. Semester 1, 2004 Week 6 CSE9020 / 13 Levels of Testing n Acceptance Testing: Testing to determine if a system is ready for implementation (or, - will the ‘user’ commit to use this ? )

14. Semester 1, 2004 Week 6 CSE9020 / 14 Types of Testing: Functional n For each component, test to confirm that the functional requirements have been satisfied. n A necessary part of all testing

15. Semester 1, 2004 Week 6 CSE9020 / 15 Black Box Testing n The components being examined as being closed – not available to be inspected. n Like when you buy a watch, or a DVD player. Then internals are ‘a black box’

16. Semester 1, 2004 Week 6 CSE9020 / 16 Glass Box Testing n As opposed to black box testing n For each component, the internals are examined, and then a decision is made as to the best way of making up a detailed test specification

17. Semester 1, 2004 Week 6 CSE9020 / 17 Boundary Value Analysis n Guidelines: –Break up the system in to components that have to be tested –Determine the boundary values for these components –Design test cases based on the boundary values n Limitation: Works well with independent variables (or components) with well defined boundaries

18. Semester 1, 2004 Week 6 CSE9020 / 18 Equivalence Classes n Motivations: Completeness with minimal or no redundancies n An equivalence class is a partition of a set n Guidelines: –Identify equivalence classes –Design test cases for each equivalence class n Example: A variable can take only values from 0 to 9 inclusive.

19. Semester 1, 2004 Week 6 CSE9020 / 19 Decision Tables n Condition/Action rules are presented in a tabular form n Test Cases: Conditions are inputs, Actions are outputs 12345 Fixed rate accountTTFFF Variable rate accountFFTTF Consumption < 100 KWHTFTF Consumption >= 100 KWHFTFT Minimum monthly chargeX Schedule A billingXX Schedule B billingX Other treatmentX

20. Semester 1, 2004 Week 6 CSE9020 / 20 Automated vs. Manual Testing n Manual –Every test case is executed manually –Comparison of actual and expected is done manually

21. Semester 1, 2004 Week 6 CSE9020 / 21 Automated vs. Manual Testing n Manual –Suitable for interactive tests –Tedious and time consuming –Simpler

22. Semester 1, 2004 Week 6 CSE9020 / 22 Automated vs. Manual Testing n Automated –A test harness runs the test scripts –Analysis of the results is automated and the results are logged –Testing effort is reduced and may be more efficient in the longer term n Note: The actual test scripts are not generated automatically

23. Semester 1, 2004 Week 6 CSE9020 / 23 Defect Fixing Strategies n Document Defects –full and complete information so the defect can be reproduced –Pass it back to the developer n Fix –Document the fix –Pass it back to the tester n Re-test, regression tests (this is where automated testing can be effective)

24. Semester 1, 2004 Week 6 CSE9020 / 24 Test Process n Develop a test plan (and agree on it) n Specify the test procedure n Perform boundary value analysis / equivalence classes etc.

25. Semester 1, 2004 Week 6 CSE9020 / 25 Test Process n Design and implement test cases n Run the tests n Analyse and document results n Fix Defects, Run the tests, compare the results, and agree that the fixes have worked

26. Semester 1, 2004 Week 6 CSE9020 / 26 Testing Web Sites n Are essentially client/server applications n Consider: –Interaction between.html pages –internet connections –firewalls –applications that run in web pages (client) applets javascript.net approach

27. Semester 1, 2004 Week 6 CSE9020 / 27 Testing Web Sites –applications that run on the server cgi scripts database interfaces logging applications dynamic page generators –variety of servers, browsers (versions) –expected loads on the server, expected performance on client, and server investigate web testing tools n http://www.softwareqatest.com/qatweb1.html

28. Semester 1, 2004 Week 6 CSE9020 / 28 Testing Web Sites n Who is the target audience? –Browsers, connection speed intranet (fast) or internet (may be sloooooow!) –Usability guidelines Pages should be 3-5 screens max., otherwise provide internal links within the page Use a consistent layout, navigation Make pages browser-independent, or generate for a specific browser Include page owner, revision date, contact link NO dead-end pages!

29. Semester 1, 2004 Week 6 CSE9020 / 29 Testing Web Sites n On-going –Performance Testing –Link Testing –Security Testing –HTML validation (browsers change!)

30. Semester 1, 2004 Week 6 CSE9020 / 30 In Summary... n Why? n Who? n When? n What? n How?

31. Semester 1, 2004 Week 6 CSE9020 / 31 Bottom Line n Testing is expensive n Testing is necessary n You are never going to test everything –use Risk Analysis to determine where testing should be focused

32. Semester 1, 2004 Week 6 CSE9020 / 32 Bottom Line n Your Case Study Project is not complete until your client has completed the Acceptance Testing to their satisfaction, and then signed- off the project !

33. Semester 1, 2004 Week 6 CSE9020 / 33 Bottom Line n Most Essential Make sure that –the system –and the group members maintain versioning control (link Software Testing and Quality Assurance with Risks and Ethics)

34. Semester 1, 2004 Week 6 CSE9020 / 34 More Maintenance Pilot Entry: Left inside main tyre almost needs replacement Mechanic Entry: Almost replaced left inside main tyre Pilot Entry: Test flight OK, except autoland very rough Mechanic entry Autoland not installed on this aircraft